topic UTF-8 and alternative to $hex. format in SAS Programming

UTF-8 and alternative to $hex. format

xxformat_com — Sun, 08 May 2022 18:38:59 GMT

Here the first character is 56, the second one is C3A9, the third one is 72 and the fourth one is 6F:

How do we know that the first character is not 56C3 for example?

Is there a new format or feature which would allow to identify explicitely each character?

Eg. get a value like 56-C3A9-72-6F in a similar way as the e8601da format as alternative to b8601da format for ISO dates.

This would be useful as this kind of code helps identifying specific characters... to replace them.

I've seen the kpropdata function but not tested it. It could cover most of the common situations but if a format solution exists, it would be good to know about it.

Re: UTF-8 and alternative to $hex. format

yabwon — Sun, 08 May 2022 20:08:43 GMT

Try this:

data  _null_;
  text = "12ŻÓŁĆ34";
  do i = 1 to klength(text);
    x = ksubstr(text,i,1);
    t = putc(x,cats("hex",length(x)*2,"."));
    put t @@;
  end;
  put;
run;

log shows:

1    data  _null_;
2      text = "12ŻÓŁĆ34";
3      do i = 1 to klength(text);
4        x = ksubstr(text,i,1);
5        t = putc(x,cats("hex",length(x)*2,"."));
6        put t @@;
7      end;
8      put;
9    run;

31 32 C5BB C393 C581 C486 33 34
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds

Bart

Re: UTF-8 and alternative to $hex. format

yabwon — Sun, 08 May 2022 20:30:49 GMT

You can also just build yourself a format:

proc FCMP outlib=work.f.p;
  function myHex(text $) $;
    length ret $ 32767;
    do i = 1 to klength(text);
      x = ksubstr(text,i,1);
      ret = catx("-", ret, putc(x,cats("hex",length(x)*2,".")));
    end;
    return(ret);
  endsub;
run;

options cmplib=work.f;

proc format;
  value $ myHex (default=200)
  other = [myHex()]
  ;
run;

data  _null_;
  text = "12ŻÓŁĆ34";
  put text $myHex.;
run;

[EDIT] one more approach, more "readable"

proc FCMP outlib=work.f.p;
  function myHex(text $) $;
    length ret $ 32767 x $ 4 t $ 8;
    do i = 1 to klength(text);
      x = ksubstr(text, i, 1);

      select(length(x));
        when (1)  t = put(x, $hex2.);
        when (2)  t = put(x, $hex4.);
        when (3)  t = put(x, $hex6.);
        otherwise t = put(x, $hex8.);
      end;

      ret = catx("-", ret, t);
    end;
    
    return(ret);
  endsub;
run;

Bart