topic Re: Unicode Output in SAS Procedures

Unicode Output

OS2Rules — Tue, 22 Nov 2011 19:26:08 GMT

Hi All:

I have been asked to output a file (just text) in Unicode (UTF-8).

My code is:

data work02;

file "&outfile" encoding="utf-8" notitles lrecl=400;

set work01;

text1 = compress(text,'"');

put @001 text1 $utf8x400.;

run;

and it seems to run OK.

When I look at the resulting file, there only seems to be a couple of funny characters at the biginning of the first record and nothing else different. How can I confirm that the results are in Unicode format?

Thanks in advance.

Unicode Output

art297 — Tue, 22 Nov 2011 19:40:01 GMT

There are some examples at: http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a000211297.htm

One easy test would simply be to reverse the process, i.e., to import the resulting file using the same encoding (i.e., utf-8).

Of course, if the file is really comprised of multiple fields, I would think that you would have to have exported it as multiple fields in order for the end user to be able to accurately treat it as being in unicode format. My limited experience (1 time) dealing with that format was with a file that contained nulls between every field.

Unicode Output

chang_y_chung_hotmail_com — Wed, 23 Nov 2011 14:37:08 GMT

All the 7-bit ascii characters are also valid utf-8 encoded unicode characters -- utf-8 encoding scheme was specifically designed to be this way. As long as your input characters are ascii (no extended characters), your output should be the same. hth.

Unicode Output

OS2Rules — Mon, 28 Nov 2011 19:09:10 GMT

All:

Thanks for all the suggestions....

Now - another (minor) issue.

How do I remove the trailing blanks from the output record written in the Unicode format (as noted above). The client wants the trailing blanks removed...

If I use just "$utf8x." as the output format, the default length is 8 chars.

Anyone ???

Re: Unicode Output

Tom — Mon, 28 Nov 2011 19:24:44 GMT

I do not think that you need to use the $utf8x format. The ENCODING option on the FILE statement should be enough. You can use the $VARYING format to write varying lengths to the output file.

data _null_;
  file "&outfile" encoding="utf-8" lrecl=400;
  set work01;
  text1 = compress(text,'"');
  len=length(text1);
  put text1 $varying400. len; 
run;

Note the two bytes at the beginning of the file is the signal to SAS and other programs what encoding is being used. Try different encodings and you should see different values in those first two bytes.

Re: Unicode Output

OS2Rules — Mon, 28 Nov 2011 19:28:34 GMT

Sorry - but I have to answer my own question here.... (after a call to SAS).

Just adding the format modifier ":" (full colon) before the unicode output format worked.

So the format becomes ":$utf8x400."

Re: Unicode Output

art297 — Mon, 28 Nov 2011 21:49:19 GMT

Why sorry? You got the answer to your question and, better yet, shared it with the rest of us. Mark your last post as the one that had the correct answer.