topic Issue in reading Japanese character with UTF-8 encoding, current SAS env has Latin1 Western encoding in SAS Programming

Issue in reading Japanese character with UTF-8 encoding, current SAS env has Latin1 Western encoding

RaSha — Fri, 17 Apr 2020 07:20:08 GMT

Hey There,

I am trying to read a column holding Japanese character from an excel sheet via infile or proc import statement but getting boxes in output. Tried to convert input sheet to unicode txt and UTF-8 csv but no luck. Current system encoding is Latin1 western (ISO). Appreciate your help. Many thanks

Sample code -

filename temp '<location>/Bookutf8.csv' encoding='UTF-8';
data test;
infile temp dlm=',' dsd firstobs=2 encoding='utf-8'
input variable1 $ variable2 $;
run;

Re: Issue in reading Japanese character with UTF-8 encoding, current SAS env has Latin1 Western enco

andreas_lds — Fri, 17 Apr 2020 07:45:59 GMT

If the data contains chars that don't exist in Latin1, you must change the encoding of you sas-session to read the data properly.

Re: Issue in reading Japanese character with UTF-8 encoding, current SAS env has Latin1 Western enco

RaSha — Fri, 17 Apr 2020 11:46:09 GMT

Hi.. Thanks for your reply..

I tried to use encoding statement in data step but it didn't work and I also tried to add the statement in configuration file. After which the metadata server threw errors and didn't connect ..

Any sample code which you can suggest may work?

Thanks in advance

Re: Issue in reading Japanese character with UTF-8 encoding, current SAS env has Latin1 Western enco

RaSha — Fri, 17 Apr 2020 14:51:38 GMT

Re: Issue in reading Japanese character with UTF-8 encoding, current SAS env has Latin1 Western enco

Tom — Fri, 17 Apr 2020 16:14:03 GMT

If your SAS session is running using a single byte encoding, like WLATIN1, then there are only 256 possible characters it can represent. And none of them are Japanese characters.

You probably need to change the encoding for the whole SESSION, not just how this one file is being read. That must be done when you start SAS, you cannot do it in the program itself.

Check with your local SAS support how you can start SAS using UTF-8 encoding or UNICODE support. If you are running SAS Display Manager then your typically use a different command (or click on a different ICON in Windows). If you are using Enterprise Guide or SAS/Studio you connect to a different SAS application server.

Re: Issue in reading Japanese character with UTF-8 encoding, current SAS env has Latin1 Western enco

ChrisNZ — Sat, 18 Apr 2020 00:24:25 GMT

I can't verify as my session is UTF8, but try to set the dataset's encoding to UTF8, like you did for the filename.

You won't be able to display the Japanese characters in your wlatin1 SAS session though.

Also check that the variable lengths are sufficient for storing multi-byte characters.