topic Re: Abnormal characters in sas file in SAS Programming

Abnormal characters in sas file

Eileen1496 — Sat, 16 Sep 2023 14:28:06 GMT

Hi all,

I got a 4gb SAS dataset with company information. The company name column is very important to me. However, I found though most of the observations are in normal format, e.g. "Birly Co", there are many rows has values like "ÃƒÂ". Do you know if this is a problem in the original dataset that I cannot solve as a data user, or it is a problem of encoding on my computer?

I actually tried encoding it to utf-8 or utf-16le, utf-16be in python, but still I got such weird characters.

Thank you!

Re: Abnormal characters in sas file

Kurt_Bremser — Sat, 16 Sep 2023 15:46:15 GMT

What is your session encoding?

Re: Abnormal characters in sas file

Eileen1496 — Sat, 16 Sep 2023 16:18:21 GMT

Hi Kurt, what do you mean by "session encoding"? I tried utf-8, utf-16le,
utf-16be in python. In sas I just open the file directly because it is
originally in sas7bdat format. And it also shows the abnormal chars.

Re: Abnormal characters in sas file

Kurt_Bremser — Sat, 16 Sep 2023 17:48:49 GMT

In SAS, run

proc options option=encoding value;
run;

to see how the session encoding is set.

Re: Abnormal characters in sas file

Eileen1496 — Wed, 20 Sep 2023 13:24:40 GMT

Hi Kurt! It return the following to me:

SAS (r) Proprietary Software Release 9.4 TS1M6

Option Value Information For SAS Option ENCODING
Value: WLATIN1
Scope: SAS Session
How option value set: Config File
Config file name:
C:\Program Files\SASHome\SASFoundation\9.4\nls\en\sasv9.cfg

Should I change anything so this problem can be fixed?

Re: Abnormal characters in sas file

Kurt_Bremser — Wed, 20 Sep 2023 18:00:49 GMT

You will need to have a session started with utf-8 encoding to properly use the dataset. Talk with your SAS administrators how to get one.

Re: Abnormal characters in sas file

Eileen1496 — Wed, 20 Sep 2023 18:08:37 GMT

Dear Kurt, thanks for pointing out the problem! I would never realize that alone probably, I will talk with them.

Re: Abnormal characters in sas file

Eileen1496 — Mon, 25 Sep 2023 14:17:06 GMT

Dear Kurt, thanks for pointing out the problem! I would never realize that alone probably, I will talk with them.