topic Re: SAS Code in EG get garbled in SAS Enterprise Guide

SAS Code in EG get garbled - UTF-8 encoding

ChaseLiu — Thu, 04 May 2017 11:30:33 GMT

Hi Communities.

My code which saved in UTF-8 format get garbled when open in SAS EG.

I googled and searched, but don't find any solution to solve this.

Besides, when open this code in SAS DM, it is displayed normally.

Re: SAS Code in EG get garbled

Kurt_Bremser — Wed, 03 May 2017 06:04:49 GMT

How did you save SAS code as UTF-8? .sas files are plain ASCII.

Re: SAS Code in EG get garbled

ChaseLiu — Wed, 03 May 2017 06:33:34 GMT

Hi Kurt,

The code is developed in external editor using UTF-8 encoding.

Re: SAS Code in EG get garbled

Kurt_Bremser — Wed, 03 May 2017 06:37:02 GMT

@ChaseLiu wrote:

Hi Kurt,

The code is developed in external editor using UTF-8 encoding.

Why, in $DEITY's name? What's wrong with the SAS Enhanced Editor?

The fact that you have that problem clearly illustrates the error of your ways.

Re: SAS Code in EG get garbled

ChaseLiu — Wed, 03 May 2017 07:12:10 GMT

SAS Enhanced Editor is too slow and outdated compared with those modern IDEs. Neither SAS EG could compare with it.

Besides, Many modern IDEs support customized function which great enchane the productivity.

I do not think it's the way that cause my problems. instead, it's the way SAS EG works.

I never encounter such a problem in traditional SAS.

Re: SAS Code in EG get garbled

Kurt_Bremser — Wed, 03 May 2017 07:20:59 GMT

@ChaseLiu wrote:

SAS Enhanced Editor is too slow and outdated compared with those modern IDEs.

Grmmpf. Too slow? I've never experienced any delay when typing in the EE in EG (the Enhanced Editor in Base SAS and Enterprise Guide is the same, literally the same dll if you have local SAS). Any delay I get is when I send programs for execution over the network, and the runtime of code itself. And in what aspect is the EE "outdated"?

Re: SAS Code in EG get garbled

ChaseLiu — Wed, 03 May 2017 09:02:51 GMT

What I said 'Slow', I mean developing code is slow.

I'm here to find a solution rather than argue with such a question. If you think SAS is not outdated, then be it.

Re: SAS Code in EG get garbled

Kurt_Bremser — Wed, 03 May 2017 09:33:41 GMT

How did you open the code in SAS DM? Do you work across platforms (eg SAS with DM on UNIX, EG (naturally) on Windows)?

Seeing what you are doing in your code, one workaround (and significant improvement of the code) would be to store the lookup in a dataset, create a format dynamically from that, and use that format in a single put statement instead of that long if-then-else chain. That would remove the special UTF characters (which I guess are part of the strings) from the code.

And anytime you have a if-then-else chain, look at the select data step statement. Won't solve your immediate problem, but makes code more readable.

Re: SAS Code in EG get garbled

Tom — Wed, 03 May 2017 12:13:18 GMT

Is there a setting in Enterprise Guide to tell it to use UTF-8 encoding?

Re: SAS Code in EG get garbled

ChaseLiu — Wed, 03 May 2017 12:53:14 GMT

Hi Kurt,

Thanks for your suggestion, I think your advice would be a work around. But I do not think this is a long-term solution.

What confused me a lot is that while I can open code normally in PC SAS but not in EG. This is very weird and I think there should have some ways to work out.

Re: SAS Code in EG get garbled

ChaseLiu — Wed, 03 May 2017 12:56:55 GMT

This is exactly what I'm asking.

while I'm using EG in windows environment, it seems that EG default the encoding to windows default encoding.

Re: SAS Code in EG get garbled - UTF-8 encoding

ChrisHemedinger — Thu, 04 May 2017 11:30:51 GMT

You can tell SAS Enterprise Guide to open a file with encoding. File->Open->Program, and then in the file dialog there is a drop down menu on the Open button:

You'll be presented with a long list of possible encoding options.

Usually EG will detect the encoding automatically, but some editors don't always save the proper cues for others to pick it up.

Re: SAS Code in EG get garbled

Tom — Wed, 03 May 2017 13:51:56 GMT

What version of EG does that menu appear? I do not see it in 6.1 M1HF5 (6.100.0.4180) (64-bit)

Re: SAS Code in EG get garbled

ChrisHemedinger — Wed, 03 May 2017 14:09:31 GMT

Hi @Tom - it was added a long time ago, I think in EG 5.1.

2 step process depicted:

Re: SAS Code in EG get garbled

CaseySmith — Wed, 03 May 2017 17:03:00 GMT

Are you opening your .sas program through your local file system or through the SAS server (Files location)? Have you tried saving your .sas file in UTF-8 with a byte order mark (BOM) to see if that allows EG to discover the encoding?

Re: SAS Code in EG get garbled

ChaseLiu — Thu, 04 May 2017 01:53:36 GMT

Hi Chris,

Thanks for your great solution. It's working!

Just like you said, It's because EG could not detect the encoding in my SAS program by default.

Re: SAS Code in EG get garbled

Kurt_Bremser — Thu, 04 May 2017 05:39:24 GMT

@ChaseLiu wrote:

It's because EG could not detect the encoding in my SAS program by default.

Which happens because your IDE does not supply the BOM that tells other software that the file is UTF coded. You might look if there's an option for that.

Re: SAS Code in EG get garbled

ChaseLiu — Thu, 04 May 2017 06:57:28 GMT

@Kurt_Bremser You are right. The editor save file in UTF-8 without BOM by default. Thanks for you advice! I think my confuse has been completely addressed.

Re: SAS Code in EG get garbled - UTF-8 encoding

DumptyHumpty — Fri, 26 Jan 2018 13:34:38 GMT

I had a related issue with thousands of *.sas files from external sources in UTF8 that were incorrectly handled by the EG due to the missing Byte Order Mark (BOM) in the files. Opening and saving them manually in EG with the correct encoding information was not an option.

I added the BOM programmatically which made EG detect the encoding correctly. Due to my personal preferences I coded it in linux bash. Powershell should look similiar.

 echo -ne '\xEF\xBB\xBF' > prog_utf8_BOM.sas && cat prog_utf8.sas >> prog_utf8_BOM.sas &&  mv prog_utf8_BOM.sas prog_utf8.sas

Re: SAS Code in EG get garbled

NMad — Mon, 27 Aug 2018 10:00:35 GMT

"Use of a BOM is neither required nor recommended for UTF-8, but may be encountered in contexts where UTF-8 data is converted from other encoding forms that use a BOM or where the BOM is used as a UTF-8 signature"

https://www.unicode.org/versions/Unicode6.0.0/ch02.pdf