Default file encoding in EG 8.6

[Erdnis]

Hi

After upgrading from EG 8.5 to 8.6, I've noticed that all code files are now stored and opened in UTF-8-BOM instead of the original encoding which in our case is  wlatin1/cp1252. I know I can change the encoding of the file when saving it, but is there a way to start EG with a cp1252 as the default encoding for code files. Server is running wlatin1, so this is not about setting the encoding of the server. That works - however running utf-8 files on a wlatin1 server is not a good idea. 

If the code file is UTF-8 and someone enters a new value with a special character, say æ, ø å, then the server won't understand it and insert a weird character. The log reports this first when reading the a utf-8 encoded file:

NOTE: A byte-order mark in the file "codefile.sas" (for fileref "#LN00005") indicates that the data is encoded in "utf-8".  This encoding will be used to process the file.

This means that if a value in any row uses a special character - it will be replaced with an unreadable character. If this get persisted to a dataset you'll end up in all sorts of trouble later on. This i because file is UTF-8 and server is wlatin1. 

The problem is also the other way - opening a cp1252-file in EG 8.6 it will try to open it with UTF-8-encoding, and it will cripple any special characters that cannot be translated from cp1252 to utf-8 with this character: �. If you then save the file then you are in trouble. Again.

I believe EG 8.5 stored files with cp1252/wlatin1, and it probably honored the encoding (haven't checked), but in 8.6 it will try to set the encoding to UTF-8 BOM regardless.

EG is running on Windows 11 - if it matters (it could - I don't know). Changing the server encoding is not an option. The only possible solution I see right now is to downgrade to 8.5.

[FredrikHansson]

I have the same problem after upgrading to 8.6. I hope we can get a quick fix for this.

[Tom]

Did either of you try opening a ticket with SAS support?

What do they say?

[FredrikHansson]

I did. Half an hour ago. 🙂
I will update this thread when I get an answer.

[FredrikHansson · Accepted solution]

I've got an answer. I haven't read it in detail yet, but it seems like its not the answer I was hoping for...

Hello Fredrik,

Thank you for contacting SAS Technical Support and for your inquiry regarding the default file encoding in SAS Enterprise Guide 8.6.

Starting with this version, the default encoding for new program files has been updated to UTF-8. We understand that this change may affect workflows where a different encoding was previously used.

As a workaround we propose the following, (this example is for Norwegian characters, adapt it as required):

1) From EG 8.6, select File > Open > My Computer, and change the "Open as" to be "Program with encoding", as per following screenshot

2) Browse to the code required, and select, where you get will get a pop up as following:

3) Select "Code with encoding" and set encoding to "Western European (Windows) Code Page 1252)". You will then see the ANSI code working as expected, as per the following screenshot:

An alternative workaround would be to convert the program files to UTF-8.

We encourage reviewing encoding settings when working with existing files to ensure alignment with your current environment.

Please don’t hesitate to reach out if we can assist with anything. I hope this answers any questions you have, but if you need anything further then please let us know,

Regards,

[Tom]

Yikes.  What if the program file does have a BOM to indicate it was encoded using something other than UTF-8?  You could probably create such a file using NOTEPAD and test.

This is why I generally avoid placing non 7-bit ASCII characters into program files.  In fact nothing that is not between space ('20'x) and tilde ('7E'f).  If I need to reference some other character in code I try to use hex literals.