Default file encoding in EG 8.6

[Erdnis]

Hi

After upgrading from EG 8.5 to 8.6, I've noticed that all code files are now stored and opened in UTF-8-BOM instead of the original encoding which in our case is  wlatin1/cp1252. I know I can change the encoding of the file when saving it, but is there a way to start EG with a cp1252 as the default encoding for code files. Server is running wlatin1, so this is not about setting the encoding of the server. That works - however running utf-8 files on a wlatin1 server is not a good idea. 

If the code file is UTF-8 and someone enters a new value with a special character, say æ, ø å, then the server won't understand it and insert a weird character. The log reports this first when reading the a utf-8 encoded file:

NOTE: A byte-order mark in the file "codefile.sas" (for fileref "#LN00005") indicates that the data is encoded in "utf-8".  This encoding will be used to process the file.

This means that if a value in any row uses a special character - it will be replaced with an unreadable character. If this get persisted to a dataset you'll end up in all sorts of trouble later on. This i because file is UTF-8 and server is wlatin1. 

The problem is also the other way - opening a cp1252-file in EG 8.6 it will try to open it with UTF-8-encoding, and it will cripple any special characters that cannot be translated from cp1252 to utf-8 with this character: �. If you then save the file then you are in trouble. Again.

I believe EG 8.5 stored files with cp1252/wlatin1, and it probably honored the encoding (haven't checked), but in 8.6 it will try to set the encoding to UTF-8 BOM regardless.

EG is running on Windows 11 - if it matters (it could - I don't know). Changing the server encoding is not an option. The only possible solution I see right now is to downgrade to 8.5.

[FredrikHansson]

I have the same problem after upgrading to 8.6. I hope we can get a quick fix for this.

[Tom]

Did either of you try opening a ticket with SAS support?

What do they say?

[FredrikHansson]

I did. Half an hour ago. 🙂
I will update this thread when I get an answer.

[FredrikHansson]

I've got an answer. I haven't read it in detail yet, but it seems like its not the answer I was hoping for...

Hello Fredrik,

Thank you for contacting SAS Technical Support and for your inquiry regarding the default file encoding in SAS Enterprise Guide 8.6.

Starting with this version, the default encoding for new program files has been updated to UTF-8. We understand that this change may affect workflows where a different encoding was previously used.

As a workaround we propose the following, (this example is for Norwegian characters, adapt it as required):

1) From EG 8.6, select File > Open > My Computer, and change the "Open as" to be "Program with encoding", as per following screenshot

2) Browse to the code required, and select, where you get will get a pop up as following:

3) Select "Code with encoding" and set encoding to "Western European (Windows) Code Page 1252)". You will then see the ANSI code working as expected, as per the following screenshot:

An alternative workaround would be to convert the program files to UTF-8.

We encourage reviewing encoding settings when working with existing files to ensure alignment with your current environment.

Please don’t hesitate to reach out if we can assist with anything. I hope this answers any questions you have, but if you need anything further then please let us know,

Regards,

[Tom]

Yikes.  What if the program file does have a BOM to indicate it was encoded using something other than UTF-8?  You could probably create such a file using NOTEPAD and test.

This is why I generally avoid placing non 7-bit ASCII characters into program files.  In fact nothing that is not between space ('20'x) and tilde ('7E'f).  If I need to reference some other character in code I try to use hex literals.

[osmelbrito]

Frankly, I can’t believe whoever decided at SAS thought it was a good idea to change the default program encoding with the upgrade to version 8.6. This has completely disrupted my existing program flows—scripts that ran fine before are now throwing errors or displaying garbled characters, all because of an encoding change I never asked for. What’s worse, they didn’t even bother to provide an option to let users select or revert the encoding of their files.

The so-called “solution” that SAS Support recommends doesn’t even appear as an option in the version I’m running. I’m left without any straightforward way to adjust the encoding, and the documentation is useless in this regard. Did anyone at SAS actually consider how this would impact users with established, working code bases? This is incredibly frustrating and frankly unacceptable for a company of this size and reach.

[SASKiwi]

I'd suggest putting your concerns in a Tech Support track if you haven't already. You'll gain better visibility that way. Please post their response back into this thread.

[osmelbrito]

I already did it

[FredrikHansson]

@Erdnis Could you please (if possible) unmark this reply as a solution? I guess this workaround is sufficient for some users, but for many (most?) users it's not acceptable.

[Erdnis]

@FredrikHansson It's a solution, although not a very good one. Unmarked anyway.

Personally I downgraded to 8.5, and wont deal with it anymore. That's my solution.

[FredrikHansson]

So the solution above didn't really solve your your problem then? 😉
Thank you for unmarking it. 🙏

I also have to downgrade. Using the workaround 20 times a day is not possible.
How did you manage to downgrade? 🤔 My installation depot contains EG8.31 and I don't want to go that far back.

[Erdnis]

Well, the solution would probably work, but you have to be very careful when operating on cp1512 vs utf-8-files. 

I like to think that I downgrade to 8.5, but I'm now running EG 8.3 update 6. I won't spend time researching how to get to 8.5.

[andreas_lds]

It is even worse: https://communities.sas.com/t5/Administration-and-Deployment/Default-file-encoding-in-EG-8-6-continu....

I'd love to see an option to set the default encoding. The IT is using utf-8-bom on most code files, the statisticians still have their code in latin1.

Avoiding national language chars in labels would render them useless.