Solaris | Encoding

[MariaD]

Hi @LeonidBatkhan ,

On sasv9.cfg ENCODING settings is already as LATIN1.

Regards, 

[Kurt_Bremser]

The change needs to be made in sasconf/Lev1/SASApp/sasv9_usermods.cfg, after which your SAS services need to be restarted.

[ChrisNZ]

So you see 2 folders on the server? Histórico and Histórico?

When is the second one created?  After the libname statement is run?

[MariaD]

Hi @ChrisNZ ,

The first one was created through SAS Enterprise Guide, under Files Section (New Folder and then renamed to "Histórico"). After we created through SAS Enterprise Guide, we validate using PuTTY and we discovered that the folder created appears as "Histórico". Using the same user, but through PuTTY interface, we create a new folder using mkdir command, in this case the folder created appears as " Histórico".

Regards, 

[Tom]

---

@MariaD wrote:

Hi @ChrisNZ ,

The first one was created through SAS Enterprise Guide, under Files Section (New Folder and then renamed to "Histórico"). After we created through SAS Enterprise Guide, we validate using PuTTY and we discovered that the folder created appears as "Histórico". Using the same user, but through PuTTY interface, we create a new folder using mkdir command, in this case the folder created appears as " Histórico".

Regards, 

---

By PUTTY do you mean that you used that software to open a terminal window on your unix system and typed the unix command mkdir at the command prompt? What encoding was your terminal window using?  What happens if you use SAS command to make the directory?

data _null_;
  infile 'mkdir /path with non 7bit ascii letters' pipe;
  input;
  put _infile_;
run;

[MariaD]

Hi @Tom ,

Follows the results:

NOTE: The infile 'mkdir /path/test' is:
      Pipe command="mkdir /path/test"

NOTE: 0 records were read from the infile 'mkdir /path/test'.
NOTE: DATA statement used (Total process time):
      real time           0.08 seconds
      cpu time            0.01 seconds

By terminal, such PuTTY or command, folder or files with special characters are created find and we can navigate through it.

One more point, we can't see SAS tables with special characters on name under LIBNAME. We know this is not the best practices, but sometimes our business users create this type of file too. 

Regards, 

[Kurt_Bremser]

Make it a habit (and beat this into the heads of all your users) that you NEVER use anything other than

• the 26 letters of the standard alphabet
• the 10 digits of the decimal system
• underlines

in directory and file names on servers. NEVER. It only causes problems and serves no real purpose.

Client settings can cause all kinds of problems when trying to access files with UTF characters in the name.

Anecdote:

My MacBook failed to read mp3 files on my family NAS because an idiot system (Windows) wrote the single quotes in some titles as UTF single quotes, which appeared as funny character sequences in the Finder. Probably because the NAS coughed up on it.

[ChrisNZ]

@MariaD 

I agree with @Kurt_Bremser: IT is still a mess. Just make life easier for yourself by avoiding silly issues.

And I should add that his native language (like mine) uses "accented" letters too, so his reply is not born out of intolerance for funny squiggles. 

[MariaD]

Hi @Kurt_Bremser ,

Yes, you are correct but some business user uses latin characters on folders and files, even when we tried to suggest to not to use. Our previous SAS server was Linux, so we have a lot of files and folders with this situation.

Regards,

[Tom]

First step is to run SAS using UTF-8 encoding.  This has to be set when the SAS session starts, you cannot change it later.

Then try making a file or directory using some non-7 bit ASCII character, like your example o with accent.

Then try seeing what the name looks like in the filesystem.  For example you might try reading the output of the LS command.

data test;
  infile
"cd /usuarios/nuevo_usuario/
;ls -d His*
"  pipe truncover ;
   input filename $30. ;
   put _n_= filename=  +1 filename $hex60. ;
run;

Now you should be able to see the names of the two different His... directories and the actual character codes.

Here is my attempt to see the difference in what bytes that name has when using UTF-8 vs LATIN encoding by running the same data step in two different SAS sessions. 

Notice how just pasting the UTF-8 text into this forum causes the UTF-8 character to be interpreted as two independent non-7bit ASCII characters.

22    data _null_;
23      length encoding name $20;
24      encoding=getoption('encoding');
25      name = 'Histórico';
26      len=length(name);
27      klen=klength(name);
28      put (_all_) (=) / name $hex. ;
29    run;

encoding=UTF-8 name=Histórico len=10 klen=9
48697374C3B37269636F20202020202020202020

1317  data _null_;
1318    length encoding name $20;
1319    encoding=getoption('encoding');
1320    name = 'Histórico';
1321    len=length(name);
1322    klen=klength(name);
1323    put (_all_) (=) / name $hex. ;
1324  run;

encoding=WLATIN1 name=Histórico len=9 klen=9
48697374F37269636F2020202020202020202020

So when using UTF-8 the character is coded as C3 B3 and when running WLATIN1 the character is coded as F3.