How to migrate SAS 9.4 latin1 codes into Viya utf-8

idziemianczyk — Wed, 17 Aug 2022 07:43:37 GMT

In SAS9.4 there is a rule 1 byte = 1 character.

Viya comes with Utf-8 and this rule no longer working. In utf-8 special characters can use 2,3,4 bytes, and some of the "standard" functions from SAS9.4 are not working in Viya.

K-functions are described here:

https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2018/1902-2018.pdf

and here:

https://documentation.sas.com/doc/en/vdmmlcdc/8.1/nlsref/p0ydidiqibf0vbn1jljj1cv9men9.htm

When we run the following code in SAS9.4 and SAS Viya 3.5 we'll see different results:

DATA test;
  name = 'éàè@&ùµ';
  len = LENGTH(name);
  sub = SUBSTR(name,5);
  klen = KLENGTH(name);  
  ksub = KSUBSTR(name,5);
RUN;  
proc print data=test; run;

The result in SAS9.4

The result in SAS Viya 3.5

As we can see, very popular functions like LENGTH or SUBSTR return the wrong results in Viya and we should use KLENGTH and KSUBSTR.

The question is:

Are there any other ways to migrate code from SAS9.4 to Viya3.5 without changing all non-K-Functions to K-Functions?

At that moment it seems that backward compatibility doesn't work between SAS 9.4 code and SAS Viya 3.5.

Re: How to migrate SAS 9.4 latin1 codes into Viya utf-8

andreas_lds — Wed, 17 Aug 2022 10:18:06 GMT

From the docs of length:

The LENGTH function returns a length in bytes. The KLENGTH function returns a length in a character-based unit.

So length returns the correct value in both environments, unfortunately many sas user (including myself) have ignored the fact, that the result depends on the encoding. Our 9.4 environment hast two SASApps, one for latin1 the other one for utf-8.

Are there any other ways to migrate code from SAS9.4 to Viya3.5 without changing all non-K-Functions to K-Functions?

Afaik there is no other way.

topic How to migrate SAS 9.4 latin1 codes into Viya utf-8 in SAS Programming

How to migrate SAS 9.4 latin1 codes into Viya utf-8

Re: How to migrate SAS 9.4 latin1 codes into Viya utf-8