How to fix the encoding problem

A_n_ — Thu, 17 Feb 2022 19:20:10 GMT

Hi all,

I use SAS Studio and have the problem with encoding of sas dataset.

data dt;
set inlib.dt;
run;

I get the error and 0 row in work.dt dataset.

ERROR: Some character data was lost during transcoding in the dataset WORK.DT. Either the data contains characters that are not representable in the new encoding or truncation occurred during transcoding.

So, I run the next code. And we can see that encodings are equal.

%let dsn=inlib.dt;
%let dsid=%sysfunc(open(&dsn,i));
%let encoding=%sysfunc(attrc(&dsid,encoding)).;
%let rc=%sysfunc(close(&dsid));
%put The encoding for data set &dsn is: &encoding.;
%put The SAS session encoding is: &sysencoding.;

==> Log:

    %let dsn=inlib.dt;
    %let dsid=%sysfunc(open(&dsn,i));
    NOTE: Data file INLIB.DT.DATA is in a format that is native to another host, or the file encoding does not match the session encoding. Cross Environment Data Access will be used, which might require additional CPU resources and might reduce performance.
    WARNING: Some character data was lost during transcoding in the dataset INLIB.DT. Either the data contains characters that are not representable in the new encoding or truncation occurred during transcoding.
    %let encoding=%sysfunc(attrc(&dsid,encoding)).;
    %let rc=%sysfunc(close(&dsid));
    %put The encoding for data set &dsn is: &encoding.;
    The encoding for data set inlib.dt is: utf-8 Unicode (UTF-8).
    %put The SAS session encoding is: &sysencoding.;
    The SAS session encoding is: utf-8

SAS session encoding and data set encoding are the same.

What's wrong? What to do?

inlib.dt contains the Cyrillic symbols. I can open this dataset in SAS Studio and all symbols display ok.

Thank you!

Re: How to fix the encoding problem

Tom — Thu, 17 Feb 2022 19:29:06 GMT

So either the creator of the dataset provided invalid UTF-8 bytes in the file.

Or the other issue mentioned in the error message is happening. Is the file corrupted?

Re: How to fix the encoding problem

A_n_ — Thu, 17 Feb 2022 20:15:43 GMT

I think you are right about invalid UTF-8 bytes in the file.
I try with other dataset. And it's okay.

The first dataset doesn't seem corrupted because I can open it in SAS Studio and can use "proc print".

So, I think the problem is decided 🙂 Need the new correct dataset.

Thanks.

topic Re: How to fix the encoding problem in SAS Programming

How to fix the encoding problem

Re: How to fix the encoding problem

Re: How to fix the encoding problem