How to solve this? ERROR: Some character data was lost during transcoding in the dataset XXXX.

[tmartinma]

libname lib 'address';

data sample;
set lib.dataset1;
where variable='13';
run;

Tried running these simple line of codes to extract part of the dataset.

But encountered these weird error codes repeatedly.

Only part of the dataset/program can go through. Which leads to an incomplete result dataset with lots of missing records.

See below for the log.

1                                                          The SAS System                            09:45 Tuesday, January 19, 2021

1          ;*';*";*/;quit;run;
2          OPTIONS PAGENO=MIN;
3          %LET _CLIENTTASKLABEL='libname';
4          %LET _CLIENTPROCESSFLOWNAME='Process Flow';
5          %LET _CLIENTPROJECTPATH='XXX';
6          %LET _CLIENTPROJECTPATHHOST='XXX';
7          %LET _CLIENTPROJECTNAME='XXX';
8          %LET _SASPROGRAMFILE='';
9          %LET _SASPROGRAMFILEHOST='';
10        
11         ODS _ALL_ CLOSE;
12         OPTIONS DEV=PNG;
13         GOPTIONS XPIXELS=0 YPIXELS=0;
14         FILENAME EGSR TEMP;
15         ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR
16             STYLE=HtmlBlue
17             STYLESHEET=(URL="file:///C:/Program%20Files/SASHome/SASEnterpriseGuide/7.1/Styles/HtmlBlue.css")
18             NOGTITLE
19             NOGFOOTNOTE
20             GPATH=&sasworklocation
21             ENCODING=UTF8
22             options(rolap="on")
23         ;
NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR
24        
25         GOPTIONS ACCESSIBLE;
26        
27         libname lib 'xxxx';
NOTE: Libref lib was successfully assigned as follows:
      Engine:        V9
      Physical Name: xxxx
28        
29         data xxxx;
30         set xxx.xxxx;
NOTE: Data file xxx.xxxx.DATA is in a format that is native to another host, or the file encoding does not match the session
      encoding. Cross Environment Data Access will be used, which might require additional CPU resources and might reduce
      performance.
31         where xxx='13';
32         run;

ERROR: Some character data was lost during transcoding in the dataset xxx.xxx. Either the data contains characters that are
       not representable in the new encoding or truncation occurred during transcoding.
NOTE: The DATA step has been abnormally terminated.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 38067 observations read from the data set xxx.xxxx.
      WHERE xxx='13';
WARNING: The data set xxx.xxxx may be incomplete.  When this step was stopped there were 38067 observations and 32 variables.
NOTE: DATA statement used (Total process time):
      real time           0.94 seconds
      cpu time            0.81 seconds
     

33        
34        
35        
36         GOPTIONS NOACCESSIBLE;
37         %LET _CLIENTTASKLABEL=;
2                                                          The SAS System                            09:45 Tuesday, January 19, 2021

38         %LET _CLIENTPROCESSFLOWNAME=;
39         %LET _CLIENTPROJECTPATH=;
40         %LET _CLIENTPROJECTPATHHOST=;
41         %LET _CLIENTPROJECTNAME=;
42         %LET _SASPROGRAMFILE=;
43         %LET _SASPROGRAMFILEHOST=;
44        
45         ;*';*";*/;quit;run;
46         ODS _ALL_ CLOSE;
47        
48        
49         QUIT; RUN;
50        

[ElisabettaC]

Hi,
Try to overwrite the table with:
data lib.dataset1;
set lib.dataset1;
run;

and then apply the data step:

data sample;
set lib.dataset1;
where variable='13';
run;

ElisabettaC

[BrunoMueller · Accepted solution]

Editor's note (10/9/23): This blog post provides additional context and help.

This message often comes from incompatible encoding between your SAS dataset and the SAS encoding.

Use Proc CONTENTS on your input dataset to see the encoding used and see how it compares to your current SAS encoding.

To display the SAS encoding use:

proc options option=encoding define;
run;

To be able to read the data anyway, you can use the following code:

data newdata;
set olddata(encoding=any);
run;

Please be aware, that the chars that could not be converted will leave you with some strange text.

If you know the encoding of the SAS datasets you can use the KPROPDATA function to convert it like so:

data newdata;
set olddata(encoding=any);
str = KPROPDATA(v1, "UESC", "UTF-8"); 
run;