topic Re: ERROR: Some character data was lost during transcoding in SAS Programming

ERROR: Some character data was lost during transcoding

fengyuwuzu — Tue, 08 Jun 2021 19:00:20 GMT

When I tried to re-run an old program I got the following error: ERROR: Some character data was lost during transcoding in the dataset BASE_P.DAT_RAND. Either the data contains characters that are not representable in the new encoding or truncation occurred during transcoding.

Does any one know what the problem is? Thanks.

27         PROC SORT data=base_p.Dat_rand out=Dat_rand; BY SUBNUM; RUN;
NOTE: Data file BASE_P.DAT_RAND.DATA is in a format that is native to another host, or the file encoding does not match the session 
      encoding. Cross Environment Data Access will be used, which might require additional CPU resources and might reduce 
      performance.

ERROR: Some character data was lost during transcoding in the dataset BASE_P.DAT_RAND. Either the data contains characters that are 
       not representable in the new encoding or truncation occurred during transcoding.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 9 observations read from the data set BASE_P.DAT_RAND.
WARNING: The data set WORK.DAT_RAND may be incomplete.  When this step was stopped there were 0 observations and 12 variables.
WARNING: Data set WORK.DAT_RAND was not replaced because this step was stopped.
NOTE: PROCEDURE SORT used (Total process time):
      real time           0.03 seconds
      cpu time            0.01 seconds

Re: ERROR: Some character data was lost during transcoding

fengyuwuzu — Tue, 08 Jun 2021 19:06:38 GMT

I also got the following error:

ERROR: File BASE_P.FORMATS.CATALOG was created for a different operating system.
NOTE: Format CVTYPE was not found or could not be loaded.
ERROR: File BASE_P.FORMATS.CATALOG was created for a different operating system.
NOTE: Format $CNY was not found or could not be loaded.

....

If I ask for xpt files and then convert to SAS files, will this help fix this issue?

Re: ERROR: Some character data was lost during transcoding

Tom — Tue, 08 Jun 2021 19:29:25 GMT

The XPT extension is ambiguous.

You cannot transport catalogs in XPORT files. You can in CPORT files instead. So if they use PROC CPORT they can make a transport file that includes format catalos and/or SAS datasets. They could also use PROC FORMAT to generate a dataset that describes the formats using the CNTLOUT= option and transfer that dataset.

To handle the transcoding issue make sure you are running your SAS session using either same encoding as the datasets you are trying to read. Or you are using UTF-8 encoding (also called "unicode support").

Re: ERROR: Some character data was lost during transcoding

fengyuwuzu — Tue, 08 Jun 2021 19:37:10 GMT

update: I found a machine with 64bit SAS, and the format errors are gone.

Now I only have the transcoding error.

Re: ERROR: Some character data was lost during transcoding

Tom — Tue, 08 Jun 2021 19:59:08 GMT

Transcoding is tricky. You need to do some research to see what is happening.

First what encoding is you current SAS session using? Look at the SYSENCODING macro variable. Or the ENCODING option. To change that you need to change how you are starting SAS. Are you running SAS using Display Manager? Enterprise Guide? SAS/Studio? From the command line as background job? Something else? The way to change the system encoding differs between those different methods of submitting SAS code.

Second what encoding does the dataset you are trying to read use? Run PROC CONTENTS on the dataset.

Re: ERROR: Some character data was lost during transcoding

fengyuwuzu — Tue, 08 Jun 2021 20:09:04 GMT

Than you. I checked the data and the encoding was UTF-8 Unicode.

My SAS encoding is WLATIN1

I am running SAS in both SAS EG 7.15 and SAS 9.4 in windows

Re: ERROR: Some character data was lost during transcoding

Tom — Tue, 08 Jun 2021 20:46:35 GMT

So you are probably using a PC to be able to run Enterprise Guide.

When you start SAS directly do you click on an ICON (or select one from the START menu)? Which one do you use? When they installed SAS on my PC they made different commands for launching it using WLATIN1 and using UTF-8.

With Enterprise Guide you would need to change the application server it is connecting to in order to allow you to run SAS code to one that is using UTF-8.

Re: ERROR: Some character data was lost during transcoding

fengyuwuzu — Wed, 09 Jun 2021 00:16:11 GMT

Thank you Tom.

I found I also have SAS 9.4 (Unicode support) and there is no error with it.