BookmarkSubscribeRSS Feed
fengyuwuzu
Pyrite | Level 9

When I tried to re-run an old program I got the following error: ERROR: Some character data was lost during transcoding in the dataset BASE_P.DAT_RAND. Either the data contains characters that are not representable in the new encoding or truncation occurred during transcoding.

 

Does any one know what the problem is? Thanks.

 

27         PROC SORT data=base_p.Dat_rand out=Dat_rand; BY SUBNUM; RUN;
NOTE: Data file BASE_P.DAT_RAND.DATA is in a format that is native to another host, or the file encoding does not match the session 
      encoding. Cross Environment Data Access will be used, which might require additional CPU resources and might reduce 
      performance.

ERROR: Some character data was lost during transcoding in the dataset BASE_P.DAT_RAND. Either the data contains characters that are 
       not representable in the new encoding or truncation occurred during transcoding.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 9 observations read from the data set BASE_P.DAT_RAND.
WARNING: The data set WORK.DAT_RAND may be incomplete.  When this step was stopped there were 0 observations and 12 variables.
WARNING: Data set WORK.DAT_RAND was not replaced because this step was stopped.
NOTE: PROCEDURE SORT used (Total process time):
      real time           0.03 seconds
      cpu time            0.01 seconds
7 REPLIES 7
fengyuwuzu
Pyrite | Level 9

I also got the following error:

ERROR: File BASE_P.FORMATS.CATALOG was created for a different operating system.
NOTE: Format CVTYPE was not found or could not be loaded.
ERROR: File BASE_P.FORMATS.CATALOG was created for a different operating system.
NOTE: Format $CNY was not found or could not be loaded.

....

 

If I ask for xpt files and then convert to SAS files, will this help fix this issue? 

Tom
Super User Tom
Super User

The XPT extension is ambiguous.

You cannot transport catalogs in XPORT files.  You can in CPORT files instead.  So if they use PROC CPORT they can make a transport file that includes format catalos and/or SAS datasets.   They could also use PROC FORMAT to generate a dataset that describes the formats using the CNTLOUT= option and transfer that dataset.

 

To handle the transcoding issue make sure you are running your SAS session using either same encoding as the datasets you are trying to read.  Or you are using UTF-8 encoding (also called "unicode support").

fengyuwuzu
Pyrite | Level 9

update: I found a machine with 64bit SAS, and the format errors are gone. 

Now I only have the transcoding error. 

 

Tom
Super User Tom
Super User

Transcoding is tricky.  You need to do some research to see what is happening.

First what encoding is you current SAS session using?  Look at the SYSENCODING macro variable.  Or the ENCODING option.   To change that you need to change how you are starting SAS.  Are you running SAS using Display Manager?  Enterprise Guide? SAS/Studio? From the command line as background job?  Something else?  The way to change the system encoding differs between those different methods of submitting SAS code.

 

Second what encoding does the dataset you are trying to read use?  Run PROC CONTENTS on the dataset.

fengyuwuzu
Pyrite | Level 9

Than you. I checked the data and the encoding was UTF-8 Unicode. 

 

 

fengyuwuzu_0-1623182714670.png

 

My SAS encoding is WLATIN1 

fengyuwuzu_1-1623182826817.png

I am running SAS in both SAS EG 7.15 and  SAS 9.4 in windows 

Tom
Super User Tom
Super User

So you are probably using a PC to be able to run Enterprise Guide.

When you start SAS directly do you click on an ICON (or select one from the START menu)?  Which one do you use?  When they installed SAS on my PC they made different commands for launching it using WLATIN1 and using UTF-8.

image.png

With Enterprise Guide you would need to change the application server it is connecting to in order to allow you to run SAS code to one that is using UTF-8.

fengyuwuzu
Pyrite | Level 9

Thank you Tom. 

I found I also have SAS 9.4 (Unicode support) and there is no error with it. 

 

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 7 replies
  • 2714 views
  • 2 likes
  • 2 in conversation