I have a csv file with some non-English characters in it. Using data infile in regular SAS, once the foreign character is encountered, the system stops reading in more data and ignores the remainder of the dataset, resulting in a large chunk of the file not imported. I then tried the same code under SAS 9.4 with unicode support, which seems to read in the correct number of rows. The problem is when I try to save it as a dataset on a pre-defined library: "ERROR: Some character data was lost during transcoding in the dataset lib1.dat1. Either the data contains characters that are not representable in the new encoding or truncation occurred during transcoding." and "WARNING: The data set lib1.data1 may be incomplete. When this step was stopped there were x observations and y variables." x is way less than the number of records in the dataset in the work library. Is this because dataset is currently in unicode and needs to be converted to regular coding somehow before it can be saved and used? Thanks.
... View more