topic Re: Encoding error? in SAS Programming

Encoding error?

Miracle — Mon, 09 Mar 2015 05:24:41 GMT

Dear all,

How are you?

Can I please ask what does this note mean after I run the SAS import procedure?

NOTE: A byte order mark in the file "C:\Users\kwong\Documents\RF2015.csv" (for fileref "#LN00643") indicates that the data is encoded in "utf-16le". This encoding will be used to process the file.

How do I resolve it? I'm running SAS9.3 TS Level 1M0 on W32_7PRO platform.

Thank you very much.

Re: Encoding error?

Miracle — Mon, 09 Mar 2015 06:19:06 GMT

Dear all,

I found this thread

and rewrote my SAS code as

filename rf2015 "C:\Users\kwong\Documents\RF2015.csv" encoding="utf-16";

PROC IMPORT OUT= WORK.rf2015

DATAFILE= rf2015

DBMS=CSV dbms=dlm REPLACE;

delimiter='09'x;

GETNAMES=YES;

DATAROW=2;

GUESSINGROWS=2147483647;

RUN;

When I checked the log window, the corresponding note on encoding has gone and the data seems fine.

Can someone please confirm if my code is right?

Thank you very much.

Re: Encoding error?

Kurt_Bremser — Mon, 09 Mar 2015 08:11:55 GMT

I'm a little concerned with the two dbms= options; apart from that, the code looks good. You could try to reduce the guessingrows, as these tend to be very bad for performance (the lookup for correct data types will take longer than the actual import).

If the structure of the csv file will not be subject to changes that your program needs to automatically adapt to, I suggest you take the data step code created by proc import and use that in the future.