Encoding error?

Accepted Solution Solved
Reply
Regular Contributor
Posts: 226
Accepted Solution

Encoding error?

Dear all,

How are you?

Can I please ask what does this note mean after I run the SAS import procedure?

NOTE: A byte order mark in the file "C:\Users\kwong\Documents\RF2015.csv" (for fileref "#LN00643") indicates that the data is encoded in "utf-16le".  This encoding will be used to process the file.

How do I resolve it? I'm running SAS9.3 TS Level 1M0 on W32_7PRO platform.


Thank you very much.


Accepted Solutions
Solution
‎03-09-2015 04:11 AM
Super User
Posts: 7,433

Re: Encoding error?

I'm a little concerned with the two dbms= options; apart from that, the code looks good. You could try to reduce the guessingrows, as these tend to be very bad for performance (the lookup for correct data types will take longer than the actual import).

If the structure of the csv file will not be subject to changes that your program needs to automatically adapt to, I suggest you take the data step code created by proc import and use that in the future.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers

View solution in original post


All Replies
Regular Contributor
Posts: 226

Re: Encoding error?

Dear all,

I found this thread

and rewrote my SAS code as

filename rf2015 "C:\Users\kwong\Documents\RF2015.csv" encoding="utf-16";

PROC IMPORT OUT= WORK.rf2015

            DATAFILE= rf2015

            DBMS=CSV dbms=dlm REPLACE;

  delimiter='09'x;

  GETNAMES=YES;

  DATAROW=2;

  GUESSINGROWS=2147483647;

RUN;


When I checked the log window, the corresponding note on encoding has gone and the data seems fine.


Can someone please confirm if my code is right?


Thank you very much.

Solution
‎03-09-2015 04:11 AM
Super User
Posts: 7,433

Re: Encoding error?

I'm a little concerned with the two dbms= options; apart from that, the code looks good. You could try to reduce the guessingrows, as these tend to be very bad for performance (the lookup for correct data types will take longer than the actual import).

If the structure of the csv file will not be subject to changes that your program needs to automatically adapt to, I suggest you take the data step code created by proc import and use that in the future.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 2 replies
  • 496 views
  • 0 likes
  • 2 in conversation