BookmarkSubscribeRSS Feed
xyxu
Quartz | Level 8

I am trying to import a large CSV file into SAS. I tried to use both 

proc import datafile = "rawdata.csv" dbms = csv out = mydata replace; getnames = yes;  run;

 

and 

 

data mydata;
	infile "rawdata.csv" delimiter = ',' dsd missover lrecl = 32767 firstobs = 2; ...; run;

These two commands appear to generate the same dataset. However, only the second command leads to error messages in log. The messages are as follows:

xyxu_0-1585351569950.png

To investigate the issue, I open the raw file in MS Access and go to row 2322547. The issue seems to be that a character variable has weird values:

xyxu_1-1585352133241.png

And in the imported dataset, this value becomes 

xyxu_2-1585352174614.png

I tried to use encoding='utf-8' but it does not help. How can I correctly import this dataset?

 

 

 

 

s

 

3 REPLIES 3
Tom
Super User Tom
Super User

Most likely you have poorly formed file. Perhaps it has some embedded line breaks in the middle of one the fields? This can cause the INPUT statement, either the one generated by PROC IMPORT or the one you wrote for your own data step, to get out of alignment with the values on the lines.

 

If that is your problem it is a common problem and has been answered on this forum many times.  You might be able to read it if the line breaks in the fields just use single carriage return or single linefeed and the actual real end of line codes in the file use both a carriage return and a linefeed.  If not then you need to pre-process the file to fix the issue by removing the extra line breaks.

PS Why did you paste photographs of text? You can just copy and paste the lines of text from the SAS log.

xyxu
Quartz | Level 8

Thank, Tom. I found that the issue is due to hidden linebreakers in the string.  

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 3 replies
  • 533 views
  • 2 likes
  • 3 in conversation