topic Import dirty csv data in SAS Programming

Import dirty csv data

Xusheng — Sat, 31 Mar 2018 15:38:57 GMT

Hi, I'm trying to import the csv file that I downloaded from Bloomberg. I tried the method that I used before but it does not work properly (Error message in log). I attached the code and screenshot of csv file and SAS result here. Any suggestion for improving the code? Thanks a lot!

PROC IMPORT OUT= Bloomberg																/*Import three inventory data*/				
			DATAFILE= "E:\one_drive\OneDrive\Thesis\2017_Xusheng\Update1\1-Price\Data\Inventory\Bloomberg.csv"   
            DBMS=CSV REPLACE;				
     GETNAMES=YES;
     DATAROW=2; 
Run;

CSV fileSAS Result Log error message

Re: Import dirty csv data

Tom — Sat, 31 Mar 2018 15:44:15 GMT

The only errors your log are showing is when the DATE column contains '#VALUE1' instead of a date.

Sounds like you could just ignore the errors since they will only result in the DATE column having a missing value.

Otherwise write you own data step to read the CSV file. You will then have complete control over how to handle those values.

PS You posted a picture of a spreadsheet and labeled it a CSV file. A CSV is a text file. Don't post pictures of text, just post the text. It is clearer, smaller and easier than posting photos. And if you do post a photo then post a photo of the actual object and not a version that has been transformed by being read into a program like Excel.

Re: Import dirty csv data

Xusheng — Sat, 31 Mar 2018 15:50:15 GMT

Thank you for replying, yes, I read the error message and did not
figure out how it affects my result. The date and event names are good but
the problem is with the numbers. Probably I will just create my own data
step.

I will keep in mind regarding the posting pictures.

Re: Import dirty csv data

Tom — Sat, 31 Mar 2018 15:58:02 GMT

We cannot tell from the photos what issues you might have with the numbers. Post just a few rows of the actual data in the CSV file and of that data after PROC IMPORT read them (again as text and not photos) and explain it what way that are not correct.

Re: Import dirty csv data

Ksharp — Sun, 01 Apr 2018 11:19:48 GMT

Try another option.

PROC IMPORT OUT= Bloomberg /*Import three inventory data*/
DATAFILE= "E:\one_drive\OneDrive\Thesis\2017_Xusheng\Update1\1-Price\Data\Inventory\Bloomberg.csv"
DBMS=CSV REPLACE;
GETNAMES=YES;

guessingrows=32767;

DATAROW=2;
Run;

Re: Import dirty csv data

Xusheng — Sun, 01 Apr 2018 18:17:41 GMT

Thank you, Tom. I've found the problem. It was inside my data. Invalid value in the date column ruins the rest of the data. Thank you.

Re: Import dirty csv data

Xusheng — Sun, 01 Apr 2018 18:18:24 GMT

Thank you, Ksharp. I've found the problem and fixed that.