topic proc import won't read all columns in SAS Procedures

proc import won't read all columns

GreggB — Thu, 15 Dec 2011 19:07:02 GMT

I have a csv file with 214 columns. SAS is only reading the the first 125. the log says "Number of names found is greater than number of variables found." I have imported this file numerous times using the same code I'm using today but this is the first time i've done it using version 9.3. Am I missing something?

proc import datafile="G:\Departments\Research\test scores\GT\1112\GIFT.csv"

out=mydata

dbms=dlm replace;

delimiter=',';

getnames=yes;

run;

proc import won't read all columns

art297 — Thu, 15 Dec 2011 19:34:04 GMT

Without seeing the file one can only guess. My first guess would be that the file begins with some incomplete records. I would try adding a guessingrows=32000; statement and see if that corrects the situation.

proc import won't read all columns

GreggB — Thu, 15 Dec 2011 19:44:32 GMT

well.....that's interesting. I used guessingrows and it imported 200 columns....only 14 to go. I have 11,000 rows. Maybe increasing the number to 50,000? ...hmmm

proc import won't read all columns

art297 — Thu, 15 Dec 2011 19:49:16 GMT

If the records span multiple rows you may have to increase the number. The maximum is:

2147483647

proc import won't read all columns

art297 — Thu, 15 Dec 2011 19:51:38 GMT

Could also be a record length issue. You could, I think, assign the file with a filename statement and set lrecl to some very large number. What does proc import curently show it to be set at in your log? And, in reality, how wide is the longest record in the file?

proc import won't read all columns

GreggB — Thu, 15 Dec 2011 19:55:26 GMT

max record length is 1177. can you use lrecl=xxxxx in a proc import?

proc import won't read all columns

art297 — Thu, 15 Dec 2011 20:11:57 GMT

I don't think so, but you can always use function key F4 to get the actual code that was submitted, adust it manually, and resubmit it.

proc import won't read all columns

GreggB — Thu, 15 Dec 2011 20:23:35 GMT

for whatever reason, I saved the external csv file as xls and all columns imported. It's a much bigger file but I'll let our IT director worry about that.

proc import won't read all columns

Ksharp — Fri, 16 Dec 2011 02:55:55 GMT

Did you try getnames=no; to avoid invalid variable name as the reason can't input all the variables.?

Re: proc import won't read all columns

DaveH — Thu, 14 Nov 2013 17:31:36 GMT

I have experienced this unhelpful message: it occurs for me when the number of characters in the first line of the file (variable names) in longer than ~32,000. Essentially, it reads those first 32k characters and those variables names come through to the SAS dataset correctly. SAS gives up for the other variable names and just names them var0001....

One work around for this is to -cut- up the input text file by columns using a command line tool (e.g. cut in linux) so that SAS can read in the manageable chunks, then merge those SAS datasets together. SAS doesn't hit a limit with my data in terms of having that number of variables, just a flawed proc import.

PS. I'm using SAS 9.3