Help using Base SAS procedures

proc import won't read all columns

Reply
Super Contributor
Posts: 268

proc import won't read all columns

I have a csv file with 214 columns.  SAS is only reading the the first 125.  the log says "Number of names found is greater than number of variables found."  I have imported this file numerous times using the same code I'm using today but this is the first time i've done it using version 9.3.  Am I missing something?

proc import datafile="G:\Departments\Research\test scores\GT\1112\GIFT.csv"

out=mydata

dbms=dlm replace;

delimiter=',';

getnames=yes;

run;

PROC Star
Posts: 7,363

proc import won't read all columns

Without seeing the file one can only guess.  My first guess would be that the file begins with some incomplete records.  I would try adding a guessingrows=32000; statement and see if that corrects the situation.

Super Contributor
Posts: 268

proc import won't read all columns

well.....that's interesting.  I used guessingrows and it imported 200 columns....only 14 to go.  I have 11,000 rows.  Maybe increasing the number to 50,000? ...hmmm

PROC Star
Posts: 7,363

proc import won't read all columns

If the records span multiple rows you may have to increase the number.  The maximum is:

2147483647

PROC Star
Posts: 7,363

proc import won't read all columns

Could also be a record length issue.  You could, I think, assign the file with a filename statement and set lrecl to some very large number.  What does proc import curently show it to be set at in your log?  And, in reality, how wide is the longest record in the file?

Super Contributor
Posts: 268

proc import won't read all columns

max record length is 1177.  can you use lrecl=xxxxx  in a proc import?

PROC Star
Posts: 7,363

proc import won't read all columns

I don't think so, but you can always use function key F4 to get the actual code that was submitted, adust it manually, and resubmit it.

Super Contributor
Posts: 268

proc import won't read all columns

for whatever reason, I saved the external csv file as xls and all columns imported.  It's a much bigger file but I'll let our IT director worry about that.

Super User
Posts: 9,682

proc import won't read all columns

Did you try getnames=no; to avoid invalid variable name as the reason can't input all the variables.?

N/A
Posts: 1

Re: proc import won't read all columns

I have experienced this unhelpful message: it occurs for me when the number of characters in the first line of the file (variable names) in longer than ~32,000. Essentially, it reads those first 32k characters and those variables names come through to the SAS dataset correctly. SAS gives up for the other variable names and just names them var0001....

One work around for this is to -cut- up the input text file by columns using a command line tool (e.g. cut in linux) so that SAS can read in the manageable chunks, then merge those SAS datasets together. SAS doesn't hit a limit with my data in terms of having that number of variables, just a flawed proc import.

PS. I'm using SAS 9.3

Ask a Question
Discussion stats
  • 9 replies
  • 6662 views
  • 3 likes
  • 4 in conversation