I have a pipe-delimited dataset that contains about 50 million observations. Unfortunately, not all of them contain the same number of variables, i.e., there are a couple 1000 observations that contain "extra" pipes. So while most observations only contain values for 50 variables, these "right-shifted" observations may have values that go through var60 or var70 (with the first 10 or 20 variables being empty). Since I need to know the number of variables in there to write an infile statement to import this dataset, is there a quick way to find out how many there are?
I have tried a simple proc import with guessingrows = max and while that worked, it took about 12 hours to run. Does proc import have an option to force all variables to character before importing? I'm running SAS EG and am aware of manually setting attributes for Excel files but not for .txt files.
... View more