topic Re: infile problem in SAS Procedures

infile problem

sasbegginer — Mon, 15 Feb 2010 17:50:55 GMT

Hi all,

I am using a simple infile statement to read in about 4 million records. For some reason sas stop at reading 700,00 and does not read the full file in. As the txt file is about 4gb i can not open it to see why. Having a look at the last record in the table i can not see why it is stopping. Any ideas? Thanks in advance

Re: infile problem

deleted_user — Mon, 15 Feb 2010 21:52:03 GMT

In case you are using Windows, you can find some file-splitter utility. Anyway, a piece of your log would be useful.

Re: infile problem

sbb — Mon, 15 Feb 2010 23:16:08 GMT

Agreed - SAS log with DATA step code would be helpful - and you may find setting END=EOF on the INFILE useful while doing some code like:

IF EOF THEN PUTLOG _INFILE_;

or use some HEX type SAS format to display the last record as SAS interprets the end-of-file condition.

Scott Barry
SBBWorks, Inc.

Re: infile problem

Doc_Duke — Tue, 16 Feb 2010 00:44:54 GMT

If you are on a PC and your text file includes a CTRL-Z, SAS will stop reading it. CTRL-Z is the (antique) end-of-file indicator left over from V6 days.

Re: infile problem

Patrick — Tue, 16 Feb 2010 09:08:49 GMT

If the issue is caused by a control character then eventually the following code will help:

data want;
infile ;
input @;
_infile_=COMPRESS(_infile_, , "c");
input ;
run;

If this works and you want to find the line with the control character then:

if length(_infile_) ne length(COMPRESS(_infile_, , "c")) should do the job.

You could of course also use find() or indexc() - but I assume compress will work faster.

HTH
Patrick

Re: infile problem

data_null__ — Tue, 16 Feb 2010 12:03:42 GMT

Perhaps the option

IGNOREDOSEOF
is used in the context of I/O operations on variable record format files. When this option is specified, any occurrence of ^Z is interpreted as character data and not as an end-of-file marker.

Re: infile problem

martha_sas — Tue, 16 Feb 2010 16:13:31 GMT

You aren't using Learning Edition, are you?
If so, you are limited in the number of observations, so you could be hitting that limit.