BookmarkSubscribeRSS Feed
sasbegginer
Calcite | Level 5
Hi all,

I am using a simple infile statement to read in about 4 million records. For some reason sas stop at reading 700,00 and does not read the full file in. As the txt file is about 4gb i can not open it to see why. Having a look at the last record in the table i can not see why it is stopping. Any ideas? Thanks in advance
6 REPLIES 6
deleted_user
Not applicable
In case you are using Windows, you can find some file-splitter utility. Anyway, a piece of your log would be useful.
sbb
Lapis Lazuli | Level 10 sbb
Lapis Lazuli | Level 10
Agreed - SAS log with DATA step code would be helpful - and you may find setting END=EOF on the INFILE useful while doing some code like:

IF EOF THEN PUTLOG _INFILE_;

or use some HEX type SAS format to display the last record as SAS interprets the end-of-file condition.

Scott Barry
SBBWorks, Inc.
Doc_Duke
Rhodochrosite | Level 12
If you are on a PC and your text file includes a CTRL-Z, SAS will stop reading it. CTRL-Z is the (antique) end-of-file indicator left over from V6 days.
data_null__
Jade | Level 19
Perhaps the option

IGNOREDOSEOF
is used in the context of I/O operations on variable record format files. When this option is specified, any occurrence of ^Z is interpreted as character data and not as an end-of-file marker.
Patrick
Opal | Level 21
If the issue is caused by a control character then eventually the following code will help:

data want;
infile ;
input @;
_infile_=COMPRESS(_infile_, , "c");
input ;
run;

If this works and you want to find the line with the control character then:

if length(_infile_) ne length(COMPRESS(_infile_, , "c")) should do the job.

You could of course also use find() or indexc() - but I assume compress will work faster.

HTH
Patrick
martha_sas
SAS Employee
You aren't using Learning Edition, are you?
If so, you are limited in the number of observations, so you could be hitting that limit.

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 6 replies
  • 1772 views
  • 0 likes
  • 7 in conversation