BookmarkSubscribeRSS Feed
park2039
Calcite | Level 5

I am trying to read a text file generated in Mainframe.  I used EBCDIC format with recfm=f option.  However, SAS stops before it reaches the end-of-file. The input file has 8.2 mil records but SAS stop after it reads 6.2 mil.  Does anyone have a solution to this problem? 

Thanks.

19 REPLIES 19
art297
Opal | Level 21

Not sure if it would be relevant for EBCDIC, but have you tried the ignoredoseof option?

art297
Opal | Level 21

Actually, it may be quite relevant.  Take a look at http://www.sascommunity.org/wiki/IGNOREDOSEOF_Option

park2039
Calcite | Level 5

I tried but it produced the same result.

FriedEgg
SAS Employee

Are you using any truncover or missover options?  Have you checked to make sure that your layout is accurate to the file in question, especially in respect to the final length you a defining for a line?  If the read is stopping on a consistent line have you inspected the line in question for some annomily?

Tom
Super User Tom
Super User

Is it  6.2 out of 8.2 lines in the flat file or observations in the output file?

How did you count the lines in the flat file?

How did you move the file from the mainframe?

park2039
Calcite | Level 5

It is observations. Each line contains one observation.  The record counts were gievn in advance by the person who created the flat file.   The file was moved using FTP.

Tom
Super User Tom
Super User

Can you verify that it was moved as BINARY rather then ASCII?  Does it have any PIB or other fields that would contain binary data?

Was it originally fixed length records on the mainframe?

Do you get messages in the SAS log about SAS moving to another record to satisfy the input statements?  Does it say that it read the same number of lines as the observations created.  Does the data appear to get out of align with the specifications?

I like to use the LIST statement to look at the data.

You might try :

data _null_;

   infile ....... ;

   if _n_ <= 10 or 6200000 < _n_ ;

  input ;

   list;

run;

Where "62000000" is a small number of records before the point where it is stopping.

park2039
Calcite | Level 5

I assume it was moved as binary. The file was created in a server then moved to mainframe from where I copied to PC. It had fixed length records from the start. 62000000 is a larger number of records at the point when it stopped.

art297
Opal | Level 21

Have you tried it including a truncover option?  It is possible that all of the records are there, but that some are missing some data, thus reading from the next record.

park2039
Calcite | Level 5

I tried trunover option but it did not work.

Peter_C
Rhodochrosite | Level 12

park2039

please could you provide the sas notes in the log of the step which failed to read all the data

peterC

park2039
Calcite | Level 5

The SAS finished the job w/o generating error message. So there is no error in the log. It is as if there were only 6.2 mil records in the input file.  So if the record count was not given in advance there is no way of knowing the SAS finished before it reached the end of file. 

art297
Opal | Level 21

Did you try it with the truncover option?

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 19 replies
  • 4116 views
  • 0 likes
  • 8 in conversation