Reading a flat file from Mainframe

[park2039]

I am trying to read a text file generated in Mainframe.  I used EBCDIC format with recfm=f option.  However, SAS stops before it reaches the end-of-file. The input file has 8.2 mil records but SAS stop after it reads 6.2 mil.  Does anyone have a solution to this problem? 

Thanks.

[art297]

Not sure if it would be relevant for EBCDIC, but have you tried the ignoredoseof option?

[art297]

Actually, it may be quite relevant.  Take a look at http://www.sascommunity.org/wiki/IGNOREDOSEOF_Option

[park2039]

I tried but it produced the same result.

[art297]

Take a look at: http://support.sas.com/techsup/technote/ts642.html

[FriedEgg]

Are you using any truncover or missover options?  Have you checked to make sure that your layout is accurate to the file in question, especially in respect to the final length you a defining for a line?  If the read is stopping on a consistent line have you inspected the line in question for some annomily?

[Tom]

Is it  6.2 out of 8.2 lines in the flat file or observations in the output file?

How did you count the lines in the flat file?

How did you move the file from the mainframe?

[park2039]

It is observations. Each line contains one observation.  The record counts were gievn in advance by the person who created the flat file.   The file was moved using FTP.

[Tom]

Can you verify that it was moved as BINARY rather then ASCII?  Does it have any PIB or other fields that would contain binary data?

Was it originally fixed length records on the mainframe?

Do you get messages in the SAS log about SAS moving to another record to satisfy the input statements?  Does it say that it read the same number of lines as the observations created.  Does the data appear to get out of align with the specifications?

I like to use the LIST statement to look at the data.

You might try :

data _null_;

   infile ....... ;

   if _n_ <= 10 or 6200000 < _n_ ;

  input ;

   list;

run;

Where "62000000" is a small number of records before the point where it is stopping.

[park2039]

I assume it was moved as binary. The file was created in a server then moved to mainframe from where I copied to PC. It had fixed length records from the start. 62000000 is a larger number of records at the point when it stopped.

[art297]

Have you tried it including a truncover option?  It is possible that all of the records are there, but that some are missing some data, thus reading from the next record.

[park2039]

I tried trunover option but it did not work.

[Peter_C]

park2039

please could you provide the sas notes in the log of the step which failed to read all the data

peterC

[park2039]

The SAS finished the job w/o generating error message. So there is no error in the log. It is as if there were only 6.2 mil records in the input file.  So if the record count was not given in advance there is no way of knowing the SAS finished before it reached the end of file. 

[art297]

Did you try it with the truncover option?