XPT2LOC Macro Cutting Data at 300 observations

[CycleTiger]

I've been using the XPT2LOC macro to read in V8 transport files.  Appears to be working OK  but it is cutting the data at 300 observations.  Anyone know how to fix this?

[RW9]

Sorry, not debugging that macro.  It seems to be reading the raw file, and I don't think that is really what you want.  SAS provides several means to read these files, are they XPT's or cport files?

XPT:
https://communities.sas.com/t5/SAS-Programming/Convert-XPT-to-SAS-Datasets/td-p/97872

For cport files use the cimport procedure.  There is no reason to be using code to read raw files when there are procedures which do it for you.

[CycleTiger]

They're version 8 XPT files and my understanding is that cimport doesn't work (it didn't like them when tried) - the only way I've found to read these in is using the XPT2LOC macro.  Any suggestions on reading in V8 XPT files v welcome!  

[RW9]

If cimport does not work, then use xport per the link I provided.  One of those should work, if it doesn't then maybe the file is corrupt.

[Tom · Accepted solution]

There is nothing in that macro that would truncate the table.

First check the OBS system option.  or just run

options obs=max;

Then check the Export file.  The structure is published (although parts are binary) so you could just look at the file.

I know the V5 xport format does not include the number of observations in the dataset header (which was a serious problem from datasets with short records) but I believe that the V8 xport format does.  So check the content of the file and see if it says how many observations there should be. Perhaps the dataset really only has 300 observations and the error was in the process that generated the file.  If the header says there are more than 300 then check if the size of the file is consistent with that. Perhaps it was truncated.

Remember that those files need to be treated as BINARY files, even though the start of the file makes them look like text.  If you moved the file via FTP then perhaps carriage returns were deleted or insert and the structure has been corrupted.

[CycleTiger]

Thanks Tom - update on the issue.  These are V8 transport files and do contain header information including having NOBS set as 300 for all the export files (irrespective of how many records they contained).  We're assuming that this has been an issue with the creation of the XPT file and have passed back to the data supplier.  In the meantime we are using an amended version of the XPT2LOC macro  where we've set NOBS to a constant rather than using the value read from the file.