topic Re: Sequential read of binary file using IML in SAS/IML Software and Matrix Computations

Sequential read of binary file using IML

IanWakeling — Wed, 16 Oct 2019 13:04:25 GMT

Following on from my question yesterday, I am now successfully reading data from a binary file, but I can only do this with the random access style syntax along the lines of "input >n" to read something at byte position n.

If I try syntax like " input <8 x IEEE8.; " to read the next 8 bytes as an IEEE double, it does not advance the file pointer, so any similar input statements that follow keep re-reading the same 8 bytes. I have tried using a trailing @ but this makes no difference. Is there perhaps a hold control for binary files that I am not aware of?

Also, is there a way of reading a block of data into a matrix, instead of reading numbers one at a time?

Re: Sequential read of binary file using IML

Ksharp — Wed, 16 Oct 2019 13:17:44 GMT

I remembered @art297 wrote a paper about reading binary file(a picture) , you need special informat .

Re: Sequential read of binary file using IML

Rick_SAS — Wed, 16 Oct 2019 13:25:32 GMT

I don't know. I have never read binary data into IML variables.

In my experience, it is easier to use the DATA step for operations such as this. If this is something that is one step of a larger IML program, you can use the SUBMIT-ENDSUBMIT block to call the DATA step from within PROC IML. You can even pass parameters from IML variables into the DATA step.

Re: Sequential read of binary file using IML

IanWakeling — Wed, 16 Oct 2019 14:09:53 GMT

I think I am probably sticking with a pure IML solution as I have something that works, it's is just not quite as fast as I hoped. The binary files are large, around 350MB to 1GB. Each contains 5 matrices and 5 vectors (not all of which will be required), and it just seemed more efficient to read what I want directly into IML for further processing.

Re: Sequential read of binary file using IML

Rick_SAS — Thu, 17 Oct 2019 15:35:52 GMT

OK. One possible idea regarding the speed: If you need to run several programs on the same data, you can read the binary data into matrices once, then save them as a SAS data set (preferred) or use the STORE statement to save the matrices. If you do that, subsequent access of the data will be faster. Of course, if you are trying to read data that are generated every morning by some non-SAS process and only analyzed by one IML program, this idea won't save you any time.