Can I write to stdin and read from stdout in the same data step?

ScottBass — Wed, 01 Feb 2017 22:31:56 GMT

SAS 9.3 on Windows

I have an external program that can both read from stdin and write to stdout in the same call.

The external program is something like this:

exe keyfile col1 col2 output_type [input_file | -] [output_file | -]

If input file and output file are not specified (or "-"), the program reads from and writes to stdin and stdout respectively.

The transient external files can contain sensitive information, plus they can be big. If I could avoid the external files that would be a nice-to-have.

I tried something like:

options noquotelenmax; 
filename tmp pipe "&cryptPgmEncrypt &cryptKeysDir\&cryptKeyFile 1 10 OVERRIDE";
options quotelenmax;  

data _null_;
   set mysasdataset;
   infile tmp sharebuffers;  * also tried w/o sharebuffers ;
   file tmp;
   put @1 recnum z10.;
   input @1 recnum_e $13.;
   put recnum_e;
run;

But SAS didn't like that:

ERROR: Random access not allowed.

I'm not seeing the random access - it looks pretty sequential to me.

I also tried this, same results:

data _null_;
   infile tmp end=eof2;
   file tmp;
   do until (eof1);
      set mysasdataset end=eof1;
      put @1 recnum z10.;
   end;
   do until (eof2);
      input;
      put _infile_;
   end;
   stop;
run;

Wondering if the output would be buffered somehow?

Of course, the workaround is to use external files, but I'm curious if the above can be done (on Windows)?

Re: Can I write to stdin and read from stdout in the same data step?

mkeintz — Wed, 01 Feb 2017 05:36:19 GMT

I didn't see random access either, but you are reading and writing to the same pipe.

Look at what happens with this program, which (in the second data step) reads and writes to the same external disk file. In particular it reads the first half of the file, and writes only half of that - after modifying content. It neither reads nor writes the second half of the file.

data _null_;
  file 'c:\temp\t.txt';
  do n=100 to 119; put n z3.; end;
run;

data _null_;
  infile 'c:\temp\t.txt';
  file 'c:\temp\t.txt';
  input txt $3. ;
  substr(txt,1,1)='X';
  if mod(_n_,2)=0 then put txt;
  if _n_>=10 then stop;
run;

data _null_;
  infile 'c:\temp\t.txt';
  input txt $3.;
  put _n_=z2. txt;
run;

Take a look at the difference between the before and after for the file t.txt. You'll see 20 lines in each , with lines 2,4,6,8,10 modified, and others left as is.

Now, since the PUT statement is only executed 5 times, how did it know to write those five records in alternative locations, rather than consecutive locations? It has to be because a kind of "random" access to those lines was being supported, i.e. the INPUT statement advanced the destination of the PUT statement since the same file is being read and written.

Maybe you wouldn't want to call that random access, but then why are all the other lines left in the file? After all, if the destination of PUT was a new file it would have only 5 lines, not 20. Lines are effectively being logically "inserted".

Even if the file is stored as a sequential file, if it's the object of both INFILE and FILE, then apparently SAS needs to think of it as having random access. Who knew?

And I guess this "random access" is supported for a single FILE used as both INPUT source and PUT target, but apparently a single PIPE is not similarly supported.

Maybe you need two pipes for the external process - one to designate the process' stdin and one for its stdout.

topic Can I write to stdin and read from stdout in the same data step? in SAS Programming

Can I write to stdin and read from stdout in the same data step?

Re: Can I write to stdin and read from stdout in the same data step?