topic Re: read same file twice inside datastep in SAS Programming

read same file twice inside datastep

Maze — Fri, 11 Oct 2019 13:50:58 GMT

Hi everyone,

I´m having trouble reading the same file twice inside a datastep.

data WORK.OUT;
	infile datalines;
	length INP $40;
	input INP $;
	infile dummy filevar=INP end=EOF;
	do while(^EOF);
	  input VAR1 VAR2 VAR3;
	  output;
	end;
datalines;
/path/to/file/file1.txt
/path/to/file/file2.txt
/path/to/file/file2.txt
/path/to/file/file3.txt
;

Each file is read and output once even though file2.txt should have been output twice.

The code is just exemplary. The real code depends on a list of items in a dataset for which (depending on a parameter) different external files should be read. If the same file occurs two times in a row it is not read again.

If files alternate like below, they are read every time:

/path/to/file/file1.txt

/path/to/file/file2.txt

/path/to/file/file3.txt

/path/to/file/file2.txt

What am I missing? Do I need to reset a pointer or the infile? Or is this not intended?

Any help is appreciated!

/edit: formatting...

Re: read same file twice inside datastep

Tom — Fri, 11 Oct 2019 14:41:45 GMT

Yes. You need to run the INFILE statement twice to "clear" it. So point it to some other file between each real file in your list. Try using the bit bucket on Unix.

Something like this:

data WORK.OUT;
  infile datalines;
  length INP filevar $40;
  input INP $;
  do filevar=inp,'/dev/null';
    infile dummy filevar=filevar end=EOF;
    do while(^EOF);
      input VAR1 VAR2 VAR3;
      output;
    end;
  end;
datalines;
/path/to/file/file1.txt
/path/to/file/file2.txt
/path/to/file/file2.txt
/path/to/file/file3.txt
;

Re: read same file twice inside datastep

data_null__ — Fri, 11 Oct 2019 14:23:39 GMT

From the documentation

FILEVAR=variable

specifies a variable whose change in value causes the INFILE statement to close the current input file and open a new one. When the next INPUT statement executes, it reads from the new file that the FILEVAR= variable specifies. Like automatic variables, this variable is not written to the data set.

Why do you want to read the file twice?

Re: read same file twice inside datastep

yabwon — Fri, 11 Oct 2019 15:10:39 GMT

If I may, I think the TEMP filename would be a bit more OS independent.

All the best

Bart

filename f TEMP;
filename f LIST;
data _null_; /* force to create file */
  file f ;
  put    ;
  stop   ;
run;

data WORK.OUT;
	infile datalines;
	length INP $ 100.;
	input INP $ ; 

  infile dummy filevar=INP end=EOF;
  do while(^EOF);
    input VAR1 VAR2 VAR3;
    output;
  end;
  INP = pathname('f');
  infile dummy filevar=INP end=EOF;
  
datalines;
E:/SAS_WORK_5400/file1.txt
E:/SAS_WORK_5400/file2.txt
E:/SAS_WORK_5400/file2.txt
E:/SAS_WORK_5400/file3.txt
;
run;

filename f CLEAR;

Re: read same file twice inside datastep

Maze — Mon, 14 Oct 2019 05:46:07 GMT

Thanks everyone for the pointers, doc is pretty clear, I don´t know why I did not see that.

Why do you want to read the file twice?

I have a dataset with table names that include a structure tag. Based on that tag I read the according structure file to generate a DDL script (structures may repeat!).

Re: read same file twice inside datastep

Tom — Mon, 14 Oct 2019 05:53:54 GMT

Using TEMP to allocate a name is a good idea.

It is interesting that it works using two different INFILE statements. Note that they both have to use the same variable for the FILEVAR= option and they also have to both use the same filref (DUMMY in the example). Otherwise the second one does not cause the first to forget the filename it was reading before.