Repeating files in FILEVAR (INFILE statment) fails.

Reply
Occasional Contributor
Posts: 12

Repeating files in FILEVAR (INFILE statment) fails.


Hey All,

     I have a SAS dataset data set (Directories.sasb7dat) with a column of file locations (variable = 'Path'').

     OBS     Path

          1     C:\Project\File_1.txt

          2     C:\Project\File_2.txt

          3     C:\Project\File_2.txt

          4     C:\Project\File_3.txt

          5     C:\Project\File_4.txt

     In a subsequent data step, when I try to use the 'Path' variable with the FILEVAR option, SAS does not start over on File_2.txt (see below).

     infile temp filevar=Path lrecl=10000 end=last line=_L_;

     On the "last" will still be 1 and _L_ will still point to the last line at OBS=3. I would like it if SAS could start over reading this file. Make sense?

Help?

Huey

Respected Advisor
Posts: 4,655

Re: Repeating files in FILEVAR (INFILE statment) fails.

In the documentation, it states that a new file is opened every time the filevar= variable changes value. Between obs 2 and 3, the filevar= value doesn't change. That's why the file is not reread. If the two identical obs were not consecutive, the file would be read twice.

PG

PG
Occasional Contributor
Posts: 12

Re: Repeating files in FILEVAR (INFILE statment) fails.

Thanks for the reply. And I completely agree with you.

However, it is hard to believe that the best solution is to simply "anit-sort" my 'Directory' dataset. Where would I be if I didn't have enough unique directories to permute them as you state? Up a creek.

I've looked all over the SAS Language Reference for ways to close the file referenced by INFILE, or to deassign the reference, but no dice.

The only work around I've found is to assign "Path" to a dummy file (e.g. "\...\Empty.txt") between each SET observations. This seems to reset the filevar in a workable manner.

I'm dropping this approach, and skinning the cat differently. *Sigh*

Thanks again,

Huey

Respected Advisor
Posts: 4,655

Re: Repeating files in FILEVAR (INFILE statment) fails.

Another way to read many files is to list them in a filename statement. For example :

data myFiles;
length path $100;
input path $;
datalines;
C:\Project\File_1.txt
C:\Project\File_2.txt
C:\Project\File_2.txt
C:\Project\File_3.txt
C:\Project\File_4.txt
;

proc sql noprint;
select quote(trim(path)) into Smiley Tongueaths separated by "," from myFiles;
quit;

filename myFiles (&paths.);
data want;
infile myFiles;
run;

If you have a reasonable number of files, this would work.

PG

PG
Respected Advisor
Posts: 3,777

Re: Repeating files in FILEVAR (INFILE statment) fails.

What are ou

Huey wrote:

Thanks for the reply. And I completely agree with you.

However, it is hard to believe that the best solution is to simply "anit-sort" my 'Directory' dataset. Where would I be if I didn't have enough unique directories to permute them as you state? Up a creek.

I've looked all over the SAS Language Reference for ways to close the file referenced by INFILE, or to deassign the reference, but no dice.

The only work around I've found is to assign "Path" to a dummy file (e.g. "\...\Empty.txt") between each SET observations. This seems to reset the filevar in a workable manner.

I'm dropping this approach, and skinning the cat differently. *Sigh*

Thanks again,

Huey

What are you doing?  You haven't show any code.  From what little you have said I don't think you understand what FILEVAR does.  When the value of filevar changes the currently open file is closed and the new one is opened.  If you want to read all the records in a file you need to do that before reading another FILEVAR name from SET.  Something like this.

data;

   set filenames;

   infile dummy filevar=filename end=eof;

   do while(not eof);

      input ....;

      output;

      end;

   run;

Occasional Contributor
Posts: 12

Re: Repeating files in FILEVAR (INFILE statment) fails.

@ data _null_,

Thanks for your constructive, and truly positive input, I appreciate it.

This is my task:

    

     I have a big dataset (Directories.sasb7dat) items/observations with a variable that holds the directory to a chunk of HTML that is used to represent the item/observation in Website A.

     For each item/observation, I need to restructure the associated HTML to fit nicely into Website B.

     Since SAS will not store a character variable over 32,*** characters, I thought a good solution would be to write the restructrured HTML for  Website B, while I was reading the HTML from Website A.

     This method works great, except that some of the items/observations point to the same directory, and happen to follow one another in the "Directories" dataset.

My very explicit query was this:

     How to force SAS to close a file used in an infile statement? It's been stated many times, in fact even before you posted, that the FILEVAR variable value needs to change in order for SAS to close a file pointed to by the FILEVAR variable. So the question can be restated as: How to force SAS to close a file used in an infile statement, without changing the FILEVAR variable?

I honestly don't know what else you want from me,

Huey

Respected Advisor
Posts: 3,777

Re: Repeating files in FILEVAR (INFILE statment) fails.

Ok I get now.  Sorry to be dense.  I think you can do the same thing PGstats is suggesting without changing your filename data set something like this.

data;

   set filenames;

   infile dummy filevar=filename end=eof;

   do while(not eof);

      input ....;

      output;

      end;

   filename = 'NULL';

   infile dummy filevar=filename;

   run;

As mentioned by PG NULL or whatever you call it will need to exist.

Super User
Super User
Posts: 6,502

Re: Repeating files in FILEVAR (INFILE statment) fails.

The file need not "really" exist if you point to operating system defined null file.

For Unix this is '/dev/null' and for Windows it is 'nul:' .

Super User
Posts: 5,092

Re: Repeating files in FILEVAR (INFILE statment) fails.

I don't know if this would work, but have you tried injecting an unused change in the source data?  Something like this:

another_file = 'C:\some\other\file';

infile temp filevar=another_file lrecl=10000 end=last line=_L;

infile temp filevar=path lrecl=10000 end=last line=_L;

Don't read anything from the extra file.  Just use it to change the source data set by adding the extra INFILE statement.

Super User
Posts: 5,092

Re: Repeating files in FILEVAR (INFILE statment) fails.

Sorry to say, I finally had a chance to test this and it didn't work.  I even tried adding an INPUT statement to read from the extra data source, and the results were bizarre but unsatisfactory.  Evidently, each INFILE statements tracks separately the file that it last referenced.

:smileycry:

Occasional Contributor
Posts: 12

Re: Repeating files in FILEVAR (INFILE statment) fails.

Thanks for your efforts Astounding. Let it be known if you find a remedy!

- Huey

Respected Advisor
Posts: 4,655

Re: Repeating files in FILEVAR (INFILE statment) fails.

Here is how it can be done, without too much pain, at least on Windows.

data files;
set Directories;
output;
path = "NUL";
output;

run;

filename toto dummy;

data _null_;
set files;
infile toto filevar=path end=done;
do while (not done);
input;
put _infile_;
end;
run;

PG

PG
Occasional Contributor
Posts: 12

Re: Repeating files in FILEVAR (INFILE statment) fails.

PG,

    

     That's definitely a viable work around... Every other observation points to a "NUL" destination. If you look at my first response to you, that's also how I ultimately got around it, except that I pointed to an existing "Empty.txt"

     I am curious about whether SAS errs or not when the FILEVAR variable points to a file that does not exist? That fear is the reason why I created an empty file to point to SAS.

     I honestly appreciate your effort and input on this. Smiley Happy

- Huey

Respected Advisor
Posts: 4,655

Re: Repeating files in FILEVAR (INFILE statment) fails.

Yes, I just did the test, if the file doesn't exist, you get the error message :

ERROR: Physical file does not exist, filename.

PG

PG
Occasional Contributor
Posts: 12

Re: Repeating files in FILEVAR (INFILE statment) fails.

PGStats,

     Your idea works. I suspect you got an ERROR: in your test run because a variable length issue on the "Path" variable, or something...?

     In any case, the "NUL" directory is a darn good idea, and I've modified your code to make the below (which works!):

          data _null_;

               /* Re-assign INFILE FILEVAR value using the "NUL" directory. This closes whatever file came before it!*/

                    length Path $ 300; /* Arbitrarilly long to handle upcoming directory text.*/  

                    Path="NUL";

                    infile temp filevar=Path;

               /* Get on with the program...*/

                    set Directories;

                    infile temp filevar=Path end=done;
                    do while (not done);
                          input;
                          put _infile_;
                    end;
          run;

Thanks --> PGStats, Tom,

Huey

Ask a Question
Discussion stats
  • 16 replies
  • 773 views
  • 3 likes
  • 5 in conversation