Explanation of behaviour when conditionally executing an infile statement

[Amir]

Hi,

Using Enterprise Guide 8.2, SAS 9.4.

I'm looking for an explanation of the behaviour shown herein, e.g., documented behaviour (with a link).

I want to check the existence of a file using the fileexist() function before I attempt to read it conditionally using the infile statement, as follows:

%let my_file = /file/does/not/exist;

data tmp;
   length record $ 1000;

   /* check if file exists */
   file_found = fileexist("&my_file");

   putlog file_found;

   /* if file exists then read contents */
   if file_found then
   do;
      infile "&my_file" end = last_record;

      do until (last_record);

         input;

         record = _infile_;

         output;
      end;
   end;
run;

The code works OK when the file exists, but when a file does not exist I get the error:

28         %let my_file = /file/does/not/exist;
29         
30         data tmp;
31            length record $ 1000;
32         
33            /* check if file exists */
34            file_found = fileexist("&my_file");
35         
36            putlog file_found;
37         
38            /* if file exists then read contents */
39            if file_found then
40            do;
41               infile "&my_file" end = last_record;
42         
43               do until (last_record);
44         
45                  input;
46         
47                  record = _infile_;
48         
49                  output;
50               end;
51            end;
52         run;

ERROR: Physical file does not exist, /file/does/not/exist.
NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set WORK.TMP may be incomplete.  When this step was stopped there were 0 observations and 2 variables.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds

I have achieved this in the past using macro code, but I don't see why the error message is appearing when the infile statement is not being executed in data step code as it is an executable statement as opposed to a declarative one (see list at end of Executable and Declarative Statements).

I can use alternative coding methods, but as I said at the start, I'm looking for an explanation of the behaviour shown herein, e.g., documented behaviour (with a link).

Thanks & kind regards,

Amir.

[Kurt_Bremser]

If you use a literal file name, the existence of the file is checked at compile time; you can see this because the PUT of your file_found variable is never executed.

You can avoid this by making the filename dynamic through the FILEVAR= option:

data tmp;
fname = "&my_file.";
length record $ 1000;
file_found = fileexist(fname);
put file_found=;
if file_found
then do;
  infile dummy filevar=fname end=last_record;
  do until (last_record);
    input;
    record = _infile_;
    output;
  end;
end;
run;

[Amir]

Thanks @Kurt_Bremser.

I deduced the same about compile time action when I couldn't see the value of variable file_found in the log, but assumed the infile statement was being executed, as opposed to the literal file name being checked, as you mentioned.

I'll see if anyone else has anything to add.

Thanks & kind regards,

Amir.

[Tom · Accepted solution]

There are actions that happen before the data step starts running for INFILE statement.  Are you at the end of the file yet (END= option) for example.  In particular when the name is fixed the file is checked for existence. Note that to SAS the value of the macro variable is a fixed string since it does not know you used a macro variable to generate it.

Use the FILEVAR= option to have the filename be dynamic and SAS cannot (and so will not) check if the file exists before the data step starts.

%let my_file = /file/does/not/exist;
data tmp;
   length record $ 1000;
   filename="&my_file";
   if fileexist(filename) then do;
     infile dummy filevar=filename truncover ;
     input record $char1000. ;
  end;
  else do;
    putlog "File not found. " filename= :$quote. ;
    stop;
  end;
run;

[Amir]

Thanks @Tom.

Apologies for the delayed response. I think linking the behaviour with that of how the end= option is processed helps compartmentalise how to think of this, even though the more general file-specification is not an optional argument.

I also tried using the filename() function, but encountered similar issues even though a fileref was used in the infile statement. For example, in the changed code below I commented out the code in the do-end block of the if statement and the file_found flag is displayed in the log, but uncommenting the code does not display the file_found flag, even though the explicit file reference is first assigned to a data step variable before being used in the filename() function before being indirectly referenced in the infile statement using a fileref. This seems like a lot of evaluation before running the data step.

%let my_file = /file/does/not/exist;

data tmp;
   length record $ 1000;

   my_file = "&my_file";
   file_found = filename('in_file',my_file);

   putlog file_found=;

   /* if file exists then read contents */
   if file_found eq 0 then
   do;
/*      infile in_file end = last_record;*/
/**/
/*      do until (last_record);*/
/**/
/*         input;*/
/**/
/*         record = _infile_;*/
/**/
/*         output;*/
/*      end;*/
   end;
run;

Thanks & kind regards,

Amir.

[Tom]

Not sure what you are saying here.

If you use a fileref in the INFILE statement I would assume that the existence of the FILEREF will be checked when the data step is compiled (or really during what ever happends before it start really running) and that the fileref would be opened for read access.  If you then tried to redefine the fileref using the FILENAME() function while the step is running I would assume you would get an error saying the fileref cannot be redefined because it is already in use (by this data step!).

[Amir]

Hi @Tom,

Following the suggestions already given about using the filevar= option, I thought this degree of separation from using the literal file name in the infile statement would also work for a fileref that is assigned using the filename() function, but this was evidently not the case. I wasn't trying to redefine the fileref at any point in the data step.

As you implied, there seems to be more going on than just compile-time actions and run-time actions.

Thanks & kind regards,

Amir.