DATA Step, Macro, Functions and more

unexpected behavior in fileexist function in Macro

Reply
Occasional Contributor
Posts: 8

unexpected behavior in fileexist function in Macro

Using Base SAS 9.4M4 on Linux, I want to create a zero-byte file in the SAS WORK directory.  After this target file is defined with a FILENAME statement I then want to verify it exists using the FILEEXIST function.  And since this is Macro, it uses the %SYSFUNC Macro function for the test.  In the SAS log below, I have two Macros: %write1 and %write2.  They both use the same logic, yet for some reason I can not understand, the %write1 Macro fails the FILEEXIST function test.  

 

Following the FILEEXIST function test for the target file in Macro %write1, I have a _null_ Data Step that opens and closes this non-existent file!

 

I've tried to anticipate the possibility of a timing issue between the Macro compile phase and the Macro execution phase, but I think that problem is eliminated since the %LET to define the target file is done prior to defining the %write1 Macro.

 

Any insights are appreciated.  The log is first, followed by the source program.

 

Best,

Randy

 

<snip>


NOTE: SAS initialization used:
      real time           0.02 seconds
      cpu time            0.01 seconds
      
1          options mtrace mlogic mprint;
2          %put ===> PURPOSE: Create a zero-byte file called "arbitrary_file.txt" in the SAS WORK directory;
===> PURPOSE: Create a zero-byte file called "arbitrary_file.txt" in the SAS WORK directory
3          %put;
 
4          %put ===> SAS WORK is %sysfunc(getoption(work));
===> SAS WORK is /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6
5          %put;
 
6          
7          %let file1 = %sysfunc(getoption(work))/arbitrary_file.txt;
8          %macro write1;
9             %put ===> &file1;
10            data _null_;
11            filename mm_file1 "&file1";
12            file mm_file1;
13            put ;
14         
15            %if %sysfunc(fileexist(&file1))
16            %then %do;
17               %put ===> file &file1 exists.;
18         	  %return;
19            %end;
20         
21            %else %do;
22               %put ===> file &file1 DOES NOT exists.;
23            %end;	
24         
25            run;
26         %mend write1;
27         
28         %let file2 = %sysfunc(getoption(work))/another_arbitrary_file.txt;
29         %macro write2;
30            data _null_;
31            filename mm_file2 "&file2";
32            file mm_file2;
33            put "The name of this file is:" /
34                "&file2/another_arbitrary_file.txt";
35            run;
36         
37         %let rc2 = %sysfunc(fileexist(&file2));
38            %if %sysfunc(fileexist(&file2))
39            %then %do;
40               %put ===> file &file2 exists.;
41            %end;
42         
43            %else %do;
44               %put ===> file &file2 DOES NOT exists.;
45            %end;
46         
47            %put ===> %nrstr(&rc2) resolves to: &rc2;	
48            run;
49         %mend write2;
50         %write1;
MLOGIC(WRITE1):  Beginning execution.
MLOGIC(WRITE1):  %PUT ===> &file1
===> /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt
MPRINT(WRITE1):   data _null_;
MPRINT(WRITE1):   filename mm_file1 "/tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt";
MPRINT(WRITE1):   file mm_file1;
MPRINT(WRITE1):   put ;
MLOGIC(WRITE1):  %IF condition %sysfunc(fileexist(&file1)) is FALSE
MLOGIC(WRITE1):  %PUT ===> file &file1 DOES NOT exists.
===> file /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt DOES NOT exists.
MPRINT(WRITE1):   run;

NOTE: The file MM_FILE1 is:
      Filename=/tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt,
      Owner Name=DIR\thomas.betancourt,
      Group Name=DIR\domain^users,
      Access Permission=-rw-r-----,
      Last Modified=02Jan2018:18:30:57

NOTE: 1 record was written to the file MM_FILE1.
      The minimum record length was 0.
      The maximum record length was 0.

NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds
      

MLOGIC(WRITE1):  Ending execution.
51         %write2;
MLOGIC(WRITE2):  Beginning execution.
MPRINT(WRITE2):   data _null_;
MPRINT(WRITE2):   filename mm_file2 "/tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/another_arbitrary_file.txt";
MPRINT(WRITE2):   file mm_file2;
MPRINT(WRITE2):   put "The name of this file is:" / 
"/tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/another_arbitrary_file.txt/another_arbitrary_file.txt";
MPRINT(WRITE2):   run;

NOTE: The file MM_FILE2 is:
      Filename=/tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/another_arbitrary_file.txt,
      Owner Name=DIR\thomas.betancourt,
      Group Name=DIR\domain^users,
      Access Permission=-rw-r-----,
      Last Modified=02Jan2018:18:30:57

NOTE: 2 records were written to the file MM_FILE2.
      The minimum record length was 25.
      The maximum record length was 95.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds
      

MLOGIC(WRITE2):  %LET (variable name is RC2)
MLOGIC(WRITE2):  %IF condition %sysfunc(fileexist(&file2)) is TRUE
MLOGIC(WRITE2):  %PUT ===> file &file2 exists.
===> file /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/another_arbitrary_file.txt exists.
MLOGIC(WRITE2):  %PUT ===> %nrstr(&rc2) resolves to: &rc2
===> &rc2 resolves to: 1
MPRINT(WRITE2):   run;
MLOGIC(WRITE2):  Ending execution.
52         
53         proc sql;
54               select xpath into :file_one
55         	  from dictionary.extfiles
56         	  where fileref = "MM_FILE1";
57            quit;
NOTE: The PROCEDURE SQL printed page 1.
NOTE: PROCEDURE SQL used (Total process time):
      real time           0.02 seconds
      cpu time            0.03 seconds
      

58         
59         %let len1 = %length(&file1);
60         %let len2 = %length(&file_one);
61         
62         %put ===> path from %nrstr(&file1) is:     &file1 with length: &len1;
===> path from &file1 is:     /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt with length: 60
63         %put ===> path from %nrstr(&file_one) is:  &file_one with length: &len2;
===> path from &file_one is:  /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt                                          
                                                                                                                               
with length: 60
NOTE: SAS Institute Inc., SAS Campus Drive, Cary, NC USA 27513-2414
NOTE: The SAS System used:
      real time           0.06 seconds
      cpu time            0.05 seconds
      

The source program:

options mtrace mlogic mprint;
%put ===> PURPOSE: Create a zero-byte file called "arbitrary_file.txt" in the SAS WORK directory;
%put;
%put ===> SAS WORK is %sysfunc(getoption(work));
%put;

%let file1 = %sysfunc(getoption(work))/arbitrary_file.txt;
%macro write1;
   %put ===> &file1;
   data _null_;
   filename mm_file1 "&file1";
   file mm_file1;
   put ;

   %if %sysfunc(fileexist(&file1))
   %then %do;
      %put ===> file &file1 exists.;
	  %return;
   %end;
   
   %else %do;
      %put ===> file &file1 DOES NOT exists.;
   %end;	  

   run;
%mend write1;

%let file2 = %sysfunc(getoption(work))/another_arbitrary_file.txt;
%macro write2;
   data _null_;
   filename mm_file2 "&file2";
   file mm_file2;
   put "The name of this file is:" /
       "&file2/another_arbitrary_file.txt"; 
   run;
   
%let rc2 = %sysfunc(fileexist(&file2));
   %if %sysfunc(fileexist(&file2))
   %then %do;
      %put ===> file &file2 exists.;
   %end;
   
   %else %do;
      %put ===> file &file2 DOES NOT exists.;
   %end;
   
   %put ===> %nrstr(&rc2) resolves to: &rc2;	     
   run;   
%mend write2;
%write1;
%write2;

proc sql;
      select xpath into :file_one 
	  from dictionary.extfiles
	  where fileref = "MM_FILE1";
   quit;

%let len1 = %length(&file1);
%let len2 = %length(&file_one);

%put ===> path from %nrstr(&file1) is:     &file1 with length: &len1;
%put ===> path from %nrstr(&file_one) is:  &file_one with length: &len2;
Super User
Posts: 22,875

Re: unexpected behavior in fileexist function in Macro

Posted in reply to rbetancourt

Following the FILEEXIST function test for the target file in Macro %write1, I have a _null_ Data Step that opens and closes this non-existent file!

 

Your data step does not end/close before you check if the file exists. Since SAS is likely writing to a temp file first and then renaming/copying it over. 

 

Adding a RUN after your data _null_ step but before you call the FEXIST function seems to solve this issue. 

If you have a use case where you need to check if the file exists while running, it may help to elaborate on what you're trying to achieve. 

There's also the FEXIST function, if that has any added benefit to you. 

 

I also made the macro variable GLOBAL but I don't think that's required.

 

%global file1;
%let file1 = /folders/myfolders/arbitrary_file.txt;

%macro write1;
    %put ===> &file1.;

    data _null_;
        filename mm_file1 "&file1.";
        file mm_file1;
        put;
    run;

    %if %sysfunc(fileexist(&file1.)) %then
        %do;
            %put ===> file &file1. exists.;
        %end;
    %else
        %do;
            %put ===> file &file1. DOES NOT exists.;
        %end;

%mend write1;

%write1;

 

%WRITE2 has two RUN statements which are not required and the %RETURN doesn't add much here either that I can see, but you may have simplified your code for this forum. 

 

FILENAME statements are global statements and typically not included in a data step. You can create the file reference directly without resorting to macro variables. 

 

filename mm_file1  "%sysfunc(getoption(work))/another_arbitrary_file.txt";

data _null_;

file mm_file1;

put ;

run;

The documentation has a bunch of examples along these lines that may be helpful, specifically:

http://documentation.sas.com/?docsetId=mcrolref&docsetTarget=p1hogk0ekd1z62n18xx76ou16unt.htm&docset...

 

And the full list of what's available:

https://communities.sas.com/t5/SAS-Communities-Library/SAS-9-4-Macro-Language-Reference-Has-a-New-Ap...

 

Spoiler

rbetancourt wrote:

Using Base SAS 9.4M4 on Linux, I want to create a zero-byte file in the SAS WORK directory.  After this target file is defined with a FILENAME statement I then want to verify it exists using the FILEEXIST function.  And since this is Macro, it uses the %SYSFUNC Macro function for the test.  In the SAS log below, I have two Macros: %write1 and %write2.  They both use the same logic, yet for some reason I can not understand, the %write1 Macro fails the FILEEXIST function test.  

 

Following the FILEEXIST function test for the target file in Macro %write1, I have a _null_ Data Step that opens and closes this non-existent file!

 

I've tried to anticipate the possibility of a timing issue between the Macro compile phase and the Macro execution phase, but I think that problem is eliminated since the %LET to define the target file is done prior to defining the %write1 Macro.

 

Any insights are appreciated.  The log is first, followed by the source program.

 

Best,

Randy

 

<snip>


NOTE: SAS initialization used:
      real time           0.02 seconds
      cpu time            0.01 seconds
      
1          options mtrace mlogic mprint;
2          %put ===> PURPOSE: Create a zero-byte file called "arbitrary_file.txt" in the SAS WORK directory;
===> PURPOSE: Create a zero-byte file called "arbitrary_file.txt" in the SAS WORK directory
3          %put;
 
4          %put ===> SAS WORK is %sysfunc(getoption(work));
===> SAS WORK is /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6
5          %put;
 
6          
7          %let file1 = %sysfunc(getoption(work))/arbitrary_file.txt;
8          %macro write1;
9             %put ===> &file1;
10            data _null_;
11            filename mm_file1 "&file1";
12            file mm_file1;
13            put ;
14         
15            %if %sysfunc(fileexist(&file1))
16            %then %do;
17               %put ===> file &file1 exists.;
18         	  %return;
19            %end;
20         
21            %else %do;
22               %put ===> file &file1 DOES NOT exists.;
23            %end;	
24         
25            run;
26         %mend write1;
27         
28         %let file2 = %sysfunc(getoption(work))/another_arbitrary_file.txt;
29         %macro write2;
30            data _null_;
31            filename mm_file2 "&file2";
32            file mm_file2;
33            put "The name of this file is:" /
34                "&file2/another_arbitrary_file.txt";
35            run;
36         
37         %let rc2 = %sysfunc(fileexist(&file2));
38            %if %sysfunc(fileexist(&file2))
39            %then %do;
40               %put ===> file &file2 exists.;
41            %end;
42         
43            %else %do;
44               %put ===> file &file2 DOES NOT exists.;
45            %end;
46         
47            %put ===> %nrstr(&rc2) resolves to: &rc2;	
48            run;
49         %mend write2;
50         %write1;
MLOGIC(WRITE1):  Beginning execution.
MLOGIC(WRITE1):  %PUT ===> &file1
===> /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt
MPRINT(WRITE1):   data _null_;
MPRINT(WRITE1):   filename mm_file1 "/tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt";
MPRINT(WRITE1):   file mm_file1;
MPRINT(WRITE1):   put ;
MLOGIC(WRITE1):  %IF condition %sysfunc(fileexist(&file1)) is FALSE
MLOGIC(WRITE1):  %PUT ===> file &file1 DOES NOT exists.
===> file /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt DOES NOT exists.
MPRINT(WRITE1):   run;

NOTE: The file MM_FILE1 is:
      Filename=/tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt,
      Owner Name=DIR\thomas.betancourt,
      Group Name=DIR\domain^users,
      Access Permission=-rw-r-----,
      Last Modified=02Jan2018:18:30:57

NOTE: 1 record was written to the file MM_FILE1.
      The minimum record length was 0.
      The maximum record length was 0.

NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds
      

MLOGIC(WRITE1):  Ending execution.
51         %write2;
MLOGIC(WRITE2):  Beginning execution.
MPRINT(WRITE2):   data _null_;
MPRINT(WRITE2):   filename mm_file2 "/tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/another_arbitrary_file.txt";
MPRINT(WRITE2):   file mm_file2;
MPRINT(WRITE2):   put "The name of this file is:" / 
"/tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/another_arbitrary_file.txt/another_arbitrary_file.txt";
MPRINT(WRITE2):   run;

NOTE: The file MM_FILE2 is:
      Filename=/tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/another_arbitrary_file.txt,
      Owner Name=DIR\thomas.betancourt,
      Group Name=DIR\domain^users,
      Access Permission=-rw-r-----,
      Last Modified=02Jan2018:18:30:57

NOTE: 2 records were written to the file MM_FILE2.
      The minimum record length was 25.
      The maximum record length was 95.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds
      

MLOGIC(WRITE2):  %LET (variable name is RC2)
MLOGIC(WRITE2):  %IF condition %sysfunc(fileexist(&file2)) is TRUE
MLOGIC(WRITE2):  %PUT ===> file &file2 exists.
===> file /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/another_arbitrary_file.txt exists.
MLOGIC(WRITE2):  %PUT ===> %nrstr(&rc2) resolves to: &rc2
===> &rc2 resolves to: 1
MPRINT(WRITE2):   run;
MLOGIC(WRITE2):  Ending execution.
52         
53         proc sql;
54               select xpath into :file_one
55         	  from dictionary.extfiles
56         	  where fileref = "MM_FILE1";
57            quit;
NOTE: The PROCEDURE SQL printed page 1.
NOTE: PROCEDURE SQL used (Total process time):
      real time           0.02 seconds
      cpu time            0.03 seconds
      

58         
59         %let len1 = %length(&file1);
60         %let len2 = %length(&file_one);
61         
62         %put ===> path from %nrstr(&file1) is:     &file1 with length: &len1;
===> path from &file1 is:     /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt with length: 60
63         %put ===> path from %nrstr(&file_one) is:  &file_one with length: &len2;
===> path from &file_one is:  /tmp/SAS_workB8620000F9C1_slrs01va6f1clw6/arbitrary_file.txt                                          
                                                                                                                               
with length: 60
NOTE: SAS Institute Inc., SAS Campus Drive, Cary, NC USA 27513-2414
NOTE: The SAS System used:
      real time           0.06 seconds
      cpu time            0.05 seconds
      

The source program:

options mtrace mlogic mprint;
%put ===> PURPOSE: Create a zero-byte file called "arbitrary_file.txt" in the SAS WORK directory;
%put;
%put ===> SAS WORK is %sysfunc(getoption(work));
%put;

%let file1 = %sysfunc(getoption(work))/arbitrary_file.txt;
%macro write1;
   %put ===> &file1;
   data _null_;
   filename mm_file1 "&file1";
   file mm_file1;
   put ;

   %if %sysfunc(fileexist(&file1))
   %then %do;
      %put ===> file &file1 exists.;
	  %return;
   %end;
   
   %else %do;
      %put ===> file &file1 DOES NOT exists.;
   %end;	  

   run;
%mend write1;

%let file2 = %sysfunc(getoption(work))/another_arbitrary_file.txt;
%macro write2;
   data _null_;
   filename mm_file2 "&file2";
   file mm_file2;
   put "The name of this file is:" /
       "&file2/another_arbitrary_file.txt"; 
   run;
   
%let rc2 = %sysfunc(fileexist(&file2));
   %if %sysfunc(fileexist(&file2))
   %then %do;
      %put ===> file &file2 exists.;
   %end;
   
   %else %do;
      %put ===> file &file2 DOES NOT exists.;
   %end;
   
   %put ===> %nrstr(&rc2) resolves to: &rc2;	     
   run;   
%mend write2;
%write1;
%write2;

proc sql;
      select xpath into :file_one 
	  from dictionary.extfiles
	  where fileref = "MM_FILE1";
   quit;

%let len1 = %length(&file1);
%let len2 = %length(&file_one);

%put ===> path from %nrstr(&file1) is:     &file1 with length: &len1;
%put ===> path from %nrstr(&file_one) is:  &file_one with length: &len2;

 

Respected Advisor
Posts: 4,574

Re: unexpected behavior in fileexist function in Macro

@rbetancourt

And to add to what @Reeza wrote: It's normally best to first try and write as much as you can without macro and only "macrotize" once you've got a working "base SAS" version.

 

Below such a "base SAS" version to build upon.

%let myfile=%sysfunc(pathname(work))\some_file7.txt;


data _nulll_;

  /* this bit checks for existence of the file in data step execution phase */
  rc=fileexist("&myfile");
  if rc=1 then
    do;
      put "*** File Exist" / "*** File is: &myfile";
    end;
  else
    do;
      put "*** File does Not Exist" / "*** File is: &myfile";
    end;

  /* this bit creates the file during data step compilation phase */ 
  stop;
  file "&myfile";

run; 

http://documentation.sas.com/?docsetId=lrcon&docsetTarget=p08a4x7h9mkwqvn16jg3xqwfxful.htm&docsetVer...

Occasional Contributor
Posts: 8

Re: unexpected behavior in fileexist function in Macro

Thank you Reeza.  This is very helpful.  And also thanks for pointing out the documentation examples.

 

Best,

Randy

Respected Advisor
Posts: 4,574

Re: unexpected behavior in fileexist function in Macro

Posted in reply to rbetancourt

@rbetancourt

You're creating the file via a SAS data step. Not sure why you then also have to check if this data step created the file as if not then the data step would fail and throw an error. You don't have to use macro syntax for this check.

 

For %write1(): Macro code executes before SAS code - by RUN group. Compare where you've placed the RUN statement in the two macros.

Highlighted
Occasional Contributor
Posts: 8

Re: unexpected behavior in fileexist function in Macro

Thank you Patrick--this is very helpful.  

Ask a Question
Discussion stats
  • 5 replies
  • 165 views
  • 2 likes
  • 3 in conversation