Combine datasets in macro loop when some are missing

[dwarden3]

Hello,

I have a macro that runs a regression over a set of variables. It outputs datafiles from the regressions, manipulates them, and then I have code that joins them all together. However, some of the regressions do not converge, so the output datasets are not generated. Because of this, the combined dataset is not generated because it is trying to use nonexistent datasets to make itself. 

Here is the code I use to combine the datasets: 

data Combined;
	set
	%do i = 1 %to &numo;
		tt3_&i 
	%end; ;
	*ends i; 
run; 

&numo is the number of variables to be run (1,2,3,4...), so tt3_&i  is the output dataset and might be tt3_121, etc. 

I am getting errors like this: 

ERROR: File WORK.TT3_16.DATA does not exist.
ERROR: File WORK.TT3_34.DATA does not exist.
ERROR: File WORK.TT3_64.DATA does not exist.
ERROR: File WORK.TT3_72.DATA does not exist.
ERROR: File WORK.TT3_104.DATA does not exist.
ERROR: File WORK.TT3_109.DATA does not exist.
ERROR: File WORK.TT3_125.DATA does not exist.

How can I either have the code ignore these missing datasets/errors or create dummy datasets when those don't exist so that the combination step works. 

Thanks in advance! 

[PaigeMiller]

The much simpler way to do this is to not use a macro loop. Instead

data Combined;
	set tt3_:;
run;

This may not work in all situations, for example if you have data set tt3_500 and you want the loop to end at 200, then that won't work, and maybe you need to go back to using a macro loop with modifications.

[dwarden3]

Thank you. There are instances where I would want &numo to be 200 instead of all 500 (or whatever number), so I went with the other solution, but this is still very helpful! 

[Quentin · Accepted solution]

I just learned from another post here, that there is a system option NODSNFERR which can be set so that references to non-existing data sets are not an error.

NODSNFERR
specifies that SAS ignore the error message and continue processing if a reference is made to a SAS data set that does not exist. The data set reference is treated as if _NULL_ had been specified.

Another option would be to test for existence with:

data Combined;
	set
	%do i = 1 %to &numo;
		%if %sysfunc(exist(tt3_&i)) %then tt3_&i  ;
	%end; ;
	*ends i; 
run;

[dwarden3]

Thank you! I knew there was a way to suppress the errors, but I couldn't find it. 

I decided to go with the sysfunc solution since I don't necessarily want to change system options. 

[A_Kh]

Another interesting and useful system option.

In the second example,

%macro combine;
data Combined;
	set
	%do i = 1 %to &numo;
		%if %sysfunc(exist(tt3_&i)) %then tt3_&i  
	%end; 
    ;
run;
%mend; 

%if ..%then expression produces a warning message "WARNING: Missing semicolon between %THEN clause and END has been assumed.".
Any idea about how to avoid this warning in the log?

[Quentin]

You need a semicolon to end the %IF statement:

%if %sysfunc(exist(tt3_&i)) %then tt3_&i  ;   %*<--- semicolon here to end %IF statement ;

As a style issue, I usually code as:

%if %sysfunc(exist(tt3_&i)) %then %do;
  tt3_&i
%end;

The later style makes it clearer which semicolons are part of the macro language, rather than SAS language semicolons.

[A_Kh]

Cool! I see the difference now, used to code the latter way and no semicolon at the end.