Saving the files found using the list_files macro into a global macro variable

[serrld113]

I need to find all the SAS files in all the sub-directories in a drive and save them into a macro variable that I can use outside the macro. I found this example below that prints the names of all the files of said directory into the log. I know that it saves them locally into &name, but I need to use name outside this macro. I tried changing %local to %global and the macro ran forever. I also tried saving name into another variable within the macro but also failed. I'd much rather save these file names into a table, but a macro variable will also work. Any help would be appreciated.

I'm using the following macro that I found here:

https://documentation.sas.com/?docsetId=mcrolref&docsetTarget=n0js70lrkxo6uvn1fl4a5aafnlgt.htm&docse...

This is it below:

%macro list_files(dir,ext);
%local filrf rc did memcnt name i;
%let rc=%sysfunc(filename(filrf,&dir));
%let did=%sysfunc(dopen(&filrf));

%if &did eq 0 %then %do; 
%put Directory &dir cannot be open or does not exist;
%return;
%end;

%do i = 1 %to %sysfunc(dnum(&did));

%let name=%qsysfunc(dread(&did,&i));

%if %qupcase(%qscan(&name,-1,.)) = %upcase(&ext) %then %do;
%put &dir\&name;
%end;
%else %if %qscan(&name,2,.) = %then %do; 
%list_files(&dir\&name,&ext)
%end;

%end;
%let rc=%sysfunc(dclose(&did));
%let rc=%sysfunc(filename(filrf));

%mend list_files;
%list_files(c:\temp,sas);

[PaigeMiller]

I have a somewhat different macro that saves the results in a SAS data set. From there, you can do whatever you want with the data set.

%macro list_files(dir=,outdsn=files,all_subdirs=N);
data _null_;
	if upcase("&all_subdirs")='N' then call symputx('str',cat("'dir ",'"',"%trim(&dir)"" /b'"));
	else if upcase("&all_subdirs")='Y' then call symputx('str',cat("'dir ",'"',"%trim(&dir)"" /b /s'"));
run;
filename tmp pipe &str;
data &outdsn;
  infile tmp dlm="¬" missover;
  input filenames :$200.;
run;
%mend;

 (Works only on Windows) 

[serrld113]

I really appreciate this, but it didn't output a dataset for me. I added my directory path and changed all_subdirs = "Y"

[PaigeMiller]

---

@serrld113 wrote:
I really appreciate this, but it didn't output a dataset for me. I added my directory path and changed all_subdirs = "Y"

---

Works for me.

Note: for your own benefit, as well as the benefit of others, it is never sufficient to say something didn't work and provide no additional information. If it didn't work, give us details about what you did and what happened. Otherwise, the conversation goes "didn't work" "works for me" "doesn't work for me" "works fine for me"

[serrld113]

I apologize. In my reply I included that I've added my directory as well as changed the option for all_subdirs to Y. I've also tried my directory between quotation marks and without. Thanks again.

[PaigeMiller]

We need to see your code, or the log, or both. That's the only way to figure out what happens when code doesn't work.

[ballardw]

I don't know about your system but I have some project that if the paths are kept in the values with the filename then the length of single macro variable would be exceeded.

Plus are your paths and file names really clean, like no spaces or odd characters in the folder or file names?

If not, then you might have some fun using such an ugly variable.

Anything that can be done with macro variable and much more can be done with values in a data set. I would, for any purpose I can envision using such, go with @PaigeMiller's suggestion.

[serrld113]

I added that I prefer a data set over a macro variable.

[serrld113]

The one thing that kinda works is adding a data step after the %put statement in my original post. However, it will only output the last SAS file the loop finds. I was wondering if there was a dynamic way to keep adding to the variable as the loop runs. 

      %if %qupcase(%qscan(&name,-1,.)) = %upcase(&ext) %then %do;
			%put &dir\&name;

/* this is the part I add */
				data files;
			  		filenames = "&dir\&name";
				run;

		%end;

[Tom]

Don't put the file names into macro variables. Put them into a dataset.  A macro variable has a maxium length of 64K.  Even a moderate number of names could easily overflow that limit.

There is a great idea for doing the recursion into the subdirectories by using the MODIFY statement in a data step posted in this thread: https://communities.sas.com/t5/SAS-Programming/listing-all-files-within-a-directory-and-subdirectori...

Here is an example.  First create a dataset with the structure you want and one observation for each top level directory you want to read.

data filelist;
  length root filename $256 dir level 8;
  input root;
  retain filename ' ' level 0 dir 1;
cards4;
c:\temp
;;;;

Now run this data step to scan all of the files in the directory(s).

data filelist;
  modify filelist;
  rc1=filename('tmp',catx('\',root,filename));
  rc2=dopen('tmp');
  dir = not not rc2;
  replace;
  if dir;
  level=level+1;
  filename0=filename;
  do i=1 to dnum(rc2);
    filename=catx('\',filename0,dread(rc2,i));
    output;
  end;
  rc3=dclose(rc2);
run;

If you want to limit to files with a particular extension it is probably easier after you have the dataset already.

proc print data=filelist;
  where filename like '%.sas7bdat';
run;

Results

Obs     root      filename               dir    level

 4     c:\temp    one.sas7bdat            0       1
 6     c:\temp    two.sas7bdat            0       1
 7     c:\temp    sub1\three.sas7bdat     0       2