listing all files within a directory and subdirectories

[ilikesas]

Hi,

I am studying the supprt SAS example List All Files within a Directory Including Subdirectories and this is a macro that can not only list all files of a certain type within a directory, but also within the subdirectories of that directory.

I was trying to create a table that would include all these fies, but couldn't - I tried placing data; ... run; in different places but was getting an error all the time. I wll be very grateful for your help, especially that I feel that I might have missed some importnat part of which I am not even aware.

Thank you! 

[Patrick · Accepted solution]

The code below should give you a start.

%macro list_files(dir,ext);
  %local filrf rc did memcnt name i;
  %let rc=%sysfunc(filename(filrf,&dir));
  %let did=%sysfunc(dopen(&filrf));      

   %if &did eq 0 %then %do; 
    %put Directory &dir cannot be open or does not exist;
    %return;
  %end;

   %do i = 1 %to %sysfunc(dnum(&did));   

   %let name=%qsysfunc(dread(&did,&i));

      %if %qupcase(%qscan(&name,-1,.)) = %upcase(&ext) %then %do;
        %put &dir\&name;

        data _tmp;
          length dir $512 name $100;
          dir=symget("dir");
          name=symget("name");
        run;
        proc append base=want data=_tmp;
        run;quit;

      %end;
      %else %if %qscan(&name,2,.) = %then %do;        
        %list_files(&dir\&name,&ext)
      %end;

   %end;
   %let rc=%sysfunc(dclose(&did));
   %let rc=%sysfunc(filename(filrf));     

%mend list_files;
%list_files(c:\temp,sas)

[bravo433]

Hi Patrick,

This is a great macro. I am working within an environment where I don't have access to any Unix/DOS shell commands so I am totally reliant on SAS code for everything.

[ChrisNZ]

You just need to replace one line.

%macro list_files(dir,ext);
  %local filrf rc did memcnt name i;
  %let rc=%sysfunc(filename(filrf,&dir));
  %let did=%sysfunc(dopen(&filrf));      

   %if &did eq 0 %then %do; 
    %put Directory &dir cannot be open or does not exist;
    %return;
  %end;

   %do i = 1 %to %sysfunc(dnum(&did));   

   %let name=%qsysfunc(dread(&did,&i));

      %if %qupcase(%qscan(&name,-1,.)) = %upcase(&ext) %then %do;
        FILE="&dir\&name";  output;
      %end;
      %else %if %qscan(&name,2,.) = %then %do;        
        %list_files(&dir\&name,&ext)
      %end;

   %end;
   %let rc=%sysfunc(dclose(&did));
   %let rc=%sysfunc(filename(filrf));     

%mend list_files;

data WANT;
  length FILE $200;
  %list_files(c:\temp,sas)
run;

[rogerjdeangelis]

/* T008390 Ingenious code to recursively list all directories, sub-directories and files into a SAS datasets

Here is what is going on

  1. Build a template to hold the root, path and dir(depth)
  2. Modifily that datsets as new objects are found
  3. Use SAS dir commands to traverse directories

data template;
   length root path $200 dir 8;
   call missing(path,dir);
   input root;
 cards4;
D:\haven
D:\tym
;;;;
run;quit;


 data template;
   modify template;
   rc=filename('tmp',catx('/',root,path));
   dir=dopen('tmp');
   replace;
   if dir;
   path0=path;
   do _N_=1 to dnum(dir);
     path=catx('/',path0,dread(dir,_N_));
     output;
     end;
   rc=dclose(dir);
 run;

proc print data=template;
run;quit;

Obs      ROOT      PATH                             DIR

  1    D:\haven                                      1
  2    D:\tym                                        1
  3    D:\haven    old                               1
  4    D:\haven    pdf                               1
  5    D:\haven    r.sas7bdat                        0
  6    D:\haven    wps.sas7bdat                      0
  7    D:\tym      mta_incidence.sas7bdat            0
  8    D:\tym      pdf                               1
  9    D:\tym      tym_addavg.sas7bdat               0
 10    D:\tym      tym_asma_nit.sas7bdat             0
 11    D:\tym      tym_asma_wek.sas7bdat             0
 12    D:\tym      tym_estimate.sas7bdat             0
 13    D:\tym      tym_percent.sas7bdat              0
 14    D:\tym      tym_series_addmod.sas7bdat        0
 15    D:\tym      tym_series_addmon.sas7bdat        0
 16    D:\tym      tym_series_season.sas7bdat        0
 17    D:\tym      tym_series_series.sas7bdat        0
 18    D:\tym      tym_series_twoser.sas7bdat        0
 19    D:\tym      tym_series_twoser001.sas7bdat     0
 20    D:\tym      tym_series_twoser001.zip          0
 21    D:\haven    old/r.sas7bdat                    0
 22    D:\haven    old/testdta - Copy.sas7bdat       0
 23    D:\haven    old/testdta.dta                   0
 24    D:\haven    old/testr.sas7bdatx               0
 25    D:\haven    old/testwps.sas7bdat              0
 26    D:\haven    old/txzips.sas7bdatx              0
 27    D:\haven    old/wps.sas7bdat                  0
 28    D:\haven    old/wps.sas7bdatz                 0
 29    D:\haven    old/wpsx.sas7bdat                 0
 30    D:\haven    pdf/old                           1
 31    D:\haven    pdf/sas7bdat.pdf                  0
 32    D:\tym      pdf/tym_series_slyser.pdf         0
 33    D:\haven    pdf/old/testdircode.txt           0