How to: List all the files in a folder

[MS_Bhati]

Hi, I am using SAS University Edition and trying to get names of all the file available in a Folder (Folder inside Shared Folder). So that, I can prepared a Macro to import and append all the data sets in one. 
I need help to get names for all the CSV files available in my Folder.

[ChrisNZ]

This question has been asked over and over. Maybe this topic should be a sticky @AnnaBrown @BeverlyBrown ?

Search   list files   for example.

Out of many replies, here is a clever way, devised by Tom.

Spoiler

data FILELIST;
  format DIR LEVEL 3. PATH FILENAME $256. LASTMOD datetime.;
  retain DIR 0 LEVEL -1 FILENAME ' ' LASTMOD .;
  input PATH;
cards;
c:\temp\
x:\ 
run;
data FILELIST;
  retain SEP "%sysfunc(ifc(&sysscp=WIN,\,/))";
  modify FILELIST;
  RC    = filename('tmp',catx(SEP, PATH, FILENAME));
  DIRID = dopen('tmp');
  DIR   = (DIRID>0);
  if DIR then do;
    PATH     = catx(SEP, PATH, FILENAME);
    FILENAME = ' ';
    if SEP='/' then LASTMOD  = input(dinfo(DIRID, doptname(DIRID, 5)), nldatm100.);
    replace;
    DIR  =0;
    LEVEL=LEVEL+1;
    do I=1 to dnum(DIRID);
      FILENAME=dread(DIRID, I);
      output;
    end;
    RC=dclose(DIRID);
  end;
  else if scan(FILENAME,-1,'.') ne 'lck' then do;
    FID=fopen('tmp','i',0,'b');
    %* == FOPTNAME values for single files ==
         Unix                Win
    1  Filename            Filename
    2  Owner name          RECFM
    3  Group name          LRECL
    4  Access Permission   File Size (bytes)
    5  Last Modified       Last Modified
    6  File Size (bytes)   Create Time      ;
    if FID  then do;
      LASTMOD = input(finfo(FID, foptname(FID, 5)), nldatm100.);
      SIZE    = input(finfo(FID, foptname(FID, ifn(SEP='/',6,4))), 32.);
      RC      = fclose(FID);
      replace;
    end;
  end;
run;

[ScottBass]

Yet another approach...

https://github.com/scottbass/SAS/blob/master/Macro/dirlist.sas

[SASJedi]

I do this enough to have a macro for it. There are some dependencies on other macros I've written. You can try it yourslef :

/* Get the necessary macro files */
filename getmacro url "https://raw.githubusercontent.com/SASJedi/sas-macros/master/exist.sas";
%include getmacro;
filename getmacro url "https://raw.githubusercontent.com/SASJedi/sas-macros/master/fileattribs.sas";
%include getmacro;
filename getmacro url "https://raw.githubusercontent.com/SASJedi/sas-macros/master/translate.sas";
%include getmacro;
filename getmacro url "https://raw.githubusercontent.com/SASJedi/sas-macros/master/findfiles.sas";
%include getmacro;

/* Get syntax help in the log */
%FindFiles(?)

/* List files in the s:/workshop and all sub-folders, save as a SAS data set */
%FindFiles(s:/workshop,,work.myFiles)

[Kurt_Bremser · Accepted solution]

Use the built-in SAS functions:

data filenames;
length fref $8 fname $200;
did = filename(fref,'/folders/myfolders');
did = dopen(fref);
do i = 1 to dnum(did);
  fname = dread(did,i);
  output;
end;
did = dclose(did);
did = filename(fref);
keep fname;
run;

[librasonali]

hi Kurt ,

May you please explain me the above approach ??
thank you in advance

[Kurt_Bremser]

It first creates a file reference for the directory (FILENAME), opens the directory and creates a handle (DOPEN), and then loops over all (DNUM) entries in the directory and reads each (DREAD). At the end, it deassigns the handle and the file reference.

I recommend studying the documentation of all these functions.

[Kevin_Viel]

I wrote a macro that allows recursive functionality and adds FINFO data.  This works on both Windows and Linux, even when the X statement cannot be used.  On LSAF, the datetime stamps are moot since the file is copied to the transient directory.

HTH,

Kevin

[Kurt_Bremser]

---

@Kevin_Viel wrote:

I wrote a macro that allows recursive functionality and adds FINFO data. 

---

You're not the first one 😉

[Kevin_Viel]

No doubt and I did not search, but it might make a suitable SUG paper if one does not already exist or does not have some functionalities.  I use it quite frequently.  I have read entire directories in minutes, which is fast given the structure and volume, but I am sure it could be faster.

[Kurt_Bremser]

I would have presented it this year in Washington (was an invited speaker), but COVID ...

Actually, a complete session about interacting with the base operating system, with and without XCMD.

I plan it for the next in-person Global Forum.

[LittlesasMaster]

You can also see an example from SAS Documentation , a dynamic way of doing this 

A Macro which gets you Sub Dir and File Names

%macro list_files(dir,ext);
%local filrf rc did memcnt name i;
%let rc=%sysfunc(filename(filrf,&dir));
%let did=%sysfunc(dopen(&filrf));

%if &did eq 0 %then %do; 
%put Directory &dir cannot be open or does not exist;
%return;
%end;

%do i = 1 %to %sysfunc(dnum(&did));

%let name=%qsysfunc(dread(&did,&i));

%if %qupcase(%qscan(&name,-1,.)) = %upcase(&ext) %then %do;
%put &dir\&name;
%end;
%else %if %qscan(&name,2,.) = %then %do; 
%list_files(&dir\&name,&ext)
%end;

%end;
%let rc=%sysfunc(dclose(&did));
%let rc=%sysfunc(filename(filrf));

%mend list_files;
%list_files(c:\temp,sas)

[ScottBass]

---

@LittlesasMaster wrote:

You can also see an example from SAS Documentation , a dynamic way of doing this 

A Macro which gets you Sub Dir and File Names

%macro list_files(dir,ext);
%local filrf rc did memcnt name i;
%let rc=%sysfunc(filename(filrf,&dir));
%let did=%sysfunc(dopen(&filrf));

%if &did eq 0 %then %do; 
%put Directory &dir cannot be open or does not exist;
%return;
%end;

%do i = 1 %to %sysfunc(dnum(&did));

%let name=%qsysfunc(dread(&did,&i));

%if %qupcase(%qscan(&name,-1,.)) = %upcase(&ext) %then %do;
%put &dir\&name;
%end;
%else %if %qscan(&name,2,.) = %then %do; 
%list_files(&dir\&name,&ext)
%end;

%end;
%let rc=%sysfunc(dclose(&did));
%let rc=%sysfunc(filename(filrf));

%mend list_files;
%list_files(c:\temp,sas)

---

This macro looping would be orders of magnitude slower than using corresponding functions in a data step, although it does have the benefit of being recursive.

[Peter_C]

I know
It's been done (to death😎)
But for such a simple request it's nice to provide the simple solution:
Read all csv files in one folder into one dataset.....
Data resultDS( compress= yes) ;
Length file filename col1-col100 $300 ;
INFILE "yourfolder/?*.csv" lrecl= 32000 dsd filename= file truncover ;
Input col1-col100 ;
Filename=file ;
Run ;

[Kevin_Viel]

That is a simple approach if the number of files is small and the file sizes are also small.  It is a good demonstration. For a single level, i.e. not recursive, Kurt snippet is a great approach.  Have you tried this with other file formats, life PDF?  I suspect it would still work.  I also wonder about using *.*?