topic Re: List of Files in Directory and Subdirectories with File Attributes in SAS Programming

List of Files in Directory and Subdirectories with File Attributes

suncawy — Tue, 03 Nov 2020 00:23:17 GMT

I'm trying to get a list of files in a directory and the subdirectories with file attributes, in particular the size of each file.

the files are listed in the main directory, but there a bunch of subdirectories with files. I've been using the following code, but

have not figured out how to get the size of the file, it only gives the directory and file. I've used the FINFO function, but doesn't give the size.

%macro list_files(dir,ext);
%local filrf rc did memcnt name i;
%let rc=%sysfunc(filename(filrf,&dir));
%let did=%sysfunc(dopen(&filrf));

%if &did eq 0 %then %do;
%put Directory &dir cannot be open or does not exist;
%return;
%end;

%do i = 1 %to %sysfunc(dnum(&did));

%let name=%qsysfunc(dread(&did,&i));

/* %if %qupcase(%qscan(&name,-1,.)) = %upcase(&ext) %then %do;*/
%if %qscan(&name,2,.) ne %then %do;
%put &dir/&name;

data _tmp;
length dir $512 name $100;
dir=symget("dir");
name=symget("name");

size=FINFO(OPEN(symget("name")), 'FILE SIZE (BYTES)');
run;

proc append base=want

data =_tmp;
run;quit;

%end;

%else %if %qscan(&name,2,.) = %then %do;
%list_files(&dir/&name,&ext)
%end;

%end;

%let rc=%sysfunc(dclose(&did));
%let rc=%sysfunc(filename(filrf));

%mend list_files;
%list_files(/gpfsFS2/sasdata/adhoc/pofin/nps/npsts/prod/cera/data,*)

Re: List of Files in Directory and Subdirectories with File Attributes

SASKiwi — Tue, 03 Nov 2020 01:38:26 GMT

Try the FOPEN function. The OPEN function is for SAS datasets.

Also I think you need to assign a FILEREF for the file you want to open then refer to that, not the actual file name:

rc=filename("curdirfl", "c:\temp\MytestFile.txt");
fid=fopen("curdirfl", "i") ;

Re: List of Files in Directory and Subdirectories with File Attributes

Kurt_Bremser — Tue, 03 Nov 2020 10:06:15 GMT

If you have XCMD enabled, use the proper external command (find with .ls) in a FILENAME PIPE and read the output.

If not, here's a very neat piece of code from @Tom that does it in a single data step:

data filelist;
  length dname filename $256 dir level 8 lastmod size 8;
  format lastmod datetime20.;
  input dname;
  retain filename ' ' level 0 dir 1;
cards4;
/folders/myfolders
;;;;

data filelist;
  modify filelist;
  rc1=filename('tmp',catx('/',dname,filename));
  rc2=dopen('tmp');
  dir = not not rc2;
  if not dir then do;
    fid=fopen('tmp','i',0,'b');
    lastmod=input(finfo(fid,foptname(fid,5)),NLDATM100.);
    size=input(finfo(fid,foptname(fid,6)),32.);
    fid=fclose(fid);
  end;
  else do;
    dname=catx('/',dname,filename);
    filename=' ';
    lastmod=input(dinfo(rc2,doptname(rc2,5)),NLDATM100.);
  end;
  replace;
  if dir;
  level=level+1;
  do i=1 to dnum(rc2);
    filename=dread(rc2,i);
    output;
  end;
  rc3=dclose(rc2);
run;

Re: List of Files in Directory and Subdirectories with File Attributes

sathya66 — Tue, 22 Dec 2020 10:03:58 GMT

Can we get the owner of the file/directory as well (ie:who created/modified)

Re: List of Files in Directory and Subdirectories with File Attributes

Kurt_Bremser — Tue, 22 Dec 2020 11:10:38 GMT

You can get the owner, but not the modifier if that is different from the owner (if a file is group-and/or world-writable). Modifiers are not recorded in any operating system I know of, unless you run some kind of auditing software; in which case, the modifier would still not be stored in file metadata, but in a separate database.

The owner is file info item #2 (at least on UNIX).

See this example code to get a feel for the construct:

data check;
length fref $8;
fid = filename(fref,"insert your file path here");
fid = fopen(fref);
do i = 1 to foptnum(fid);
  opt = foptname(fid,i);
  value = finfo(fid,opt);
  output;
end;
fid = fclose(fid);
fid = filename(fref);
run;

This will show you all available information items for your platform, and the values for the file selected.

Re: List of Files in Directory and Subdirectories with File Attributes

DarrylLawrence — Fri, 08 Oct 2021 21:46:15 GMT

This is a great solution.

Do you know how I can add the owner (person who created the files) to the report?

Re: List of Files in Directory and Subdirectories with File Attributes

Kurt_Bremser — Sat, 09 Oct 2021 05:52:53 GMT

Please show your code.

Re: List of Files in Directory and Subdirectories with File Attributes

DarrylLawrence — Sat, 09 Oct 2021 07:40:10 GMT

This is the code I am usign - copied from this blog

I need to add the owner of the files - so we can send out reports to the owners to delete obsolete files.

data filelist;
length dname filename $256 dir level 8 lastmod size 8 user $32 ;
format lastmod datetime20.;
input dname;
retain filename ' ' level 0 dir 1;
cards4;
/sas/analytics/sasdata/users
/sas/analytics/projects
;;;;

data filelist;
modify filelist;
rc1=filename('tmp',catx('/',dname,filename));

rc2=dopen('tmp');
dir = not not rc2;
if not dir then do;
fid=fopen('tmp','i',0,'b');
lastmod=input(finfo(fid,foptname(fid,5)),NLDATM100.);
size=input(finfo(fid,foptname(fid,6)),32.);
fid=fclose(fid);
end;
else do;
dname=catx('/',dname,filename);
filename=' ';
lastmod=input(dinfo(rc2,doptname(rc2,5)),NLDATM100.);
end;
replace;
if dir;
level=level+1;
do i=1 to dnum(rc2);
filename=dread(rc2,i);
output;
end;
rc3=dclose(rc2);
run;

Re: List of Files in Directory and Subdirectories with File Attributes

Kurt_Bremser — Sat, 09 Oct 2021 08:45:11 GMT

Add it here:

if not dir then do;
  fid=fopen('tmp','i',0,'b');
  owner = finfo(fid,foptname(fid,2));
  lastmod=input(finfo(fid,foptname(fid,5)),NLDATM100.);
  size=input(finfo(fid,foptname(fid,6)),32.);
  fid=fclose(fid);
end;

Set a proper length for the variable.

Re: List of Files in Directory and Subdirectories with File Attributes

DarrylLawrence — Sat, 09 Oct 2021 19:44:17 GMT

Thank you, it worked!

Re: List of Files in Directory and Subdirectories with File Attributes

hellohere — Sun, 20 Feb 2022 11:59:53 GMT

How to use it?!

14815 data check;
14816 length fref $8;
14817 fid = filename(fref,"d:\");
14818 fid = fopen(fref);
14819 do i = 1 to foptnum(fid);
14820 opt = foptname(fid,i);
14821 value = finfo(fid,opt);
14822 output;
14823 end;
14824 fid = fclose(fid);
14825 fid = filename(fref);
14826 run;

Re: List of Files in Directory and Subdirectories with File Attributes

torvyle — Thu, 06 Oct 2022 20:08:58 GMT

How does one use this code? Is it only replacing the "/folders/myfolders" with the desired directory?

Thanks

Re: List of Files in Directory and Subdirectories with File Attributes

Kurt_Bremser — Fri, 07 Oct 2022 06:20:02 GMT

@torvyle wrote:

How does one use this code? Is it only replacing the "/folders/myfolders" with the desired directory?

Thanks

You insert the root directories of the trees you want to search in in the DATALINES block of the first data step (replace /folders/myfolders, as that was the data directory used in the no longer available SAS University Edition).