topic Re: list of files and modified date in a dateset in SAS Programming

list of files and modified date in a dateset

sathya66 — Tue, 07 Jul 2020 11:11:18 GMT

All,

trying to get list of files and modified date in a dataset . list of files is working but not modified date.

filename myDir "/work" ;

data test (keep=filename modified);
	did=dopen("myDir") ;
	filecount=dnum(did) ;
	
	do i=1 to filecount ;
	fid = fopen('myDir');
		filename=dread(did,i) ;
		modified=finfo(fid,"Last Modified");
		put filename= ;
		output ;
			fid=fclose(fid);
	end ;
	rc=dclose(did) ;

run ;

Re: list of files and modified date in a dateset

Jagadishkatam — Tue, 07 Jul 2020 11:25:01 GMT

please try dictionary.tables as below from libname , you will see column memname with list and modate with modified date

libname adam '~path of datasets';

proc sql;
create table want as select * from dictionary.tables where libname='ADAM' ;
quit;

Re: list of files and modified date in a dateset

Jagadishkatam — Tue, 07 Jul 2020 11:26:52 GMT

Alternatively via data step

data want;
set sashelp.vtable;
where  libname='ADAM' ;
keep modate memname;
run;

Re: list of files and modified date in a dateset

sathya66 — Tue, 07 Jul 2020 11:30:11 GMT

Hi Thanks for this.
but I am looking for list of all files(ex: text,csv,datset,etc ) and their modified date in a directory to read them into a dateset.

Re: list of files and modified date in a dateset

Jagadishkatam — Tue, 07 Jul 2020 11:40:41 GMT

please try this code as well

data test (keep=filename modified);
length fref $8;
     rc=filename(fref,"/work");
 if rc = 0 then do;
 did = dopen(fref);
 rc = filename(fref);
 end;
dnum = dnum(did);
do i = 1 to dnum;
filename = dread(did, i);/* If this entry is a file, then output. */
fid = mopen(did, filename);
modified=finfo(fid,'Last Modified');  
if fid > 0 then output;
end; 
rc = dclose(did);
run;

Re: list of files and modified date in a dateset

Jagadishkatam — Tue, 07 Jul 2020 11:41:35 GMT

I just sent another code, which will provide the output you are expecting

Re: list of files and modified date in a dateset

Kurt_Bremser — Tue, 07 Jul 2020 11:55:58 GMT

Make your code well-behaving by cleaning up all references it creates:

data test (keep=fname modified);
length fref $8;
if filename(fref,"/work") = 0
then do;
  did = dopen(fref);
  if did ne 0
  then do;
    dnum = dnum(did);
    do i = 1 to dnum;
      fname = dread(did, i);
      fid = mopen(did, fname);
      if fid ne 0
      then do;
        modified = finfo(fid,'Last Modified');  
        output;
        fid = dclose(fid);
      end;
    end;
    did = dclose(did);
  end;
  rc = filename(fref);
end;
run;

as otherwise it will have "memory leaks" by accruing file references and file handles.

Re: list of files and modified date in a dateset

s_lassen — Tue, 07 Jul 2020 13:16:58 GMT

Actually, I do not think "cleaning up" is strictly necessary in a data step. It seems that SAS takes case of cleaning up when the data step ends. For instance:

1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 72         
 73         filename x "%sysfunc(pathname(WORK))";
 74         
 75         data _null_;
 76           did=dopen('x');
 77           put _ALL_;
 78         run;
 
 did=1 _ERROR_=0 _N_=1
 NOTE: DATA statement used (Total process time):
       real time           0.00 seconds
       cpu time            0.00 seconds
       
 
 79         
 80         data _null_;
 81           did=dopen('x');
 82           put _ALL_;
 83         run;
 
 did=1 _ERROR_=0 _N_=1

So what we see is that the directory handle number (DID=1) get reused in the second step, because it becomes deallocated as the data step ends.

In macros, om the other hand, and probably also in SAS/SCL programming, cleanup is not automatic, and is good practice.

Re: list of files and modified date in a dateset

Kurt_Bremser — Tue, 07 Jul 2020 13:37:57 GMT

Yes, but ...

On UNIX systems it is standard to set the number of available file handles for a user to something like 2000 per default, and that can quickly be used in a step that works through a "industrially populated" directory. Or once one starts to work with code that reads multiple directories in one step.

Re: list of files and modified date in a dateset

s_lassen — Tue, 07 Jul 2020 19:52:57 GMT

Yes, of course! You can hit the ceiling in a single datastep if you do not close the handles. Thanks!