topic List all files in folder and see information last modify date in SAS Programming

List all files in folder and see information last modify date

Ronein — Thu, 14 Mar 2024 08:28:11 GMT

Hello

I want to perform the following task:

1-List all files in specific folder

A-Get the files names without the end (It means that require to see the field fname without the end "sas7bdat")

B-Get field of last modified date (Field Last Modified). Currently this field is char and the request is to be date time

2-Delete all files that last modified date is before 31DEC2022

/**********SAS -list all files in specific folder with modified date information********************/
/**********SAS -list all files in specific folder with modified date information********************/
/**********SAS -list all files in specific folder with modified date information********************/
%let filesLocation=/usr/local/SAS/SASUsers/LabRet/Credit_Scoring_Users/R_R;
/* get full list of files from &filesLocation. (including all subdirectories) */

/* just to be sure the file "files"  is "empty" */
data files_to_get;  
run;
data files_to_get;
  length root dname $ 2048 filename $ 256 dir level 8;
  root = "&filesLocation.";
  retain filename dname ' ' level 0 dir 1;
  label 
    filename = "file"
    dname = "folder"
    ;
run;
data files_to_get;
  modify files_to_get;
  rc1=filename('tmp',catx('/',root,dname,filename));
  rc2=dopen('tmp');
  dir = 1 & rc2;
  if dir then 
    do;
      dname=catx('/',dname,filename);
      filename=' ';
    end;
  replace;
  if dir;
  level=level+1;
  do i=1 to dnum(rc2);
    filename=dread(rc2,i);
    output;
  end;
  rc3=dclose(rc2); 
run;
/* get files info in "Long" format */
data files_to_get_dates;
  set files_to_get(where=(dir=0));
  fname="tempfile";
  rc=filename(fname, catx('/',root,dname,filename));
  if rc = 0 and fexist(fname) then
     do;
        fid=fopen(fname);
        infonum=foptnum(fid);
           do i=1 to infonum;
              infoname=foptname(fid, i);
              infoval=finfo(fid, infoname);
              output;
           end;
        fid=fclose(fid);
     end;
  rc=filename(fname);
  drop dir--i;
run;
/* transforms files info into "Wide" format */
proc transpose data = files_to_get_dates(rename=(filename=fn)) out=files_to_get_dates_T(drop =_name_);
by root dname fn notsorted;
id infoname;
var infoval;
run;

Re: List all files in folder and see information last modify date

yabwon — Thu, 14 Mar 2024 09:05:09 GMT

I guess you get the inspiration here:

https://communities.sas.com/t5/SAS-Programming/List-all-files-in-a-directory-and-dates-they-were-last-modified/m-p/867418#M342591

? Right?

1) Before executing the code(!!!) you showed run this:

options validvarname=v7;

to ensure variables names in transposed dataset will be "the SAS way"! (especially if you are running EG).

2) Run the code you showed.

3) Having the transposed table not empty, run the following:

data selected_files;
  set files_to_get_dates_T;
  where input(Last_Modified,date9.) < '31DEC2022'd
    AND
    upcase(scan(fn,-1,".")) =: 'SAS7BDAT'
  ;
run;

data _null_;
  set selected_files;
  length f $ 8;
  rc = filename(f,Filename);
  put f= rc=;
  if 0=rc then
    do;
      rcdel=fdelete(f);
      rcdelTxt=sysmsg();
      if rcdel then put rcdelTxt /;
               else put "File: " Filename "deleted syccessfully";
    end;
run;

The first data step creates "final list of files to delete", check it if there are no files you eventually want to keep!

The second one deleted pointed files.

If you expect to have indexes on your files I suggest to modify WHERE condition to:

upcase(scan(fn,-1,".")) in: ('SAS7BDAT', 'SAS7BIDX')

Bart

Re: List all files in folder and see information last modify date

LinusH — Thu, 14 Mar 2024 09:11:34 GMT

There's a Swedish saying - "crossing the river to get water".

I thin the information you are looking for is contained in SAS dictionary.tables (moddate):

proc sql;
	create table lastmod as
	select *
	from dictionary.tables
	where libname = "&YOURLIBREF";
quit;

Re: List all files in folder and see information last modify date

yabwon — Thu, 14 Mar 2024 09:19:19 GMT

BTW.

Instead copy+pasting the code (and increasing chance to "mes something up") I suggest you to try the BasePlus package and it's %direAndFiles() macro which does the job of looking up files in directories.

%dirsAndFiles(C:\SAS_WORK\
,ODS=work.result2
,keepDirs=0
,details=1
,fileExt=sas7bdat
)

Bart

Re: List all files in folder and see information last modify date

Patrick — Thu, 14 Mar 2024 10:44:06 GMT

If you can issue OS commands (=option XCMD set) then below is likely the least code option.

/* list *.sas7b* files older than 12Dec2022 */
filename listing pipe 'find /usr/local/SAS/SASUsers/LabRet/Credit_Scoring_Users/R_R -type f -name "*.sas7b*" ! -newermt "12/31/2022" -printf "%h|%f|%CF\n"';
data want;
  infile listing truncover dlm='|';
  input path:$300. name$50. modification_date:yymmdd10.;  format modification_date date9.;
run;

/* delete *.sas7b* files older than 12Dec2022 */
filename deletion pipe 'find /usr/local/SAS/SASUsers/LabRet/Credit_Scoring_Users/R_R -type f -name "*.sas7b*" ! -newermt "12/31/2022" -delete';
data _null_;
  infile deletion truncover dlm='|';
  input;
  put _infile_;
run;