Delete files if more then 5 in directory

[Citrine10]

Hi there

I have code that checks and lists files in a directory. I would like to do a check to see if there are less than 5 files it should continue else if there are more than 5 files it should delete the extra files and exit. 

How do I implement that?

[japelin]

Is the file you are referring to a SAS data set or a file on the OS?
Also, is the code SAS? Is it a shell script (command, batch, PowerShell, etc.)?
What exactly are the extra files? Are they sorted by date, by name, or by file size?
Does it terminate when it's done checking one target directory? Or does it check recursively?

The question is too vague, so please organize it a little better.

[Citrine10]

there are .log files and .xlsx files in the directory. It is a file directory. 

Yes the code is SAS.

Files are not sorted by anything. I dont need any sorting etc. 

[japelin]

I'm not sure what kind of code you have, but you can use
1. save the list of files into a dataset with their file paths.
2. If there are more than 5 obs, delete them based on the file path.

For example:

/* start create test files */
%Macro MTestFileCreate;
  %do i=1 %to 10;
  filename f "C:\Temp\extrafile&i..log";
  data _null_;
    file f;
    put 'sample';
  run;
  %end;
%Mend;
%MTestFileCreate;
data filelist;
  length path $200;
  do i=1 to 10;
    path=cats("C:\Temp\extrafile",i,".log");
    output;
  end;
  drop i;
run;
/* end create test files */
data _null_;
  set filelist;
  rc=filename('del',path);
  if _n_>5 then rc=fdelete('del');
run;

[Citrine10]

Hi

thank you for the above. I tried the code but it does not work. it does not delete the files 😞

Alternatively, I tried getting only 5 records from the list of files and thereafter decided to remove the remaining files however I am struggling with the delete and the if statement. 

[japelin]

How are you getting the file list in your code?
If you show the code, I can advise you.

[Citrine10]

I have a macro that lists all the files and I call it 

%macro list_files(dir,ext);
	%local filrf rc did memcnt name i;
	%let rc=%sysfunc(filename(filrf,&dir));
	%let did=%sysfunc(dopen(&filrf));
	%if %sysfunc(exist(tables)) = 1 %then
		%do;
			proc sql;
				drop table tables;
			quit;
		%end;
	%if &did eq 0 %then
		%do;
			%put Directory &dir cannot be open or does not exist;
			%return;
		%end;
	%do i = 1 %to %sysfunc(dnum(&did));
		%let name=%qsysfunc(dread(&did,&i));
		%if %qupcase(%qscan(&name,-1,.)) = %upcase(&ext) %then
			%do;
				%put &dir\&name;
				data _tmp;
					length dir $512 name $100 ext $100;
					dir=symget("dir");
					name=symget("name");
					ext=upcase("&ext.");
				run;
				proc append base=tables data=_tmp;
				run;
				quit;
			%end;
		%else %if %qscan(&name,2,.) = %then
			%do;
			%end;
	%end;
	%let rc=%sysfunc(dclose(&did));
	%let rc=%sysfunc(filename(filrf));
	proc sql;
		drop table _TMP;
	quit;
%mend list_files;

------------------------------------------------------------------

I call the macro :

include "/C:/list_files.sas";

/*file paths*/
%let path_in= /myfiles;

/*List of files*/
%list_files("&path_in",xlsx)

proc sql;
create table allfiles as
select * from tables;
quit;

proc sql;
create table FileCount as
select count(name) into: Filecount from tables;
quit;

Now this is where I need it to do a check to see if there are more then 5 files in the directory then delete them and exit, if there are less than 5 files, then continue

[Quentin]

If the directory has more than 5 files, and you want to delete the "extra" files, how do you determine which files to delete?

[Citrine10]

Yes correct. it can be any of the files

[Quentin]

I suggest reviewing:

https://blogs.sas.com/content/sgf/2018/07/17/delete-sas-logs-admin/

[Citrine10]

Thank you Quentin, that is very useful. Do you know where I can put a check to see if there are more than 5 files then delete them and exit else continue?

[Quentin]

I would approach this in chunks.

Are you happy with your %listfiles macro?  If so, then you've got a dataset that lists all the files in the directory.

I would next write a %deletefile() macro which allows you to pass it a list of files to be deleted.  I would use the blog post as a guide for approaches to writing that macro.

Once you have %deletefiles done, then you can run your code like you have it to get a dataset of files, and you can check that dataset to see if it has more then 5 files, and if it does, you can use %deletefiles to delete the extras.

Or since you said you don't care which files are deleted, maybe you could do something quick like (untested pseudo-code):

data _null_;
    set tables; *your dataset listing all the files found in the directory;
    if _n_>5;
    rc=filename(fname, cats(dir,name,ext)); 
    rc=fdelete(fname);
run;

[Citrine10]

thank you Quentin, you are really helping me get there 🙂

i have a macro that deletes a file from a directory and when testing with one file, it works however when I try to pass it through the code it doesnt work. It only shows me the remaining list which should be deleted yet it doesnt delete

%macro delfile(file);
%if %sysfunc(fileexist(&file)) ge 1 %then %do;
   %let rc=%sysfunc(filename(temp,&file));
   %let rc=%sysfunc(fdelete(&temp));
%end; 
%else %put The file &file does not exist;
%mend check;


/*lists all files to delete*/
data list;
set allfiles; *your dataset listing all the files found in the directory;
if _n_>5;
rc=filename(fname, cats(dir,name,ext)); 
rc=fdelete(fname);
%delFile(&fullpath);
run;
 

[japelin]

how about this before use %delfile.

I think filename and fdelete function in datastep needs quoted fileref.

data list;
set allfiles; *your dataset listing all the files found in the directory;
if _n_>5;
rc=filename('fname', cats(dir,name,ext)); 
rc=fdelete('fname');

run;

[Reeza · Accepted solution]

How to delete files from a list of files:

*list of files to take an action on;
data test;
infile datalines;
input file_nm $50.;
datalines;
/home/users/temp1.sas7bndx
/home/users/temp2.sas7bndx
/home/users/temp3.sas7bndx
/home/users/temp4.sas7bndx
/home/users/temp5.sas7bndx
/home/users/temp6.sas7bndx
;
run;

*action to do per file - delete file in this case;
%macro check(del_file);
%if %sysfunc(fileexist(&del_file)) ge 1 %then %do;
%let rc=%sysfunc(filename(temp, &del_file));
%let rc=%sysfunc(fdelete(&temp));
%end;
%else %put The file &file does not exist;
%mend check;

*call macro for each file;
data _null_;
set test;

str = catt('%check(', file_nm, ');');

call execute(str);

run;