topic Re: Delete files if more then 5 in directory in SAS Procedures

Delete files if more then 5 in directory

Citrine10 — Mon, 24 Jan 2022 09:18:38 GMT

Hi there

I have code that checks and lists files in a directory. I would like to do a check to see if there are less than 5 files it should continue else if there are more than 5 files it should delete the extra files and exit.

How do I implement that?

Re: Delete files if more then 5 in directory

japelin — Mon, 24 Jan 2022 09:43:01 GMT

Is the file you are referring to a SAS data set or a file on the OS?
Also, is the code SAS? Is it a shell script (command, batch, PowerShell, etc.)?
What exactly are the extra files? Are they sorted by date, by name, or by file size?
Does it terminate when it's done checking one target directory? Or does it check recursively?

The question is too vague, so please organize it a little better.

Re: Delete files if more then 5 in directory

Citrine10 — Mon, 24 Jan 2022 09:45:07 GMT

there are .log files and .xlsx files in the directory. It is a file directory.

Yes the code is SAS.

Files are not sorted by anything. I dont need any sorting etc.

Re: Delete files if more then 5 in directory

japelin — Mon, 24 Jan 2022 10:13:01 GMT

I'm not sure what kind of code you have, but you can use
1. save the list of files into a dataset with their file paths.
2. If there are more than 5 obs, delete them based on the file path.

For example:

/* start create test files */
%Macro MTestFileCreate;
  %do i=1 %to 10;
  filename f "C:\Temp\extrafile&i..log";
  data _null_;
    file f;
    put 'sample';
  run;
  %end;
%Mend;
%MTestFileCreate;
data filelist;
  length path $200;
  do i=1 to 10;
    path=cats("C:\Temp\extrafile",i,".log");
    output;
  end;
  drop i;
run;
/* end create test files */
data _null_;
  set filelist;
  rc=filename('del',path);
  if _n_>5 then rc=fdelete('del');
run;

Re: Delete files if more then 5 in directory

Citrine10 — Mon, 24 Jan 2022 10:29:19 GMT

thank you for the above. I tried the code but it does not work. it does not delete the files 😞

Alternatively, I tried getting only 5 records from the list of files and thereafter decided to remove the remaining files however I am struggling with the delete and the if statement.

Re: Delete files if more then 5 in directory

japelin — Mon, 24 Jan 2022 10:39:36 GMT

How are you getting the file list in your code?
If you show the code, I can advise you.

Re: Delete files if more then 5 in directory

Citrine10 — Mon, 24 Jan 2022 10:56:22 GMT

I have a macro that lists all the files and I call it

%macro list_files(dir,ext);
	%local filrf rc did memcnt name i;
	%let rc=%sysfunc(filename(filrf,&dir));
	%let did=%sysfunc(dopen(&filrf));
	%if %sysfunc(exist(tables)) = 1 %then
		%do;
			proc sql;
				drop table tables;
			quit;
		%end;
	%if &did eq 0 %then
		%do;
			%put Directory &dir cannot be open or does not exist;
			%return;
		%end;
	%do i = 1 %to %sysfunc(dnum(&did));
		%let name=%qsysfunc(dread(&did,&i));
		%if %qupcase(%qscan(&name,-1,.)) = %upcase(&ext) %then
			%do;
				%put &dir\&name;
				data _tmp;
					length dir $512 name $100 ext $100;
					dir=symget("dir");
					name=symget("name");
					ext=upcase("&ext.");
				run;
				proc append base=tables data=_tmp;
				run;
				quit;
			%end;
		%else %if %qscan(&name,2,.) = %then
			%do;
			%end;
	%end;
	%let rc=%sysfunc(dclose(&did));
	%let rc=%sysfunc(filename(filrf));
	proc sql;
		drop table _TMP;
	quit;
%mend list_files;

------------------------------------------------------------------

I call the macro :

include "/C:/list_files.sas";

/*file paths*/
%let path_in= /myfiles;

/*List of files*/
%list_files("&path_in",xlsx)

proc sql;
create table allfiles as
select * from tables;
quit;

proc sql;
create table FileCount as
select count(name) into: Filecount from tables;
quit;

Now this is where I need it to do a check to see if there are more then 5 files in the directory then delete them and exit, if there are less than 5 files, then continue

Re: Delete files if more then 5 in directory

Quentin — Mon, 24 Jan 2022 12:34:54 GMT

If the directory has more than 5 files, and you want to delete the "extra" files, how do you determine which files to delete?

Re: Delete files if more then 5 in directory

Citrine10 — Mon, 24 Jan 2022 12:37:54 GMT

Yes correct. it can be any of the files

Re: Delete files if more then 5 in directory

Quentin — Mon, 24 Jan 2022 13:26:45 GMT

I suggest reviewing:

https://blogs.sas.com/content/sgf/2018/07/17/delete-sas-logs-admin/

Re: Delete files if more then 5 in directory

Citrine10 — Mon, 24 Jan 2022 13:46:57 GMT

Thank you Quentin, that is very useful. Do you know where I can put a check to see if there are more than 5 files then delete them and exit else continue?

Re: Delete files if more then 5 in directory

Quentin — Mon, 24 Jan 2022 14:19:41 GMT

I would approach this in chunks.

Are you happy with your %listfiles macro? If so, then you've got a dataset that lists all the files in the directory.

I would next write a %deletefile() macro which allows you to pass it a list of files to be deleted. I would use the blog post as a guide for approaches to writing that macro.

Once you have %deletefiles done, then you can run your code like you have it to get a dataset of files, and you can check that dataset to see if it has more then 5 files, and if it does, you can use %deletefiles to delete the extras.

Or since you said you don't care which files are deleted, maybe you could do something quick like (untested pseudo-code):

data _null_;
    set tables; *your dataset listing all the files found in the directory;
    if _n_>5;
    rc=filename(fname, cats(dir,name,ext)); 
    rc=fdelete(fname);
run;

Re: Delete files if more then 5 in directory

Citrine10 — Mon, 24 Jan 2022 14:35:12 GMT

thank you Quentin, you are really helping me get there 🙂

i have a macro that deletes a file from a directory and when testing with one file, it works however when I try to pass it through the code it doesnt work. It only shows me the remaining list which should be deleted yet it doesnt delete

%macro delfile(file);
%if %sysfunc(fileexist(&file)) ge 1 %then %do;
   %let rc=%sysfunc(filename(temp,&file));
   %let rc=%sysfunc(fdelete(&temp));
%end; 
%else %put The file &file does not exist;
%mend check;


/*lists all files to delete*/
data list;
set allfiles; *your dataset listing all the files found in the directory;
if _n_>5;
rc=filename(fname, cats(dir,name,ext)); 
rc=fdelete(fname);
%delFile(&fullpath);
run;

Re: Delete files if more then 5 in directory

japelin — Mon, 24 Jan 2022 15:18:48 GMT

how about this before use %delfile.

I think filename and fdelete function in datastep needs quoted fileref.

data list;
set allfiles; *your dataset listing all the files found in the directory;
if _n_>5;
rc=filename('fname', cats(dir,name,ext)); 
rc=fdelete('fname');

run;

Re: Delete files if more then 5 in directory

Reeza — Mon, 24 Jan 2022 17:07:16 GMT

How to delete files from a list of files:

*list of files to take an action on;
data test;
infile datalines;
input file_nm $50.;
datalines;
/home/users/temp1.sas7bndx
/home/users/temp2.sas7bndx
/home/users/temp3.sas7bndx
/home/users/temp4.sas7bndx
/home/users/temp5.sas7bndx
/home/users/temp6.sas7bndx
;
run;

*action to do per file - delete file in this case;
%macro check(del_file);
%if %sysfunc(fileexist(&del_file)) ge 1 %then %do;
%let rc=%sysfunc(filename(temp, &del_file));
%let rc=%sysfunc(fdelete(&temp));
%end;
%else %put The file &file does not exist;
%mend check;

*call macro for each file;
data _null_;
set test;

str = catt('%check(', file_nm, ');');

call execute(str);

run;

Re: Delete files if more then 5 in directory

Citrine10 — Wed, 26 Jan 2022 09:01:54 GMT

I figured it is not deleting because the rc variable is another format and I'm guessing it is unable to identify it:

Re: Delete files if more then 5 in directory

Citrine10 — Wed, 26 Jan 2022 09:01:28 GMT

I changed it now to reference the variable 'fullpath' but it still doesnt delete the files:

Re: Delete files if more then 5 in directory

japelin — Tue, 25 Jan 2022 07:04:55 GMT

Is it possible to specify one of those full paths and delete it with the following code?

filename fname 'specify fullpath';
data _null_;
  rc=fdelete('fname');
  put rc=;
run;

If rc is 0, then the file has been deleted.
Could you try setting the return value of the filename function to "rc0" instead of "rc" and check if the filename function is being handled correctly?