topic Re: how to exclude a directory with a find command within a pipe command%0D%0A in SAS Enterprise Guide

how to exclude a directory with a find command within a pipe command%0D%0A

alepage — Tue, 08 Jul 2025 13:32:20 GMT

How to exclude the 3 subdirectories

dircf, diraf, dirdrl

from the find command

%let cleandspath=/dwh_actuariat/sasdata/Data_Retention/;
%let dircf=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_CleanFiles;
%let diraf=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_Anonymized;
%let dirdrl=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_list;

filename oscmd pipe "find &cleandspath -type f -name '*_prm*'  ";
     

data test;
length text $1000.;
	infile oscmd;
    input;
   text= _infile_;
run;

Re: how to exclude a directory with a find command within a pipe command%0D%0A

yabwon — Tue, 08 Jul 2025 13:47:34 GMT

how about:

%let cleandspath=/dwh_actuariat/sasdata/Data_Retention/;
%let dircf=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_CleanFiles;
%let diraf=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_Anonymized;
%let dirdrl=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_list;

filename oscmd pipe "find &cleandspath -type f -name '*_prm*'  ";

data test;
  length text $1000.;
	infile oscmd;
  input;
  text= _infile_;
  if NOT (text in: ("&dircf." "&diraf." "&dirdrl."));
run;

for start?

Or maybe even with less macrovariables:

%let cleandspath=/dwh_actuariat/sasdata/Data_Retention/;

filename oscmd pipe "find &cleandspath -type f -name '*_prm*'  ";

data test;
  length text $1000.;
	infile oscmd;
  input;
  text = _infile_;
  if NOT (text in: ("&cleandspath.Data_Retention_CleanFiles" 
                    "&cleandspath.Data_Retention_Anonymized" 
                    "&cleandspath.Data_Retention_list"
         ));
run;

Bart

Re: how to exclude a directory with a find command within a pipe command%0D%0A

alepage — Tue, 08 Jul 2025 20:20:14 GMT

Hello,

I can not filter the information into the test dataset. I need to find the good unix command to do what I want, because, I am going to delete those file after.

I am looking for something like below but it is like either I am able to filter the directories but not search for file containing _prm

filename oscmd pipe  "find /dwh_actuariat/sasdata/Data_Retention/
                     -type f -name '*_prm*'
                     -path '/dwh_actuariat/sasdata/Data_Retention/Data_Retention_CleanFiles' -prune -o -print 
                     -path '/dwh_actuariat/sasdata/Data_Retention/Data_Retention_Anonymized' -prune -o -print
                     -path '/dwh_actuariat/sasdata/Data_Retention/Data_Retention_list' -prune -o -print 
                     -path '/dwh_actuariat/sasdata/Data_Retention/temp' -prune -o -print
                     -path '/dwh_actuariat/sasdata/Data_Retention/Audit' -prune -o -print
                     -path '/dwh_actuariat/sasdata/Data_Retention/Validation' -prune -o -print 
                     -path '/dwh_actuariat/sasdata/Data_Retention/sas2001b' -prune -o -print ";

data test;
length text $1000.;
	infile oscmd;
    input;
   text= _infile_;
run;

Re: how to exclude a directory with a find command within a pipe command%0D%0A

Kurt_Bremser — Tue, 08 Jul 2025 20:37:30 GMT

Pipe the output from find into grep with the -v option to exclude lines with a specific pattern.

Re: how to exclude a directory with a find command within a pipe command%0D%0A

alepage — Wed, 09 Jul 2025 02:26:20 GMT

the option -type -name ‘*_prm*’ is working but not with the -path option

or the path option is working but not the -type option

please provide an example with the grep v option

-type f -name '*_prm*'
                     -path '/dwh_actuariat/sasdata/Data_Retention/Data_Retention_CleanFiles' -prune -o -print

Re: how to exclude a directory with a find command within a pipe command%0D%0A

Tom — Wed, 09 Jul 2025 02:54:42 GMT

Why not just use SAS code instead? https://github.com/sasutils/macros/blob/master/dirtree.sas

You are asking for help with a UNIX command on a SAS forum. Someone gave you an answer for how to filter the output of the UNIX command using normal SAS code. Why isn't that good enough?

Hint on using the find command. Look at the many expressions you can use with the find command. In particular the -prune option.

Here is an example I used in the past to ignore .snapshot directories that were created automatically on some network attach disk drives.

find ....... -name \.snapshot -prune ....

Re: how to exclude a directory with a find command within a pipe command%0D%0A

Patrick — Wed, 09 Jul 2025 05:59:39 GMT

You could just use the ! to exclude paths.

Using the following prompts with Copilot:

Amend to exclude paths /a/b/c and /a/b/x from find /a -type f -name '*_prm*'

I've got this answer:

find /a -type f -name '*_prm*' ! -path "/a/b/c/*" ! -path "/a/b/x/*"

Applied to the code you shared this could look like:

filename oscmd pipe "find &cleandspath -type f -name '*_prm*' ! -path '&dircf/*' ! -path '&diraf/*' ! -path '&dirdrl/*' ";

I've tested the syntax and it works as desired. It will exclude all content under the paths that are listed as excluded (the NOT exclamation mark).

Re: how to exclude a directory with a find command within a pipe command%0D%0A

Kurt_Bremser — Wed, 09 Jul 2025 09:17:13 GMT

I created some directories in my personal folder on my Macbook:

> find test
test
test/uvw
test/abc
test/def
test/xyz

Then I ran this command:

find test|grep -v xyz|grep -v uvw

and got this result:

test
test/abc
test/def

Re: how to exclude a directory with a find command within a pipe command%0D%0A

yabwon — Wed, 09 Jul 2025 09:30:21 GMT

Why can't you create a temporary dataset?

Having such a data set with list of files you want to delete seems like a perfect fit for running FDELETE() function:

data _null_;
    SET test;
    fname="tempfile";
    rc=filename(fname, strip(text));
    if rc = 0 and fexist(fname) then
       rc=fdelete(fname);
    rc=filename(fname);
run;

Or in fact you could do it "on the fly":

%let cleandspath=/dwh_actuariat/sasdata/Data_Retention/;

filename oscmd pipe "find &cleandspath -type f -name '*_prm*'  ";

data _null_;
  length text $1000.;
	infile oscmd;
  input;
  text = _infile_;
  if NOT (text in: ("&cleandspath.Data_Retention_CleanFiles" 
                    "&cleandspath.Data_Retention_Anonymized" 
                    "&cleandspath.Data_Retention_list"
         ));


    fname="tempfile";
    rc=filename(fname, strip(text));
    if rc = 0 and fexist(fname) then
       rc=fdelete(fname);
    rc=filename(fname);

run;

Bart

Re: how to exclude a directory with a find command within a pipe command%0D%0A

alepage — Wed, 09 Jul 2025 12:45:06 GMT

This script works perferctly. However, I would like to understand the signification of the exclamation mark

Re: how to exclude a directory with a find command within a pipe command%0D%0A

Kurt_Bremser — Wed, 09 Jul 2025 14:35:33 GMT

Read for yourself: find

It's in the OPERATORS section.