how to exclude a directory with a find command within a pipe command%0D%0A

[alepage]

How to exclude the 3 subdirectories

dircf,  diraf, dirdrl

from the find command 

%let cleandspath=/dwh_actuariat/sasdata/Data_Retention/;
%let dircf=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_CleanFiles;
%let diraf=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_Anonymized;
%let dirdrl=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_list;

filename oscmd pipe "find &cleandspath -type f -name '*_prm*'  ";
     

data test;
length text $1000.;
	infile oscmd;
    input;
   text= _infile_;
run;

[yabwon]

how about:

%let cleandspath=/dwh_actuariat/sasdata/Data_Retention/;
%let dircf=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_CleanFiles;
%let diraf=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_Anonymized;
%let dirdrl=/dwh_actuariat/sasdata/Data_Retention/Data_Retention_list;

filename oscmd pipe "find &cleandspath -type f -name '*_prm*'  ";

data test;
  length text $1000.;
	infile oscmd;
  input;
  text= _infile_;
  if NOT (text in: ("&dircf." "&diraf." "&dirdrl."));
run;

for start?

Or maybe even with less macrovariables:

%let cleandspath=/dwh_actuariat/sasdata/Data_Retention/;

filename oscmd pipe "find &cleandspath -type f -name '*_prm*'  ";

data test;
  length text $1000.;
	infile oscmd;
  input;
  text = _infile_;
  if NOT (text in: ("&cleandspath.Data_Retention_CleanFiles" 
                    "&cleandspath.Data_Retention_Anonymized" 
                    "&cleandspath.Data_Retention_list"
         ));
run;

?

Bart

[alepage]

Hello,

I can not filter the information into the test dataset.  I need to find the good unix command to do what I want, because, I am going to delete those file after.

I am looking for something like below but it is like either I am able to filter the directories but not search for file containing _prm

filename oscmd pipe  "find /dwh_actuariat/sasdata/Data_Retention/
                     -type f -name '*_prm*'
                     -path '/dwh_actuariat/sasdata/Data_Retention/Data_Retention_CleanFiles' -prune -o -print 
                     -path '/dwh_actuariat/sasdata/Data_Retention/Data_Retention_Anonymized' -prune -o -print
                     -path '/dwh_actuariat/sasdata/Data_Retention/Data_Retention_list' -prune -o -print 
                     -path '/dwh_actuariat/sasdata/Data_Retention/temp' -prune -o -print
                     -path '/dwh_actuariat/sasdata/Data_Retention/Audit' -prune -o -print
                     -path '/dwh_actuariat/sasdata/Data_Retention/Validation' -prune -o -print 
                     -path '/dwh_actuariat/sasdata/Data_Retention/sas2001b' -prune -o -print ";

data test;
length text $1000.;
	infile oscmd;
    input;
   text= _infile_;
run;

[Kurt_Bremser]

Pipe the output from find into grep with the -v option to exclude lines with a specific pattern.

[alepage]

the option -type -name ‘*_prm*’ is working but not with the -path option 

or the path option is working but not the -type option 

please provide an example with the grep v option

-type f -name '*_prm*'
                     -path '/dwh_actuariat/sasdata/Data_Retention/Data_Retention_CleanFiles' -prune -o -print

[Kurt_Bremser]

I created some directories in my personal folder on my Macbook:

> find test
test
test/uvw
test/abc
test/def
test/xyz

Then I ran this command:

find test|grep -v xyz|grep -v uvw

and got this result:

test
test/abc
test/def

[Patrick · Accepted solution]

You could just use the ! to exclude paths. 

Using the following prompts with Copilot:

Amend to exclude paths /a/b/c and /a/b/x from find /a -type f -name '*_prm*' 

I've got this answer:

find /a -type f -name '*_prm*' ! -path "/a/b/c/*" ! -path "/a/b/x/*"

Applied to the code you shared this could look like:

filename oscmd pipe "find &cleandspath -type f -name '*_prm*' ! -path '&dircf/*' ! -path '&diraf/*' ! -path '&dirdrl/*' ";

I've tested the syntax and it works as desired. It will exclude all content under the paths that are listed as excluded (the NOT exclamation mark).

[alepage]

This script works perferctly. However, I would like to understand the signification of the exclamation mark

[Kurt_Bremser]

Read for yourself: find 

It's in the OPERATORS section.

[yabwon]

Why can't you create a temporary dataset? 

Having such a data set with list of files you want to delete seems like a perfect fit for running FDELETE() function:

data _null_;
    SET test;
    fname="tempfile";
    rc=filename(fname, strip(text));
    if rc = 0 and fexist(fname) then
       rc=fdelete(fname);
    rc=filename(fname);
run;

Or in fact you could do it "on the fly":

%let cleandspath=/dwh_actuariat/sasdata/Data_Retention/;

filename oscmd pipe "find &cleandspath -type f -name '*_prm*'  ";

data _null_;
  length text $1000.;
	infile oscmd;
  input;
  text = _infile_;
  if NOT (text in: ("&cleandspath.Data_Retention_CleanFiles" 
                    "&cleandspath.Data_Retention_Anonymized" 
                    "&cleandspath.Data_Retention_list"
         ));


    fname="tempfile";
    rc=filename(fname, strip(text));
    if rc = 0 and fexist(fname) then
       rc=fdelete(fname);
    rc=filename(fname);

run;

Bart

[Tom]

Why not just use SAS code instead? https://github.com/sasutils/macros/blob/master/dirtree.sas

You are asking for help with a UNIX command on a SAS forum.  Someone gave you an answer for how to filter the output of the UNIX command using normal SAS code.  Why isn't that good enough?

Hint on using the find command.  Look at the many expressions you can use with the find command.  In particular the -prune option.

Here is an example I used in the past to ignore .snapshot directories that were created automatically on some network attach disk drives.

find ....... -name \.snapshot -prune ....