Finding Log files that have been created on the current day. On Linux mach

[SASHunter]

HI,

I have in my program the following that works just fine, but I want it to do something more:

filename indata pipe "ls -1 &logdir | grep .log";

where &logdir is assigned as the log directory that I want to search. I want to find only those log files that were created today. This is what I tried:

filename indata pipe "ls -1 &logdir | grep '&mdate' ";

where &mydate is equal to say - Mar 22

Of course this doesn't work!! I could use any suggestions.

From the command line I can use the following:

ls -l /cpspb/prod/log/mar11/ | grep "$mydate" , if the variable mydate has been assigned first.

Thanks, Nancy

[Peter_C]

instead of
filename indata pipe "ls -1 &logdir | grep '&mdate'";
try
filename indata pipe "ls -1 &logdir | grep ""&mdate"" ";

[SASHunter]

I changed the program to have the following:

filename indata pipe "ls -l &logdir | grep ""&newdate"" ";

where &logdir and &newdate resolve to the following:

34 filename indata
SYMBOLGEN: Macro variable LOGDIR resolves to /cpspb/prod/log/mar11/
SYMBOLGEN: Macro variable NEWDATE resolves to Mar 23
34 ! pipe "ls -l &logdir | grep ""&newdate"" ";
35

But this still is not working. This is the error I get. Like it can't read the files that it found.

NOTE: The infile INDATA is:
Pipe command="ls -l /cpspb/prod/log/mar11/ | grep "Mar 23" "

ERROR: Physical file does not exist, /cpspb/prod/log/mar11/-rw-rw-r-- 1 cpsmod
cpspb 2581 Mar 23 14:38 blaisechgdaypi.log.
RULE: ----+----1----+----2----+----3----+----4----+----5----+----6----+----7
3 The SAS System 14:38 Wednesday, March 23,>

1 -rw-rw-r-- 1 cpsmod cpspb 2581 Mar 23 14:38 blaisechgdaypi.log 65
lname= linectr=0
f2r=-rw-rw-r-- 1 cpsmod cpspb 2581 Mar 23 14:38 blaisechgdaypi.log
file2read=/cpspb/prod/log/mar11/-rw-rw-r-- 1 cpsmod cpspb 2581 Mar 23 14:38 b
laisechgdaypi.log done=0 _ERROR_=1 _N_=1
NOTE: 1 record was read from the infile INDATA.
The minimum record length was 65.
The maximum record length was 65.


If I do this same command directly on the command line, I get the following:

/home/hunte022> ls -l /cpspb/prod/log/mar11/ | grep 'Mar 23'
-rw-rw-r-- 1 cpsmod cpspb 2581 Mar 23 14:38 blaisechgdaypi.log


Does anyone see what I am doing wrong?

Thanks, Nancy

[sbb]

Move your "filter logic" (grep component) to operate as part of the SAS DATA step.

And, with that suggestion, have you at least verified the SAS FILENAME and associate specifications work correctly without the "grep" component?

Scott Barry
SBBWorks, Inc.

[SASHunter]

I am still struggling with this code!!

I'm not sure what you mean by 'Move your "filter logic" (grep component) to operate as part of the SAS DATA step.'

The grep command is part of the filename statement. As the following:
filename indata pipe "ls -lt --full-time &logdir | grep &testmdate ";

38 filename indata
SYMBOLGEN: Macro variable LOGDIR resolves to /cpspb/prod/log/mar11/
SYMBOLGEN: Macro variable TESTMDATE resolves to 2011-03-28
38 ! pipe "ls -lt --full-time &logdir | grep &testmdate ";

The file is definitely in the log directory, but I get this ERROR:

NOTE: The infile INDATA is:

Pipe command="ls -lt --full-time /cpspb/prod/log/mar11/ | grep 2011-03-28
"

ERROR: Physical file does not exist, /cpspb/prod/log/mar11/-rw-rw-r-- 1 cpsmod
cpspb 6237 2011-03-28 08:10:35.000000000 -0400
err_list_SASLOG_Summarizer_28MAR11.rtf.
RULE: ----+----1----+----2----+----3----+----4----+----5----+----6----+----7
1 -rw-rw-r-- 1 cpsmod cpspb 6237 2011-03-28 08:10:35.000000000 -040
71 0 err_list_SASLOG_Summarizer_28MAR11.rtf 110
lname= linectr=0
f2r=-rw-rw-r-- 1 cpsmod cpspb 6237 2011-03-28 08:10:35.000000000 -0400 err_
list_SASLOG_Summarizer_28MAR11.rtf

/-------------------------------------------------------------------------------------------------------------------------/
If I do the following at the prompt in Linux, I get what I am expecting. It finds the three files that have today's date associated with them.

/home/hunte022> ls -t1 --full-time /cpspb/prod/log/mar11 | grep '2011-03-28'
-rw-rw-r-- 1 cpsmod cpspb 6237 2011-03-28 08:10:35.000000000 -0400 err_list_SASLOG_Summarizer_28MAR11.rtf
-rw-rw-r-- 1 cpsmod cpspb 191973 2011-03-28 08:10:35.000000000 -0400 error_checker.log
-rw-rw-r-- 1 cpsmod cpspb 4084 2011-03-28 08:10:35.000000000 -0400 misclist_SASLOG_Summarizer_28MAR11.rtf
/home/hunte022>

What can I be doing wrong?

Thanks, Nancy

[TimB_SAS]

Try this to see whether it finds the files as you need:

filename foo pipe "find &logdir -maxdepth 1 -type f -name '*.log' -mtime -1 -exec ls -l '{}' \;" ;

data _null_ ;
infile foo ;
input ;
put _infile_ ;
run ;

The -MTIME option can be a bit tricky, you made need to specify -2 in place of -1 to get all the expected files, check the man page on find for details.

[RalphGöstenmeier]

You need "ls -l" (the letter l) for displaying the date, but not for extracting the filename.

So maybe this helps:

filename indata pipe "ls -1 &logdir | grep '&mdate' | cut -c52-";

"cut" display's from columns52 and more, so this is only the filename.

[SASHunter]

Thanks for every ones help.

I have finally figured out what I needed to change in the program.

I have the FILENAME statement that now looks like this:

filename indata pipe "ls -lt --full-time &logdir | grep .log | grep &testmdate ";

where it resolves to:
21 filename indata
SYMBOLGEN: Macro variable LOGDIR resolves to /cpspb/prod/log/mar11/
SYMBOLGEN: Macro variable TESTMDATE resolves to 2011-03-28
21 ! pipe "ls -lt --full-time &logdir | grep .log | grep
21 ! &testmdate ";

And inside the dataset, I have the following that I had to change in the INPUT statement:

30 data mylist;
31
32 length lname $55;
33
34 INFILE indata lrecl=200 pad;
35 retain linectr 0;
36
37 input @73 f2r $80.; * Start reading at this character on the
37 ! line;
38
39 file2read="&logdir"||f2r;
SYMBOLGEN: Macro variable LOGDIR resolves to /cpspb/prod/log/mar11/
40
41 infile dummy filevar=file2read filename=lname end=done
42 lrecl=200 truncover;
43
44 run;

--------------------------------------------------------------------------------------------------------------

This now does the 'ls' command and then does the two greps to narrow my search.

And I start reading only at position 73 to find the file name.

Thanks for the other examples. I will test them also.

Nancy

[SASHunter]

Hi all,

Just a little addition to this post:

I tried Ralph's recommendation also. This worked, when adding the 'cut' at the end of the FILENAME statement.

filename indata pipe
"ls -lt --full-time &logdir | grep .log | grep &testmdate | cut -c73-";

and changing the INPUT statement back to what I had.
from:
INPUT @73 f2r $80.;
to:
INPUT f2r $80.;

-------------------------------------------------------------------------------------------------------------------
The FILENAME resolves to:
21 filename indata pipe
SYMBOLGEN: Macro variable LOGDIR resolves to /cpspb/prod/log/mar11/
SYMBOLGEN: Macro variable TESTMDATE resolves to 2011-03-30
22 "ls -lt --full-time &logdir | grep .log | grep &testmdate | cut
22 ! -c73-";

and it finds the one file that has today's date in it.

Thanks so much for the suggestions.

Nancy