topic Re: Using function mopen to extract the log owner, while the log is still being created in SAS Programming

Using function mopen to extract the log owner, while the log is still being created

J111 — Sun, 12 Feb 2023 11:19:29 GMT

Hello,

when using the code below, can we get the information regarding the log owner (function mopen) even while the log is still being written ?

Thanks

%let logs_folder=;
%let logs_from_dt=;
%macro logs_get_files(logs_folder=, logs_from_dt=);
data LOGS_FILES(keep=folder file_name file_size created_dttm owner);
length
Folder $200
File_Name $100
File_Size
Created_DTTM 8;
rc=filename('_dir_',"&logs_folder",'','encoding="utf-8"');
did=dopen('_dir_');
if (did lt 1) then do;
put "ERR" "OR: Unable to open &logs_folder as a directory.";
stop;
end;
do i=1 to dnum(did);
folder="&logs_folder";
file_name = scan(dread(did,i), -1, '\/');
File attributes;
fid=mopen(did,file_name);
type=ifc(fid > 0,'F','D');
if (type='F') then do;
ext = lowcase(scan(file_name,-1,'.'));
if ext='log' then do;
file_size = input(finfo(fid,'File Size (bytes)'),32.);
created_dttm = input(finfo(fid,'Create Time'),NLDATM30.);
owner= strip(finfo(fid,'Owner Name'));
if datepart(created_dttm)>="&logs_from_dt"d then ;output;
end;
end;
fid=fclose(fid);
end;
did=dclose(did);
rc=filename('_dir_','');
format File_Size comma32. Created_DTTM datetime22.;
run;
%mend;
%logs_get_files(logs_folder=/sas/logs/, logs_from_dt=13NOV2022);

Re: Using function mopen to extract the log owner, while the log is still being created

Quentin — Sun, 12 Feb 2023 12:43:41 GMT

Hi,

Did you try it? It may depend on the OS. I tried it on windows, using test code like:

proc printto log="Q:\junk\foologs\mylog5.log" new ;
run ;

%logs_get_files(logs_folder=Q:\junk\foologs, logs_from_dt=13NOV2022)

proc printto ;
run ;

And it did not work. fid=mopen(did,file_name) ; returned 0 for the active log file which is still being written to.

But it's possible it is OS dependent. Might be worth trying on linux. The success of this code probably depends on how the OS handles files that are currently open for writing.

Note you have an unrelated typo (extra semicolon) in your code:

if datepart(created_dttm)>="&logs_from_dt"d then ;output;  *Do not want the semicolon after then ;

If you can't get it working, one option would be to wrap your code in a PROC PRINTTO block. With that, when you close the PROC PRINTTO it will stop writing to the log file, and then you should be able to get the file's metadata. So the below code is able to get the attributes of mylog6.log, because it is not active:

proc printto log="Q:\junk\foologs\mylog6.log" new ;
run ;

*Main code here ;

data foo ;
  set sashelp.class ;
run ;

proc printto ;
run ;

%logs_get_files(logs_folder=Q:\junk\foologs, logs_from_dt=13NOV2022)

Re: Using function mopen to extract the log owner, while the log is still being created

J111 — Sun, 12 Feb 2023 13:07:39 GMT

We want to read 100 logs according to their owner, while some are still being created.

What is the best way to do it ?

Thanks

Re: Using function mopen to extract the log owner, while the log is still being created

J111 — Sun, 12 Feb 2023 13:15:57 GMT

Shall try to clarify:-

Lets say we have 2 owners of logs A & B

If the owner is A we want to open the log and look into its details

If the owner is B we want to skip this log and continue until we find anther log with owner A

We want to know if the owner is A or B even if the log is still being created

There are 100 logs in Linux Folder.

Please advise

Re: Using function mopen to extract the log owner, while the log is still being created

Quentin — Sun, 12 Feb 2023 13:47:51 GMT

I think your current attempt makes sense. What OS are your working on? It didn't work for me on windows, but did you try it on linux?

If that code doesn't work, do you have XCMD enabled? If so, then you can see if running LS or a similar OS command will be able to retrieve the file owner information of an active log file. If it does, you can use filename pipe to execute the command and then parse the results.

To test, I would create a SAS job that SLEEPS for 30 minutes, and then open a command prompt and see if OS commands can tell you the owner of the log file that is actively being written to.

Re: Using function mopen to extract the log owner, while the log is still being created

J111 — Sun, 12 Feb 2023 13:51:32 GMT

Thanks

One more detail - the program is running with EGUIDE

Re: Using function mopen to extract the log owner, while the log is still being created

Tom — Sun, 12 Feb 2023 16:59:24 GMT

Is the XCMD option active in your SAS session?

If so it should be possible to see the owner of a file in Unix just use the normal ls command.

The owner is the third value on the line and the filename is the last.

data log_files;
   length dummy $1 owner $30 filename $256 ;
   infile "ls /some/directory/*.log" pipe truncover ;
   input (2*dummy owner 5*dummy) (:) filename $256. ;
  drop dummy;
run;

Re: Using function mopen to extract the log owner, while the log is still being created

ballardw — Sun, 12 Feb 2023 17:00:10 GMT

@J111 wrote:

Lets say we have 2 owners of logs A & B

We want to know if the owner is A or B even if the log is still being created

"If the log is still being created" points me to process id system tools and/or metadata, not open the file.

I question the utility of attempting to read a file that is still being created. That means the content is changing as you look at it. So, just what to you want to find in these open logs?

Re: Using function mopen to extract the log owner, while the log is still being created

ChrisNZ — Mon, 13 Feb 2023 01:49:08 GMT

I suspect it's a Windows machine since Unix wouldn't care that the files are in use.

Switch /q adds the owner to the command.
How the line is parsed will depend on your local Windows settings.

data log_files; ... infile "dir &path /q" pipe truncover ; ... run;

Re: Using function mopen to extract the log owner, while the log is still being created

J111 — Mon, 13 Feb 2023 05:52:41 GMT

Many thanks for all the replies.

ERROR: Insufficient authorization to access PIPE.

We are looking into ERROR lines in the log, during the next step

We shall try to run the program while a log is being created

Re: Using function mopen to extract the log owner, while the log is still being created

Quentin — Mon, 13 Feb 2023 13:31:42 GMT

@J111 wrote:

Many thanks for all the replies.

1.

ERROR: Insufficient authorization to access PIPE.

This means you don't have the system option XCMD turned on. In order to execute native OS commands from SAS, your admin will need to turn on XCMD.