topic Re: How to get the complete file listing on a unix server and put this info in a sas dataset in SAS Enterprise Guide

How to get the complete file listing on a unix server and put this info in a sas dataset

alepage — Tue, 23 Jan 2024 22:07:54 GMT

Hello,

The task that I need to carry out is to put in a SAS dataset, the complete file listing of a Unix Server. I need to refine my search with file extension (spde file, base dataset, text file, csv file, xlsx file) . I need to know tha age of each file.

I am looking for a Unix command that I could use with a x command in SAS.

Please provide base script.

Re: How to get the complete file listing on a unix server and put this info in a sas dataset

SASJedi — Tue, 23 Jan 2024 22:15:51 GMT

Check out this post:
Solved: Importing File name and Modified date into dataset - SAS Support Communities

Re: How to get the complete file listing on a unix server and put this info in a sas dataset

Kurt_Bremser — Wed, 24 Jan 2024 07:14:13 GMT

My WUSS 2022 presentation contains a program which recursively traverses a directory tree.

Re: How to get the complete file listing on a unix server and put this info in a sas dataset

Kurt_Bremser — Wed, 24 Jan 2024 07:16:26 GMT

Alternatively, use the UNIX find command with INFILE PIPE and parse the return for the modified date.

Re: How to get the complete file listing on a unix server and put this info in a sas dataset

Patrick — Wed, 24 Jan 2024 10:16:53 GMT

If you're allowed to execute OS commands out of SAS (option XCMD set) and it's a Unix/Linux environment then I'd certainly use the Unix find command for this via a filename pipe.

The find command is by nature recursive. Code along the line of below should do the job (not tested! - but I've used similar logic in the past successfully).

filename listing pipe 'find / \( -name "*.sas7b" -o -name "*.csv" \) -type f -printf "%h|%f|%CF\n"';
data want;
  infile listing truncover dlm='|';
  input path:$300. name$50. modification_date:yymmdd10.;
  format modification_date date9.;
run;

Ideally first use a command prompt like Putty to fully test the find command syntax you need ...and only run it against a sub-folder and not the whole file system -> find /folder1/folder2/...

The \n at the end of -printf adds a newline character at the end of each match to ease reading the result with a SAS data step.

Also be aware that you will only find files in folders for which the user under which the process runs got a least the permission to list the files.

Re: How to get the complete file listing on a unix server and put this info in a sas dataset

alepage — Wed, 24 Jan 2024 15:38:16 GMT

Hello Mr. Bremser,

We are starting a new project called Data Retention. The goal of this project is to trace in our production server, all datasets or table such as base SAS dataset, SPDE file, Text file, CSV file, Excel files and so on. The first step is to make a listing of all those file, their location and the creation Date.

Which of your unix command is the best for this purpose.

Best Regards

Re: How to get the complete file listing on a unix server and put this info in a sas dataset

Kurt_Bremser — Wed, 24 Jan 2024 16:39:09 GMT

UNIX does not keep the creation date, only the timestamp of the last modification.

@Patrick has already shown an example for the usage of the find command. You should issue the command first from a commandline (log on to the server with ssh) to see if the command itself is correct. Then proceed to run it from within SAS.

As also mentioned, you can only list directories for which you (the user under which the command is issued) have read permission.