I have a UNIX server that contain the following folder structure:
Root: /sas/config/master.data/
1st Layer (Possible folders):
/sas/config/master.data/12312016
/sas/config/master.data/06302017
/sas/config/master.data/08312017
/sas/config/master.data/johncena
/sas/config/master.data/brock123
/sas/config/master.data/elonmusk
2nd Layer
/sas/config/master.data/johncena/09302019
/sas/config/master.data/johncena/04302012
/sas/config/master.data/brock123/02282013
/sas/config/master.data/elonmusk/01312020
/sas/config/master.data/elonmusk/03312021
Do notice that only those english name subfolders (eg. johncena) are being pointed out here assuming that it contains a MMDDYYYY folder inside. I can achieve this by checking it with anydate function (eg. folderdate=input(fname,anydtdte.);). Also, the folder master.data contains a dot (may cause issue if i understand correctly).
By looking at the folder structure above, we can tell that only the root directory is fixed. After the root directory, there are 2 more levels that are dynamic which require further looping.
Ultimately, I will need to store this into a SAS Dataset so that I can assign it into macro for generating mv -f command to either remove it or move or copy to another directory within the UNIX server.
Now my trouble is, I have a script that I referred to SAS documentation but I couldn't find the right way and logic to scan unix directory dynamically with a check for 2 additional layers (refer to 2nd Layer explanation.) despite me knowing that there is a function to check if the folder contains a valid date format or not.
The UNIX directory scan script as below. I can execute it but it will only show in log, not storing into a table with the full directory for me to further process. Would this be the most efficient script for what I am aimed to do?
%macro drive(dir,ext);
%local filrf rc did memcnt name i;
/* Assigns a fileref to the directory and opens the directory */
%let rc=%sysfunc(filename(filrf,&dir));
%let did=%sysfunc(dopen(&filrf));
/* Make sure directory can be open */
%if &did eq 0 %then %do;
%put Directory &dir cannot be open or does not exist;
%return;
%end;
/* Loops through entire directory */
%do i = 1 %to %sysfunc(dnum(&did));
/* Retrieve name of each file */
%let name=%qsysfunc(dread(&did,&i));
/* Checks to see if the extension matches the parameter value */
/* If condition is true print the full name to the log */
%if %qupcase(%qscan(&name,-1,.)) = %upcase(&ext) %then %do;
%put &dir/&name;
%end;
/* If directory name call macro again */
%else %if %qscan(&name,2,.) = %then %do;
%drive(&dir/%unquote(&name),&ext)
%end;
%end;
/* Closes the directory and clear the fileref */
%let rc=%sysfunc(dclose(&did));
%let rc=%sysfunc(filename(filrf));
%mend drive;
/* First parameter is the directory of where your files are stored. */
/* Second parameter is the extension you are looking for. */
%drive(/sas/config/master.data/,sas)
You can use the MODIFY statement functionality of a DATA step to scan through a directory tree without have to actually write a recursive function.
Use this macro.
https://github.com/sasutils/macros/blob/master/dirtree.sas
If you only want to go down two levels use the MAXDEPTH option.
%dirtree
/*---------------------------------------------------------------------------
Build dataset of files in directory tree(s)
----------------------------------------------------------------------------*/
(/sas/config/master.data /* Pipe delimited directory list (default=.) */
,out=dirtree /* Output dataset name */
,maxdepth=2 /* Maximum tree depth */
);
To get list of directories that are two levels below your starting path you could use:
data level2;
set dirtree;
where level=2 and type='D';
run;
/* If directory name call macro again */
The way to test this is to use the dopen() function
You can use the MODIFY statement functionality of a DATA step to scan through a directory tree without have to actually write a recursive function.
Use this macro.
https://github.com/sasutils/macros/blob/master/dirtree.sas
If you only want to go down two levels use the MAXDEPTH option.
%dirtree
/*---------------------------------------------------------------------------
Build dataset of files in directory tree(s)
----------------------------------------------------------------------------*/
(/sas/config/master.data /* Pipe delimited directory list (default=.) */
,out=dirtree /* Output dataset name */
,maxdepth=2 /* Maximum tree depth */
);
To get list of directories that are two levels below your starting path you could use:
data level2;
set dirtree;
where level=2 and type='D';
run;
Good news: We've extended SAS Hackathon registration until Sept. 12, so you still have time to be part of our biggest event yet – our five-year anniversary!
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.
Ready to level-up your skills? Choose your own adventure.