topic Reading filenames, date and time from a directory in SAS Programming

Reading filenames, date and time from a directory

SandySKY — Sun, 08 Apr 2018 08:02:00 GMT

In Unix I want to list all log files names in a folder along with last modified date and time separately.

(date and time can be in any format)

Expected output should be

file_name date time

file1 08-04 13:16

file2 08-04 13:17

I tried below code, but not able to get date and time.

%let subdir = /abc/jht/temp/ ;
filename dir pipe "ls -1 &subdir | grep *.log ";
data Pfile;

length filename $ 300 size $ 20 time $ 20;
infile dir truncover;
input filename size time$;
run;

Re: Reading filenames, date and time from a directory

Ksharp — Sun, 08 Apr 2018 12:37:28 GMT

Not sure if it could work .

infile dir truncover  expandtabs ;

Re: Reading filenames, date and time from a directory

Shmuel — Sun, 08 Apr 2018 13:26:12 GMT

It is not clear - is date and time part of the filename or do you want to extract them from the unix file metadata?

Look at the documentation of next functions:

DOPEN - open a directory

DREAD - read a filename in the directory

FOPEN - open a file

FATTR - get file attributes

FCLOSE - close the file

DCLOSE - clse the directory

Next code demonstartes usage of those functions:

%let path = /folders/myshortcuts/My_Folders/flat/;
filename mydir "&path";
data _NULL_;
  did = dopen('mydir');
  dnum = dnum(did);
  put did= dnum=;
  do i=1 to 4; /*dnum; */
     fname = dread(did,i);
     fnx = "&path"||fname;
     rc=filename('tmp',fname,,,'mydir');
     fid = fopen('tmp');
     put i= rc= fid= fnx=;
     if fid then do;
        fattr = finfo(fid,'Last Modified'); 
        lm = input(fattr,datetime18.);        
        put i= fattr=  lm= datetime18.;
      /*  
        infonum = foptnum(fid); put infonum=;
        do i=1 to infonum;
           infoname = foptname(fid,i);
           fattr = finfo(fid,foptname); put i= infoname = fattr=;
        end; 
      */
        fid=fclose(fid);
     end;
     rc=filename('tmp');
     rc=0; _syserr_ = 0;
  end;
run;

Re: Reading filenames, date and time from a directory

NiyiAwe1 — Wed, 23 Oct 2019 15:28:01 GMT

Try this:

/*MACRO TO GET ALL FILE NAMES AND DETAILS WITHIN A DIRECTORY*/
	%MACRO GET_FILE_NAMES(FILE_DIR	,COMMON_FILE_NAME ,FILE_FORMAT	,DATA_OUT);

		FILENAME LS PIPE "dir &FILE_DIR.&COMMON_FILE_NAME.*.&FILE_FORMAT.";
		DATA FILE_NAMES (KEEP=LIST_FILE); 
			INFILE LS PAD TRUNCOVER EXPANDTABS ; 
			INPUT FILE_NAMES $10000.;
			LIST_FILE = COMPRESS('"'||FILE_NAMES||'"');
		RUN;
		PROC SQL;	SELECT LIST_FILE 	AS LIST_FILE		INTO:	LIST_FILE_1 - 		FROM FILE_NAMES; QUIT;
		PROC SQL; 	SELECT COUNT(*) 	AS NUM_OF_SEARCHES	INTO: 	NUM_OF_SEARCHES		FROM FILE_NAMES;	QUIT;

		%DO Z = 1 %TO &NUM_OF_SEARCHES.;
			DATA RAW_WANT;			
				FILENAME LISA &&LIST_FILE_&Z..; 
				INFILE LISA TRUNCOVER ;
				FORMAT FILE_NAME $20000.;
				FILE_NAME 	= 	&&LIST_FILE_&Z..;
				FLIE_SIZE	=	FINFO(FOPEN('LISA') 		,'FILE SIZE (BYTES)');
				FORMAT MOD_DATE DATETIME.;
   				MOD_DATE	=	INPUT(FINFO(FOPEN('LISA')	,'LAST MODIFIED'), DATETIME.);     
   			    RIGHTS		=	FINFO(FOPEN('LISA')			,'ACCESS PERMISSION');      
		 		OWNER		=	FINFO(FOPEN('LISA')			,'OWNER NAME');      
			RUN;

			PROC APPEND DATA = RAW_WANT BASE = WANT_1 FORCE; RUN;
		%END;
		DATA &DATA_OUT.; 	SET WANT_1; 		RUN;
		PROC DATASETS;		DELETE WANT_1 RAW_WANT FILE_NAMES;	RUN;
	%MEND GET_FILE_NAMES;

/*INPUT PARAMETERS HERE*/
	%GET_FILE_NAMES (FILE_DIR  			= \Users\abc\Desktop\ 		/* SPECIFY THE DIRECTORY LOCATION HERE*/
				   , COMMON_FILE_NAME 	= HAVE_NAME     			/* FILE COMMON NAME, YOU CAN PUT "*" HERE TO SELECT ALL FILE NAMES*/
				   , FILE_FORMAT 		= txt 						/* FILE FORMAT,  YOU CAN PUT "*" HERE TO SELECT ALL FILE FORMATS WITHIN THE DIRECTORY*/
				   , DATA_OUT			= WANT);

Re: Reading filenames, date and time from a directory

hollarl11 — Mon, 26 Sep 2022 13:42:47 GMT

Does anyone have code similar to this that would loop through multiple all sub-directories within the path?

Re: Reading filenames, date and time from a directory

PaigeMiller — Mon, 26 Sep 2022 13:49:29 GMT

@hollarl11 wrote:

Does anyone have code similar to this that would loop through multiple all sub-directories within the path?

Answer from @Ksharp here: https://communities.sas.com/t5/SAS-Programming/Import-all-TSV-files-in-all-subfolders/m-p/834126

Re: Reading filenames, date and time from a directory

Tom — Mon, 26 Sep 2022 14:09:26 GMT

Use this %DIRTREE() macro.

You can pass it multiple top level nodes if you want (separated by pipe).

You can limit the number of levels of subdirectories in scans.

%dirtree
/*---------------------------------------------------------------------------
Build dataset of files in directory tree(s)
----------------------------------------------------------------------------*/
(directory    /* Pipe delimited directory list (default=.) */
,out=dirtree  /* Output dataset name */
,maxdepth=120 /* Maximum tree depth */
);