topic how to get list of files available in Directory including sub folder in SAS in SAS Data Management

how to get list of files available in Directory including sub folder in SAS

sivastat08 — Wed, 24 Jan 2018 11:57:07 GMT

Hi , i was trying to get list of files(.csv) available in Directory including sub folder in SAS,please help me.

code

%macro list_files(dir,ext);
%local filrf rc did memcnt name i;
%let rc=%sysfunc(filename(filrf,&dir));
%let did=%sysfunc(dopen(&filrf));

%if &did eq 0 %then %do;
%put Directory &dir cannot be open or does not exist;
%return;
%end;

%do i = 1 %to %sysfunc(dnum(&did));

%let name=%qsysfunc(dread(&did,&i));

/* %if %qupcase(%qscan(&name,-1,.)) = %upcase(&ext) %then %do;*/
%if %qscan(&name,2,.) ne %then %do;
%put &dir\&name;

data _tmp;
length dir $512 name $100;
dir=symget("dir");
name=symget("name");
run;
proc append base=want data=_tmp;
run;quit;

%end;
%else %if %qscan(&name,2,.) = %then %do;
%list_files(&dir\&name,&ext)
%end;

%end;
%let rc=%sysfunc(dclose(&did));
%let rc=%sysfunc(filename(filrf));

%mend list_files;
%list_files("C:\Users\sivakumar.sivaraj\Documents\Dataset",*)

Error - ''The meaning of an identifier after a quoted string might change in a future SAS release. Inserting white space
between a quoted string and the succeeding identifier is recommended''

1 ;*'
_
49
1 ! ;*";*/;quit;run;
NOTE 49-169: The meaning of an identifier after a quoted string might change in a future SAS release. Inserting white space
between a quoted string and the succeeding identifier is recommended.

2 OPTIONS PAGENO=MIN;
3 %LET _CLIENTTASKLABEL='Program'
_______________________________
49
3 ! ;

NOTE 49-169: The meaning of an identifier after a quoted string might change in a future SAS release. Inserting white space
between a quoted string and the succeeding identifier is recommended.

4 %LET _CLIENTPROCESSFLOWNAME='Process Flow';
5 %LET _CLIENTPROJECTPATH='';
6 %LET _CLIENTPROJECTNAME='';
7 %LET _SASPROGRAMFILE=;
8
9 ODS _ALL_ CLOSE;
10 OPTIONS DEV=ACTIVEX;
11 GOPTIONS XPIXELS=0 YPIXELS=0;
12 FILENAME EGSR TEMP;
13 ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR
14 STYLE=HtmlBlue
15 STYLESHEET=(URL="file:///C:/Program%20Files/SASHome/SASEnterpriseGuide/7.1/Styles/HtmlBlue.css")
NOTE: The quoted string currently being processed has become more than 262 characters long. You might have unbalanced quotation
marks.

Re: how to get list of files available in Directory including sub folder in SAS

RW9 — Wed, 24 Jan 2018 12:17:50 GMT

Sorry, I can't read that. What software are you using, is it Windows/Unix/Server based? Simplest method is to use the operating system if you have access, so for windows:

filename tmp pipe 'dir "c:/somewhere/*.csv" /b /s';

data want;
  infile tmp dlm="¬";
  length cmd_line $2000;
  input cmd_line;
run;

Want then contains the basics directory listing (/b) with all subdirectories (/s) only csv files. Without more on the rest of the process its hard to say.

Re: how to get list of files available in Directory including sub folder in SAS

sivastat08 — Wed, 24 Jan 2018 12:22:22 GMT

Thanks for quick replay.

Re: how to get list of files available in Directory including sub folder in SAS

rickpaulos — Tue, 20 Nov 2018 21:39:45 GMT

Nice & simple, BUT try it with sub folders with spaces in the folder names. I'd like to see a fix for that beside renaming others' folders.

I use X to call dos commands and read in the results:

OPTIONS NOXWAIT;

X "cd c:\folder\sub folder\";

X "dir *.csv /b /s > files.txt";

DATA MyListOfcsvFiles;

INFILE 'files.txt' LRECL=256 PAD MISSOVER;

INPUT filenames $ 1-256; *256 is still the max path length allowed in dos/windows *;

RUN;

Re: how to get list of files available in Directory including sub folder in SAS

Tom — Tue, 20 Nov 2018 21:47:59 GMT

PAD MISSOVER?? There is the TRUNCOVER option available for the INFILE statement now (only been around for about the last 30 years I think).

Note that the trick of using a strange character for the delimiter should eliminate the need for TRUNCOVER.

Of course both will have issues with filenames with leading spaces. For that you could use $CHAR or $VARYING informat or just use the _INFILE_ automatic variable.

And for trailing spaces you will need to keep track of how many characters were on the original source line.

Re: how to get list of files available in Directory including sub folder in SAS

rickpaulos — Tue, 20 Nov 2018 22:51:52 GMT

You learn what your teachers are comfortable with. 30 years ago? That's about when I started learning SAS for DOS and I was assisted by older people who probably never read about updates. The local references were the thick v6 manuals that cost so much, they are still around. Truncover is not in those. And I prefer to avoid gimmicks in any programming language. Keep it simple and easy to read and understand. If you want cryptic gimmicks, try R.

Putting spaces in folder names, file names, Access column names, etc will mess up some languages for sure. SAS is no exception.

Re: how to get list of files available in Directory including sub folder in SAS

RW9 — Wed, 21 Nov 2018 09:04:07 GMT

So you reply to a post from nearly a year ago to tell me that in your special circumstance whereby you are putting special characters in a path - something which is highly recommended not to do - the code doesn't work?

Re: how to get list of files available in Directory including sub folder in SAS

AngusLooney — Fri, 23 Nov 2018 08:29:05 GMT

I've got a fleshed out macro that generates a dataset for the files in a directory and optionally subdirectories with file name, size, data and time, he the full and reletive paths. No issues with spaces etc.

Works for Windows using the file pipe mechanism (hence needs XCMDS), I have a Linux port of it knockign around as well.

Any use?

Long had a plan to move the the dread stuff, to avoid XCMDs, but there isn't as much functionality down that route IMHO.

Re: how to get list of files available in Directory including sub folder in SAS

AngusLooney — Fri, 23 Nov 2018 08:31:24 GMT

Spaces in paths will always come back to annoy you, but you can cope with them, just ends up riving you mad with macro resolution, single and double quotes etc.

In columns names, best avoided.

Re: how to get list of files available in Directory including sub folder in SAS

NiyiAwe1 — Wed, 23 Oct 2019 15:27:15 GMT

This MACRO should help too:

/*MACRO TO GET ALL FILE NAMES AND DETAILS WITHIN A DIRECTORY*/
	%MACRO GET_FILE_NAMES(FILE_DIR	,COMMON_FILE_NAME ,FILE_FORMAT	,DATA_OUT);

		FILENAME LS PIPE "dir &FILE_DIR.&COMMON_FILE_NAME.*.&FILE_FORMAT.";
		DATA FILE_NAMES (KEEP=LIST_FILE); 
			INFILE LS PAD TRUNCOVER EXPANDTABS ; 
			INPUT FILE_NAMES $10000.;
			LIST_FILE = COMPRESS('"'||FILE_NAMES||'"');
		RUN;
		PROC SQL;	SELECT LIST_FILE 	AS LIST_FILE		INTO:	LIST_FILE_1 - 		FROM FILE_NAMES; QUIT;
		PROC SQL; 	SELECT COUNT(*) 	AS NUM_OF_SEARCHES	INTO: 	NUM_OF_SEARCHES		FROM FILE_NAMES;	QUIT;

		%DO Z = 1 %TO &NUM_OF_SEARCHES.;
			DATA RAW_WANT;			
				FILENAME LISA &&LIST_FILE_&Z..; 
				INFILE LISA TRUNCOVER ;
				FORMAT FILE_NAME $20000.;
				FILE_NAME 	= 	&&LIST_FILE_&Z..;
				FLIE_SIZE	=	FINFO(FOPEN('LISA') 		,'FILE SIZE (BYTES)');
				FORMAT MOD_DATE DATETIME.;
   				MOD_DATE	=	INPUT(FINFO(FOPEN('LISA')	,'LAST MODIFIED'), DATETIME.);     
   			    RIGHTS		=	FINFO(FOPEN('LISA')			,'ACCESS PERMISSION');      
		 		OWNER		=	FINFO(FOPEN('LISA')			,'OWNER NAME');      
			RUN;

			PROC APPEND DATA = RAW_WANT BASE = WANT_1 FORCE; RUN;
		%END;
		DATA &DATA_OUT.; 	SET WANT_1; 		RUN;
		PROC DATASETS;		DELETE WANT_1 RAW_WANT FILE_NAMES;	RUN;
	%MEND GET_FILE_NAMES;

/*INPUT PARAMETERS HERE*/
	%GET_FILE_NAMES (FILE_DIR  			= \Users\abc\Desktop\ 		/* SPECIFY THE DIRECTORY LOCATION HERE*/
				   , COMMON_FILE_NAME 	= HAVE_NAME     			/* FILE COMMON NAME, YOU CAN PUT "*" HERE TO SELECT ALL FILE NAMES*/
				   , FILE_FORMAT 		= txt 						/* FILE FORMAT,  YOU CAN PUT "*" HERE TO SELECT ALL FILE FORMATS WITHIN THE DIRECTORY*/
				   , DATA_OUT			= WANT);