List files of a directory

[japfvg]

Dear Experts:

Hope you're doing great, asking for your help again.

I'm trying to list all csv files inside a directory and put them in a dataset, I'm using this code:

%let ruta="c:\tmp\"
filename myDir &ruta ;

data test (keep=filename);
	did=dopen("myDir") ;
	filecount=dnum(did) ;
	do i=1 to filecount ;
		filename=dread(did,i) ;
		if scan(filename,2,".")="csv" then 
			filename= &ruta. || '\' || filename;
		output ;
	end ;
	rc=dclose(did) ;
run ;

The code is not working when I'm trying to include the path and the filename and not sure why's hapenning this:

filename= &ruta. || '\' || filename;

Any help or ideas would be greatly appreciated.

Regards!

Fer

[Kurt_Bremser]

%let ruta="c:\tmp\" /* missing semicolon here */
filename myDir &ruta ;

data test (keep=filename);
	did=dopen("myDir") ;
	filecount=dnum(did) ;
	do i=1 to filecount ;
		filename=dread(did,i) ;
		if scan(filename,2,".")="csv" then 
			filename= &ruta. || '\' || filename; /* will create two successive backslashes *
		output ;
	end ;
	rc=dclose(did) ;
run ;

After making corrections, please post the complete log from that step.

[japfvg]

Hi Kurt

Thanks for the correction, the final code is:

%let ruta="\\crgldept01p.cri.bns\MOVEIT\CR\TsysSchemeFiles\";
filename myDir &ruta ;
data test (keep=filename);
	did=dopen("myDir") ;
	filecount=dnum(did) ;
	do i=1 to filecount ;
		filename=dread(did,i) ;
		*if scan(filename,2,".")="csv" then put filename ;
		if scan(filename,2,".")="csv" then 
		filename= &ruta. || filename;
		output ;
	end ;
	rc=dclose(did) ;
run ;

And the log is:

1                                                          The SAS System                           16:36 Tuesday, February 16, 2021

1          ;*';*";*/;quit;run;
2          OPTIONS PAGENO=MIN;
3          %LET _CLIENTTASKLABEL='list_files';
4          %LET _CLIENTPROCESSFLOWNAME='Flujo del proceso';
5          %LET _CLIENTPROJECTPATH='C:\FVG\SAS\prueba.egp';
6          %LET _CLIENTPROJECTPATHHOST='LPC0PHWAVMEXBT';
7          %LET _CLIENTPROJECTNAME='prueba.egp';
8          %LET _SASPROGRAMFILE='';
9          %LET _SASPROGRAMFILEHOST='';
10         
11         ODS _ALL_ CLOSE;
12         OPTIONS DEV=PNG;
13         GOPTIONS XPIXELS=0 YPIXELS=0;
14         FILENAME EGSR TEMP;
15         ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR
16             STYLE=HTMLBlue
17             STYLESHEET=(URL="file:///C:/Program%20Files/SASHome/SASEnterpriseGuide/7.1/Styles/HTMLBlue.css")
18             NOGTITLE
19             NOGFOOTNOTE
20             GPATH=&sasworklocation
21             ENCODING=UTF8
22             options(rolap="on")
23         ;
NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR
24         
25         GOPTIONS ACCESSIBLE;
26         %let ruta="\\crgldept01p.cri.bns\MOVEIT\CR\TsysSchemeFiles\";
27         filename myDir &ruta ;
28         
29         data test (keep=filename);
30         	did=dopen(myDir) ;
31         	filecount=dnum(did) ;
32         	do i=1 to filecount ;
33         		filename=dread(did,i) ;
34         		*if scan(filename,2,".")="csv" then put filename ;
35         		if scan(filename,2,".")="csv" then
36         		filename= &ruta. || filename;
37         		output ;
38         	end ;
39         	rc=dclose(did) ;
40         run ;

NOTE: Numeric values have been converted to character values at the places given by: (Line):(Column).
      30:12   
NOTE: Variable myDir is uninitialized.
NOTE: Argument 1 to function DNUM(0) at line 31 column 12 is invalid.
ERROR: Invalid DO loop control information, either the INITIAL or TO expression is missing or the BY expression is missing, zero, 
       or invalid.
did=0 myDir=. filecount=. i=1 filename=  rc=. _ERROR_=1 _N_=1
NOTE: Mathematical operations could not be performed at the following places. The results of the operations have been set to 
      missing values.
      Each place is given by: (Number of times) at (Line):(Column).
      1 at 31:12   
NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set WORK.TEST may be incomplete.  When this step was stopped there were 0 observations and 1 variables.
WARNING: Data set WORK.TEST was not replaced because this step was stopped.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
2                                                          The SAS System                           16:36 Tuesday, February 16, 2021

      cpu time            0.00 seconds
      

41         
42         
43         GOPTIONS NOACCESSIBLE;
44         %LET _CLIENTTASKLABEL=;
45         %LET _CLIENTPROCESSFLOWNAME=;
46         %LET _CLIENTPROJECTPATH=;
47         %LET _CLIENTPROJECTPATHHOST=;
48         %LET _CLIENTPROJECTNAME=;
49         %LET _SASPROGRAMFILE=;
50         %LET _SASPROGRAMFILEHOST=;
51         
52         ;*';*";*/;quit;run;
53         ODS _ALL_ CLOSE;
54         
55         
56         QUIT; RUN;
57         

Now it seems the "myDir" variable is not working...

Thanks again.

Fer

[andreas_lds]

---

@japfvg wrote:

...     

Now it seems the "myDir" variable is not working...

Thanks again.

Fer

---

Interesting: code and log don't match. "myDir" is not a variable, but the name of a file-reference, so you have to use

did=dopen("myDir"); /* copied form the code */

instead of

did=dopen(myDir) ; /* copied from the log */

[Tom]

You didn't submit the code you posted.

You said the program included this line:

	did=dopen("myDir") ;

But the SAS log shows this line

30         	did=dopen(myDir) ;

Can you see the difference?

Also define your variables before using them, otherwise SAS will just guess how you want them defined by how they are first used in the code.

Also to test the extension on a file scan from right to left. Otherwise you will get the wrong answer for filenames that include periods in them.  Also if  your statements take more than on one line indent the secondary lines otherwise it is difficult for humans to review the code.

if lowcase(scan(filename,-1,"."))="csv" then 
  filename= &ruta. || filename
;

[ballardw · Accepted solution]

I suggest that you print out some of the results for Filename and share them here.

I suspect that you may have some names truncated.

When a variable such as FILENAME does not have an explicit length assigned when created then the first use sets the length for the variable. So when you attempt to prefix some text to the same variable there isn't enough room.

Run this code and check the result in the log:

data example;
   filename = 'something.csv';
   filename = 'c:\path'||'\'||filename;
   put filename=;
run;

You will see a result of

filename=c:\path\somet

Counting characters will show you that the result has the same number of characters as "something.csv".

Solution: define a length for the variable that is long enough to hold the longest expected value before the first use. Possibly:

data test (keep=filename);
	did=dopen("myDir") ;
	filecount=dnum(did) ;
        length filename $ 256;
	do i=1 to filecount ;
		filename=dread(did,i) ;
		if scan(filename,2,".")="csv" then 
			filename= &ruta. || '\' || filename;
		output ;
	end ;
	rc=dclose(did) ;
run ;

Depending on the values involved you may also occasionally find that the || operator uses the full length of a variable and pads things with spaces.

You may want to try using the CATX function to stick things together

filename= Catx('\',&ruta.,filename);

which removes trailing spaces and inserts the first parameter between all the non-blank strings present. Which can make creating longer paths with multiple values lots easier to follow than all the ||'\'|| codes.