topic Re: Macro for path and list of files in SAS Programming

Macro for path and list of files

dustychair — Mon, 06 Jun 2022 18:42:17 GMT

%macro fileread_rf(d);

filename indata pipe 'dir P:\Results\RFs\&d./b';
/*to read the name of files in a folder*/
data file_list_&d._rf;
length fname_rf $50;
infile indata truncover;
input fname_rf $50.;
call symput ('num_files_&d._rf',_n_);
run;

%do j=1 %to &&num_files_&d._rf;
data _null_;
set file_list_&d._rf;
if _n_=&j;
call symput ('filein_rf',fname_rf);
run;

proc import file="P:\Results\RFs\&d.\&filein_rf"
out=rf_&d._&j.
dbms=csv;
run;

%end;
%mend fileread_rf; 
%fileread_rf(ALG);
%fileread_rf(MAT);

Hi,

I am sure there are more problems in my macro but for now i am struggling with two of them.

1. "filename indata pipe 'dir P:\Results\RFs\&d./b'"

I think this statement is wrong because when i run it I don't get the list of files in ALG or MAT folders.

2. num_files_&d._rf

I am also gettung error for this statement. ERROR: Symbolic variable name NUM_FILES_&D._RF must contain only letters, digits, and underscores

could you please help me to fix it?

Thanks

Re: Macro for path and list of files

Tom — Mon, 06 Jun 2022 18:57:22 GMT

The macro processor will ignore macro code that is embedded in strings bounded by single quotes. Use double quote characters instead.

filename indata pipe "dir P:\Results\RFs\&d. /b";

Do not use the ANCIENT function CALL SYMPUT(). The only time you should EVER use that is if it is REQUIRED that the generated macro variable has leading and/or trailing spaces in it. Use the normal CALL SYMPUTX() function instead. And do not try to use an & character in the name of the macro variable.

call symputx("num_files_&d._rf",_n_);

Re: Macro for path and list of files

ballardw — Mon, 06 Jun 2022 18:57:28 GMT

You have an problem the macro variable &d will not resolve inside single quotes.

filename indata pipe 'dir P:\Results\RFs\&d./b'; call symput ('num_files_&d._rf',_n_);

As previously suggested: Options Mprint and show the entire log when you get error messages. Copy the text from the log, open a text box on the forum and PASTE the text .

Instead of looping over dataset that way look at the CALL EXECUTE statement in the data set syntax.

Or write code statements to a program file and use %include to execute the statements.

Re: Macro for path and list of files

PaigeMiller — Mon, 06 Jun 2022 19:06:47 GMT

In your previous thread, I and others specifically asked to see the log (all of it) after you turn on the option to debug macros. Please follow those instructions.

Your first question: you need double quotes, like this:

filename indata pipe "dir P:\Results\RFs\&d./b";

and

call symput ("num_files_&d._rf",_n_);

Re: Macro for path and list of files

Tom — Mon, 06 Jun 2022 19:10:05 GMT

Do all of the CSV files with similar names have the same structure? Same set of variables in the same order.

If so then you can just read them all at once into ONE dataset. This will eliminate your earlier problem where you were trying to modify the generated datasets.

Here is a sketch.

data rf_&d ;
  length fname filename $200;
  input "P:\Results\RFs\&d.\*" dsd truncover filename=fname;
  input @ ;
* Reset line counter and skip the header line ;
  if fname ne lag(fname) then do;
      number=0;
      delete ;
  end;
  filename = scan(fname,-1,'/\');
  number+1;
  input ..... ;
run;

Just complete the INPUT statement to read in the variables. Add a FORMAT statement if some of the variable need to have special formats attached to display properly, normally only needed for DATE, TIME or DATETIME values.

Re: Macro for path and list of files

JOL — Mon, 06 Jun 2022 19:15:12 GMT

First error is because your file path with the macro variable reference needs to be in double quotes.

1. filename indata pipe "dir P:\Results\RFs\&d./b";

Second error, looks like you are trying to create a single macro variable named NUM_FILES_&D._RF, based on your CALL SYMPUT statement and that is not proper naming. If you mean for the &D to resolve to create a macro variable NUM_FILES_ALG_RF, you must use double quotes on you CALL SYMPUT

2. call symput ("num_files_&d._rf",_n_);

Re: Macro for path and list of files

AllanBowe — Sun, 12 Jun 2022 13:43:08 GMT

You could do this without PIPE or XCMD, in a platform agnostic way - see: https://core.sasjs.io/mp__dirlist_8sas.html