topic Re: folder reading by macro in SAS Programming

folder reading by macro

sahoositaram555 — Sun, 05 Apr 2020 16:36:16 GMT

Hi, I have 5 folders with different dates which have some sas datasets.

trying to read the folder so that later i can pick the sas dataset of m interest.

below code throws an errors.

ERROR: Argument 1 to function PUTN referenced by the %SYSFUNC or %QSYSFUNC macro function is not a number.
ERROR: Invalid arguments detected in %SYSCALL, %SYSFUNC, or %QSYSFUNC argument list. Execution of %SYSCALL statement or %SYSFUNC
or %QSYSFUNC function reference is terminated.

%let dt=%sysfunc(putn("2019-05-10"d, yymmdd10.));
%put &dt;

any ideas how to read folders with yyyy-mm-dd format to pick the sas datasets ?

Re: folder reading by macro

yabwon — Sun, 05 Apr 2020 16:49:57 GMT

Wrong date format, try:

%let dt=%sysfunc(putn("10may2019"d, yymmdd10.));
%put &dt;

SAS date literal is in the format "DDMmmYYYY"d

All the best

Bart

Re: folder reading by macro

sahoositaram555 — Sun, 05 Apr 2020 16:55:46 GMT

Hi @yabwon,
Folders i have are in YYYY-MM-DD eg: 2019-05-10. Is there any other way out to read the folder and pick the dataset of my interest or shall i convert the folder name to 10may2019 by applying date9.? Kindly suggest

Re: folder reading by macro

yabwon — Sun, 05 Apr 2020 16:59:49 GMT

Could you give some more detailed example? I'm not sure if I correctly understood your request.

Do you have folder like: "C:\my_folder\my_subfolder2019-05-10" and want to read data set from that folder?

Bart

Re: folder reading by macro

PaigeMiller — Sun, 05 Apr 2020 17:08:42 GMT

%let dt=%sysfunc(putn("10may2019"d, yymmddd10.));
%put &=dt;

produces the result 2019-05-10

that's what you want, right?

Re: folder reading by macro

sahoositaram555 — Sun, 05 Apr 2020 17:12:33 GMT

Hi @yabwon, Thanks for your concern. i have 6 folders "C:\my_folder\my_subfolder\2019-05-10" to "C:\my_folder\my_subfolder\2019-11-10" . Every folder contains some ****.sas7bdat datasets. I'm in the process of creating a macro to pick 2 .sas7bdat dataset of my interest from each folder which is required for merging in later stage.

so, i was trying to use %let dt=%sysfunc(putn("2019-05-10"d, yymmdd10.)) for start date to use it in do loop. but some how error comes up because the date is not being read by sas. Hope I'm clear.

Re: folder reading by macro

PaigeMiller — Sun, 05 Apr 2020 17:14:35 GMT

@sahoositaram555

did you even try running the code from @yabwon or myself above, it seems to give you exactly what you are asking for.

Re: folder reading by macro

yabwon — Sun, 05 Apr 2020 17:23:43 GMT

Hi,

try with this example:

/* generate test data in folders */

options dlcreatedir;
libname x "C:\my_folder\";
libname x "C:\my_folder\my_subfolder\";
libname x "C:\my_folder\my_subfolder\2019-05-10"; 
data x.class1; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-06-10";
data x.class2; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-07-10";
data x.class3; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-08-10";
data x.class4; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-09-10";
data x.class5; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-10-10";
data x.class6; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-11-10";
data x.class7; set sashelp.class; run;
libname x clear;


/* macro-loop to assign libraries */

%macro generateLibrary(start, number);

%do i = 1 %to &number.;
  %let dt=%sysfunc(intnx(month, &start.,%eval(&i. - 1) ,S), yymmdd10.);
  libname lib&i "C:\my_folder\my_subfolder\&dt.";
%end;

%mend generateLibrary;

%generateLibrary('10may2019'd, 7);

and adopt it to your needs

All the best

Bart

Re: folder reading by macro

Tom — Sun, 05 Apr 2020 17:25:29 GMT

To use a date literal, "xxxxxx"d, the string inside the quotes needs to be something that the DATE informat can understand. Your mistake is trying to use year month day style strings as date literals. If you already have the date in yymmdd10 style then you don't need to use the PUTN() function call at all.

%let dt=2019-05-10;
libname want "c:\my_folder\my_subfolder\&dt" ;

Re: folder reading by macro

PaigeMiller — Sun, 05 Apr 2020 17:33:23 GMT

Well, that's so simple its frightening. Why didn't I think of that?

Re: folder reading by macro

sahoositaram555 — Sun, 05 Apr 2020 17:44:59 GMT

Thank you both @PaigeMiller and @Tom for your thoughts .
I actually have them in YYYY-MM-DD format. The errors i was getting as i was using them in %let statement without quotes.
Thanks for making me clarify on this.

Re: folder reading by macro

sahoositaram555 — Sun, 05 Apr 2020 19:45:42 GMT

Hi @yabwon, Thank you for sharing a sample code. I should have mentioned it before that the difference isn't one. Though i know that in my last post i had mentioned 2019-05-10 to 2019-11-10, but it's not exactly with of equal difference(-1) between folders. The difference varies.
i understand the statement %let dt=%sysfunc(intnx(month, &start.,%eval(&i. - 1) ,S), yymmdd10.); will only work when difference of folders are only a month.
So, any other way of dealing this if the difference between folders not consistent,unlike 1 month as assumed?
And it throws error when the library names generated from the statement libname lib&i "C:\my_folder\my_subfolder\&dt."; is taken for any further proc process.
eg: instead of C:\my_folder\my_subfolder\ the sample i have is
"\\XXX-sas-zz1\my_folder\sub_folder. So when i use a procedure
proc sort data=lib.&i;by PatientInitials;run; it throws error as i have \\XXX-sas-zz1 and due to the "-" delimiters sas is unable to recognize the path and throws error.

Re: folder reading by macro

Tom — Sun, 05 Apr 2020 20:06:30 GMT

It is hard to give an answer when you haven't explained the actual problem.

If you know the folder names then just use them.

Are you saying you don't know the folder names and you need to try to figure them out?

Can't you just ask the operating system to tell you? If you are using Windows then use DIR command, if Unix then ls command. Read the output into a dataset and you know the folder names.

If you cannot do that (perhaps the NOXCMD option is set?) and you want to find all of the folders for a range of dates then just loop over that range of dates and check if the folder exists.

%do date=%sysevalf("01MAY2018"d) %to %sysevalf("01JAN2010"d);
  %let dt=%sysfunc(putn(&date,yymmdd10.));
  %let path=c:\myfolders\&dt ;
  %if %sysfunc(fileexist("&path")) %then %do;
    .... what ever you need to do when you find one ...
  %end;
%end;

Re: folder reading by macro

yabwon — Mon, 06 Apr 2020 08:22:57 GMT

Hi @sahoositaram555

1) You can create a list of subFolders like below:

/* generate test data in folders */

options dlcreatedir;
libname x "C:\my_folder\";
libname x "C:\my_folder\my_subfolder\";
data x.class01 x.class02 x.class03; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-05-10"; 
data x.class1; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-06-10";
data x.class2; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-07-10";
data x.class3; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-08-10";
data x.class4; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-09-10";
data x.class5; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-10-10";
data x.class6; set sashelp.class; run;
libname x "C:\my_folder\my_subfolder\2019-11-10";
data x.class7; set sashelp.class; run;
libname x clear;

/* extract the list of subdirectories */

data exactFolderNames;
   length exactFolderName $ 512;
   keep exactFolderName;
   base = "C:\my_folder\my_subfolder";

   rc=filename("_TMP_", base);
   did=dopen("_TMP_"); 

   do i = 1 to dnum(did);
        subfolder = dread(did, i);
        
        rc = filename("_sTMP_", catx("/", base, subfolder));
        subdid = dopen("_sTMP_");
        if subdid = 0 then continue; /* to ignore files */
 
        /* put (_all_) (=/) / ;*/
        exactFolderName = dinfo(subdid, "Directory");
        output;
        rc = dclose(subdid);
        rc = filename("_sTMP_");
   end;

   did=dclose(did);
   rc=filename("_TMP_");
run;

All the best

Bart

Re: folder reading by macro

Kurt_Bremser — Mon, 06 Apr 2020 13:18:28 GMT

@sahoositaram555 wrote:

Hi, I have 5 folders with different dates which have some sas datasets.

trying to read the folder so that later i can pick the sas dataset of m interest.

below code throws an errors.

ERROR: Argument 1 to function PUTN referenced by the %SYSFUNC or %QSYSFUNC macro function is not a number.
ERROR: Invalid arguments detected in %SYSCALL, %SYSFUNC, or %QSYSFUNC argument list. Execution of %SYSCALL statement or %SYSFUNC
or %QSYSFUNC function reference is terminated.

%let dt=%sysfunc(putn("2019-05-10"d, yymmdd10.));
%put &dt;

any ideas how to read folders with yyyy-mm-dd format to pick the sas datasets ?

You do not need to read the directory to get the names of SAS datasets. Assign a library to the directory and query DICTIONARY.TABLES (or SAHELP.VTABLE).

To see what your problem is, and how it can be avoided, see this:

%let dt=%sysfunc(putn("10may2019"d, yymmdd10.)); 
%put &dt;
%let dt=%sysfunc(putn(%sysfunc(inputn(2019-05-10,yymmdd10.)),yymmddd10.)); 
%put &dt;
%let dt=2019-05-10;
%put &dt.;

All three assignments create the same result.