topic Re: Need help trouble shooting program to produce multiple folders with a macro. in SAS Programming

Need help trouble shooting program to produce multiple folders with a macro.

madhatter — Mon, 18 May 2015 10:13:52 GMT

My goal is to create A program that will create all the macros and File folders that are required for a given project, with minimal upfront coding. I have multiple annual projects that I will be doing this for and most of these projects use the same data sources and formats each year; so my ultimate goal is to use one macro to create all the variables and report names.

My current issue ( I am sure I will have more before I am through) is that I do have enough knowledge of macros to identify why my program is not producing the correct file folders. The SASA program and log are below (. . Before I ran this program. I checked that any folders that I was trying to created DID NOT exist; this included deleting the files the program was creating in the wrong directory; exited SAS; and started it new.

The log indicates hat The directory "C:\PROG EVAL 2017" exists (But it did not and does not); and that C:\PROG EVAL 2017\Data and C:\PROG EVAL 2017\Data\GFU. were created BUT they were not. The C:\CPSASLIB\PROGEVAL2017 folder was created and the LIBREF was assigned correctly. What the program did do is create the following folders C:\PROG EVAL 2015\PE HEAD\2017; C:\PROG EVAL 2015\PE HEAD\2017\DATA; C:\PROG EVAL 2015\PE HEAD\2017\DATA\GFU and C:\PROG EVAL 2015\PE HEAD\EVAL.

Any help will be appreciated.

Thanks

LOG

NOTE: SAS (r) Proprietary Software 9.3 (TS1M2)

Licensed to STATE BOARD FOR TECH & COMP EDU - ADMIN, Site 70049135.

NOTE: This session is executing on the W32_7PRO platform.

NOTE: Enhanced analytical products:

SAS/STAT 12.1, SAS/ETS 12.1, SAS/OR 12.2, SAS/IML 12.1, SAS/QC 12.1

NOTE: SAS initialization used:

real time 1.07 seconds

cpu time 0.59 seconds

1 /*THIS IS A PROGRAM THAT ESTABLISHES THE MACROS THAT WILL BE USED FOR THE PROGRAM

2 EVALUATION SAS PROGRAMS*/

4 /*Change The PE_TERM Value To The Starting Summer Term*/

5 %let PE_TERM=20143;

7 /*****************************************************/

9 /***************FIXED MACROS DO NOT CHANGE*******************/

10 %LET PE_YR=%SYSFUNC(INT(%EVAL(&PE_TERM/10)))+3;/*Resolves to the year the eval is conducted

10 ! 2015*/

11 %LET ST_YR=%sysfunc(MOD(%sysfunc(INT(%EVAL(&&PE_TERM/10))),100))+1;/*resolves to the 2-Digit

11 ! start yrr for the grads*/

12 %LET JK=%eval(&st_yr)+2; /*resolves to the correct YR for the libname*/

14 /*****************************************************/

16 OPTIONS NONUMBER NODATE NOBYLINE mprint mlogic SYMBOLGEN;

17 TITLE;

19 /*CREATES the MACRO FOR MAIN FOLDER*/

20 %Let PXDIR=C:\PROG EVAL %EVAL(&PE_YR);

SYMBOLGEN: Macro variable PE_YR resolves to 2014+3

22 /*CREATES MACRO FOR SAS LIBRARY FOLDER PATH. */

23 %Let fpath=C:\CPSASLIB\PROGEVAL%EVAL(&PE_YR);

SYMBOLGEN: Macro variable PE_YR resolves to 2014+3

24 /*****************************************************/

25 /*CREATES THE NEW PE FOLDER AND SUBFOLDERS*/

26 %macro chk_dir(dir=) ;

27 options noxwait;

28 %local rc fileref ;

29 %let rc = %sysfunc(filename(fileref,&dir)) ;

30 %if %sysfunc(fexist(&fileref)) %then

31 %put NOTE: The directory "&dir" exists ;

32 %else

33 %do ;

34 %sysexec md &dir ;

35 %put %sysfunc(sysmsg()) The directory has been created. ;

36 %end ;

37 %let rc=%sysfunc(filename(fileref)) ;

39 %mend chk_dir ;

41 %chk_dir(DIR=&PXDIR) ;

MLOGIC(CHK_DIR): Beginning execution.

SYMBOLGEN: Macro variable PXDIR resolves to C:\PROG EVAL 2017

MLOGIC(CHK_DIR): Parameter DIR has value C:\PROG EVAL 2017

MPRINT(CHK_DIR): options noxwait;

MLOGIC(CHK_DIR): %LOCAL RC FILEREF

MLOGIC(CHK_DIR): %LET (variable name is RC)

SYMBOLGEN: Macro variable DIR resolves to C:\PROG EVAL 2017

SYMBOLGEN: Macro variable FILEREF resolves to #LN00012

MLOGIC(CHK_DIR): %IF condition %sysfunc(fexist(&fileref)) is TRUE

MLOGIC(CHK_DIR): %PUT NOTE: The directory "&dir" exists

SYMBOLGEN: Macro variable DIR resolves to C:\PROG EVAL 2017

NOTE: The directory "C:\PROG EVAL 2017" exists

MLOGIC(CHK_DIR): %LET (variable name is RC)

MLOGIC(CHK_DIR): Ending execution.

42 %chk_dir(DIR=&PXDIR\DATA);

MLOGIC(CHK_DIR): Beginning execution.

SYMBOLGEN: Macro variable PXDIR resolves to C:\PROG EVAL 2017

MLOGIC(CHK_DIR): Parameter DIR has value C:\PROG EVAL 2017\DATA

MPRINT(CHK_DIR): options noxwait;

MLOGIC(CHK_DIR): %LOCAL RC FILEREF

MLOGIC(CHK_DIR): %LET (variable name is RC)

SYMBOLGEN: Macro variable DIR resolves to C:\PROG EVAL 2017\DATA

SYMBOLGEN: Macro variable FILEREF resolves to #LN00013

MLOGIC(CHK_DIR): %IF condition %sysfunc(fexist(&fileref)) is FALSE

MLOGIC(CHK_DIR): %SYSEXEC md &dir

SYMBOLGEN: Macro variable DIR resolves to C:\PROG EVAL 2017\DATA

MLOGIC(CHK_DIR): %PUT %sysfunc(sysmsg()) The directory has been created.

WARNING: Physical file does not exist, C:\PROG EVAL 2017\DATA. The directory has been created.

MLOGIC(CHK_DIR): %LET (variable name is RC)

MLOGIC(CHK_DIR): Ending execution.

43 %chk_dir(DIR=&PXDIR\GFU);

MLOGIC(CHK_DIR): Beginning execution.

SYMBOLGEN: Macro variable PXDIR resolves to C:\PROG EVAL 2017

MLOGIC(CHK_DIR): Parameter DIR has value C:\PROG EVAL 2017\GFU

MPRINT(CHK_DIR): options noxwait;

MLOGIC(CHK_DIR): %LOCAL RC FILEREF

MLOGIC(CHK_DIR): %LET (variable name is RC)

SYMBOLGEN: Macro variable DIR resolves to C:\PROG EVAL 2017\GFU

SYMBOLGEN: Macro variable FILEREF resolves to #LN00014

MLOGIC(CHK_DIR): %IF condition %sysfunc(fexist(&fileref)) is FALSE

MLOGIC(CHK_DIR): %SYSEXEC md &dir

SYMBOLGEN: Macro variable DIR resolves to C:\PROG EVAL 2017\GFU

MLOGIC(CHK_DIR): %PUT %sysfunc(sysmsg()) The directory has been created.

WARNING: Physical file does not exist, C:\PROG EVAL 2017\GFU. The directory has been created.

MLOGIC(CHK_DIR): %LET (variable name is RC)

MLOGIC(CHK_DIR): Ending execution.

44 %CHK_DIR(DIR=C:\CPSASLIB\PROGEVAL%EVAL(&PE_YR));

MLOGIC(CHK_DIR): Beginning execution.

SYMBOLGEN: Macro variable PE_YR resolves to 2014+3

MLOGIC(CHK_DIR): Parameter DIR has value C:\CPSASLIB\PROGEVAL2017

MPRINT(CHK_DIR): options noxwait;

MLOGIC(CHK_DIR): %LOCAL RC FILEREF

MLOGIC(CHK_DIR): %LET (variable name is RC)

SYMBOLGEN: Macro variable DIR resolves to C:\CPSASLIB\PROGEVAL2017

SYMBOLGEN: Macro variable FILEREF resolves to #LN00015

MLOGIC(CHK_DIR): %IF condition %sysfunc(fexist(&fileref)) is FALSE

MLOGIC(CHK_DIR): %SYSEXEC md &dir

SYMBOLGEN: Macro variable DIR resolves to C:\CPSASLIB\PROGEVAL2017

MLOGIC(CHK_DIR): %PUT %sysfunc(sysmsg()) The directory has been created.

WARNING: Physical file does not exist, C:\CPSASLIB\PROGEVAL2017. The directory has been created.

MLOGIC(CHK_DIR): %LET (variable name is RC)

MLOGIC(CHK_DIR): Ending execution.

SYMBOLGEN: Macro variable JK resolves to 15+2

46 /*CREATE THE NEW THE LIB REF*/

48 LIBNAME PE%EVAL(&JK) "&fpath";

SYMBOLGEN: Macro variable FPATH resolves to C:\CPSASLIB\PROGEVAL2017

NOTE: Libref PE17 was successfully assigned as follows:

Engine: V9

Physical Name: C:\CPSASLIB\PROGEVAL2017

50 proc format;

50 ! value BGRAD 0-5= '#e13300'

51 6-1000= 'BLACK';

NOTE: Format BGRAD has been output.

52 RUN;

NOTE: PROCEDURE FORMAT used (Total process time):

real time 0.06 seconds

cpu time 0.03 seconds

Re: Need help trouble shooting program to produce multiple folders with a macro.

RW9 — Mon, 18 May 2015 10:27:05 GMT

Hi,

Couple of things from my side:

1) If this is setting up a directory structure and file, my recommendation would be that you have IT do this. They would copy the necessary information across, and assign access rights per the study. This would be the ideal process, and would always be my preferred approach. Systems setup should be done and controlled by those best in the know. Also, there should be version control setup there etc. so not just a matter of dirs and files.

2) If the above is not possible, then you will have to take on the role of administrator yourself. Some extra work. Create a folder structure, add in the files. Then when a new project comes in, copy that template across. Of course you would need to setup version control etc. assign access rights and such like yourself.

These would be the better, and worse case scenarios. I wouldn't in any way do this through SAS, especially with your quote " I do have enough knowledge of macros to". Your IT guys should be able to do this pretty easily as they will have the tools, for instance 6 months down the line if you want to move folders, add new ones, allow other access to certain folders etc. then your back to stage one. They have the tools to make changes and cascade down across a fixed structure.

Some other things to remember, where is your SAS installed? If its on a server then the C: drive will not be your local C: drive. I would also avoid spaces and special characters in your path setup. As for your macro, if you want to debug it then write the program do it it once. Work on small parts of it, see what is happening, use options symblogen, mprint, mlogic.

Re: Need help trouble shooting program to produce multiple folders with a macro.

madhatter — Mon, 18 May 2015 22:08:47 GMT

Correct, this is setting up directories and files , SAS 9.3. on a PC, so IT is not going to do it. And copying and renaming folders and files is so lame. I am setting up my major recurring Semester Based studies to run with ONE or TWO macros that set the parameters for the data acquisition from the various sources; the parameters and variables names for the analysis; the titles, column headers, etc. for the reports; and the output file name for the reports. I have been fairly successful with that and getting better. My problem with the macros in my example program for creating the actual folders and subfolders I need is that I do not understand why my SAS LOG log states "SYMBOLGEN: Macro variable DIR resolves to C:\PROG EVAL 2017\GFU/MLOGIC(CHK_DIR): %PUT %sysfunc(sysmsg()) The directory has been created.WARNING: Physical file does not exist, C:\PROG EVAL 2017\GFU. The directory has been created." But the directory is not created; at least where it is suppose to be created. What was created were folders named: EVAL, 2017\GFU, and 2017\DATA in the C:\PROG EVAL 2015\PE HEAD folder, which is the folder where this SAS program resides. I guess that I can take the cheap way and create SAS ref lib folders and then clear the ref lib name; but was hoping would be able to see the error(s) with my macros.

rlWy successful with that

Re: Need help trouble shooting program to produce multiple folders with a macro.

Reeza — Tue, 19 May 2015 02:06:45 GMT

Around Line 23 - is this what you were expecting?

SYMBOLGEN:  Macro variable PE_YR resolves to 2014+3

Also, I think there's a simpler way with option dlcreatedir

http://blogs.sas.com/content/sasdummy/2013/07/02/use-dlcreatedir-to-create-folders/

And/Or using DCREATE function rather than system commands.

Re: Need help trouble shooting program to produce multiple folders with a macro.

Tom — Tue, 19 May 2015 04:02:31 GMT

Not really much in the way of a macro here, but perhaps you mean you are attempting to create macro VARIABLES ?

Your code at the top is way too complicated and you are creating macro variables with plus signs in them. Don't you want the macro variables to just have 2 or 4 digit year values? Then you can use them much more easily in the rest of the program without all of those %EVAL() calls. Since it looks like we are just talking about integer math here you can do that very easily in %LET statements. Note that you might have issues with leading zeros on the 2 digit years if this program is still running in another 80 years.

%let PE_TERM=20143;

PE_YR - 4 digit year that evaluation is conducte

ST_YR - 2 digit start year for the grads

JK - 2 digit year to use in libref

%let pe_yr=%eval(%substr(&pe_term,1,4)+3);

%let st_yr=%eval(%substr(&pe_term,3,2)+1);

%let jk=%eval(&st_yr+2);

PE_TERM=20143 PE_YR=2017 ST_YR=15 JK=17

Not sure what the rest of the program is trying to do.

Looks like you want to create some directories and at least one of them you want point a libref to.

What do you want to do with the others? Are you going to point filerefs to them? Or other librefs?

Do you also need the paths to be stored in macro variables or are the LIBREF/FILEREF definitions enough? How is the down stream program going to reference the environment?

So here is macro that will take as input the 4 digit year. It then use that four digit year to make two macro variables and four directories. Plus it will make a LIBREF using the last two digits of the year.

%macro make_dirs(yr);

%global PXDIR FPATH;

%let pxdir=C:\PROG EVAL &yr;

%let fpath=C:\CPSASLIB\PROGEVAL&yr;

data _null_;

rc=system(catx(' ','md',quote("&pxdir")));

rc=system(catx(' ','md',quote("&pxdir\DATA")));

rc=system(catx(' ','md',quote("&pxdir\GFU")));

rc=system(catx(' ','md',quote("&yr")));

run;

libname PE%substr(&yr,3,2) "&fpath" ;

%mend make_dirs ;

%make_dirs(&pe_yr);

Re: Need help trouble shooting program to produce multiple folders with a macro.

Kurt_Bremser — Tue, 19 May 2015 06:28:34 GMT

Seems to me the simplest solution will be this:

Do_not_use_blanks_in_file_or_directory_names.

Re: Need help trouble shooting program to produce multiple folders with a macro.

RW9 — Tue, 19 May 2015 07:51:12 GMT

Oh, so this isn't a commercial thing, just something for yourself. Then that is different. I would still recommend creating a template and then copy/pasting it - even if its just you (think how much time you have taken to create the macro), setup a version control system - TortoiseSVN if your on windows is easy, then you can copy through that. Otherwise you will lose something eventually. Also, all my other points excluding IT are still valid.

+1 to KurtBremser. Its never recommended to have spaces (or specials) in filename/paths at any time.

Re: Need help trouble shooting program to produce multiple folders with a macro.

madhatter — Wed, 20 May 2015 23:24:59 GMT

Tom. What you provided was most helpful. I am applying this to another current annual projects, ; One tracks 6 student cohorts (Fall 2008...Fall 2013) thru the 2014 fall term, reporting how many were retained/transferred/graduated in each intervening term; next year the 2014 cohort will be added; then 2015, ad infinitum. This project's directory structure is: a main folder (year); subfolders for each cohort; and six sub-folders for each cohort. Office policy dictates all project data/programs be maintained in a project folder on the server. Copying and editing last years 30 folders, files & program names and then editing the program fileref names appropriately, as was previously done, would been quicker than developing this process using macros....this year. But extrapolating the copy & edit time for future years & the increasing number of cohorts, no way. And I have learned a lot. After I finish this year's reports, I'm going to do a program to create the folders for the new year and copy the requisite SAS program files from the past year, all correctly named; and do it with an array, Do Loop, and macros; I'm sure you'll see other questions from me here in the future as I stumble along. Again, thanks for taking the time to offer some very useful information/guidance rather than pontificate;, it is appreciated.