Question about using %include in a macro to include another macro

[Doxastic]

Very often when I use %include within a macro, to invoke another macro or a set of macros, when I execute the main macro stops running at the %include.

So, for instance.  If my macro looked like:

%macro Test();

     Data Output; set input;

          Var="TEST";

     run;

     %include "C:\user\Sub_Macros"

     Data Final_output; set Sub_Macros_Input;

          Var2="Finished";

     run;

%mend;

The data step for "Final_output" won't run.  The macro just ends at the %include....  it's really annoying.

[Tom]

In that case it is the missing semi-colon that is making the data step not run.

Did it also complain that it could not find the files named DATE and FINAL_OUTPUT ?

[Doxastic]

Adding the semicolon doesn't help....

It just stop executing after the %include.  There's no errors, it just stops.

[Tom]

Unbalanced quotes?

Or worse the include file includes and ENDSAS statement?

[Doxastic]

I have no idea if %include has ENDSAS built into it.... nor do I know how to figure that out.

[Reeza]

Does the %include run on its own? Without being inside a macro?

[Doxastic]

Yes.  %include runs on its own.  It also runs inside the macro.  SAS stops executing AFTER the %include.  So it runs that piece, it just stops after.

[Tom]

There is no need to define a macro inside the definition of another macro.  You are just asking the computer to do extra work since now it has to re-define the macro every time you run the outer macro.

%macro Test();

     Data Output; set input;

          Var="TEST";

     run;

     * Call SUB_MACRO ;

     %SUB_MACRO;

      Data Final_output; set Sub_Macros_Input;

          Var2="Finished";

     run;

%mend;

* Include definition of SUB_MACRO ;

%include "C:\user\Sub_Macros";

*  Call main macro;

%test;

[Doxastic]

I do place macros within macros all the time.  I don't like 2000 lines in a macro.... It's just confusing.  And there will often be pieces of a macro that themselves can be macro-tized for use in a different macro or just code in general.

I use SAS to write ETLs on flat files.  So I have really long SAS programs.  Putting everything in one macro would make some macros confusingly long....

On top of that, they won't be dynamic anymore.  Here's how I'm using %include (this time):

%macro Import(FilePath);

%get_filenames(&FilePath.);

Data rename_var(keep= Campaign_Names); set campaign_file_names;  Campaign_Names=VAR1; run;

data Camp_Total; set rename_var; count+1; run;

data Camp_Macro_Names(keep=Var); set Camp_Total;

   Var=cat('%let campy',count,'=',Campaign_Names,';');

run;

data _null_;

  set Camp_Macro_Names;

  FILE "E:\Shared\Progressive\Campaigns\Specify Campaigns_&today..txt" DLM=',';

  PUT Var;

run;

%include "E:\Shared\Progressive\Campaigns\Specify Campaigns_&today..txt";

proc import datafile="&FilePath.\&&campy&i...csv"

                 out=tempy&i

  dbms=dlm replace;

          delimiter=',';

      getnames = no;

  datarow=1;

run;

proc sql;

  create table camp&i as

  select a.* from tempy&i a

  where var1 ne .

     order by var2;

quit;

proc datasets; delete tempy&i; run; quit;

%end;

data campaigns_rerun(keep= HH_id  airing_date live_offset Network daypart); set

%read2

;

      rename var6 = HH_id;

      rename var4 = airing_date;

      rename var7 = live_offset;

   rename var16 = Network;

   rename var20 = daypart;

run;

%end;

%mend Import;

Everything after the %include won't run.  SAS stops executing after the %include and it doesn't error out. 

[Tom]

Sounds like a data issue.  You are using the values of a data step variable to write %LET statements. Most likely the content of the file contains something that should be macro quoted.

Note that you are not including any macro definitions here.  You are including some lines of code with %LET statements to define some macro variables.

And not even lines of code from some other code file, but lines of code generated from data.

Note that you can replace the four data steps and the %INCLUDE with a single data _NULL_ that uses CALL SYMPUTX to create the macro variables.

data _null_ ;

  set campaign_file_names;

  call symputx(cats('campy',_n_),var1);

run;

There are more issues after that also as you seem to want to loop of these macro variables since you later reference &&CAMPY&I but there is no macro logic to do the looping.

Also you are calling a macro in the middle of a SET statement.  Perhaps you meant to reference a macro variable instead at that point?

[Doxastic]

I'll try that again... but that's what I first tried and I couldn't get it working.  I'll get back in a bit to let you know if it works.

[Tom]

Looks like you just want to read all of the CSV files in a folder. You seem to be assuming they all have the same format, so why not just write the data step to read them instead of letting PROC IMPORT guess what is in the file?

From looking at your program it looks like something like this should work.

%macro Import(FilePath);

data campaigns_rerun ;

  length filename fname $256 ;

  infile "&FilePath\*.csv" dsd filename=fname truncover ;

  input skip1 @;

  if skip1=. then delete ;

  filename = fname ;

  length skip2-skip19 $200 airing_date 8 HH_id $10 live_offset $10

         Network $20 daypart $10

  ;

  informat airing_date anydtdte. ;

  format airing_date date9.;

  input skip2 skip3 airing_date skip5 HH_id live_offset

        skip8-skip15 Netowrk skip17-skip19 daypart

  ;

  drop skip: ;

run;

%mend Import;

[Doxastic]

Thank you all for your help, btw!

@Tom:  I didn't think to do it how you are doing it.  And honestly, I don't really understand it.  If I can get it working this way, I'd like to keep it just because I get it.

And I don't think the problem is with the %include anymore....  I have no idea.  I thought it was the %include because it's happened before.  But when I got your symput code working, and macro-tized it, it failed again.  Here's the code as it is now:

%macro Import_Campaigns(FilePath, ncampaigns, Data_type);

%get_filenames(&FilePath.); /* Gets file names from a directory and puts them into a SAS dataset */

%Create_Mac_Var(campy, memname);  /*  Tom's symput code in macro form ("campy" and "Var1" are macro variables)

/*******************   This is where thing seem to be going wrong.  Either in the step above or the step below.... idk  */

%if &Data_type.="csv" %then %do i=1 %to &ncampaigns.;

     proc import datafile="&FilePath.\&&campy&i.."

                 out=tempy&i

  dbms=dlm replace;

          delimiter=',';

      getnames = no;

  datarow=1;

     run;

proc sql;

  create table campy&i as

  select a.* from tempy&i a

  where var1 ne .

     order by var2;

     quit;

proc datasets; delete tempy&i; run; quit;

%end;

%if &Data_type.="pipe" %then %do i=&start. %to &stop.;

     proc import datafile="&FilePath.\&&camp&i...txt"

                 out=tempy&i

  dbms=dlm replace;

          delimiter='|';

      getnames = no;

  datarow=4;

     run;

proc sql;

  create table campy&i as

  select a.* from tempy&i a

  where var1 ne .

     order by var2;

     quit;

proc datasets; delete tempy&i; run; quit;

%end;

%if &Data_type.="csv" %then %do;

data campaigns_rerun(keep= HH_id  airing_date live_offset Network daypart); set

%read2

;

      rename var6 = HH_id;

      rename var4 = airing_date;

      rename var7 = live_offset;

   rename var16 = Network;

   rename var20 = daypart;

run;

%end;

%if &Data_type.="pipe" %then %do;

data campaigns_rerun(keep= HH_id  airing_date live_offset Network daypart); set

%read2

;

      rename var1 = HH_id;

      rename var2 = airing_date;

      rename var3 = live_offset;

   rename var4 = Network;

   rename var5 = daypart;

run;

%end;

%mend Import_Campaigns;

And here's the log for that entire length of code.....

23588  %Import_Campaigns(E:\Shared\Campaigns\temp\csv, 5, csv);

SYMBOLGEN:  Macro variable FILEPATH resolves to E:\Shared\Campaigns\temp\csv

SYMBOLGEN:  Macro variable LOCATION resolves to E:\Shared\Campaigns\temp\csv

MPRINT(GET_FILENAMES):   filename _dir_ "E:\Shared\Progressive\Campaigns\temp\csv";

MPRINT(GET_FILENAMES):   data campaign_file_names(keep=memname);

MPRINT(GET_FILENAMES):   handle=dopen( '_dir_' );

MPRINT(GET_FILENAMES):   if handle > 0 then do;

MPRINT(GET_FILENAMES):   count=dnum(handle);

MPRINT(GET_FILENAMES):   do i=1 to count;

MPRINT(GET_FILENAMES):   memname=dread(handle,i);

MPRINT(GET_FILENAMES):   output campaign_file_names;

MPRINT(GET_FILENAMES):   end;

MPRINT(GET_FILENAMES):   end;

MPRINT(GET_FILENAMES):   rc=dclose(handle);

MPRINT(GET_FILENAMES):   run;

NOTE: The data set WORK.CAMPAIGN_FILE_NAMES has 5 observations and 1 variables.

NOTE: Compressing data set WORK.CAMPAIGN_FILE_NAMES increased size by 100.00 percent.

      Compressed is 2 pages; un-compressed would require 1 pages.

NOTE: DATA statement used (Total process time):

      real time           0.00 seconds

      cpu time            0.00 seconds

MPRINT(GET_FILENAMES):   filename _dir_ clear;

NOTE: Fileref _DIR_ has been deassigned.

MPRINT(IMPORT_CAMPAIGNS):  ;

MPRINT(IMPORT_CAMPAIGNS):   data _null_ ;

MPRINT(IMPORT_CAMPAIGNS):   set campaign_file_names;

MPRINT(IMPORT_CAMPAIGNS):   call symputx(cats('campy',_n_),memname);

MPRINT(IMPORT_CAMPAIGNS):   run;

NOTE: There were 5 observations read from the data set WORK.CAMPAIGN_FILE_NAMES.

NOTE: DATA statement used (Total process time):

      real time           0.00 seconds

      cpu time            0.00 seconds

SYMBOLGEN:  Macro variable DATA_TYPE resolves to csv

SYMBOLGEN:  Macro variable DATA_TYPE resolves to csv

SYMBOLGEN:  Macro variable DATA_TYPE resolves to csv

SYMBOLGEN:  Macro variable DATA_TYPE resolves to csv

[RW9]

I would personally put your if statements in quotes and trim them e.g.:

%if %trim(&data_type)="csv".  Whats happening is the if statement is not true and hence none of the other code operates.

As Tom previously mentioned however, you don't need to do a lot of that code.  If you look at a datastep with an infile statement you can have multiple delimiters (there are examples on here of that).  The other code as well can also be trimmed.  Am leaving for the day, however if its still open I add some code.

[Doxastic]

This sounds interesting but I definitely don't understand how you'd do that