Thanks to the SAS Community, I was able to create a Do loop that calls a macro and substitutes a text string, one at a time. Long story short, while it works great, when the # of iterations starts to climb, the iteration per second balloons. The root cause is that at the end of each iteration of the called macro, the resulting values are appended as a row in a results dataset. Reading/writing that dataset slows as its size increases. I think I can resolve this by splitting the input dataset (testcomb2) containing the text string into multiple smaller datasets. So, instead of 1.6 million rows to be run in a row, I could create 16 smaller datasets of 100,00 rows; each set of 100,000 would be a discrete completion of the Do loop. That way, the size of the results file never gets to big to bog down. I found some code that should accomplish splitting into distinct datasteps. I suppose I could simply repeat the code for the iteration loop 16 times, each time naming a different one of the smaller input sets and creating a results dataset with a slightly different name, but I am sure smarter people than me can suggest a more eloquent method, especially since it needs to be scalable (1.6 million this run, 3.2 million (32 sets of 100,00) the next, 500,000 (5 sets of 100,000) etc, the next. Perhaps, I don't need to split into smaller datasteps, simply tell the loop to process List Number 1-100,000, end the loop, then repeat the loop for 100,001 through 200,000, etc.... Or, perhaps I don't need to split either the input dataset, or the # of iterations in the loop. Rather, when the results dataset reaches 100,00 obs, it spits that out as finished and starts a new one (with a slightly different name counter) Here is the loop. **************************************************************************************;
** G-6: LOOPING MARCO **;
**************************************************************************************;
%ODSOff
%macro test;
proc sql noprint;
select count(*) into :n
from testcomb2
;
quit;
%do i=1 %to &n;
data _null_;
set testcomb2;
where list_number = &i ;
call symputx('changelist',combos2);
run;
%US13_TARGET_ANALYSIS;
%end;
%mend;
%test
%ODSOn
... View more