Macro String getting too long

[david27]

Hello,

I have this:

/*Have*/
%let output=(n1h2_output*n1h2-b3) + (n2h2_output*n2h2-b3);
%let n1h2=(n1h1_n1h2*n1h1-b2) + (n2h1_n1h2*n2h1-b2);
%let n2h2=(n1h1_n2h2*n1h1-b2) + (n2h1_n2h2*n2h1-b2);
%let n1h1=(a_n1h1*a-b1);
%let n2h1=(a_n2h1*a-b1);

%do h=2 %to 1 %by -1;
  %do n=1 %to 2;
    %let output=%sysfunc(tranwrd(&output.,*n&n.h&h.-,*(&&n&n.h&h..)-));
  %end;
%end;

The problem is, the macro string "output" gets too long.

I provided a short example here but for me the macro string goes longer than 65K characters.

Can someone please advise me to code in more efficient/better way?

Thank You

[Oligolas]

Hi,

you could split the variable or write the code to a file

I'll recommend you to use a dataset to store the results rather than the formulas in macro variables.

[Shmuel]

It seems that you have entered an endless loop, assigning OUTPUT using loop with &OUTPUT:

%let output=(n1h2_output*n1h2-b3) + (n2h2_output*n2h2-b3);
   /*======*/
%let n1h2=(n1h1_n1h2*n1h1-b2) + (n2h1_n1h2*n2h1-b2);
%let n2h2=(n1h1_n2h2*n1h1-b2) + (n2h1_n2h2*n2h1-b2);
%let n1h1=(a_n1h1*a-b1);
%let n2h1=(a_n2h1*a-b1);
%do h=2 %to 1 %by -1;
  %do n=1 %to 2;
    %let output=%sysfunc(tranwrd(&output.,*n&n.h&h.-,*(&&n&n.h&h..)-));
       /*=====*/               /*======*/
  %end;
%end;

Beyond, it is probably a part of a macro program. 

What are you trying to do?

[david27]

It basically helps me create a big formula. Longer than 65K.

Yes i am using output from previous output. But its not infinite loop it has boundaries.

[Shmuel]

try to replace line:

%let output=%sysfunc(tranwrd(&output.,*n&n.h&h.-,*(&&n&n.h&h..)-));

with:

%let output=%sysfunc(tranwrd(strip(&output.),*n&n.h&h.-,*(&&n&n.h&h..)-));

If it doesn't help  replace the double %DO loop with a datastep, writing the formula to file 

(like formula.sas) and then do %include "<path>/formula.sas"; in your program.

next may be a help to create the formula without the loops:

73         %let n1h2 = A;
 74         %let n2h2 = B;
 75         %let n1h1 = C;
 76         %let n2h1 = D;
 77         %macro m;
 78         %do h=2 %to 1 %by -1;
 79           %do n=1 %to 2;
 80             %put *n&n.h&h.- , *(&&n&n.h&h..)-));
 81           %end;
 82         %end;
 83         %mend;
 84         %m;
 *n1h2- , *(A)-))
 *n2h2- , *(B)-))
 *n1h1- , *(C)-))
 *n2h1- , *(D)-))

enter the right values instead A B C D into TRANWRD function and STRIP(each_part);  

you need just 4 lines without loops.

[Shmuel]

My first option solves the problem:

72         
 73         %macro m;
 74         %let output=(n1h2_output*n1h2-b3) + (n2h2_output*n2h2-b3);
 75         %let n1h2=(n1h1_n1h2*n1h1-b2) + (n2h1_n1h2*n2h1-b2);
 76         %let n2h2=(n1h1_n2h2*n1h1-b2) + (n2h1_n2h2*n2h1-b2);
 77         %let n1h1=(a_n1h1*a-b1);
 78         %let n2h1=(a_n2h1*a-b1);
 79         %do h=2 %to 1 %by -1;
 80           %do n=1 %to 2;
 81             %let output=%sysfunc(tranwrd(strip(&output.),*n&n.h&h.-,*(&&n&n.h&h..)-));
 82           %end;
 83         %end;
 84         %mend;
 85         %m;
 86         %put OUTPUT=&output;
 OUTPUT=strip(strip(strip(strip((n1h2_output*((n1h1_n1h2*((a_n1h1*a-b1))-b2) + (n2h1_n1h2*((a_n2h1*a-b1))-b2))-b3) + 
 (n2h2_output*((n1h1_n2h2*((a_n1h1*a-b1))-b2) + (n2h1_n2h2*((a_n2h1*a-b1))-b2))-b3)))))
 87  

[david27]

Thank You @Shmuel  for helping me here.

But the solution you provided is quite similar to what I have(except for the strip function you used.)

The solution that you are suggesting will not work for long strings.

The macro "H" can start from 10 and the macro "N" could go upto 20. As a result the string gets too long.

I think we have to find a way without creating output in an iterative fashion and put it out in an external file, which I can later do an include upon(something like formula.sas as per your suggestion)

[Shmuel]

You iterate just 2X2=4 times by loops.

Check the length after each iteration by

   %let len = %length(&output); %put LAN=&len;

and check log, does it realy flows out of max possible length; 

[Tom]

So instead of changing the references to NnHh to the value of &NnHh just change it to a reference instead. Then your macro variables will only grow by one character per reference.  Note that you need to change the references in ALL of the macro variables, not just in OUTPUT macro variable.

%let output=(n1h2_output*n1h2-b3) + (n2h2_output*n2h2-b3);
%let n1h2=(n1h1_n1h2*n1h1-b2) + (n2h1_n1h2*n2h1-b2);
%let n2h2=(n1h1_n2h2*n1h1-b2) + (n2h1_n2h2*n2h1-b2);
%let n1h1=(a_n1h1*a-b1);
%let n2h1=(a_n2h1*a-b1);

data _null_ ;
  length mvar $32 ;
  call symputx('output',tranwrd(symget('output'),'*n','*&n'));
  do h=2 to 1 by -1;
    do n=1 to 2;
      mvar=cats('n',n,'h',h);
      call symputx(mvar,tranwrd(symget(mvar),'*n','*&n'));
    end;
  end;
run;

%put N1H2=%superq(n1h2);
%put N2H2=%superq(n2h2);
%put N1H1=%superq(n1h1);
%put N2H1=%superq(n2h1);
%put OUTPUT=%superq(output);


%put OUTPUT=&output;

Compare the raw and evaluated versions and notice the difference in length.

N1H2=(n1h1_n1h2*&n1h1-b2) + (n2h1_n1h2*&n2h1-b2)

N2H2=(n1h1_n2h2*&n1h1-b2) + (n2h1_n2h2*&n2h1-b2)

N1H1=(a_n1h1*a-b1)

N2H1=(a_n2h1*a-b1)

OUTPUT=(n1h2_output*&n1h2-b3) + (n2h2_output*&n2h2-b3)

OUTPUT=(n1h2_output*(n1h1_n1h2*(a_n1h1*a-b1)-b2) + (n2h1_n1h2*(a_n2h1*a-b1)-b2)-b3) + (n2h2_output*(n1h1_n2h2*(a_n1h1*a-b1)-b2) + (n2h1_n2h2*(a_n2h1*a-b1)-b2)-b3)

Note you might need to use regular expression instead of TRANWRD() to find and replace the variable references.

[Tom]

Here is RegEx version.  Replaces patterns that have *NdHd- with *(&NdHd)-.

data _null_ ;
  length mvar $32 ;
  rexid = prxparse('s/\*(n\d+h\d+)-/\*(&$1)-/i');
  do h=2 to 1 by -1;
    do n=1 to 2;
      mvar=cats('n',n,'h',h);
      call symputx(mvar,prxchange(rexid,-1,symget(mvar)));
    end;
  end;
  call symputx('output',prxchange(rexid,-1,symget('output')));
run;

[david27]

Thank You very much @Tom  @Shmuel @Oligolas  for helping me here.

But I think, Shell Scripting is the solution here:

The reason for that is, eventually the macro string output is going to be longer than 65K

data _null_ ;
file "/code/location/on/server/formula.sas";
put  "output=" "&output.";
run;

%do h=10 %to 1 %by -1;
%do n=1 %to 20;
	systask 
		command "sed -i 's/*n&n.h&h.-/*(&&n&n.h&h..)-/g' /code/location/on/server/formula.sas"
		wait
		status=status
		shell;
	%put &=status.;
%end;
%end;

[Tom]

Note that using the nested macro variable references I showed the result of evaluating &OUTPUT can be longer than 64K even when the actual length of the macro variable value is not.

I don't really see how calling SED is going to help, but then I don't understand what you are trying to do.  It looks like you are generating a formula.  So you might use the macro variable to place that formula into some SAS code like this:

data want;
  set have ;
  new_var = &output ;
run;

Do you really expect to execute a formula that is longer than 64K?

[Shmuel]

May be the solution is not by creating a huge macro variable but  by splitting it into a set of formulas,

creating something like:

v_tmp1 = &formula1;
v_tmp2 = v_tmp1 * &formula2;
v_tmp3 = v_tmp2 * &formula3;
...etc...