topic Re: Iterative procedure in New SAS User

Iterative procedure

Thi_Oliveira — Mon, 27 Jul 2020 13:12:50 GMT

Good Morning, my name is Thiago and I'm trying to do an interactive procedure to calculate real base and ceiling temperatures of beta model in tree growth.

I used a procedure described in an article (image below), however I do not understand the way it is done, since my SAS code is showing errors.

SAS code:

data temp;
k=(Tc-To)/(To-Tb);
l=(0.05**2)*(To-Tb)*((Tc-To)**k);
do i=0 to 10;
L[i+1]=l/(((Tc-Tb)**(k+1))*((1-L[i])**k));
U[i+1]=(l/(((Tc-Tb)**(k+1))*((1-U[i])))**(1/k);
output;
end;

My data "temp" has the To, Tb and Tc values for the iteractive procedure.

Someone could help about that?

Thanks

Re: Iterative procedure

PaigeMiller — Mon, 27 Jul 2020 13:22:59 GMT

The obvious error in your code is that you have not assigned values to variables Tc To or Tb.

But for your future reference, you can't just simply say "SAS code is showing errors". You MUST show us the log from SAS, with nothing chopped out, so that we can see the code in the log plus all WARNINGs NOTEs and ERRORs for the step that has the problem. When providing the log, it is critical that you maintain the formatting of the log so we can see it exactly as SAS showed it to you, making it easier for us to use. To maintain the formatting of the log, click on the </> icon and paste the log as text into the window that appears. DO NOT SKIP THIS STEP.

Re: Iterative procedure

Thi_Oliveira — Mon, 27 Jul 2020 13:29:41 GMT

176  data temp;
177  k=(Tc-To)/(To-Tb);
178  l=(0.05**2)*(To-Tb)*((Tc-To)**k);
179  do i=0 to 10;
180  L[i+1]=l/(((Tc-Tb)**(k+1))*((1-L[i])**k));
ERROR: Undeclared array referenced: l.
ERROR: Undeclared array referenced: l.
ERROR: Variable l has not been declared as an array.
ERROR: Variable l has not been declared as an array.
181  U[i+1]=(l/(((Tc-Tb)**(k+1))*((1-U[i])))**(1/k);
                                                   -
                                                   79
ERROR: Undeclared array referenced: U.
ERROR: Undeclared array referenced: U.
ERROR: Variable U has not been declared as an array.
ERROR 79-322: Expecting a ).

182  output;
ERROR: Variable U has not been declared as an array.
183  end;

I'm sorry, but I forgot to share the SAS log.

Thanks PaigeMiler.

Re: Iterative procedure

PaigeMiller — Mon, 27 Jul 2020 14:23:57 GMT

In addition to the fact that I mentioned that you have not assigned values to the variables, you also need ARRAY statements to declare that L and U are arrays.

Re: Iterative procedure

Thi_Oliveira — Mon, 27 Jul 2020 14:29:44 GMT

It will be something like this:

data temp;
k=(Tc-To)/(To-Tb);
l=(0.05**2)*(To-Tb)*((Tc-To)**k);
array L{i+1}=l/(((Tc-Tb)**(k+1))*((1-L{i}**k));
do i=0 to 10 by 1;
end;
run;

The error in log still occured:

269  data temp;
270  k=(Tc-To)/(To-Tb);
271  l=(0.05**2)*(To-Tb)*((Tc-To)**k);
272  array L{i+1}=l/(((Tc-Tb)**(k+1))*((1-L{i}**k));
           -
           124
             -
             22
             76
ERROR 124-185: The variable L has already been defined.

ERROR 22-322: Syntax error, expecting one of the following: an integer constant, *.

ERROR 76-322: Syntax error, statement will be ignored.

273  do i=0 to 10 by 1;
274  end;
275  run;

NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set WORK.TEMP may be incomplete.  When this step was stopped there were 0
         observations and 6 variables.
WARNING: Data set WORK.TEMP was not replaced because this step was stopped.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.01 seconds

When you said to assign values is such as initial values? Like L=0 .......

Could you help how to write this code?

Re: Iterative procedure

ballardw — Mon, 27 Jul 2020 14:40:24 GMT

Pick a simpler formula, provide values for the variables like Tc Tb To and show us what you expect to get for output from that data step for like 3 observations.

I suspect that you want to retain L and U and OUTPUT new versions in the loop instead of attempting to create 11 variables whose names start with L and U

This may give you an idea of how to create multiple records.

data junk;
   Retain l 5;
   do i=0 to 5;
      output; /*being at the start of the loop the value
              of L going in is output*/
      l= l*5;
   end;
run;

The [ ] in a data step are used only for array references.

Re: Iterative procedure

PaigeMiller — Mon, 27 Jul 2020 14:48:25 GMT

If someone said to you: "I have a formula k=(Tc-To)/(To-Tb), can you tell me what is the value of k?", what would your first question be?

Re: Iterative procedure

Thi_Oliveira — Mon, 27 Jul 2020 15:03:01 GMT

Thanks for the answer.

The "k" and "l" are just index to converge L and U in the iterative process. Such as the steps in image below:

Because the model sometimes provide exagered Tb(base or minimum) and Tc(ceiling or maximum) temperatures, and this iteration process try to solve this problem by calculting L and U for recalculted Tb and Tc.

In the process above To is optimum temperature.

Re: Iterative procedure

Thi_Oliveira — Mon, 27 Jul 2020 16:22:42 GMT

Thanks for the answer.

The "k" and "l" are just index to converge L and U in the iterative process. Such as the steps in image below:

In the process above To is optimum temperature.

Re: Iterative procedure

PaigeMiller — Mon, 27 Jul 2020 16:36:42 GMT

@Thi_Oliveira wrote:

Thanks for the answer.

The "k" and "l" are just index to converge L and U in the iterative process. Such as the steps in image below:

Because the model sometimes provide exagered Tb(base or minimum) and Tc(ceiling or maximum) temperatures, and this iteration process try to solve this problem by calculting L and U for recalculted Tb and Tc.

In the process above To is optimum temperature.

I don't think this answers my question, which was: If someone said to you: "I have a formula k=(Tc-To)/(To-Tb), can you tell me what is the value of k?", what would your first question be?

Re: Iterative procedure

Thi_Oliveira — Mon, 27 Jul 2020 16:43:00 GMT

The "k" represents the range of temperatures where there are plant growth in the study.

Re: Iterative procedure

PaigeMiller — Mon, 27 Jul 2020 17:05:11 GMT

Well, ok, this isn't working.

You cannot calculate K if you don't have values for To, Tc and Tb. Correct?

So, you have to tell SAS what the values of To, Tc and Tb are, or nothing gets calculated.

Re: Iterative procedure

ballardw — Mon, 27 Jul 2020 17:17:42 GMT

Instead of a counted loop I think that you want a DO until (convergence). But you haven't said what the convergence limit would be. That would apparently be based on the two calculations. The iteration to track what has is happening probably should be a one record output for each iteration so you can see how values are changing. (i.e approaching convergence or not).

Re: Iterative procedure

Thi_Oliveira — Mon, 27 Jul 2020 17:18:41 GMT

I inform the values of To, Tb and Tc.

data temp;
To=14;
Tb=4;
Tc=20;
k=(Tc-To)/(To-Tb);
l=(0.05**2)*(To-Tb)*((Tc-To)**k);
do i=1 to 50 by 1;
i+1= l/(((Tc-Tb)**(k+1))*((1-i)**k));
output;
end;
run;

However the log indicates an error:

416  data temp;
417  To=14;
418  Tb=4;
419  Tc=20;
420  k=(Tc-To)/(To-Tb);
421  l=(0.05**2)*(To-Tb)*((Tc-To)**k);
422  do i=1 to 50 by 1;
423  i+1= l/(((Tc-Tb)**(k+1))*((1-i)**k));
424  output;
425  end;
426  run;

NOTE: Division by zero detected at line 423 column 7.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
NOTE: Invalid argument(s) to the exponential operator "**" at line 423 column 32.
To=14 Tb=4 Tc=20 k=0.6 l=0.0732539013 i=51 _ERROR_=1 _N_=1
NOTE: Missing values were generated as a result of performing an operation on missing values.
      Each place is given by: (Number of times) at (Line):(Column).
      49 at 423:25
NOTE: Mathematical operations could not be performed at the following places. The results of the
      operations have been set to missing values.
      Each place is given by: (Number of times) at (Line):(Column).
      1 at 423:7     49 at 423:32
NOTE: The data set WORK.TEMP has 50 observations and 6 variables.
NOTE: DATA statement used (Total process time):
      real time           0.01 seconds
      cpu time            0.01 seconds

Now, I don't know what is going on. In the article, the authors have said to start with L=0 and then proceed to the interaction until convergence is achieved. However, I still thinking that I'm not writting on right way.

Re: Iterative procedure

PaigeMiller — Mon, 27 Jul 2020 17:25:58 GMT

data temp;
To=14;
Tb=4;
Tc=20;
k=(Tc-To)/(To-Tb);
array LL l1-l50;
ll(1)=(0.05**2)*(To-Tb)*((Tc-To)**k);
do i=1 to 49 by 1;
LL(i+1)= LL(1)/(((Tc-Tb)**(k+1))*((1-LL(i))**k));
output;
end;
run;

Does this do what you want? You know the forumlas, I don't.

Re: Iterative procedure

Thi_Oliveira — Mon, 27 Jul 2020 17:50:14 GMT

Thanks for the anwser. Do you have any example to share about this DO until convergence.

I made this SAS program, but did not work again, because I can not see any changes in values.

data temp;
To=14;
Tb=4;
Tc=20;
k=(Tc-To)/(To-Tb);
l=(0.05**2)*(To-Tb)*((Tc-To)**k);
do i=1 to 50 by 1;
i+1= l/(((Tc-Tb)**(k+1))*((1-i)**k));
output;
end;
run;

Re: Iterative procedure

Thi_Oliveira — Mon, 27 Jul 2020 18:38:34 GMT

Thanks for the answer PaigeMiler. But, it isn't. I will share again the Appendix in the article.

I only have the values of To, Tb and Tc, all the other must be estimated.

Maybe I have to do a DO until code becasue I have to achieved the convergence, because in the end I only have one value of L and U (convergence), and this value I will sum in Tb (minimum temperature) and Tc (maximum temperature) to corret it.

I read a lot this example in the article but it is very confuse to understand this iterative procedure.

If we could continue to talk about this, I I appreciate it.

Re: Iterative procedure

ballardw — Mon, 27 Jul 2020 21:45:56 GMT

What do you think this this line of code does?

i+1= l/(((Tc-Tb)**(k+1))*((1-i)**k));

I seriously doubt it does what you think, such as calculate the next value of i, which would be poor practice in most cases with an iterated loop as it can lead to infinite loops.

One example of why to be careful with modifying loop index variables inside a loop. The K counter is included to show what values of I look like at the end of the loop. The Leave instruction tells SAS to exit the loop when executed. It is easy to see that without that K and the Leave instruction the loop will never end.

data example2;
   do i=1 to 10;
      i=i-1;
      k+1;
      if k>100 then leave;
      output;
   end;
run;

If you want to use a calculated value inside a loop to control whether to continue you use a DO UNTIL or DO WHILE loop.

A brief example:

data example;
   j=7;
   do until (j le 0.1);
      j = j/3;
      output;
   end;
run;

You assign (calculate an initial value). Then you test if the value "converges", i.e is in some specific range of values. Above the test if the current value of J is <= 0.1.

If the comparison is false (not "converged") then the loop is entered. The J value, the loop control variable (or multiple variables) are recalculated. The the loop starts over with another test. The output shows the value at the bottom of the loop in this case. The last value for J shown will be the one that passed the comparison or "converged".

So the structure of your program should be

Establish all of the initial values, including your convergence criteria.

Test the value(s) for convergence.

If they don't converge then in the loop modify the calculation (all those K and K+1 variables), recalculate the convergence criteria value(s).

Some hints: Do not use exactly equal as the boundary (convergence) criteria and appropriate GE or LE are safer.

Test the code with a counter like my Example2 to break out of the loop after a few values so you can see if the values you need are moving in the expected manner.

Output the values per iteration so you can see those. Sometimes you discover that you have missed adjusting one of the variables you intended to.

back to this bit of code:

l/(((Tc-Tb)**(k+1))*((1-i)**k));

when i=1 then

(1-i)**k)   is (1-1)**k
or 
0**k

Which SAS will resolve as 0. So you have

l/(((Tc-Tb)**(k+1))*0); See the problem yet?

0* anything = 0. So you have

l/0, throwing the divide by zero error.

So some return to algebra is indicated.

Re: Iterative procedure

Kurt_Bremser — Tue, 28 Jul 2020 10:16:54 GMT

This statement:

i+1= l/(((Tc-Tb)**(k+1))*((1-i)**k));

is a so-called Sum Statement, and translates to this:

i = i + (1 = l/(((Tc-Tb)**(k+1))*((1-i)**k)));

which means that you add the result of a comparison to i, so you add either a 1 or a 0; since I doubt that the complex formula on the right will ever create an exact 1, this statement does in essence nothing. See it as an honourable mention in this year's Obfuscated SAS Code Contest.

Re: Iterative procedure

FreelanceReinh — Tue, 28 Jul 2020 17:26:01 GMT

Hello @Thi_Oliveira,

Try this approach:

data want;
/* Provide initial values and derived quantities */

To=14; Tb=4; Tc=20;
h=0.05;
c=10; /* Replace 10 with the real value of c in your application. */

k=(Tc-To)/(To-Tb);
j=h**(1/c)*(To-Tb)*(Tc-To)**k; /* (use j to avoid name conflict with L) */

eps=1e-12; /* used in convergence criterion (see chk_conv below) */
maxiter=1000; /* maximum number of iterations */

retain i L U 0; /* use RETAIN also to define variable order */
link chk_conv;
format dL dU best8.;
label L='L[i]'
      U='U[i]'
      dL='L[i] - L[i-1]'
      dU='U[i] - U[i-1]';

/* Perform iteration */

do i=0 to maxiter while(not conv);
  output;
  L=j/((Tc-Tb)**(k+1)*(1-L)**k);
  U=(j/((Tc-Tb)**(k+1)*(1-U)))**(1/k);
  link chk_conv;
end;

/* Compute final results (named Tb_ and Tc_) */

Tb_=Tb+L*(Tc-Tb);
Tc_=Tc-U*(Tc-Tb); /* Note the typo in the article! */
output;
return;

/* Check convergence criterion */

chk_conv:
  dL=dif(L);
  dU=dif(U);
  if i then conv=(.<abs(dL)<eps & .<abs(dU)<eps);
  return;
run;

proc print data=want label;
run;