Proc OPTMODEL problem (derived variable from existing variable and parameter)

[KCKC]

My excel example is a simple panel data. p,m,r,a1,a2 is from raw data: p is the group variable and m is the time period.

when m=1, derived=1+（r+d)/3

else derived=derived[t-1]*(1+（r+d)/3)

After derived is calculated, (a1-a2)/derived are sum for each group p, then the sum is average across all the groups. The objective of this problem is to find out d so that the final average value equal to target value.

I wrote the following code but it seems optmodel does not allow the derived variable here. It didn't do anything and give out the message

"

NOTE: The problem has 0 variables (0 free, 0 fixed).
NOTE: The problem has 0 linear constraints (0 LE, 0 EQ, 0 GE, 0 range).
NOTE: The problem has 0 nonlinear constraints (0 LE, 0 EQ, 0 GE, 0 range).
NOTE: An optimal solution already exists for the problem received by solver.

"

proc optmodel PRESOLVER=NONE;

set I;

number ID{I};

number r{I}, a1{I}, a2{I};

number der{I}, der1{I},value;

read data testdata into I = [_n_]

ID = _n_ r = r a1 = a1 a2 = a2;

var d >=-1000 <=1000;

for {j in I} do;

if m=0 then der=1+(r+d)/3;

else der=der[j-1]*(1+(r+d)/3);

der1=(a1-a2)/der;

end;

value=sum{k in I} der1;

min g=abs((value/2)-60);

solve with NLPC OBJ g;

quit;

[RobPratt]

You are trying to use der, der1, and value as decision variables but have instead declared them as numeric parameters.  Use either VAR or IMPVAR instead.

[KCKC]

Thanks RobPratt. I just tried.

When I change to VAR, it's the same and SAS stop at initial value.

When I change to IMPVAR, it will give out the error message:

"ERROR 180-322: Statement is not valid or it is used out of proper order."

Any suggestion? Thanks a lot!

[RobPratt]

Can you please post what you tried, including the input data set?

[KCKC]

I attach my code in the following, including input dataset. My goal is explained in the original post but for you convinience I re-post here:

p,m,r,a1,a2 is from raw data: p is the group variable and m is the time period.

when m=1, derived=1+（r+d)/3

else derived=derived[t-1]*(1+（r+d)/3)

After derived is calculated, (a2-a1)/derived are sum for each group p, then the sum is average across all the groups. The objective of this problem is to find out d so that the final average value equal to target value.

Here are my codes:

data test;
input p m r a1 a2;
datalines;
1 1 0.5 1 2
1 2 1 2 3
1 3 1.5 3 4
1 4 2 4 5
1 5 2.5 5 6
2 1 1 6 7
2 2 1.5 7 8
2 3 2 8 9
2 4 2.5 9 10
2 5 3 10 11
;
run;

proc optmodel PRESOLVER=NONE;
set I;
number ID{I};
number r{I}, p{I},m{I},a1{I}, a2{I};
var der{I}, der1{I},value;
read data test into I = [_n_]
ID = _n_ r = r a1 = a1 a2 = a2 p=p m=m;
var d >=-1000 <=1000;

for {j in I} do;
if m=0 then der=1+(r+d)/3;
else der=der[j-1]*(1+(r+d)/3);
der1=(a2-a1)/der;
end;

value=sum{k in I} der1;
min g=abs((value/2)-1);

solve with NLPC OBJ g;

quit;

[RobPratt · Accepted solution]

Your m = 0 should be m = 1, and you haven't used p anywhere.  Here are three approaches that should help you proceed.

The first approach is closest to your model:

proc optmodel PRESOLVER=NONE;
    set I;
    number ID{I};
    number r{I}, p{I},m{I},a1{I}, a2{I};
    var der{I} init 1;
    read data test into I = [_n_]
        ID = _n_ r = r a1 = a1 a2 = a2 p=p m=m;
    var d >=-1000 <=1000;
    con der_con {j in I}:
        der = if m=1 then 1+(r+d)/3
                else der[j-1]*(1+(r+d)/3);
    impvar der1 {j in I} =(a2-a1)/der;
    impvar value=sum{k in I} der1;
    min g=abs((value/2)-1);
    solve;
    print d;
quit;

The second approach avoids the absolute value function, which is nondifferentiable and causes difficulty for smooth NLP solvers.

proc optmodel PRESOLVER=NONE;
    set I;
    number ID{I};
    number r{I}, p{I},m{I},a1{I}, a2{I};
    var der{I} init 1;
    read data test into I = [_n_]
        ID = _n_ r = r a1 = a1 a2 = a2 p=p m=m;
    var d >=-1000 <=1000;
    con der_con {j in I}:
        der = if m=1 then 1+(r+d)/3
                else der[j-1]*(1+(r+d)/3);
    impvar der1 {j in I} =(a2-a1)/der;
    impvar value=sum{k in I} der1;
    var gplus >= 0;
    var gminus >= 0;
    con g_con:
        (value/2)-1 = gplus - gminus;
    min g = gplus+gminus;
    solve;
    print d;
quit;

The third approach linearizes the absolute value using inequalities instead of equality:

proc optmodel PRESOLVER=NONE;
    set I;
    number ID{I};
    number r{I}, p{I},m{I},a1{I}, a2{I};
    var der{I} init 1;
    read data test into I = [_n_]
        ID = _n_ r = r a1 = a1 a2 = a2 p=p m=m;
    var d >=-1000 <=1000;
    con der_con {j in I}:
        der = if m=1 then 1+(r+d)/3
                else der[j-1]*(1+(r+d)/3);
    impvar der1 {j in I} =(a2-a1)/der;
    impvar value=sum{k in I} der1;
    var gabs >= 0;
    con g_con1:
        gabs >= (value/2)-1;
    con g_con2:
        gabs >= -(value/2)+1;
    min g = gabs;
    solve;
    print d;
quit;

[KCKC]

Thank you so much RobPratt.

I tried the first method with a little modification of the code (essentially just get rid of p which is not used anywhere):

proc optmodel PRESOLVER=NONE;

set I;

number ID{I};

number r{I},m{I},a1{I}, a2{I};

var der{I};

read data test into I = [_n_]

ID = _n_ r = r a1 = a1 a2 = a2 m=m;

var d >=-1000 <=1000;

con der_con{j in I}: der= if m=1 then 1+(r+d)/3 else der[j-1]*(1+(r+d)/3);

impvar der1{j in I}=(a2-a1)/der;

impvar value=sum{k in I} der1;

min g=abs((value/2)-1);

solve;

print d;

quit;

However, I got some error about impvar:

1321  impvar der1{j in I}=(a2-a1)/der;
      ------
      180
ERROR 180-322: Statement is not valid or it is used out of proper order.

1322  impvar value=sum{k in I} der1;
      ------
      180
ERROR 180-322: Statement is not valid or it is used out of proper order.

1323  min g=abs((value/2)-1);
                 -----
                 536
ERROR 536-782: The symbol 'value' is unknown.

1324  solve;
ERROR: No objective has been specified at line 1324 column 1.

My SAS version is SAS (r) 9.1 (TS1M3). Is it because this version did not have full support of optmodel statement/options?

Thanks anyway!

[RobPratt]

Yes, that is a very old version.  We have had several SAS/OR releases since then, and implicit variables were introduced in SAS/OR 9.22.  You should definitely upgrade to 9.3, but until then here is a workaround.  If you have an implicit variable:

impvar x {i in ISET} = some function of other variables;

Then replace with an explicit variable and constraint:

var x {i in ISET};

con x_con {i in ISET}:

     x = some function of other variables;

[KCKC]

Thanks RobPratt.

I modified my code as following:

proc optmodel;

set I;

number ID{I};

number r{I},m{I},a1{I}, a2{I};

var der{I}, value;

read data test into I = [_n_]

ID = _n_ r = r a1 = a1 a2 = a2 m=m;

var d >=1 <=100;

con der_con{j in I}: der= if m=1 then 1+(r+d)/3 else der[j-1]*(1+(r+d)/3);

con value_con: value=sum{j in I} (a2-a1)/der;

min g=abs((value/2)-1);

solve with SQP/maxiter=1000;

print der r m;

print g d;

quit;

However it didn't converge.  By printing out der, the calculation of der is not correct. with positive d, the value of first der is larger than 1, and the following der is larger than previous der within the same group. however this is not the case from the printout.

                                                 Solver                                    SQP
                                                 Objective Function                          g
                                                 Solution Status       Iteration Limit Reached
                                                 Objective Value                        5545.3
                                                 Iterations                              10000

                                                 Infeasibility                    178.17641513
                                                 Optimality Error                  90391156651
                                                 Complementarity                  0.8673629529

                                                       [1]            der       r     m

                                                         1    -4.6580E-04     0.5     1
                                                         2     2.6289E+00     1.0     2
                                                         3     5.7257E+00     1.5     3
                                                         4     1.0694E+01     2.0     4
                                                         5     2.0447E+01     2.5     5
                                                         6    -1.1184E-04     1.0     1
                                                         7     2.7604E+01     1.5     2
                                                         8     2.2540E+02     2.0     3
                                                         9     5.1963E+02     2.5     4
                                                        10     1.1087E+03     3.0     5

                                                                    g           d

                                                               5545.3     0.13264

[RobPratt]

Yes, the positive value of Infeasibility indicates that the final solution does not satisfy the constraints.  In 9.1, you should get better results with the IPNLP solver and/or the linearized formulations I sent.  Also, you should initialize der to 1 (or something positive) since otherwise the default initial value of 0 leads to division by 0 at the starting point.

[KCKC]

Thanks a lot, Rob. This is very helpful!