topic Re: what's wrong with this code in Mathematical Optimization, Discrete-Event Simulation, and OR

what's wrong with this code

OceanDream — Mon, 25 Jun 2018 04:29:40 GMT

I am using the following code to solve two unknown variables (T and delta) with two equations. If I do not use "bounds t>1" AND "optimize" option, it gives me value of T close to zero to all the observations. However, if I add these two codes, the results for all observations are 1.01 for T. I checked proc model code with simple quadratic equation to get only negative results. It worked fine. Anyone could please help me? What is wrong with my current code for this more complex equations??? I am in urgent need to solve this. Please help!!!

Additional information: I got the following information when running the code. It seems that the code did not even iterate once to solve the equations and gave error of zero?? That does not seem right.

NOTE: Optimal.
NOTE: Objective = 0.4636390687.
NOTE: The NLP solver is called.
NOTE: The Interior Point algorithm is used.
Objective Optimality
Iter Value Infeasibility Error
0 0.49897143 0 0

"proc model data=HAVE noprint out=NEED;
bounds t>1;
eq.one = (1-PD) - (probnorm((-log(PD)-(s - 0.5*(delta*delta))*T)/(delta*sqrt(T)))
-PD*exp(s*T)* probnorm((-log(PD)-(s - 0.5*(delta*delta))*T)/(delta*sqrt(T))- delta*sqrt(T)));
eq.two = delta_E*(1-PD) - delta*probnorm((-log(PD)-(s - 0.5*(delta*delta))*T)/(delta*sqrt(T)));
solve delta T /optimize solveprint ;
id companyid date pd s delta_e;
run;"

Re: what's wrong with this code

RobPratt — Mon, 25 Jun 2018 14:41:07 GMT

PROC MODEL is part of SAS/ETS, and I see that you have already posted in the SAS Forecasting and Econometrics community. You could also use PROC OPTMODEL to solve your equations. Can you please share your data?

Re: what's wrong with this code

OceanDream — Mon, 25 Jun 2018 15:03:26 GMT

Hi Rob,

Thank you for your reply. Attached is my data. I could not attach my file to the reply. It says " the contents of the attachment doesn't match its file type". Do you know what is wrong? If you don't mind, I can email you my data. Thank you so much for your help!!

"proc model data=bl.decomp5 noprint out=decomp6;
endo delta T;
bounds T>0.1;
eq.one = (1-PD) - (probnorm((-log(PD)-(s - 0.5*(delta*delta))*T)/(delta*sqrt(T)))
-PD*exp(s*T)* probnorm((-log(PD)-(s - 0.5*(delta*delta))*T)/(delta*sqrt(T))- delta*sqrt(T)));
eq.two = delta_E*(1-PD) - delta*probnorm((-log(PD)-(s - 0.5*(delta*delta))*T)/(delta*sqrt(T)));
solve delta T /optimize solveprint ;
id gvkey dealactivedate facilityid pd s delta_e beta;
run;"

Re: what's wrong with this code

RobPratt — Mon, 25 Jun 2018 19:34:25 GMT

Please try to attach a CSV or ZIP file instead.

Re: what's wrong with this code

OceanDream — Tue, 26 Jun 2018 01:13:36 GMT

Hi Rob,

Please see attached dataset. Thanks again for your help!

Re: what's wrong with this code

RobPratt — Wed, 27 Jun 2018 19:40:50 GMT

Is beta supposed to be used anywhere?

Re: what's wrong with this code

OceanDream — Thu, 28 Jun 2018 01:09:40 GMT

Thanks for checking it, Rob!

No. For my question, beta is not needed.

Did you find anything wrong with my data or the code?

Re: what's wrong with this code

RobPratt — Thu, 28 Jun 2018 15:04:38 GMT

I don't see the same behavior in the latest release (SAS 9.4M5). I get various values for T, not all 1.01. What version are you running?

When I add the OUTOBJVALS option, I see that the sum of squared errors is positive, meaning that the equations are not satisfied exactly. Do you know that there is always a solution with zero error?

Do you know other bounds on delta and T besides T > 0.1 (or did you want T > 1)?

You might also want to specify a good initial solution.

Here is a way to minimize the sum of squared errors for the same equations with PROC OPTMODEL in SAS/OR:

proc optmodel printlevel=0;
   set OBS;
   str gvkey {OBS};
   num dealactivedate {OBS}, facilityid {OBS}, pd {OBS}, s {OBS}, delta_e {OBS}, beta {OBS};   
   read data decomp5 into OBS=[_N_] gvkey dealactivedate facilityid pd s delta_e beta;   

   num pd_this, s_this, delta_e_this;
   var delta init 1;
   var T >= 0.1 <= 9 init 1;
   var error {1..2};
   min sse = sum {i in 1..2} error[i]^2;
   con eqone: error[1] = (1-pd_this) - (probnorm((-log(pd_this)-(s_this - 0.5*(delta*delta))*T)/(delta*sqrt(T)))
      -pd_this*exp(s_this*T)* probnorm((-log(pd_this)-(s_this - 0.5*(delta*delta))*T)/(delta*sqrt(T))- delta*sqrt(T)));
   con eqtwo: error[2] = delta_e_this*(1-pd_this) - delta*probnorm((-log(pd_this)-(s_this - 0.5*(delta*delta))*T)/(delta*sqrt(T)));

   num delta_sol {OBS}, T_sol {OBS}, obj {OBS};
   option nonotes;
   cofor {i in OBS} do;
      put i=;
      pd_this = pd[i];
      s_this = s[i];
      delta_e_this = delta_e[i];
      solve with nlp / ms;
      delta_sol[i] = delta.sol;
      T_sol[i] = T.sol;
      obj[i] = _OBJ_.sol;
   end;
   option notes;
   create data optmodel_out from [i] gvkey dealactivedate facilityid pd s delta_e beta delta=delta_sol T=T_sol obj;
quit;

The upper bound of 9 on T is to prevent overflow in the exponential function.

Re: what's wrong with this code

OceanDream — Fri, 29 Jun 2018 01:10:22 GMT

Hi Rob,

Thank you so much for you response. I did not get all values of 1.01 for T, but most of them are, which is still problematic. Did you get different values for each observations or the variation is not that much? I think my SAS version is 9.3..

I do not expect the solutions are always with zero error.

I only need to bound T greater than one, nothing else.

I have been running the code PROC OPTMODEL that you wrote for hours and it is still running. Is that what also happens to you?

Re: what's wrong with this code

RobPratt — Fri, 29 Jun 2018 14:04:57 GMT

I got a wide variety of T values, but none of them had zero error.

I ran on a busy 100-node grid, and it took 12 minutes. You can speed things up but might find worse solutions by omitting the multistart option. That is, replace...

solve with nlp / ms;

...with...

solve;

But I would also recommend upgrading. The last SAS 9.3 release came out in 2012, and the subsequent releases all contain performance improvements.

Re: what's wrong with this code

OceanDream — Fri, 29 Jun 2018 16:45:26 GMT

My school only provides 9.3 version of SAS.

If it is not too much to ask, could you share me with the results from running PROC MODEL using your 9.4 version?

I replaced

solve with nlp / ms;

with

solve;

using PROC OPTMODEL, but the values for T seem reasonable, but most values for delta are one, which is not reasonable...