Team & @RobPratt
Please look into the below code.
First I am assigning values to w1-w8 and b1-b3.
Then i am verifying that everything is populated in the correct fashion in the verification_dsn dataset.
I am then using proc optmodel to validate if i reach similar/same numbers for w1-w8 and b1-b3.
But everything comes out as 0.
I had this example in excel and then using solver, i reached the numbers for w1-w8 and b1-b3.
The formulas for minimize function and the constraint are exactly same.
For example: If a+b=c then c-b=a. And I am just trying to validate that.
But something is a miss here.
Please advise.
Thank You.
@mkeintz: Tagging you because I thought you must be interested in the solution.
data equation;
input y m x c;
datalines;
14 9 1 5
851 9 94 5
473 9 52 5
635 9 70 5
518 9 57 5
257 9 28 5
383 9 42 5
428 9 47 5
356 9 39 5
68 9 7 5
257 9 28 5
248 9 27 5
194 9 21 5
86 9 9 5
761 9 84 5
284 9 31 5
653 9 72 5
356 9 39 5
320 9 35 5
;run;
%let w1=0.094961;
%let w2=0.008734;
%let w3=6.21478;
%let w4=7.002409;
%let w5=0.012339;
%let w6=0.396723;
%let w7=13.94726;
%let w8=1.045836;
%let b1=0;
%let b2=0;
%let b3=0;
data verification_dsn;
set equation;
w1=0.094961;
w2=0.008734;
w3=6.21478;
w4=7.002409;
w5=0.012339;
w6=0.396723;
w7=13.94726;
w8=1.045836;
b1=0;
b2=0;
b3=0;
n1h1=x*&w1.-&b1.;
n2h1=x*&w2.-&b1.;
n1h2=((x*&w1.-&b1.)*&w3.)+((x*&w2.-&b1.)*&w4.)-&b2.;
n2h2=(n1h1*&w5.)+(n2h1*&w6.)-&b2.;
output_network=(n1h2*&w7.)+(n2h2*&w8.)-&b3.;
sq_diff=(y-output_network)**2;
minimize_function=(y-(((((x*w1-b1)*w3)+((x*w2-b1)*w4)-b2)*w7)+((((x*w1-b1)*w5)+((x*w2-b1)*w6)-b2)*w8)-b3))**2;
constraint=abs(((((x*w1-b1)*w3)+((x*w2-b1)*w4)-b2)*w7)+((((x*w1-b1)*w5)+((x*w2-b1)*w6)-b2)*w8)-b3);
run;
proc optmodel;
var w1 ,w2 ,w3 ,w4 ,w5 ,w6 ,w7 ,w8 ,b1 ,b2 ,b3;
number x ,y;
read data equation into x y;
min sq_diff=(y-(((((x*w1-b1)*w3)+((x*w2-b1)*w4)-b2)*w7)+((((x*w1-b1)*w5)+((x*w2-b1)*w6)-b2)*w8)-b3))**2;
con y = abs(((((x*w1-b1)*w3)+((x*w2-b1)*w4)-b2)*w7)+((((x*w1-b1)*w5)+((x*w2-b1)*w6)-b2)*w8)-b3);
solve;
print w1 w2 w3 w4 w5 w6 w7 w8 b1 b2 b3;
expand / iis;
quit;
As indicated in the log, you were reading only one observation:
NOTE: There were 1 observations read from the data set WORK.EQUATION.
If you add a PRINT statement after the READ DATA, you can see that only the first observation was read:
print x y;
x | y |
---|---|
1 | 14 |
Instead, you need an index set for the observations:
set OBS;
number x {OBS}, y {OBS};
read data equation into OBS=[_N_] x y;
print x y;
[1] | x | y |
---|---|---|
1 | 1 | 14 |
2 | 94 | 851 |
3 | 52 | 473 |
4 | 70 | 635 |
5 | 57 | 518 |
6 | 28 | 257 |
7 | 42 | 383 |
8 | 47 | 428 |
9 | 39 | 356 |
10 | 7 | 68 |
11 | 28 | 257 |
12 | 27 | 248 |
13 | 21 | 194 |
14 | 9 | 86 |
15 | 84 | 761 |
16 | 31 | 284 |
17 | 72 | 653 |
18 | 39 | 356 |
19 | 35 | 320 |
Similarly, the objective function needs to sum over the observations:
min sq_diff = sum{i in OBS}(y[i]-(((((x[i]*w[1]-b[1])*w[3])+((x[i]*w[2]-b[1])*w[4])-b[2])*w[7])+((((x[i]*w[1]-b[1])*w[5])+((x[i]*w[2]-b[1])*w[6])-b[2])*w[8])-b[3]))**2;
From your solution, it looks like you wanted w and b to be >= 0. Here's a compact way to declare that:
var w {1..8} >= 0, b {1..3} >= 0;
By default, decision variables have infinite lower and upper bounds. If you do not specify >= 0, it turns out the solver actually finds an optimal objective value of 0 for this problem.
I'm not sure what you intended with EXPAND / IIS because you are not using the IIS solver option. But EXPAND by itself will show you the full problem as a sanity check.
The problem appears to be nonconvex, so I recommend using the MULTISTART (or MS) option to try to avoid getting stuck in a local minimum.
Putting it all together:
proc optmodel;
var w {1..8} >= 0, b {1..3} >= 0;
set OBS;
number x {OBS}, y {OBS};
read data equation into OBS=[_N_] x y;
print x y;
min sq_diff = sum{i in OBS}(y[i]-(((((x[i]*w[1]-b[1])*w[3])+((x[i]*w[2]-b[1])*w[4])-b[2])*w[7])+((((x[i]*w[1]-b[1])*w[5])+((x[i]*w[2]-b[1])*w[6])-b[2])*w[8])-b[3]))**2;
expand;
solve with nlp / ms;
print w b;
quit;
The SAS System |
Problem Summary | |
---|---|
Objective Sense | Minimization |
Objective Function | sq_diff |
Objective Type | Nonlinear |
Number of Variables | 11 |
Bounded Above | 0 |
Bounded Below | 11 |
Bounded Below and Above | 0 |
Free | 0 |
Fixed | 0 |
Number of Constraints | 0 |
The SAS System |
Solution Summary | |
---|---|
Solver | Multistart NLP |
Algorithm | Interior Point Direct |
Objective Function | sq_diff |
Solution Status | Optimal |
Objective Value | 126.64605979 |
Number of Starts | 100 |
Number of Sample Points | 1600 |
Number of Distinct Optima | 30 |
Random Seed Used | 1679670 |
Optimality Error | 3.1111693E-8 |
Infeasibility | 0 |
Presolve Time | 0.00 |
Solution Time | 0.55 |
[1] | w | b |
---|---|---|
1 | 43.23150674 | -.0000000021 |
2 | 34.96859624 | 0.0000017686 |
3 | 102.26490433 | -.0000000082 |
4 | 80.22402775 | |
5 | 180.63897324 | |
6 | 97.11066371 | |
7 | 0.00055746 | |
8 | 0.00045163 |
The resulting solution is different than what you expected, but the optimal objective value matches, so this looks like a case of multiple optimal solutions.
I think, it is just reading the first observation of the dataset and solving the first row only.
I see in the log---
NOTE: There were 1 observations read from the data set WORK.EQUATION.
This is not intended.
I want optmodel to read in the entire dataset and then come up with the parameters.
As indicated in the log, you were reading only one observation:
NOTE: There were 1 observations read from the data set WORK.EQUATION.
If you add a PRINT statement after the READ DATA, you can see that only the first observation was read:
print x y;
x | y |
---|---|
1 | 14 |
Instead, you need an index set for the observations:
set OBS;
number x {OBS}, y {OBS};
read data equation into OBS=[_N_] x y;
print x y;
[1] | x | y |
---|---|---|
1 | 1 | 14 |
2 | 94 | 851 |
3 | 52 | 473 |
4 | 70 | 635 |
5 | 57 | 518 |
6 | 28 | 257 |
7 | 42 | 383 |
8 | 47 | 428 |
9 | 39 | 356 |
10 | 7 | 68 |
11 | 28 | 257 |
12 | 27 | 248 |
13 | 21 | 194 |
14 | 9 | 86 |
15 | 84 | 761 |
16 | 31 | 284 |
17 | 72 | 653 |
18 | 39 | 356 |
19 | 35 | 320 |
Similarly, the objective function needs to sum over the observations:
min sq_diff = sum{i in OBS}(y[i]-(((((x[i]*w[1]-b[1])*w[3])+((x[i]*w[2]-b[1])*w[4])-b[2])*w[7])+((((x[i]*w[1]-b[1])*w[5])+((x[i]*w[2]-b[1])*w[6])-b[2])*w[8])-b[3]))**2;
From your solution, it looks like you wanted w and b to be >= 0. Here's a compact way to declare that:
var w {1..8} >= 0, b {1..3} >= 0;
By default, decision variables have infinite lower and upper bounds. If you do not specify >= 0, it turns out the solver actually finds an optimal objective value of 0 for this problem.
I'm not sure what you intended with EXPAND / IIS because you are not using the IIS solver option. But EXPAND by itself will show you the full problem as a sanity check.
The problem appears to be nonconvex, so I recommend using the MULTISTART (or MS) option to try to avoid getting stuck in a local minimum.
Putting it all together:
proc optmodel;
var w {1..8} >= 0, b {1..3} >= 0;
set OBS;
number x {OBS}, y {OBS};
read data equation into OBS=[_N_] x y;
print x y;
min sq_diff = sum{i in OBS}(y[i]-(((((x[i]*w[1]-b[1])*w[3])+((x[i]*w[2]-b[1])*w[4])-b[2])*w[7])+((((x[i]*w[1]-b[1])*w[5])+((x[i]*w[2]-b[1])*w[6])-b[2])*w[8])-b[3]))**2;
expand;
solve with nlp / ms;
print w b;
quit;
The SAS System |
Problem Summary | |
---|---|
Objective Sense | Minimization |
Objective Function | sq_diff |
Objective Type | Nonlinear |
Number of Variables | 11 |
Bounded Above | 0 |
Bounded Below | 11 |
Bounded Below and Above | 0 |
Free | 0 |
Fixed | 0 |
Number of Constraints | 0 |
The SAS System |
Solution Summary | |
---|---|
Solver | Multistart NLP |
Algorithm | Interior Point Direct |
Objective Function | sq_diff |
Solution Status | Optimal |
Objective Value | 126.64605979 |
Number of Starts | 100 |
Number of Sample Points | 1600 |
Number of Distinct Optima | 30 |
Random Seed Used | 1679670 |
Optimality Error | 3.1111693E-8 |
Infeasibility | 0 |
Presolve Time | 0.00 |
Solution Time | 0.55 |
[1] | w | b |
---|---|---|
1 | 43.23150674 | -.0000000021 |
2 | 34.96859624 | 0.0000017686 |
3 | 102.26490433 | -.0000000082 |
4 | 80.22402775 | |
5 | 180.63897324 | |
6 | 97.11066371 | |
7 | 0.00055746 | |
8 | 0.00045163 |
The resulting solution is different than what you expected, but the optimal objective value matches, so this looks like a case of multiple optimal solutions.
@RobPratt :: You THE Boss.
Thank You very much for that.
I read this link numerous times but was not able to comprehend the part that you showed.
https://support.sas.com/documentation/onlinedoc/or/132/optmodel.pdf
What you showed here in the example-- where is that in this documentation of OPTMODEL?
Regards.
Glad to help. Which part do you mean?
Reading the entire dataset and then go to SOLVE part.
The READ DATA statement is documented here and is also illustrated in most of the end-of-chapter examples. You might also be interested in this book of 29 examples, almost all of which use the READ DATA statement.
Hello @RobPratt :
In the above example i did put a constraint:
con y = abs(((((x*w1-b1)*w3)+((x*w2-b1)*w4)-b2)*w7)+((((x*w1-b1)*w5)+((x*w2-b1)*w6)-b2)*w8)-b3);
Is the way I put the constraint correct or it also has to be array?
con y[i]=(((((x[i]*w[1]-b[1])*w[3])+((x[i]*w[2]-b[1])*w[4])-b[2])*w[7])+((((x[i]*w[1]-b[1])*w[5])+((x[i]*w[2]-b[1])*w[6])-b[2])*w[8])-b[3]);
I will have to add a constraint here where y for each observation is equal to value coming out of formula.
Because, if i don't do that, then Solver calculates a mean value that satisfies the solver condition and as a result the value is same for all rows.
Correct syntax is:
con c {i in OBS}: y[i] = ((((x[i]*w[1]-b[1])*w[3])+((x[i]*w[2]-b[1])*w[4])-b[2])*w[7])+((((x[i]*w[1]-b[1])*w[5])+((x[i]*w[2]-b[1])*w[6])-b[2])*w[8])-b[3];
But a best practice is to use an implicit variable if the same long expression appears more than once:
impvar yhat {i in OBS} = ((((x[i]*w[1]-b[1])*w[3])+((x[i]*w[2]-b[1])*w[4])-b[2])*w[7])+((((x[i]*w[1]-b[1])*w[5])+((x[i]*w[2]-b[1])*w[6])-b[2])*w[8])-b[3];
min sq_diff = sum{i in OBS}(y[i]-yhat[i])**2;
con c {i in OBS}: y[i] = yhat[i];
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 16. Read more here about why you should contribute and what is in it for you!
Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.
Find more tutorials on the SAS Users YouTube channel.