I was able to solve but as I scale up I have Out of memory errors ! I even removed a dimension. How to manage memory for large scale optimizations? Log: NOTE: PROCEDURE PRINTTO used (Total process time): real time 0.00 seconds cpu time 0.00 seconds 17 18 proc sql; 19 create table TABLE1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX NOTE: The query requires remerging summary statistics back with the original data. NOTE: Table WORK.TABLE1 created, with 396,971 rows and 8 columns. 29 quit; NOTE: PROCEDURE SQL used (Total process time): real time 1.48 seconds cpu time 2.07 seconds 30 31 proc sql; 32 create table TABLE_RGN as 33 select unique RGN from TABLE1; NOTE: Table WORK.TABLE_RGN created, with 7 rows and 1 columns. 33 ! quit; NOTE: PROCEDURE SQL used (Total process time): real time 0.13 seconds cpu time 0.20 seconds 34 35 proc sql; 36 create table TABLE_MN as 2 The SAS System 14:01 Tuesday, December 23, 2014 37 select unique MTH from TABLE1; NOTE: Table WORK.TABLE_MN created, with 12 rows and 1 columns. 37 ! quit; NOTE: PROCEDURE SQL used (Total process time): real time 0.11 seconds cpu time 0.15 seconds 38 39 proc sql; 40 create table TABLE_PS as 41 select unique CNTRY from TABLE1; NOTE: Table WORK.TABLE_PS created, with 92 rows and 1 columns. 41 ! quit; NOTE: PROCEDURE SQL used (Total process time): real time 0.19 seconds cpu time 0.23 seconds 41 ! 42 43 proc sql; 44 create table TABLE_CM as 45 select unique CLS from TABLE1; NOTE: Table WORK.TABLE_CM created, with 2 rows and 1 columns. 45 ! quit; NOTE: PROCEDURE SQL used (Total process time): real time 0.11 seconds cpu time 0.14 seconds 46 47 proc sql; 48 create table TABLE_ND as 49 select unique MKT from TABLE1; NOTE: Table WORK.TABLE_ND created, with 6952 rows and 1 columns. 49 ! quit; NOTE: PROCEDURE SQL used (Total process time): real time 0.26 seconds cpu time 0.35 seconds 50 51 /*****************************************************************************************/ 52 /* Dummy data set only to provide constant targets */ 53 54 proc sql; 55 create table RGN_TGT as 56 select unique RGN, 57 sum(VALUE)*1.1 as RGN_VALUE_TGT format dollar16.0, 58 1.05*(sum(VALUE)/sum(DMD)) as RGN_PRICE_TGT format dollar16.0, 59 sum(DMD) as RGN_DMD 60 from TABLE1 61 group by RGN; 3 The SAS System 14:01 Tuesday, December 23, 2014 NOTE: Table WORK.RGN_TGT created, with 7 rows and 4 columns. 62 quit; NOTE: PROCEDURE SQL used (Total process time): real time 0.11 seconds cpu time 0.10 seconds 63 64 proc sql; 65 create table MTH_TGT as 66 select unique MTH, 67 sum(VALUE) as MTH_VALUE_TGT format dollar16.0, 68 (sum(VALUE)/sum(DMD)) as MTH_PRICE_TGT format dollar16.0, 69 sum(DMD) as MTH_DMD 70 from TABLE1 71 group by MTH; NOTE: Table WORK.MTH_TGT created, with 12 rows and 4 columns. 72 quit; NOTE: PROCEDURE SQL used (Total process time): real time 0.11 seconds cpu time 0.21 seconds 73 74 data MTH_TGT; 75 set MTH_TGT; 76 if MTH in ('APR', 'MAY', 'JUN', 'JUL', 'AUG', 'SEP') then MTH_VOL_TGT = MTH_DMD*1.05; 77 else MTH_VOL_TGT = MTH_DMD*0.95; 78 run; NOTE: There were 12 observations read from the data set WORK.MTH_TGT. NOTE: The data set WORK.MTH_TGT has 12 observations and 5 variables. NOTE: DATA statement used (Total process time): real time 0.01 seconds cpu time 0.01 seconds 78 ! 79 /*****************************************************************************************/ 80 81 options missing=0; 82 proc optmodel/*presolver=none*/; 83 84 /* declare index sets and read values from data sets*/ 85 86 set <str> RGN_I; 87 read data TABLE_RGN nomiss into RGN_I = [RGN]; NOTE: There were 7 observations read from the data set WORK.TABLE_RGN. 88 89 set <str> MTH_I; 90 read data TABLE_MN nomiss into MTH_I = [MTH]; NOTE: There were 12 observations read from the data set WORK.TABLE_MN. 91 92 set <str> CNTRY_I; 93 read data TABLE_PS nomiss into CNTRY_I = [CNTRY]; NOTE: There were 92 observations read from the data set WORK.TABLE_PS. 4 The SAS System 14:01 Tuesday, December 23, 2014 94 95 set <str> CLS_I; 96 read data TABLE_CM nomiss into CLS_I = [CLS]; NOTE: There were 2 observations read from the data set WORK.TABLE_CM. 97 98 set <str> MKT_I; 99 read data TABLE_ND nomiss into MKT_I = [MKT]; NOTE: There were 6952 observations read from the data set WORK.TABLE_ND. 100 101 set RMCCM = {RGN_I, MTH_I, CNTRY_I, CLS_I, MKT_I}; 102 103 /* declare parameters and read from data sets */ 104 105 num DMD {RMCCM}init .; 106 read data TABLE1 nomiss into [RGN MTH CNTRY CLS MKT] DMD; ERROR: Out of memory during expression evaluation. NOTE: There were 1 observations read from the data set WORK.TABLE1. 107 108 num PRICE {RMCCM}init .; 109 read data TABLE1 nomiss into [RGN MTH CNTRY CLS MKT] PRICE; ERROR: The symbol 'RMCCM' has no value at line 108 column 12. NOTE: There were 1 observations read from the data set WORK.TABLE1. 110 111 num RGN_VALUE_TGT {RGN_I}init .; 112 read data RGN_TGT nomiss into RGN_I =[RGN] RGN_VALUE_TGT; NOTE: There were 7 observations read from the data set WORK.RGN_TGT. 113 114 num RGN_PRICE_TGT {RGN_I}init .; 115 read data RGN_TGT nomiss into RGN_I =[RGN] RGN_PRICE_TGT; NOTE: There were 7 observations read from the data set WORK.RGN_TGT. 116 117 num MTH_VOL_TGT {MTH_I}init .; 118 read data MTH_TGT nomiss into MTH_I =[MTH] MTH_VOL_TGT; NOTE: There were 12 observations read from the data set WORK.MTH_TGT. 119 120 /* test that the data is read*/ 121 122 print RGN_VALUE_TGT RGN_PRICE_TGT; 14:01 Tuesday, December 23, 2014 1 124 /*print DMD PRICE; */ 125 126 set RMCCMMwithPRICE = {<r,m,p,c,k> in RMCCM: price[r,m,p,c,k] ~= .}; 127 128 /* declare variables */ 129 130 num dmdAndPriceDelta init 0.3; 131 132 var x {<r,m,p,c,k> in RMCCMwithPRICE} _______________ 537 526 ERROR 537-782: The symbol 'RMCCMwithPRICE' is unknown. ERROR 526-782: An indexing set subexpression may not be a number. 133 >= DMD[r,m,p,c,k] * (1 - dmdAndPriceDelta) _ _ 537 653 ERROR 537-782: The symbol 'm' is unknown. ERROR 653-782: Subscript 1 must be a string, found a number. 5 The SAS System 14:01 Tuesday, December 23, 2014 133 ! >= DMD[r,m,p,c,k] * (1 - dmdAndPriceDelta) _ _ 537 653 ERROR 537-782: The symbol 'p' is unknown. ERROR 653-782: Subscript 2 must be a string, found a number. 133 ! >= DMD[r,m,p,c,k] * (1 - dmdAndPriceDelta) _ 537 ERROR 537-782: The symbol 'c' is unknown. _ 653 ERROR 653-782: Subscript 3 must be a string, found a number. 133 ! >= DMD[r,m,p,c,k] * (1 - dmdAndPriceDelta) _ 537 ERROR 537-782: The symbol 'k' is unknown. _ 653 ERROR 653-782: Subscript 4 must be a string, found a number. 133 ! >= DMD[r,m,p,c,k] * (1 - dmdAndPriceDelta) _ 653 ERROR 653-782: Subscript 5 must be a string, found a number. 134 <= DMD[r,m,p,c,k] * (1 + dmdAndPriceDelta); _ _ 537 653 ERROR 537-782: The symbol 'm' is unknown. ERROR 653-782: Subscript 1 must be a string, found a number. 134 ! <= DMD[r,m,p,c,k] * (1 + dmdAndPriceDelta); _ _ 537 653 ERROR 537-782: The symbol 'p' is unknown. ERROR 653-782: Subscript 2 must be a string, found a number. 134 ! <= DMD[r,m,p,c,k] * (1 + dmdAndPriceDelta); _ 537 ERROR 537-782: The symbol 'c' is unknown. _ 653 ERROR 653-782: Subscript 3 must be a string, found a number. 134 ! <= DMD[r,m,p,c,k] * (1 + dmdAndPriceDelta); _ 537 ERROR 537-782: The symbol 'k' is unknown. 6 The SAS System 14:01 Tuesday, December 23, 2014 _ 653 ERROR 653-782: Subscript 4 must be a string, found a number. 134 ! <= DMD[r,m,p,c,k] * (1 + dmdAndPriceDelta); _ 653 ERROR 653-782: Subscript 5 must be a string, found a number. 135 136 var y {<r,m,p,c,k> in RMCCMwithPRICE} _______________ 537 526 ERROR 537-782: The symbol 'RMCCMwithPRICE' is unknown. ERROR 526-782: An indexing set subexpression may not be a number. 137 >= PRICE[r,m,p,c,k] * (1 - dmdAndPriceDelta) _ _ 537 653 ERROR 537-782: The symbol 'm' is unknown. ERROR 653-782: Subscript 1 must be a string, found a number. 137 ! >= PRICE[r,m,p,c,k] * (1 - dmdAndPriceDelta) _ _ 537 653 ERROR 537-782: The symbol 'p' is unknown. ERROR 653-782: Subscript 2 must be a string, found a number. 137 ! >= PRICE[r,m,p,c,k] * (1 - dmdAndPriceDelta) _ 537 ERROR 537-782: The symbol 'c' is unknown. _ 653 ERROR 653-782: Subscript 3 must be a string, found a number. 137 ! >= PRICE[r,m,p,c,k] * (1 - dmdAndPriceDelta) _ 537 ERROR 537-782: The symbol 'k' is unknown. _ 653 ERROR 653-782: Subscript 4 must be a string, found a number. 137 ! >= PRICE[r,m,p,c,k] * (1 - dmdAndPriceDelta) _ 653 ERROR 653-782: Subscript 5 must be a string, found a number. 138 <= PRICE[r,m,p,c,k] * (1 + dmdAndPriceDelta); _ _ 537 653 ERROR 537-782: The symbol 'm' is unknown. ERROR 653-782: Subscript 1 must be a string, found a number. 7 The SAS System 14:01 Tuesday, December 23, 2014 138 ! <= PRICE[r,m,p,c,k] * (1 + dmdAndPriceDelta); _ _ 537 653 ERROR 537-782: The symbol 'p' is unknown. ERROR 653-782: Subscript 2 must be a string, found a number. 138 ! <= PRICE[r,m,p,c,k] * (1 + dmdAndPriceDelta); _ 537 ERROR 537-782: The symbol 'c' is unknown. _ 653 ERROR 653-782: Subscript 3 must be a string, found a number. 138 ! <= PRICE[r,m,p,c,k] * (1 + dmdAndPriceDelta); _ 537 ERROR 537-782: The symbol 'k' is unknown. _ 653 ERROR 653-782: Subscript 4 must be a string, found a number. 138 ! <= PRICE[r,m,p,c,k] * (1 + dmdAndPriceDelta); _ 653 ERROR 653-782: Subscript 5 must be a string, found a number. 139 140 impvar z {r in RGN_I} 141 = sum{<(r),m,p,c,k> in RMCCMwithPRICE} x[r,m,p,c,k] * y[r,m,p,c,k]; _______________ _ 537 526 650 637 828 ERROR 537-782: The symbol 'RMCCMwithPRICE' is unknown. ERROR 526-782: An indexing set subexpression may not be a number. ERROR 650-782: The subscript count does not match array 'x', 5 NE 1. ERROR 637-782: The right operand of 'SLICE' must be a set, found a number. ERROR 828-782: An index tuple must specify at least one dummy parameter. 141 ! = sum{<(r),m,p,c,k> in RMCCMwithPRICE} x[r,m,p,c,k] * y[r,m,p,c,k]; _ _ _ 537 537 650 ERROR 537-782: The symbol 'm' is unknown. ERROR 650-782: The subscript count does not match array 'y', 5 NE 1. 141 ! = sum{<(r),m,p,c,k> in RMCCMwithPRICE} x[r,m,p,c,k] * y[r,m,p,c,k]; _ _ 537 537 ERROR 537-782: The symbol 'p' is unknown. 8 The SAS System 14:01 Tuesday, December 23, 2014 141 ! = sum{<(r),m,p,c,k> in RMCCMwithPRICE} x[r,m,p,c,k] * y[r,m,p,c,k]; _ _ 537 537 ERROR 537-782: The symbol 'c' is unknown. 141 ! = sum{<(r),m,p,c,k> in RMCCMwithPRICE} x[r,m,p,c,k] * y[r,m,p,c,k]; _ _ 537 537 ERROR 537-782: The symbol 'k' is unknown. 142 143 impvar mn {m in MTH_I} 144 = sum{<r,(m),p,c,k> in RMCCMwithPRICE} x[r,m,p,c,k]; _______________ _ 537 526 650 637 ERROR 537-782: The symbol 'RMCCMwithPRICE' is unknown. ERROR 526-782: An indexing set subexpression may not be a number. ERROR 650-782: The subscript count does not match array 'x', 5 NE 1. ERROR 637-782: The right operand of 'SLICE' must be a set, found a number. 144 ! = sum{<r,(m),p,c,k> in RMCCMwithPRICE} x[r,m,p,c,k]; _ 537 ERROR 537-782: The symbol 'p' is unknown. 144 ! = sum{<r,(m),p,c,k> in RMCCMwithPRICE} x[r,m,p,c,k]; _ 537 ERROR 537-782: The symbol 'c' is unknown. 144 ! = sum{<r,(m),p,c,k> in RMCCMwithPRICE} x[r,m,p,c,k]; _ 537 ERROR 537-782: The symbol 'k' is unknown. 145 146 /* define constraints */ 147 num tgtDelta init .05; 148 149 con mn_CON {m in MTH_I}: 150 MTH_VOL_TGT * (1 - tgtDelta) 151 <= mn 152 <= MTH_VOL_TGT * (1 + tgtDelta); 153 154 /* objective function */ 155 156 var Error {RGN_I}; 157 con Error_CON {r in RGN_I}: 158 Error = z - if RGN_VALUE_TGT = . then 0 else RGN_VALUE_TGT ; 159 160 min MSE = sum{r in RGN_I} 161 Error **2; 162 9 The SAS System 14:01 Tuesday, December 23, 2014 163 /*solve;*/ 164 165 solve with nlp/iis=on; NOTE: The OPTMODEL presolver is disabled when the IIS= option is enabled. NOTE: Problem generation will use 4 threads. NOTE: Previous errors might cause the problem to be resolved incorrectly. ERROR: The implicit variable 'z' has an incomplete declaration. ERROR: The implicit variable 'mn' has an incomplete declaration. NOTE: The problem has 7 variables (7 free, 0 fixed). NOTE: The problem uses 19 implicit variables. NOTE: The problem has 0 linear constraints (0 LE, 0 EQ, 0 GE, 0 range). NOTE: The problem has 19 nonlinear constraints (0 LE, 7 EQ, 0 GE, 12 range). NOTE: Unable to create problem instance due to previous errors. NOTE: Previous errors might cause the problem to be resolved incorrectly. 14:01 Tuesday, December 23, 2014 2 The OPTMODEL Procedure Solution Summary Solver NLP Objective Function Unknown Solution Status Semantic Error Iterations 0 Presolve Time 0.00 Solution Time 0.00 166 /*expand/iis;*/ 167 168 print x y mn z; ERROR: The symbol 'x' has an incomplete declaration at line 168 column 7. 169 170 quit; NOTE: The SAS System stopped processing this step because of errors. NOTE: The PROCEDURE OPTMODEL printed pages 1-2. NOTE: PROCEDURE OPTMODEL used (Total process time): real time 16.42 seconds cpu time 16.34 seconds 171 172 173 proc printto; 174 run;
... View more