topic Re: Can SAS/OR create an MPS text file? in Mathematical Optimization, Discrete-Event Simulation, and OR

Can SAS/OR create an MPS text file?

cuencomi — Tue, 19 Mar 2019 15:44:48 GMT

Re: Can SAS/OR create an MPS text file?

RobPratt — Tue, 19 Mar 2019 16:00:46 GMT

Yes. Use the SAVE MPS statement in PROC OPTMODEL. There is also a SAVE QPS statement for quadratic programming models.

Re: Can SAS/OR create an MPS text file?

cuencomi — Fri, 22 Mar 2019 01:55:22 GMT

It saves the MPS file as a SAS dataset not a text file.

Re: Can SAS/OR create an MPS text file?

RobPratt — Fri, 22 Mar 2019 03:43:56 GMT

You can convert from data set to text file with the following DATA step:

%macro createMpsFile(ds,fn,free=1);
   option missing = '';
   data _null_ ;
      set &ds;
      file &fn;
      %if &free %then %do;
      if FIELD1 in ('NAME','ROWS','COLUMNS','RHS','BOUNDS','ENDATA','RANGES','QSECTION') then
         put @1 FIELD1 FIELD3;
      else
         put @2 FIELD1 FIELD2 FIELD3 FIELD4 FIELD5 FIELD6;
      %end;
      %else %do;
      if FIELD1 in ('NAME','ROWS','COLUMNS','RHS','BOUNDS','ENDATA','RANGES','QSECTION') then
         put @1 FIELD1 @15 FIELD3;
      else
         put @2 FIELD1 @5 FIELD2 @15 FIELD3 @25 FIELD4 @40 FIELD5 @50 FIELD6;
      %end;
   run;
   option missing = .;
%mend createMpsFile;

Re: Can SAS/OR create an MPS text file?

cuencomi — Sun, 24 Mar 2019 18:53:11 GMT

Hi Rob,

It worked! Thanks for your quick response and for sharing your expert knowledge of SAS/OR.

On another note. I work with OR folks that primarily use Java, Python, CPLEX and Gurobi tools. Coming from a SAS background, I wanted to introduce and complement these tools with SAS/OR. On a few models I tested, I ran "out of memory" with proc optmodel when the same models ran okay with CPLEX.

I know it may be just my inexperience with SAS/OR. Could you please point me to best practices and options with SAS/OR to avoid the "out of memory" problem and make SAS/OR models run more efficiently? Most of the models we use are MILP models.

Thanks,

Michael Cuenco

Re: Can SAS/OR create an MPS text file?

RobPratt — Sun, 24 Mar 2019 23:16:41 GMT

What version of SAS/OR are you using? Can you please share your PROC OPTMODEL code for one of these models?

Re: Can SAS/OR create an MPS text file?

cuencomi — Fri, 29 Mar 2019 02:47:26 GMT

SAS/OR 14.3. I will check what model I can share.

Re: Can SAS/OR create an MPS text file?

cuencomi — Fri, 29 Mar 2019 19:59:26 GMT

Here is the PROC OPTMODEL code. For the problem size there are 113 facilities and 2821 customers.

Re: Can SAS/OR create an MPS text file?

RobPratt — Mon, 01 Apr 2019 15:04:15 GMT

I can't run this without your data, but I already see something that will help. You have this constraint:

/* define constraint: distance from customer i to site j is less than MaxStemDist to meet service */
   con Service {i in CUSTOMERS, j in FACILITIES}: PDStemTravelDistance[i,j] * Assign[i,j] <= MaxStemDist;

The intent is to force Assign[i,j] = 0 when PDStemTravelDistance[i,j] > MaxStemDist. But it would be much more efficient to avoid declaring those variables in the first place. Please see this documentation example that illustrates the idea.

Another idea that might help is to omit the following constraints:

/* define constraint: if any customer i is assigned to facility j, then facility j must be used/open */
   con AssignImpliesOpen {i in CUSTOMERS, j in FACILITIES}: Assign[i,j] <= Build[j];

Although they do tighten the linear relaxation, they are not strictly required because the Capacitated constraints already enforce the logical implication: if Assign[i,j] = 1 then Build[j] = 1.

Re: Can SAS/OR create an MPS text file?

cuencomi — Wed, 03 Apr 2019 14:33:31 GMT

Rob,

Thanks for your help. I am learning a lot from you. I first tried your suggestions on two medium-sized models and they significantly reduced the memory used. I will set it up next to test on a big model.

These are two great suggestions. Would these two be amenable to build into the pre-solver?

Thanks,

Michael

Re: Can SAS/OR create an MPS text file?

cuencomi — Wed, 03 Apr 2019 16:44:13 GMT

I ran out of memory using the model with 113 facilities and 2821 customer sites. Any other steps I can take? Thanks.

NOTE: Problem generation will use 4 threads.
NOTE: The problem has 303155 variables (0 free, 0 fixed).
NOTE: The problem uses 2 implicit variables.
NOTE: The problem has 303155 binary and 0 integer variables.
NOTE: The problem has 2934 linear constraints (113 LE, 2821 EQ, 0 GE, 0 range).
NOTE: The problem has 606197 linear constraint coefficients.
NOTE: The problem has 0 nonlinear constraints (0 LE, 0 EQ, 0 GE, 0 range).
NOTE: The remaining solution time after problem generation and solver initialization is 7181.98 seconds.
NOTE: The MILP presolver value AUTOMATIC is applied.
NOTE: The MILP presolver removed 3686 variables and 0 constraints.
NOTE: The MILP presolver removed 7372 constraint coefficients.
NOTE: The MILP presolver modified 0 constraint coefficients.
NOTE: The presolved problem has 299469 variables, 2934 constraints, and 598825 constraint coefficients.
NOTE: The MILP solver is called.
NOTE: The parallel Branch and Cut algorithm is used.
NOTE: The Branch and Cut algorithm is using up to 4 threads.
Node Active Sols BestInteger BestBound Gap Time
0 1 2 1812022 0 1812022 117
0 1 2 1812022 1052148 72.22% 119
0 1 2 1812022 1063880 70.32% 198
0 1 2 1812022 1068108 69.65% 225
0 1 2 1812022 1069878 69.37% 267
0 1 2 1812022 1070901 69.21% 333
0 1 2 1812022 1071674 69.08% 402
0 1 2 1812022 1072197 69.00% 480
0 1 2 1812022 1072624 68.93% 564
0 1 2 1812022 1072975 68.88% 662
0 1 2 1812022 1073282 68.83% 764
0 1 2 1812022 1073570 68.78% 875
0 1 2 1812022 1073762 68.75% 955
0 1 2 1812022 1073951 68.72% 1072
NOTE: The MILP solver added 1671 cuts with 13817 cut coefficients at the root.
1 2 2 1812022 1073951 68.72% 1091
752 1 2 1812022 1074456 68.65% 1823
4 The SAS System 09:43 Wednesday, April 3, 2019

956 202 2 1812022 1074498 68.64% 2000
4138 3366 2 1812022 1074498 68.64% 2500
6440 5652 2 1812022 1075663 68.46% 3000
8846 8047 2 1812022 1075806 68.43% 3502
12085 11280 2 1812022 1075809 68.43% 4000
15627 14815 2 1812022 1075857 68.43% 4500
18667 17843 3 1812012 1075857 68.42% 4873
18807 17982 3 1812012 1075857 68.42% 5006
22767 21926 3 1812012 1075858 68.42% 5500
ERROR: Out of memory.
NOTE: Objective of the best integer solution found = 1812012.3024.

NOTE: PROCEDURE OPTMODEL used (Total process time):
real time 1:37:58.90
user cpu time 4:38:00.91
system cpu time 2:01.14
memory 8095586.56k
OS Memory 8186944.00k
Timestamp 04/03/2019 11:23:25 AM
Step Count 15 Switch Count 9261
Page Faults 785
Page Reclaims 0
Page Swaps 0
Voluntary Context Switches 605112
Involuntary Context Switches 341743
Block Input Operations 1
Block Output Operations 67

Re: Can SAS/OR create an MPS text file?

RobPratt — Wed, 03 Apr 2019 16:56:45 GMT

Yes, the MILP presolver deals with these, but declaring the variables and constraints (and postsolving them) still requires memory.

Re: Can SAS/OR create an MPS text file?

RobPratt — Wed, 03 Apr 2019 16:57:28 GMT

Can you please share the data?

Re: Can SAS/OR create an MPS text file?

cuencomi — Thu, 04 Apr 2019 14:10:07 GMT

Sorry, but the data is confidential and I am not allowed to share it.

Re: Can SAS/OR create an MPS text file?

RobPratt — Thu, 04 Apr 2019 14:38:34 GMT

Even obfuscated data would be useful. A successful approach using Benders decomposition is provided in this earlier post.

Re: Can SAS/OR create an MPS text file?

cuencomi — Fri, 05 Apr 2019 17:33:31 GMT

I will anonymize the data. In what format do you want me to send?

Re: Can SAS/OR create an MPS text file?

RobPratt — Fri, 05 Apr 2019 17:37:02 GMT

Ideally, you can provide the three SAS data sets that appear in the READ DATA statements.

Re: Can SAS/OR create an MPS text file?

cuencomi — Fri, 05 Apr 2019 18:47:48 GMT

One file is too large to attach.

Re: Can SAS/OR create an MPS text file?

cuencomi — Fri, 05 Apr 2019 18:48:38 GMT

here are the 2 smaller files.

Re: Can SAS/OR create an MPS text file?

RobPratt — Fri, 05 Apr 2019 19:01:33 GMT

Can you zip the large one? Or include only the needed observations with PDStemTravelDistance <= MaxStemDist.