SAS Optimization, and SAS Simulation Studio

turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-25-2016 09:41 PM

I have a working optimization model that I want to run on a really big data set (> 10million observations) and am receiving an out of memory error. I was hoping to export out the model in MPS format so that I can try running the model in other software. But, I can't find a way to create a .mps file from SAS.

Can you help me out.

Thanks,

Sashi

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-30-2016 11:38 AM

Are you using PROC OPTMODEL? Can you please show the code and log for the error?

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-30-2016 11:55 AM

Yes, I'm using PROC OPTMODEL. Below code is for a sample of 400K customers. The full dataset has 30MM customers.

Here's the code and I have attached the log file:-

proc optmodel;

/* declare sets and data indexed by sets */

set <string> Email;

set <string> Brand;

set <number> Custs;

num Circulation{Email};

num Priority{Email};

num Percentage{Brand};

num ExpectedSpend{Custs,Email,Brand};

/* declare the variable */

var Pick{Custs,Email} binary init 0;

/* maximize objective function (Total Spend) */

maximize TotalSpend = sum{i in Custs, j in Email, k in Brand} ExpectedSpend[i,j,k] * Pick[i,j] * Priority[j];

/* subject to constraints */

con CircLimit {j in Email} : sum {i in Custs} Pick[i,j] <= Circulation[j];

con indvMailconstraint {i in Custs}: sum {j in Email} Pick[i,j] = 1;

con brandSalesLimit {k in Brand}: sum {i in Custs, j in Email} ExpectedSpend[i,j,k] * Pick[i,j] * Priority[j] >= Percentage[k] * TotalSpend / 100;

/* abstract algebraic model that captures the structure of the */

/* optimization problem has been defined without referring */

/* to a single data constant */

/* populate model by reading in the specific data instance */

read data Email into Email=[Email] Circulation Priority;

read data Brand into Brand=[Brand] Percentage;

read data Customer into Custs=[Customer];

read data ExpSpend into [Customer Email Brand] ExpectedSpend[Customer, Email, Brand]=ExpSpend;

/* solve LP using primal simplex solver */

solve with lp / solver = primal_spx;

/* display solution */

print TotalSpend;

create data results.solution from [customer email]={Custs, Email} Assignment=Pick;

quit;

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-30-2016 12:25 PM

I see that you are using SAS/OR 12.1, which is four years old. We have had several releases since then, with performance improvements in each release. The latest release is SAS/OR 14.1.

You declare binary variables but then use the LP solver instead of the MILP solver. Was this intended? Also, is there a particular reason that you specify the primal simplex algorithm rather than the default dual simplex algorithm? In fact, the interior point algorithm might be best for this large-scale instance.

Can you please set the SAS FULLSTIMER option, rerun PROC OPTMODEL, and show the new log?

option fullstimer;

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-31-2016 02:49 PM

Thanks Rob... we forced it to use primal simplex for no particular reason. We were reusing older code and didn't think it made a difference. Now that you called it out, I've taken it out and just replaced it with a solve statement with the default solver. See attached log with the fullstimer option enabled.

What options do I give if I need to make it use the interior point algorithm as you suggested? I tried the following statement but it said that the IPNLP solver does not allow integer variables.

Solve with IPNLP / Solver = IPKRYLOV;

Are you able to tell me what resources does my system need in order to solve a problem that is this big? Since this is on a sample(400K), what about the full dataset of 30MM?

Can SAS generate a MPS file for my model that I can load to other solvers on the clod to take advantage of more system resources (Amazon AWS for example).. does SAS have an equivalent?

Let me know if you have any other suggestions that I can try out.

Thanks so much for your help.

Sashi

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

05-31-2016 03:17 PM

LP algorithm choice is documented here:

For example, to relax the integer variables and use the interior point algorithm:

solve with LP relaxint / algorithm=interiorpoint;

You might also try the default LP algorithm (dual simplex):

solve with LP relaxint;

Or network simplex:

solve with LP relaxint / algorithm=network;

For information about memory requirements, see:

You can save an MPS data set as described here:

SAS/OR does support parallel processing on a single machine or on a grid:

You should also consider trying the decomposition algorithm:

Note that some of this functionality has been added since SAS/OR 12.1, so please upgrade to take advantage of the added features and numerous performance improvements.