12-15-2014 05:44 AM
I am running a sas macro to enumerate combinations of large number: eg., 150 choose 20=150!/(150-20)! 20!. Even though I deleted at temporary data set using
proc dataset library=work kill;
I still have the following issues
"OUT OF RESOURCES Select"
-N Tell procedure/DATA step no more resources
-C Cancel Submitted Statement
- T. Terminate SAS
What is wrong with my SAS? Does it mean SAS cannot handle combination of large numbers?
12-15-2014 06:26 AM
Thanks RW9! I don't think the macro is generation infinite loop.
It worked perfect with smaller numbers and gave output in no time.
The macro can be found in the SUG paper in the link below: it's the %setup macro.
12-15-2014 06:44 AM
Well, looking at that paper it is from 1998 so I would assume there are better ways of doing it now, maybe even a SAS procedure. Its not something I do so I don't know without investigating. Maybe someone else can suggest (maybe posting this in Statistical procedures). Off the top of my head reading that paper it is generating, based on parameters and variables as series of loops, each inner one based on the one above. Hence if you have a lot then it would exponentially increase the number of loops it is generating. Maybe do it in a couple of steps, i.e. do the step for <= 50 as ds1, <= 100 as ds2, <= 150 as d3, then set the 3 together.
12-15-2014 09:30 AM
SAS has a limit of 15 significant digits for storing integers. Even if SAS figures out how to abbreviate the math, this computation would stretch beyond that limit:
150 * 149 * 148 * ... * 133 * 132 * 131
There are some details, such as powers of two being factored out. But even the odd numbers alone in those 20 numbers would overwhelm the limit of 15 significant digits.
12-15-2014 09:36 AM
Have you actually considered the number of records you will try to write?
Try to get a grasp of your quantity structure before running out of disk space or memory.
12-15-2014 11:03 AM
If I read your original equation right, that would amount to (using SAS code) fact(150)/fact(150-20)*fact(20). Is that correct?
12-16-2014 02:09 AM
Well, that then amounts to ~ 2*10**61 (!!!) combinations. How many Google datacenters will you rent for this to write to? As I said, you need to get a basic grasp of the volumes involved.