Out of Resources

Reply
Contributor
Posts: 72

Out of Resources

I am running a sas macro to enumerate combinations of large number: eg., 150 choose 20=150!/(150-20)! 20!. Even though I deleted at temporary data set using

proc dataset library=work kill;

run;

I still have the following issues

"OUT OF RESOURCES Select"

-R. Retry

-N Tell procedure/DATA step no more resources

-C Cancel Submitted Statement

- T. Terminate SAS

What is wrong with my SAS? Does it mean SAS cannot handle combination of large numbers?

Super User
Super User
Posts: 7,682

Re: Out of Resources

Post your code.  Could it be that your macro is generating an infinite loop?

Contributor
Posts: 72

Re: Out of Resources

Thanks RW9! I don't think the macro is generation infinite loop.

It worked perfect with smaller numbers and gave output in no time.

The macro can be found in the SUG paper in the link below: it's the %setup macro.

http://www2.sas.com/proceedings/sugi23/Posters/p177.pdf

Thanks

Super User
Super User
Posts: 7,682

Re: Out of Resources

Well, looking at that paper it is from 1998 so I would assume there are better ways of doing it now, maybe even a SAS procedure.  Its not something I do so I don't know without investigating.  Maybe someone else can suggest (maybe posting this in Statistical procedures).  Off the top of my head reading that paper it is generating, based on parameters and variables as series of loops, each inner one based on the one above.  Hence if you have a lot then it would exponentially increase the number of loops it is generating.  Maybe do it in a couple of steps, i.e. do the step for <= 50 as ds1, <= 100 as ds2, <= 150 as d3, then set the 3 together.

Contributor
Posts: 72

Re: Out of Resources

Thanks.


Super User
Posts: 9,856

Re: Out of Resources

Are you trying to calculate  density function of hypergeometric distribution  ?

Doc Rick has already answered it . use a sas build-in function to get it .

Xia Keshan

Contributor
Posts: 72

Re: Out of Resources

I am not trying to comput hypergeometric density function. I am trying to enumerate the distinct combination of 150 chose 17.

Thanks.

Super User
Posts: 5,353

Re: Out of Resources

SAS has a limit of 15 significant digits for storing integers.  Even if SAS figures out how to abbreviate the math, this computation would stretch beyond that limit:

150 * 149 * 148 * ... * 133 * 132 * 131

There are some details, such as powers of two being factored out.  But even the odd numbers alone in those 20 numbers would overwhelm the limit of 15 significant digits.

Contributor
Posts: 72

Re: Out of Resources

Thanks

Super User
Posts: 7,383

Re: Out of Resources

Have you actually considered the number of records you will try to write?

Try to get a grasp of your quantity structure before running out of disk space or memory.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Contributor
Posts: 72

Re: Out of Resources

Thanks

Super User
Posts: 7,383

Re: Out of Resources

If I read your original equation right, that would amount to (using SAS code) fact(150)/fact(150-20)*fact(20). Is that correct?

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Contributor
Posts: 72

Re: Out of Resources

yes, please.

I have 150 names that I am trying to enumerate all possibe combinations 20 of the names taken at a time.

thanks.

Super User
Posts: 7,383

Re: Out of Resources

Well, that then amounts to ~ 2*10**61 (!!!) combinations. How many Google datacenters will you rent for this to write to? As I said, you need to get a basic grasp of the volumes involved.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
SAS Super FREQ
Posts: 3,618

Re: Out of Resources

val = comb(170, 20);

Ask a Question
Discussion stats
  • 17 replies
  • 1099 views
  • 0 likes
  • 6 in conversation