topic Re: Allocating very large arrays in IML 14.1 in SAS/IML Software and Matrix Computations

Allocating very large arrays in IML 14.1

stevo642 — Mon, 27 Jul 2015 15:27:53 GMT

Last week I got some help with allocating memory for very large IML arrays. The post got deleted by accident so I thought I'd repost to assist those trying to do that.

In short, IML 14.1 supports matrices with up to 2³¹ elements on the Windows operating system.

I upgraded to 14.1 and have sufficient RAM but a simple "y = j(33000,33000,1);" wasn't working for me.

IIRC, Rick Wicklin suggested adding "-MEMSIZE 12G" to the shortcut I start SAS with.

I went a little deeper and edited that configuration into my SASv9.CFG file and now I can allocate large arrays on my Win7 16GB RAM machine. Thanks, Rick!

Incidentally, this lets me run much larger Monty Hall simulations -- up to a quarter of a billion sims -- see attached code & results below based on Rick's "DO Loop" Monty Hall example -- and remember to "switch doors"!

proc iml;

t0 = time();

NumSim = 2.5e8; /* number of simulated games */

car = randfun(NumSim, "Table", {1 1 1}/3); /* unknown door hides car */

guess = j(NumSim, 1, 1); /* WLOG, you guess door=1 */

show = choose(car=2, 3, 2); /* host shows you a goat --> if car behind 2 then 3 else 2 */

switch = choose(show=2, 3, 2); /* the door you could switch to -- if host shows 2 then 3 else 2 */

winIfSwitch = mean(switch=car); /* (P(win | switch) */

winIfStay = mean(guess=car); /* (P(win | do not switch) */

elapsedTime = time() - t0;

print "Monty Hall: NumSim = " NumSim[format=comma12. label=EMPTY] winIfStay[format=8.6 rowname=" p(WinIfStay)" label=EMPTY] winIfSwitch[format=8.6 rowname=" p(WinIfSwitch)" label=EMPTY] " Time in Seconds = " elapsedTime[format=4.1 label=EMPTY];

Monty Hall: NumSim = 250,000,000

p(WinIfStay) 0.333347

p(WinIfSwitch) 0.666653

Time in Seconds = 11.1

Re: Allocating very large arrays in IML 14.1

SteveDenham — Wed, 29 Jul 2015 17:16:44 GMT

Completely off topic-- @stevo642, how did you get access to 14.1?

Steve Denham

Re: Allocating very large arrays in IML 14.1

Rick_SAS — Wed, 29 Jul 2015 17:22:13 GMT

It was released last week!

Here's the What's New for the Analytical products: SAS 14.1 Documentation

And here's highlights: SAS Analytics: Highlights of 14.1, 13.2, 13.1, and 12.1 Releases

Re: Allocating very large arrays in IML 14.1

stevo642 — Wed, 29 Jul 2015 17:41:09 GMT

I got IML 14.1 when upgraded from 9.4 (TS1M1) to 9.4 (TS1M3)

Re: Allocating very large arrays in IML 14.1

SteveDenham — Wed, 29 Jul 2015 18:20:12 GMT

Hmm--I would have thought that there would have been a large splash headline on the Communities page...

Steve Denham

Re: Allocating very large arrays in IML 14.1

Rick_SAS — Wed, 29 Jul 2015 18:51:35 GMT

Wait for it...coming soon!

Re: Allocating very large arrays in IML 14.1

Rick_SAS — Fri, 31 Jul 2015 13:00:35 GMT

I wrote a blog post that describes this issue and the way to resolve it:

Large matrices in SAS/IML 14.1