SAS slow operation for Multiple cross-tabulations for bootstrap-based frequency

[mandan414]

Hello,

I'm operating cross tabulations like below:

proc surveyfreq data=mydata varmethod=BRR;         

                tables Var1 * Var2 * Var3

                           Var1 * Var2 * Var4

                           Var1 * Var2 * Var5 

                            ………………

                            ……………..

                            ……………..

                           Var1 * Var2 * Var9 / row cl nofreq nowt nocellpercent nototal;

                repweights BSW1-BSW1000;

                weight  weight_var;

run;

Var1 has 5 values, var2 has 2 values and var3 or var 4...… can have from 3 to 6 values. SAS operation is really slow for this. I tried only one table  (tables Var1 * Var2 * Var3 ) with 2 bootstrap weight (BSW1-BSW2) and it took SAS 1.5mins to complete it. Any suggestions?

Thank you so much.

[ballardw]

How many records are involved in this data set?

[mandan414]

Above 100,000 records. Thanks!

[Reeza]

Is your table local or on a network drive? Have you tried loading the file with the SASFILE command?
I'm assuming you have decent RAM as well, if you're using SAS University Edition the default settings are low.

[mandan414]

Reeza. My SAS is on network drive. It's for my work and I must use the network and work computer, no other choice. Thanks.

[SASKiwi]

Where does your SAS WORK library point to? I'm hoping it is your local drive. If so run a test using data copied to WORK to see if that improves performance. 

[mandan414]

SAS work library is on the network. Thanks.

[Reeza · Accepted solution]

Not to be a pest and repeat myself again, but have you tried SASFILE?
I would assume a bootstrap option would require many reads so it may help.

[mandan414]

Hi Reeza,

I tried it, it's super! Much much faster! Although, I still don't know what SASFILE exactly does. I'm going to study about it. Thank you!

[Reeza]

It loads the data into RAM so it's not being re-read constantly so it should speed things up significantly when there are multiple reads of the same file and you're using it for data intensive calculations.

[andreas_lds]

---

@mandan414 wrote:

SAS work library is on the network. Thanks.

---

This is ok, if - and only if - sas runs on a server. If sas is installed on your computer, having work on a network share is, kindly put, a really bad idea.

[mandan414]

I don't quite understand it. Could you explain it a little more? Thank you for your time.

[andreas_lds]

If you have a local installation of sas any network-traffic will, most likely, slow down the execution of you sas job. So using local drives is highly recommended.