topic Re: General Question regarding the speed to run simulation studies using SAS 9.4 in SAS/IML Software and Matrix Computations

General Question regarding the speed to run simulation studies using SAS 9.4

souwwang — Mon, 29 Sep 2014 14:21:11 GMT

I used SAS 9.4 in a laptop to run some simulation studies. At beginning, the program runs very fast, just keep running and running. After the program ran for several hours, it become much slower.

For example, at beginning, it will take the program 2 hrs to finish one simulation scenario, but it may take two days to finish similar scenario some time later. I checked the log and found that SAS will stop for several minutes and do nothing after it finishes one simulation data sets, then it will continue. Any ideas about what happened, and how I can resolve it?

Thanks for the help.

Re: General Question regarding the speed to run simulation studies using SAS 9.4

Rick_SAS — Mon, 29 Sep 2014 14:50:56 GMT

We probably need more details, but my guess is that you are appending the results of each simulation. Read this article and see if it describes what you are attempting: Simulation in SAS

If that doesn't describe what you are doing, post some of your code.

Re: General Question regarding the speed to run simulation studies using SAS 9.4

souwwang — Tue, 30 Sep 2014 13:54:35 GMT

Thanks for the reply. I read that article, actually I used DO in my simulation codes instead of BY. Your 'BY' method is efficient, but since I need Bootstrap method to calculate 95% CI, so I cannot concatenate the data sets together and use 'BY' method, since it is too big. Actually, my problem is not code specific. When the codes generate the first several hundred simulated data set using DO loop, the program runs very fast, but as time goes on, it looks like something is stored in the memory that slows down the program. That is what I observed.

Re: General Question regarding the speed to run simulation studies using SAS 9.4

Rick_SAS — Tue, 30 Sep 2014 14:45:13 GMT

Great. In SAS/IML that is one way to do it if you have big data: use a DO loop for each bootstrap resample and store the statistics for each resample in an array.

However, you need to preallocate your array for the results. It sounds like you might be using a concatenation operator inside the DO loop to dynamically increase the array. Read this article: Pre-allocate arrays to improve efficiency - The DO Loop