topic Re: Very slow run-time in SAS Programming

Very slow run-time

Cobrad — Thu, 28 Jul 2016 08:43:42 GMT

Dear SAS

We have recently installed SAS on our company network(we use a VM set-up with multiple users on the server - we use SAS analytics pro (not SAS server). mainly Base SAS and EG are used).

The run-times are extremely slow. see log:

NOTE: There were 24868185 observations read from the data set PREP_CUR.BOOK_201606.
NOTE: The data set SUMM.BOOK_SUMMARY_201606 has 2038 observations and 7 variables.
NOTE: PROCEDURE SUMMARY used (Total process time):
real time 3:49.17
cpu time 13.99 seconds

I noticed that the CPU time is much faster than the real time.

can you perhaps tell what the issue might be by looking at the run times? we also find that if 4 users run code at the same time the speed slows down even further.

My gutfeel is that the computations happen fast (CPU time), but then something is happening to make the overall process very slow.

Do you have any suggestions on how we can speed up run-time?

Regards

Conrad.

Re: Very slow run-time

RW9 — Thu, 28 Jul 2016 08:55:05 GMT

You would be best off talking to your IT and running tests through the network. 24million observations is a far amount of data and if that is being transmitted anywhere could take a substantial amount of time. You don't mention where the data resides - this should be on the server where the VM's are so that the data is not being moved around. From that little bit of info, it doesn't seem to be the processing which is taking the time, so some network problem.

Re: Very slow run-time

Cobrad — Thu, 28 Jul 2016 09:00:20 GMT

Hi

Thanks for the prompt responds.

The data resides on the VM - hence no copying data via networks.

Can it perhaps be that the CPU is very good, but the disk read/write speed is slow? From my understanding I believe that SAS requires fast disk speed?

Regards

C.

Re: Very slow run-time

RW9 — Thu, 28 Jul 2016 09:05:37 GMT

Well, for I/O purposes a faster disk is always going to speed things up, but a 3.5hr time versus 12 second process seems to be another issue, I think somehow the data and the processor aren't connected optimally. As I say, best to speak with your IT group as they can test the process, see where things are happening.

Re: Very slow run-time

Cobrad — Thu, 28 Jul 2016 09:07:52 GMT

Thanks a lot - much appreciated.

Note the real time is 3 minutes, not 3 hours.

But what I will do is to ask IT to see where things are going slow.

Thanks again!

C.

Re: Very slow run-time

RW9 — Thu, 28 Jul 2016 09:32:43 GMT

Doh, don't trust me, I can't even read! 3 minutes isn't too bad then really for 24mill records (am wondering if I read that right now), your IT might be able to suggest some things to speed it up, SSD for instance, or more memory, but overall thats not too bad.

Re: Very slow run-time

Reeza — Thu, 28 Jul 2016 11:00:57 GMT

Quick FYI - this isn't 'SAS' if you want an answer from SAS contact tech support.

Processing 24million records in 3 minutes across a network isn't bad though you don't show the actual code so we don't know what els is happening in your data step. Assuming just a quick filter?

Re: Very slow run-time

Cobrad — Thu, 28 Jul 2016 11:09:52 GMT

Thanks for the FYI.

That was a normal data step and I created one or 2 variables - nothing fancy.

I am in contact with my IT people now - we established that it is a disc read issue - they are looking to migrate us onto faster discs.

Thanks a lot for the help - I appreciate it!!