topic Re: Comparing 2 SAS servers - program runs 4 hours longer on one server - Block Input Operations ? in Administration and Deployment

Comparing 2 SAS servers - program runs 4 hours longer on one server - Block Input Operations ?

gothaggis — Mon, 13 Aug 2018 15:07:30 GMT

Have one physical server and one virtual server. Programs run on the virtual server seem to take longer. I have enabled performance logging. One thing I have noticed is that on the slower server, the value of Block Input Operations is always 0 - this is not the case on the faster physical server. The CPU clock speed is faster on the virtual server (2.3 vs 2.0ghz)

faster physical server:

NOTE: There were 87651 observations read from the data set 
NOTE: PROCEDURE PRINT used (Total process time):
      real time           34:12:29.27
      user cpu time       34:11:31.35
      system cpu time     51.16 seconds
      memory              59511.96k
      OS Memory           72964.00k
      Timestamp           08/10/2018 10:50:41 PM
      Step Count                        2  Switch Count  0
      Page Faults                       95
      Page Reclaims                     29512917
      Page Swaps                        0
      Voluntary Context Switches        2278
      Involuntary Context Switches      56795
      Block Input Operations            418712
      Block Output Operations           1298184

      69         ods csv close;
NOTE: The SAS System used:
      real time           35:07:26.82
      user cpu time       35:06:05.80
      system cpu time     1:05.83
      memory              66841.34k
      OS Memory           72964.00k
      Timestamp           08/10/2018 10:50:41 PM
      Step Count                        2  Switch Count  20
      Page Faults                       229
      Page Reclaims                     29610854
      Page Swaps                        0
      Voluntary Context Switches        47504
      Involuntary Context Switches      60589
      Block Input Operations            923088
      Block Output Operations           1669976

slower virtual server:

NOTE: There were 87651 observations read from the data set 
NOTE: PROCEDURE PRINT used (Total process time):
      real time           37:10:44.68
      user cpu time       37:09:48.54
      system cpu time     52.78 seconds
      memory              61736.18k
      OS Memory           77348.00k
      Timestamp           08/11/2018 01:42:20 AM
      Step Count                        2  Switch Count  0
      Page Faults                       0
      Page Reclaims                     15173
      Page Swaps                        0
      Voluntary Context Switches        2519
      Involuntary Context Switches      41989
      Block Input Operations            0
      Block Output Operations           1298184

67         ods csv close;
NOTE: The SAS System used:
      real time           37:59:02.81
      user cpu time       37:57:21.07
      system cpu time     1:38.17
      memory              70975.18k
      OS Memory           77348.00k
      Timestamp           08/11/2018 01:42:20 AM
      Step Count                        2  Switch Count  20
      Page Faults                       0
      Page Reclaims                     23818
      Page Swaps                        0
      Voluntary Context Switches        2749
      Involuntary Context Switches      48514
      Block Input Operations            0
      Block Output Operations           1639480

any ideas why its so much slower on the virtual server? Does it have anything to do with Block Input Operations?

Re: Comparing 2 SAS servers - program runs 4 hours longer on one server - Block Input Operations ?

Tom — Mon, 13 Aug 2018 18:10:04 GMT

Times don't look that different to me.

34/35 vs 37

Re: Comparing 2 SAS servers - program runs 4 hours longer on one server - Block Input Operations ?

gothaggis — Mon, 13 Aug 2018 18:16:36 GMT

those are hours - so 34/35 hours vs 37 hours.

Re: Comparing 2 SAS servers - program runs 4 hours longer on one server - Block Input Operations ?

SASKiwi — Mon, 13 Aug 2018 20:17:13 GMT

Looks like your virtual servers are using more CPU time - nearly 10% more. Is this true of all SAS jobs? Due you have access to SAS Environment Manager? This may give you a better understanding of the differences.

Re: Comparing 2 SAS servers - program runs 4 hours longer on one server - Block Input Operations ?

MargaretC — Mon, 13 Aug 2018 21:59:14 GMT

If the two underlying systems are the same, (same chipset; same number of cores/RAM; same connections to storage; same storage) this it is probably just the overhead of virtualization – which can be up to 15%.

It looks like there is a 10% difference in the times you have listed (2053 minutes compared to 2280 minutes) in the PROC PRINT statements. So, it looks to be just the overhead of virtualization.

Cheers,

Margaret

Re: Comparing 2 SAS servers - program runs 4 hours longer on one server - Block Input Operations ?

Kurt_Bremser — Thu, 16 Aug 2018 11:32:11 GMT

You have a proc print to csv of a dataset with less than 100000 observations, and it runs more than 30 hours (and eats more than 30 hours of CPU time)? What kind of dataset is that?

Re: Comparing 2 SAS servers - program runs 4 hours longer on one server - Block Input Operations ?

SASKiwi — Fri, 17 Aug 2018 23:21:51 GMT

@Kurt_Bremser Looks like the OP is using ODS to create a CVS using PROC PRINT. In my experience ODS is a resource hog when writing out large files. I bet a simple PROC EXPORT using DBMS = CSV would be hugely faster.

Re: Comparing 2 SAS servers - program runs 4 hours longer on one server - Block Input Operations ?

Tom — Fri, 17 Aug 2018 23:41:25 GMT

Faster than PROC EXPORT is to just use a data step.

data _null_;
  file 'myfile.csv' dsd ;
  set have ;
  put (_all_) (+0);
run;

You can even have the data step write the header row as @Data_null_ has posted multiple places including:

https://communities.sas.com/t5/Base-SAS-Programming/Output-to-delimited-format/m-p/292822#M60848

Here is a link to macro that uses that method:

https://github.com/sasutils/macros/blob/master/csv_vnext.sas

Re: Comparing 2 SAS servers - program runs 4 hours longer on one server - Block Input Operations ?

Kurt_Bremser — Mon, 20 Aug 2018 06:38:45 GMT

@SASKiwi wrote:

@Kurt_Bremser Looks like the OP is using ODS to create a CVS using PROC PRINT. In my experience ODS is a resource hog when writing out large files. I bet a simple PROC EXPORT using DBMS = CSV would be hugely faster.

In German-speaking countries, we call this "carrying the church around the cross". Using ODS which creates lots of formatting, just to completely lose that in the csv is - ahhh - well, I'm in a civil mood today, so I won't write it here.

@gothaggis if this is actually the case and you just wanted to avoid the 10 minutes of work it takes to write the data step (or even simpler: the proc export) for a csv export, let this be an object lesson in "do not take shortcuts".

And have a look at Maxim 14.