topic Re: Testing script concurrently that fully saturates all CPUs in SAS Enterprise Guide

Testing script concurrently that fully saturates all CPUs

CloudyMind — Wed, 29 Sep 2021 00:08:24 GMT

Hi SAS experts,

Our project is to upgrade CPUs from 6 cores to 16 cores and we want to show our stakeholders the improvement before and after the upgrade complete.

Therefore, I am trying to create a complex script that fully saturates all CPUs capacity and it still doesn't create a huge impact on CPU processing.

And I would really appreciate if you can give me some directions here.

Thank you


/* Start timer */
%let _timer_start = %sysfunc(datetime());



%MACRO LOOPCOMN (Output=, String1= , String2= , String3=);
DATA COMPGED_&OUTPUT ;
X= &String1;
Y= &String2;
Z= &String3;
Cost1 = compged(X,Y);
Cost2 = compged(Y,Z);
Cost3 = compged(X,Z);
Calculation1 = SUM(( compged(X,Y)),(compged(Y,Z)));
result1=soundex(X);
result2=soundex(Y);
result3=soundex(Z);

  call symput('var1',Cost1);
  call symput('var2',Cost2);
  call symput('var3',Cost3);

RUN;

%MEND ;
%LOOPCOMN 
(Output = Test1,
 String1= 'kjfkjdsfkjhsdkjfkjshkjdshkjshkjhkjhkjhkhkjhgkjfdhkjfdhkjhkjhgkjhdfkjghfdkjhkjhkurwhyiwuh', 
 String2= 'uriufkjvcbvijivkjnienijninuijnlkjnvijnr kjnkjnkjrnijnkjnvrnj ourjoijvkmlkmntrobtr' ,
 String3= 'sdfbksjgbaorgtpaorkalsjdbvnaklijfgiUAToijfelMBNVCkjfgaiurghODFLDKJGNIUEARHOijflgkjbhg')
;
%LOOPCOMN 
(Output = Test2,
 String1= 'kjfkjdsfkjhsdkjfkjshkjdshkjshkjhkjhkjhkhkjhgkjfdhkjfdhkjhkjhgkjhdfkjghfdkjhkjhkurwhyiwuhuriufkjvcbvijivkjnienijninuijnlkjnvijnr kjnkjnkjrnijnkjnvrnj ourjoijvkmlkmntrobtr', 
 String2= 'uriufkjvcbvijivkjnienijninuijnlkjnvijnr kjnkjnkjrnijnkjnvrnj ourjoijvkmlkmntrobtraorgtpaorkalsjdbvnaklijfgiUAToijfelMBNVCkjfgaiurghODFLDKJGNI' ,
 String3= 'sdfbksjgbaorgtpaorkalsjdbvnaklijfgiUAToijfelMBNVCkjfgaiurghODFLDKJGNIUEARHOijflgkjbhgrjoijvkmlkmntrobtraorgtpaorkalsjdbvnaklijfgiUAToijfelMBNVCkjfgkjfkjdsfkjhsdkjfkjshkjdshkjshkjhkjhkjhkhkjhgkjfdhkjfdhkjhkjhgkjhdfkjghfdkjhkjhkurwhyiwuhuriu')
;


%macro loop( start= , end= , by= ) ;

%local i;
%do i = 1 %to %eval(%sysfunc(Ceil(%sysevalf((&end - &start ) / &by ) ) ) +1) ;
   %let value=%sysevalf(( &start - &by ) + ( &by * &i )) ;
   %if &value <=&end %then %do;
      %put &value;
   %end;
%end ;
%mend loop ;

%loop(start = 1 , end = 10000000 , by = .000000125 )

data T;
do I=1 to 100000; 
A=sha256(prxchange('s/a/b/',-1,repeat('a',2000))); end;
run;

/* Stop timer */
data _null_;
  dur = datetime() - &_timer_start;
  put 30*'-' / ' TOTAL DURATION:' dur time13.2 / 30*'-';
run;

Re: Testing script concurrently that fully saturates all CPUs

mkeintz — Wed, 29 Sep 2021 02:28:49 GMT

I'm not quite sure what you mean by

I am trying to create a complex script that fully saturates all CPUs capacity and it still doesn't create a huge impact on CPU processing.

Can you expand on this?

Re: Testing script concurrently that fully saturates all CPUs

SASKiwi — Wed, 29 Sep 2021 03:53:50 GMT

What type of "improvements" are you expecting? As a general rule additional CPUs are not going to speed up your SAS processing. It will allow you to run more simultaneous jobs without maxing out your CPU usage.

Rather than develop an artificial test, I'd suggest you make use of one of your regular production jobs that takes at least 5-10 mins to run. Modify it so it doesn't output anything permanent, then run it in batch mode. Make a note of the % CPU used. Then run the same job again twice at the same time, three times, four times, etc. Running six instances of the same job simultaneously should heavily load a SAS app server with only six processors. Run more instances if it doesn't.

Repeat this process after the CPU upgrade and you should now see a much reduced % CPU usage. Again you can run many more instances of your test job to confirm how many it can now cope with.

Re: Testing script concurrently that fully saturates all CPUs

Kurt_Bremser — Wed, 29 Sep 2021 05:42:37 GMT

Whenever you read and/or write data, the I/O subsystem most likely becomes the limiting factor. Unless you run mostly CPU intensive regression jobs or do all processing in-memory (CAS), look at improving your storage first.