From time to time, what appear to be simple questions generate more thinking than initially anticipated. And off the back of one such question I have written this article, which refers to how generate load across multiple cores when you have a Base SAS / SAS Foundation environment and SAS/CONNECT (think a SAS compute/connect session in 9.4/Viya ).
Before diving into the detail my thanks to my colleague Mark Thomas who helped me with MP Connect code and to colleagues in Technical Support and R&D for raising this as a topic of interest. From time to time, customers ask SAS how to determine if SAS 9.4 is able to use all the cores available to it. This is sometimes as a result of their own testing which appears to be inconclusive e.g. a customer had over 12+ cores available to them but in their testing they were only seeing a handful of cores being leveraged. So what was going on? Well here are some reminders when it comes to Base SAS:
So with all this in mind, the question of whether users can generate SAS jobs that don't rely on sufficient data to generate enough CPU load, to occupy as many cores in a symmetric multiprocessing (SMP), server springs to mind. There are probably multiple solutions to this, but one that sprang to my mind was the generation of prime numbers. Is there a sufficient number of prime numbers to find within a set of numbers e.g. 1 - 10000, that would generate sufficient load so that a user/observer could see generating across all cores (or as many cores as needed). BUT we also have to ensure that the number is not too large that the test takes hours to complete. So we want to have some code which to some degree is inefficient but not too inefficient.
With help from Mark and his knowledge of MP Connect and macros, this is the code we came up with. In essence for this example we generate 10 distinct sessions (which equates to 10 of threads out of a possible 88, and yes this is a hyperthreaded host, so 88 threads = 44 cores) and sought to find the prime numbers available in the first 100,000 numbers (1-100,000) and write them to SAS datasets. If you're wondering there 9592 but our code generates 9591 into a table (can you guess which one it leaves out?). As mentioned earlier the code is meant to be inefficient but not too inefficient. So in our approach, we deliberately exclude all even numbers from the test. For greater efficiencies you can take a look these resources to get you started:
The code below is what we used for our tests:
Below is the log that it generates, when we look at the first 100,000 numbers. The interesting takeaway is that on the this specific server, it takes roughly between 53-56 seconds to create the tables at the server side.
Using the linux utility "htop" us to look at the demand on each core and if needed we can take a screenshot (as we did below). We did not test this using SAS Environment Manager, but there's a good chance that in Viya 3.5 you will see the spikes in CPU activity.
What does the CPU load look like for 10 sessions when run against a machine with 88 available threads (44 hyperthreaded cores)? Here's a screenshot when using the Htop command.
Some final thoughts and comments:
Well that's all folks. I hope you found it interesting and I'm sure that that there will be some of you who can come up with additional ways to load an SMP using Base SAS code. Feel free to share what you know in the comments section below.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.