Both multithreading and parallel processing (parallelism) are hot topics in the data analysis space. But what is the difference between the two concepts? Are there any differences at all? When people compare SAS and Python’s parallel or multithreaded computing ability, what do they mean? Hopefully this post can contextualize these phrases and how they can be used to optimize both SAS and Python programs.     Multithreading vs Parallel Processing    When you run a simple program, like a program to count from 1 to 1 billion, you are running a program on a single thread without any parallel processing. A program of that nature might look something like this in Python:    def count(firstnum, lastnum): num = firstnum while (num <= lastnum): num += 1 count(1, 1000000000) print(“Done!”)   Let’s imagine we split this code into two equally-sized chunks:    Code that counts from 1 to 500 million  Code that counts from 500 million to 1 billion    We’re effectively splitting the program into processes, and if we ran both processes at the same time, we would expect it to take about half as long to complete all the counting. Executing these 2 processes in parallel should accomplish this, but executing these processes on 2 different threads could only serve to slow down our counting program. Why is this? Multithreading and Parallel processing are not the same thing, despite their apparent similarity in “dividing and conquering” a given workload. Both can speed up certain tasks, but through separate means.    Multithreading, or the process of running one program across multiple threads, is all about concurrency. Concurrency refers to the act of splitting a problem into smaller parts that can be worked on individually.  In “doing” multithreading with this counting program, we’d start by splitting its execution into two halves (and placing these two halves on two different threads). The threads would execute concurrently, but this is not the same thing as executing simultaneously. If we output each process’s “count” number every time it’s incremented, we might get a messy output that looks something like this:    1  500000001  2  3  500000002  4  500000003  500000004  500000005  5 6 …     The program switches between our two different threads very quickly and tries to reduce the idle time of our CPU by having threads execute that are “ready”. In this case each thread has no measurable idle time with the negligible input/output of a counting program, so both are equally “ready” to resume counting. This all means that our counting program isn’t any faster multithreaded, and in fact the act of splitting into two threads could introduce a small amount of extra work that could raise the execution time. Here’s three outputs with the program on one thread, two threads, and four threads respectively:    Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.       The time varied between 28 and 36 seconds with a similar mean time of 32 shown here on each program’s output. As we can see, the splitting of our program into threads did not reduce the overall amount of time to complete the whole program.   However, multithreading can speed up a program if you’re dealing with latency. For example: when surfing the web a lot of things are happening to load a website at once; user input is checked, objects on a webpage have to load, code is run for personalization, ads are fetched, etc. In a multithreaded approach we can do all of these “things” whenever they’re ready and do other things when they aren’t. Another great use case for multithreading would be manipulating data in a table. There’s time between the input and output of items in a data table and using a multithreaded approach we can have the CPU essentially switch over to whichever thread is ready to execute on the table at a given time.     Parallel processing differs from multithreading in that our initial example of two “count” processes can actually run simultaneously. This is accomplished by running on two separate cores as opposed to the two threads on one core that we’d get with multithreading. The thing that most often confuses people is that technically you are still running on two different threads here, but the operative word is “core”. Splitting the processes into two different cores allows for truly simultaneous execution of each process, potentially halving the execution time in this particular case.        Multithreading and Parallelism in SAS    SAS9 has had multithreading and parallel processing capabilities for quite some time. Here is a link to SAS procedures in SAS Foundation that currently leverage multi-threadiness. An article from the 2010 SAS Global Forum talks about both concepts: here. The article specifically details running multiple parallel batch SAS sessions to work with many chunks of a data table, and joining the output of those processes after each is completed. While I was playing at recess in elementary school (2010), SAS employees were hard at work getting I/O parallelism to improve data step computation speeds.    SAS/CONNECT is the next logical step from SAS9 parallelism, it’s a set of tools that links a SAS client session to a SAS server session for you. This can allow for easier parallel processing as well as other features that are outside the scope of this post. For more information pertaining to SAS/CONNECT, please refer to the documentation.    In a previous post, I used CAS – the SAS Viya Cloud Analytics Service – to compute a large volume of prime numbers. The two main types of CAS are SMP (symmetric multi-processing) and MPP (massively parallel processing). The difference is the number of machines connected to your client, where SMP has one machine and MPP has multiple machines. The program I made on MPP CAS works by sending data to 3 CAS workers with 8 threads each, for a total of 24 threads. This approach uses parallelism in that the three CAS workers are different compute cores that can generate prime numbers simultaneously. It also uses multithreading to minimize latency in the input and output data tables by splitting the process into 8 threads across the multiple CPU cores. MPP CAS automatically deals with the parallel execution bit as long as I split my program into the desired number of threads (in this case, 24). In this sense, both parallelism and multithreading are baked into Viya – and by extension, SAS.      Can Python Use These Techniques?   In short, yes. Python can run on multiple threads or cores, but modules can be installed to enable this type of behavior. “Threading” is the module that allows for multithreading, and “multiprocessing” allows for parallel processing. Both can be installed and imported into most python environments, as they’ve existed for quite a while. Threading and multiprocessing have been around since Python 2.6, and the current standard release of today is 3.12. A breakdown of Python threading and multiprocessing can be found in this YouTube video.    Using a simple Python program, I was able to get multiple threads working. If you know of more complex examples of multithreading in Python, be sure to share the links in the comments of this post.    Python has something called the GIL or “Global Interpreter Lock” which locks python to execute in a single thread. This poses no issues for python multithreading as a multithreaded environment can only have one thread “active” at a time anyway. However, the multiprocessing module must avoid the GIL by splitting processes into completely separate “subprocesses” instead of threads. These individual processes must be handled by the python programmer in their code. The examples of this I’ve seen look easy enough, but manipulation of a data table in parallel stumped me as someone unfamiliar with python parallelism. Nevertheless, if you knew what you were doing you could feasibly create powerful parallel python programs using this module.      SAS Can Also Help Python Parallelize    Be on the lookout for a post by David Estreich and me, coming soon. It’ll explore the CAS Gateway Action Set, which can allow parallel execution of Python processes in a way that might be more familiar to SAS users.     Related Links:    The Moth - Threading/Concurrency vs Parallelism (danielmoth.com)  SAS Help Center: Using Parallel Processing SAS Help Center: What Is Threading Technology in SAS? SAS Help Center: Threading in Base SAS Multithreading and concurrency fundamentals (educative.io) 217-29: Threads Unraveled: A Parallel Processing Primer (sas.com) Seriously Serial or Perfectly Parallel Data Transfer with SAS Viya - Rob Collum SAS Help Center: SAS Cloud Analytic Services: Fundamentals Using MPP CAS Multi-Threading to boost prime number computing speed when using the brute force method | Global Enablement & Learning Blog (sas.com) 109-2010: Faster Results by Multi-threading DATA Steps (sas.com) SAS Help Center: What is SAS/CONNECT? threading — Thread-based parallelism — Python 3.12.5 documentation multiprocessing — Process-based parallelism — Python 3.12.5 documentation Parallel Processing in Python - GeeksforGeeks https://www.youtube.com/watch?v=AZnGRKFUU0c      Find more articles from SAS Global Enablement and Learning here. ... View more

Yq, a command-line YAML processor, is often used on SAS environments to make edits to files such as kustomize.yaml or other site configuration files. This powerful tool has received some recent upgrades, but it comes with a syntactical overhaul that may take some adjustments. This post will walk you through what yq is, what it does, and how to update from the previous version (yq v3) to the new yq v4.   So, what is yq?   As mentioned previously: yq is a command-line YAML processor. This phrase, broken down, just means yq processes (allows you to add, modify, read, delete) the content of .yaml files using commands on the linux terminal. A YAML file, for those unfamiliar, is a structured text file using key-value pairs with indentations to mark different "levels" of description. Below is an example of a YAML file about fruits:   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   We'll call this file "fruits.yaml". Here we see keys that contain more keys with values, as well as a key with two different values in a list. It can get more complicated once you get to deployment site configuration files, but this is a simple example. Really the "complication" of a YAML file is just how deep the keys within keys go, so this file will work for our purposes going forward.   What would we use yq for in fruits.yaml?   After first installing the yq command-line utility we could issue yq commands to change this file without ever having to manually open it. It's possible to do every type of modification to the file we would need using the command line alone, which is helpful for automation and scripting. You may have a script invoke yq to modify the color of our banana from "yellow" to "green". While other command-line based text editors could technically do that particular job, yq uniquely understands YAML syntax and makes it easier to make edits to one or more values at a time. Other general text editor utilities don't understand the tiered structure present in this file, which is what makes yq efficient at targeting values.   This is how yq sees this file, and how you'd reference to the values contained within using yq:   apple.color describes the value under the key "color" which is under the key "apple". banana.color describes the value under the key "color" which is under the key "banana". It is okay to have "color" as an identical key under multiple fruits, this poses no problems in a YAML file. *.color would describe values held under the "color" key under both apple, banana, and orange. Modifying this might not be useful in this particular example, but imagine you stored some connection url information under multiple keys in a site environment's config YAML file and all of them needed to be changed, yq could do that sort of edit easily.   Updating from yq v3 to yq v4   Let's say you had a bunch of yq3 commands for building out and editing fruits.yaml. The new update to yq has changed how the utility functions... and you aren't sure where to start. We'll go over what changed in general with yq4, and then some specific yq3 -> yq4 command update examples using the new logic.   First, yq3 was incredibly specialized for a particular purpose: manipulating YAML files. This hasn't changed in yq4, but the philosophy has changed somewhat. Yq3 has specific commands for adding, deleting, editing, and reading values in YAML files. Yq4 takes this approach and simplifies it to an all-in-one edit command. Realistically an edit, deletion, and addition to some value in a YAML file is just a text edit. Deleting is the same as editing to be nothing, adding is just creating a new key and editing its value, and a standard edit just changes a value into a new value. Yq3 always output the result of a command to the console, allowing you to read what you had done, so why not bundle everything into the same command? Yq4 is that main fundamental change, with a few others sprinkled in.   Reading a value in yq3:   yq r fruits.yaml 'banana.color'   yq4 equivalent:   yq '.banana.color' fruits.yaml   Editing a value in yq3:   yq w fruits.yaml 'banana.color' green   yq4 equivalent:   yq '.banana.color = "green"' fruits.yaml   Adding a new key-value in yq3:   yq w fruits.yaml 'orange.texture' bumpy   yq4 equivalent:   yq '.orange.texture = "bumpy"' fruits.yaml   Deleting a key and its values in yq3:   yq d fruits.yaml 'banana.unbiased_tastiness_score'   (this deletes the key along with its corresponding value, 100)   yq4 equivalent:   yq 'del(.banana.unbiased_tastiness_score)' fruits.yaml   In yq4, invoking "yq" by default now invokes the all-in-one "eval" function. This eval function makes edits, additions, deletions, or lets you read a file or value based on whatever comes next in the command after "yq". Following the proposed changes (which are expressed in single quotes) is the file path, which tells yq where the file to be modified will be.   Why the changes?   Yq is already heavily based on jq, a popular command-line JSON processor, and this update aims to make the syntax between the two programs more similar. Several smaller features were dropped in favor of leveraging the more powerful already-existing linux terminal tools/strategies. For example, you used to be able to "validate" a yaml file using a custom yq command, and now you would just run a general linux command to check if your file throws errors. Comparing files used to be done with a custom "compare" command in yq, and now you would just use the "diff" linux utility to show the differences between two files. Because unneeded features were dropped, development time for the utility has then been able to be focused on improvements like allowing control over YAML metadata, letting users work with multiple files at a time, creating maps and arrays with YAML, and more. The latest iteration of yq is supposed to slot better into an automated workflow or pipeline and resemble the syntax of other familiar commands more clearly. Piping from STDIN to STDOUT (the linux terminal's standard input and standard output) has been properly defined, and the addition of an "eval" and "eval-all" function allows users to make both targeted and sweeping changes across one or more files with ease.   Read more about the changes and new syntax here.   Related Links:   yq Documentation   Deploying SAS Viya 4 (course uses yq and explains the utility for use in deployments)     Find more articles from SAS Global Enablement and Learning here. ... View more

Since joining SAS I’ve worked on several individual portions of the SAS Viya on Kubernetes deployment learning course. However, I’ve never taken a step back and looked at the entire course offering until recently. This month I was tasked with “mapping out” the course’s hands-on exercises as they connect with one another to form a web of possible methods for deploying and customizing a SAS Viya environment.   The Workshop and Overview   Workshop: Deploying SAS Viya 4.   Below is the overview map:    Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.     This post aims to break down the overview image to help you understand what the workshop’s exercises have to offer.   Kubernetes Deployment and Viya Preparation     Users begin their hands-on workshop experience by assessing their environment and deciding if it is ready for a Kubernetes deployment. Kubernetes, for those unfamiliar, is a container management platform and will be part of our infrastructure needed to deploy SAS Viya. More on Kubernetes here. Once users prove the readiness of their environment, they will deploy an upstream opensource distribution of Kubernetes.   On a customer site we might assume this to be done ahead of time, but the workshop walks users through the process by which one can order SAS Viya. Our virtual learning environment with which we are performing the hands-on already has a “SAS Viya order” and we will not actually use the sample internal order that the exercise has us make. Still, it can be beneficial to learn what the order process looks like for SAS Viya.   Next, users will manage some prerequisites for their SAS Viya deployment. If users are planning to run the Deployment As Code (DAC) tool instead, this section will be handled automatically by that tool. If not, each prerequisite (colored in orange on the overview chart) will have to be manually installed and have its specifications tweaked by the user. Once users are happy with their prerequisite management, they’ll run the “ViyaARK Prereqs report” to have their environment checked for SAS Viya installation viability. Assuming the ViyaARK tool sees all prerequisites are met, SAS Viya can then be installed!   Deploying SAS Viya     This section is broken into two parts: manual deployments and non-manual or “automatic” deployments.   The workshop walks through a few different ways to deploy SAS Viya:   A “simple” manual deployment, which is entirely generic and has almost no alterations to the defaults. We use this environment as a jumping-off point to potentially create the following manual deployments: The “programming only” environment, which would only allow users of the environment to use the SAS Studio offering of SAS Viya. The deployment with authentication, an enhancement to the simple environment which allows for different users/roles to be authenticated with separate permissions and restrictions. The deployment in a second namespace, this is essentially another simple SAS Viya deployment but running on the same infrastructure at the same time as our original. It shows us that you can have multiple SAS Viya deployments running as long as they’re in separate namespaces. A SAS Viya deployment with the “sas-orchestration command”. This method for deployment has some manual steps but the bulk of the deployment is done automatically by using information detailed in the software order email we would’ve gotten after ordering SAS Viya. A SAS Viya deployment using the Deployment Operator (DepOp). This tool requires some setup but can point to a “SASDeployment Custom Resource” that contains configuration files to extract information about the desired deployment, then deploy a SAS Viya environment accordingly. A SAS Viya deployment using the Deployment As Code (DAC) tool. This tool automatically manages all SAS Viya prerequisites and deploys a simple environment with very little manual setup required.   With SAS Viya deployments being customizable, all possible modifications to a deployment cannot be covered in this workshop. Still, we cover all the ways to deploy a simple environment as well as several key pre-deployment modifications one might make in the real world.   Users are free to go through the hands-on for one or all of the different deployment types. Before starting a new deployment, the exercises make sure users delete previous deployments and clean up their environments.   Customizing a SAS Viya Deployment     Users clean up all previous deployments they may have made and begin on a clean slate with a simple SAS Viya environment created using the sas-orchestration command. The actual method of deployment shouldn’t matter, as any methods would lead to an identically customizable SAS Viya environment.   From here, users can configure several real-world deployment customizations. Among these customizations are some SAS specific configurations: namely SAS/CONNECT, SAS/ACCESS, and the SASWORK storage location. Users are also given the option to manage the Kubernetes local registry for backups, configure Hadoop, and enable “high availability” on the SAS Viya deployment to lower chances for machine failures or service downtime.   Of note in this section users are also shown how they can validate any of their deployments. This step can be performed at any time a SAS Viya deployment exists on the environment and will generate an explorable deployment report for the user.   Putting it all together   The idea behind walking through a workshop’s hands-on in this way is that users (maybe you, dear reader) can get a better idea of what to expect before having to delve into a given workshop. The hope for GEL workshops in the future is that each one will have a similar overview map designed to walk you through their learning paths before you begin. If there’s anything in this particular workshop overview that you feel is left out or needs further explanation, feel free to comment down below and I’ll do my best to prepare you for what you can expect should you book this collection.   Related Links:   Deploying SAS Viya 4   Kubernetes   Kubernetes namespaces   Upstream Opensource (Red Hat documentation)   SAS/CONNECT   SAS/ACCESS   Hadoop     Find more articles from SAS Global Enablement and Learning here. ... View more

In my last post I calculated prime numbers using MPP CAS. There was a split focus on prime number math and SAS's capacity for parallel processing. For this post I'll be taking some of those generated numbers and looking into their distribution along the number line as an example to showcase some interesting features that the SAS Visual Studio Code extension has to offer. We'll be reusing the single-machine prime number generation code and performing some basic data exploration using SAS, SQL, and Python together in one SAS notebook file. The end result will be shipped off to SAS Studio as an executable flow.   SAS Visual Studio Code Extension   The VS Code extension allows you to browse all the same libraries and files that you would in SAS Studio. For this post, we'll be creating a SAS notebook (.sasnb file) through the Visual Studio interface that will be called "primeexploration.sasnb".   The benefit of these notebooks is that you can seamlessly switch programming languages for individual cells, while adding text-only markdown content to explain each step of your work.   Note: a "cell" is like a contained block of code. The following screenshot shows 4 cells, and the language each cell has been set to use is displayed in the bottom right corner:     Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   SAS automatically wraps any Python and SQL code you input in an invisible PROC Python or PROC SQL procedure respective to the language you chose. It's possible to view this wrapper code by modifying the output to display the SAS log renderer, rather than the default HTML renderer:     Changing the presentation can be beneficial if you want to know what your code is doing behind the scenes:     There are more options and views to play around with. However, these basics will get us through the journey from VS Code --> exploration and discovery --> SAS Studio flow.   Building Our Program   In this section, we'll follow the logical steps I took and the ways in which my work was made easier by the flexibility of a SAS notebook.   The premise for the program is that I want to show how prime numbers become scarcer as you travel down the number line.   To begin, I recycled the SAS code for generating prime numbers from the previous post. The cell was of the language type "SAS" and the code was as follows:   DATA primenumbers (drop= L j k a); DO UNTIL(primes=10000); primes+1; L=0; DO j=1 TO primes; k=0; IF (MOD(primes , j)= 0) THEN DO; k+1; a=MOD(primes , j); END; IF K=1 THEN L+1; END; IF L=2 THEN OUTPUT; END; RUN;   Now that we have a dataset consisting of all the prime numbers between 1 and 10,000 (if you're curious, there are 1229) we can perform some data exploration to see what we're working with. As a quick test I wanted to show how the first 50 numbers on our number line contain more primes than the last 50. I decided to do this using SQL, as I've used it before to print portions of datasets.   The code:   SELECT * FROM work.primenumbers WHERE primes < 50; SELECT * FROM work.primenumbers WHERE primes > (10000 – 50);   The output(s):     We can see that there are 15 prime numbers in the first 50 values along the number line, and only 2 in the last 50. This is interesting, but it doesn't illustrate the point as well as examining the entire dataset would.   To do this examination, I decided to split the data into 4 quarters. Q1 is comprised of every prime less than 2500, Q2 has primes from 2500 to 5000, Q3 has primes from 5000 to 7500, and Q4 has the remaining primes between 7500 and 10000. To assign these quarter identifiers to each row I switched back to using SAS code, as that was what I found to be easiest for me:   DATA work.primedistribution; SET work.primenumbers;IF primes < 2500 THEN quarter = ‘Q1’; ELSE IF primes < 5000 AND primes >= 2500 THEN quarter = ‘Q2’; ELSE IF primes < 7500 and primes >= 5000 THEN quarter = ‘Q3’; ELSE quarter = ‘Q4’;RUN;   Finally, I had to create a chart to show the frequency of values in each quarter. Out of all 3 languages I consider myself most experienced with Python for creating graphs, so I used it!   That's the best thing to me about this extension: switching programming languages to whatever best works for the user makes experimentation much easier. I can leverage my past programming knowledge with SQL and Python in the SAS environment to take advantage of my immediate strengths as a new SAS user.   The Python code:   import matplotlib.pyplot as plt df = SAS.sd2df(‘work.primedistribution’) ax = df.quarter.value_counts().plot(kind=’barh’) SAS.pyplot(plt) ax.set_xscale(“log”)   Our Python code does need a bit of explaining. Python can't work directly with a SAS dataset, and we need to convert our data over to a dataframe that Python can manipulate. Invoking "SAS.sd2df" allows us to pass in our 'work.primedistribution' dataset that we got from the previous SAS code cell, and convert it accordingly.   SAS.pyplot tells the SAS HTML renderer how to construct the output graph. Alternatively, you could convert a SAS dataset into a Python-readable dataframe (using sd2df), manipulate it in python, convert it back into a SAS dataset (using df2sd), and display the results. However, since we aren't making sweeping changes to the data I found it easiest to use the SAS.pyplot approach.   The following is the output:     Given more time I would format this output to look more appealing, but it serves its purpose in the current form.   We see that Q1 contains the most primes, whereas each ascending quarter contains fewer than the one before it.   This result is interesting, but what if we then wanted to move this code over to SAS Studio for collaboration with our team? Maybe they could find some new insights (or make my graph prettier).   Converting Our Code to a SAS Studio Flow   SAS Studio's "flow" feature allows users to create modular programs comprised of different nodes. These nodes may be procedure steps, data steps, files, etc. That's structure is almost the same as the SAS notebook structure we've been working with. Every cell in the SAS notebook could easily become a node in a SAS Studio flow. Lucky for us, converting from a notebook to a flow takes only a few clicks:     Now that are flow is created, we can examine it in SAS Studio:     Each cell has become a program node, and we can run the flow using the "run" button to get the same results as our notebook.   Each node can be edited separately by my team, and they can fix my graph for me (thank you).   Related Links:   Base SAS + SAS/CONNECT – A simple method to generate load on any number of licensed cores   Using MPP CAS Multi-Threading to boost prime number computing speed using the brute force method     Find more articles from SAS Global Enablement and Learning here. ... View more

Prime numbers (numbers which are only divisible by one or themselves) are used in encryption, mechanical engineering, random number generation, and a variety of other applications. The way to generate a list of prime numbers is simple in concept, but gets progressively more difficult for computers as you travel up along the number line. For this reason, it’s useful as an experiment to see how Multi-Threading can be used in CAS. More importantly, it shows us how we can optimize the use of our threads to get the most speed.   Foundational Material   This blog builds upon the ideas laid out in Simon William’s blog: Base SAS + SAS/CONNECT – A simple method to generate load on any number of licensed cores   To modify Simon’s Base SAS prime number generation code in order to load it in CAS and keep track of individual threads, I’ve used portions of code from Mike Henderson’s blog: Easy Multi-Threading with SAS – STATMIKE   Both blogs are also worth a read if you’d like to better understand aspects of SAS multi-threading and/or parallel processing.   Methods for computing primes   For this experiment we’ll be looking to compute every prime number between 1 and a given integer (1000, 10000, 100000, 500000). There are a variety of algorithms for doing so: and the big three are Fermat’s “little theorem”, the Sieve of Eratosthenes, and the brute force method.   Fermat’s little theorem does not get more complex as numbers get larger, which seemingly makes it ideal for this blog. However, it essentially involves running a series of “prime tests” on a given number. The issue with these tests is that they involve exponents of incredibly large numbers, which any given coding platform will need to be specially equipped to handle beyond a certain integer size.   For example: 3^318 is an extraordinarily large integer. To illustrate what I mean, if you open up your calculator app on your machine it will tell you your input is invalid.   Furthermore, Fermat’s little theorem is more like a statistical guess instead of a definitive answer as to whether a number is prime. The guess is usually correct and the accuracy can be tuned at the cost of more computations per checked number, but there are also special numbers called Carmichael numbers that will pass the test despite not being prime.   The Sieve of Eratosthenes involves eliminating all multiples of a prime number on the number line to eventually leave yourself with just the primes.   For example: to find the prime numbers in the span of numbers between 1 to 100 you would mark all the numbers divisible by 2, then all the numbers divisible by 3, then 5, then 7, and so on.   While the Sieve of Eratosthenes and other similar prime number sieve algorithms start out very fast, they use a lot of memory to maintain a record of primes and non-primes. Due to this, they are bottlenecked by the speed at which your machine can read/write to and from memory… and so will be less useful to us at large scales when compared to the final and most simplistic method: brute force.   The brute force method is exactly as the name suggests. It is easy to implement and is a direct check of each number to see if it is divisible by any of the numbers below it.   For example: to check if the number 7 is prime, your program would check if it was divisible by 2, then 3, then 4, then 5, then 6. Because 7 is not divisible by any of the numbers before it, the number is marked as prime. To check if the number 9 is prime, your program would check if it was divisible by 2, then 3. Because 9 is divisible by 3 it is marked as not prime.   For this experiment, brute force is the method of choice. Originally I would have liked to try the Sieve of Eratosthenes algorithm, but converting it to a multithreaded approach would be a little bit more complicated because of how threads would have to share pieces of information (perhaps it will be a topic for another blog). The complexity of brute force is O(n) – which is linear – and is easier to follow for experiments. Essentially, as you search more numbers the time to compute the primes should go up relatively consistently.   Code & Approach   Code to compute prime numbers was ran on a clean deployment and both the single thread and multi-threaded approach were carried out on the same machine to ensure accuracy when comparing them.   Code for computing prime numbers on a single thread:   %LET searchspace=1000; /* initialize an ordered table of numbers 1 to searchspace */ data work.initiallist; do until(i=&searchspace); i+1; output; end; run; data randomprime; set initiallist; if _N_ <= &searchspace then do; L=0; do j=1 to i; k=0; if (mod(i , j)= 0) then do; k+1; a=mod(i , j); end; if K=1 then L+1; end; If L=2 then do; output; end; end; run;   This code creates a list of numbers, in the above example it would be 1 to 1000. Then, the program will then use the brute force algorithm to determine if each number in the list is prime, and output all prime numbers to a new table.   Code for computing prime numbers on multiple threads:   Initialization/Randomization step:   /* setup a cas session */ cas mysess; libname mycas cas sessref=mysess; /* declare number of threads and max number searched for primes */ %LET threads = 24; %LET searchspace=500000; /* initialize an ordered table of numbers 1 to searchspace */ data initiallist; do until(i=&searchspace); i+1; output; end; run; /* randomize the order of the numbers from initiallist using a uniform distribution */ PROC sql; create table MYCAS.randomizedlist as select * from work.initiallist order by rand(‘uniform’); QUIT; Computation step: /* use parallel processing in CAS to identify prime numbers using the brute force method */ proc cas; dscode=” data randomprimeMPP; set randomizedlist; part_size = int(&searchspace/(&threads)); drop part_size; thread=_threadid_; uniqueRowID = _n_ + (_threadid_ * 1000); p = (thread – 1) * part_size; s = p + part_size; drop s; if thread = &threads then s = &searchspace; row_id = _N_; if uniqueRowID <= (s * 1000) then do; L=0; do j=1 to i; k=0; if (mod(i , j)= 0) then do; k+1; a=mod(i , j); end; if K=1 then L+1; end; If L=2 then do; output; end; end; run;”; datastep.runcode / code=dscode single=’no’; This code creates a list of numbers of a desired length and randomizes their placement in that list. Then, the program runs through different equally-sized portions of the list simultaneously using parallel processing. In this environment there are 3 CAS worker nodes with 8 threads each, for a total of 24 individual threads working on computing prime numbers independently.   In a single threaded approach, the program checks each number one at a time and it will take significantly more time as the list of numbers grows. Larger numbers are harder to calculate as they have more checks needed to ensure they are or aren’t prime. Without randomization of number placement for the multi-threaded approach, thread #1 would finish significantly faster than thread #24. The randomization essentially equalizes the time each thread should take to finish, ensuring the computational load is spread evenly to avoid later threads bottlenecking the program’s speed.   Findings   Speeds for single threaded approach:   Primes from 1 to 1000 - 0.01 seconds   1 to 10,000 – 0.91 seconds   1 to 100,000 – 1 minute 37 seconds   1 to 500,000 – 40 minutes 47 seconds   Times jump for the single threaded approach drastically as the number line gets longer. If we compare to CAS multi-threaded approach with our randomization we see a substantial speed increase:   Primes from 1 to 1000 – 0.42 seconds   1 to 10,000 – 0.49 seconds   1 to 100,000 – 16.25 seconds   1 to 500,000 – 2 minutes 7 seconds   The times shown on both series of tests are for each program’s respective prime number search procedure, and their initialization/randomization steps are not taken into consideration because the differences are negligible. There seems to be an initial setup step for CAS that raises the initial times for 1000 and 10,000 prime calculations when compared to the single threaded approach, but multi-threaded prime calculating is more than worth it as the volume of processed numbers increases.   Taking a look at the load on each of the 24 threads, we see across two separate experiments that the randomization leads to relatively even distribution of primes.   Test #1 with a search space of 500,000:   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Test #2 with a search space of 500,000:     There are 41,538 prime numbers between 1 and 500,000. The program accounts for all of them and spreads the work consistently. However, several repeated experiments confirm that threads 8, 16, and 24 generally contain slightly less primes than average. This is likely related to them being the last thread in their respective worker nodes, though the underlying reason may be due to my own code rather than a quirk in CAS.   Potential Modifications   It’s possible to further modify the code to eliminate all even values from the randomized list, which predictably cuts the computation time in half (we can do this because any even number is divisible by 2 and must not be prime). The following is an approach using the randomization step’s SQL procedure:   /* randomize the order of the numbers from initiallist using a uniform distribution */ PROC sql; create table MYCAS.randomizedlist as select * from work.initiallist WHERE MOD (i, 2) = 1 order by rand(‘uniform’); QUIT;   When searching this way for primes in the numbers 1 to 500,000, the program takes around 66 seconds, or 1 minute 6 seconds. This approach is essentially a fusion between the brute force and the Sieve of Eratosthenes algorithms and could be an interesting exercise to explore in the future. It would also be possible to remove some more numbers before prime calculation: every 3 rd , every 5 th , etc. This modification has diminishing returns for computation speed once we start removing multiples of larger numbers, but removing multiples of smaller numbers would give significant benefits.   Tweaking the type of distribution for the randomization step in the MPP CAS version of the code could allow for more even spreading of prime numbers across each thread. The distribution model the PROC SQL step currently follows is a uniform distribution, but the nature of primes being more common early in the number line would mean that a weighted distribution method would suit the data better. It's unlikely that this would lead to any significant gains in computation speed, but it’s worth noting nonetheless.   It's also possible to modify the code to only check if a given number is divisible by numbers up to its square root, not up to itself. For example, when checking the number 195 we can take the square root and get 13.9, which rounds up to 14. any number combinations that multiply to 195 that contain a number beyond 14 will have already been “checked”, as numbers greater than 14 multiplied by another number greater than 14 necessarily will yield numbers greater than 195. I didn’t have the time to test this in my environment to see the exact benefits to computation speed, but it is worth noting for future experiments. In this specific scenario I was more concerned with simplicity for testing and chose to use the most basic brute force algorithm with the least number of modifications, as improvements made by CAS are the primary focus and any other prime number-specific modifications to the code could affect the results of the comparison between single thread and multi-threaded processing.   Takeaway   Computing prime numbers, in this case, is a simple exercise that could stand in for any business application in SAS that can be spread across multiple threads. Much like the randomization step in this exercise, it’s worth looking at your own data and understanding what sorts of modifications could be made to get the most out of parallel processing in CAS.   Related Links:   Base SAS + SAS/CONNECT – A simple method to generate load on any number of licensed cores Easy Multi-Threading with SAS – STATMIKE CASL: Parallel CAS Action Execution SAS Documentation: Add a Unique Row Identifier for MPP CAS SAS Documentation: ADDROWID= Data Set Option Video on Prime Generating Constant (future research) Prime Generating Constant Research Paper   Find more articles from SAS Global Enablement and Learning here. ... View more