SAS Support Communities

Starting with SAS Viya stable release 2025.02, it is now possible to run in-memory and many more multi-threaded procedures in SAS programming run-time servers. This builds on the technology developed initially for use with CAS Servers and multi-language architecture (MLA) found within SAS Viya Workbench. This can reduce the dependency for SAS Viya users to submit data processing within a CAS Server. SAS programming run-time servers can now run almost any code developed in SAS programming languages or Python. And there is now a greater opportunity to leverage the power of GPU processing for analytical procedures.   How has this come about? Well, with the introduction of the multi-language architecture (MLA) released with SAS Viya Workbench, the use of CAS actions without requiring a deployed CAS server has now become possible. These innovative features and capabilities developed initially for SAS Viya Workbench, have now been added as enhancements to the SAS programming run-time servers, including SAS compute servers, SAS/CONNECT servers, and SAS batch servers. In practice this now allows both data scientists and programmers to easily switch between SAS Viya and SAS Viya Workbench environments should they need to do so. This is because there is such a large degree of code compatibility between the environments.   In addition Python programs developed with SAS Viya Workbench using the SAS proprietary sasviya.ml Python package were not available in a SAS Viya environment. Until now, the sasviya.ml package was only available with SAS Viya Workbench. From February 2025, Python packages such as sasviya.ml will be accessible in SAS Viya environments. Examples of using the sasviya.ml package can be found in the Python section of the SAS Viya Workbench examples on GitHub.   Runtime diagrams   Let's use a few diagrams to understand what has changed, or, better, what are the new capabilities that have been added. We will use a SAS compute server to illustrate the concepts, but the same applies to SAS/CONNECT and batch servers. Remember when looking at the diagrams, it is important to note that for each user there will be a corresponding SAS programming run-time server pod. So for 50 users, there will be 50 pods. Each CAS Server can serve multiple users, so for CAS the number of pods does not increase with the number of users unless there are scaling options enabled/triggered.     Before SAS Viya 2025.02, code running in a compute server has essentially two execution paths. Traditional SAS code, i.e. the same that would run in a SAS 9 environment, runs in-process. This means that it uses the memory and CPU of the hosting SAS process.   To exploit the additional computing power provided by CAS servers, the user/programmer would code in the CAS language or use CAS procedures. That code is only interpreted by SAS, then the execution is delegated to the CAS server. That’s the only way to execute the most advanced analytical procedures.     In the past years, following a continuous delivery paradigm, we incrementally released new CAS-enabled procedures, which present a “SAS-like” syntax that lets you code CAS’ most advanced analytical procedures without having to learn the CAS language. These procedures can still be interpreted by SAS and then delegated to the CAS server, which will in turn execute the same CAS actions as if it was called by the CAS language.   Starting with SAS Viya 2025.02, these options are still available, plus we added a new execution path. The same CAS actions, that are available in the CAS server, are now embedded and executed in the SAS compute server process. These CAS-enabled procedures can now be parsed and executed without leaving the SAS process. We will discuss below the benefits and the additional considerations that this capability brings to the SAS Viya platform.   How can SAS know if CAS-enabled PROCs should be run in-process or delegated to CAS? The answer is in the data. If the input data is in a SAS library, then the CAS action will run in-process. If the input data is in a caslib hosted by a CAS session, then the CAS action will be delegated to the CAS server.     Finally, let’s focus on the SAS process and the new CAS actions executed in-process. This capability is referred to as “multi-language architecture”, or MLA, because the same CAS actions can be called by Python code, via the sasviya.ml package. As mentioned previously, this package was initially only available with SAS Viya Workbench, and it’s now included with all the SAS Viya offerings.   The Python environment is optionally configured in the SAS programming run-time servers using the standard process described the SAS Viya Platform: Integration with External Languages guide.   Deployment considerations   Updating SAS Viya to the 2025.02 stable release or later automatically brings all the binaries and components that provide these enhancements. You do not have to install any other software, just update to the latest version as usual. The new capabilities are automatically included in all SAS Viya offerings with no additional fees, but they perform a license check to verify that they are executing in a licensed environment. Every SAS Viya license created and applied until the 2025.01 stable release does not include the code that enables them, while every license created and applied starting with the 2025.02 release does. As part of your regular update to 2025.02 or later, you will automatically get a new license that includes these new components, and it will be be applied by the update process.   There is one additional step that you have to perform to fully enable these capabilities, both for new deployments and for updates.   To support the multi-language Python library for SAS Viya Enterprise (sasviya.ml), you must enable the installation of the required Python packages. This is an additional step included in the Configure Python Integration Using SAS Configurator for Open Source page of the SAS Viya Platform: Integration with External Languages guide. It’s very easy; while following the instructions from that page, simply set to “true” the value of a new setting found in the $deploy/site-config/sas-pyconfig/change-configuration.yaml file, used to enable the SAS Configurator for Open Source.   - op: replace path: /data/default_py.pip_local_packages value: "true"   Architecture considerations   The SAS Viya architecture does not change, you can keep the same deployment topologies. Currently, all of the existing components are still required, including the requirement to deploy at least one CAS server. Indeed as of February 2025, a CAS Server is still the required analytics engine for use with SAS Visual Analytics. There will be other clients, such as Model Studio, that will require data be loaded into the CAS Server. Please refer to SAS Viya documentation for more information.   It’s important to understand that, when you run CAS actions inside a SAS programming run-time (Compute) server, all the data is loaded in memory and the code executes in a single pod. For this reason, you can only benefit from vertical scalability, by allocating additional CPU cores or more Memory/RAM to the individual compute server pods. One approach to providing SAS programming run-time (Compute) server pods with varying numbers of CPU and allocated Memory/RAM is to provide users with different Compute server contexts.   In the use cases where the data being analyzed is very large, or there are a large number of users accessing the same data, a larger number of CPUs and RAM may be required. In those scenarios, you can always switch to using a distributed CAS MPP server, with its horizontal scalability only limited by the available hardware that you can throw at it. Instructing SAS to execute the code in-process or offload it to a central CAS server is as easy as declaring a SAS library as the input data, versus using a caslib associated with a CAS session.   Running CAS actions inside SAS programming run-time (Compute) servers, might require more resources than those provided by a default configuration. Internal performance testing has demonstrated that SAS servers executing CAS actions can be as fast as SMP CAS servers running the same code, provided they are allocated the same resources, and properly configured to use them.   At the same time, not everyone in your enterprise will probably need beefy, expensive server instances. A good architecture design might call for multiple compute node pools and different Compute Server contexts, with different resource capabilities – and different costs. Remember that with the features of SAS Workload Orchestrator, it is possible to implement different Compute Server contexts using queues. This will also help keeping cost in track if you want to provision GPUs for specific use cases. Which ones? Keep reading below!   A final architectural note. You might have noticed that we wrote, above: “when you run CAS actions inside a SAS programming run-time (Compute) server, all the data is loaded in memory”. This behavior is inherited from CAS, and, just like regular CAS servers, SAS uses temporary disk space to back its process memory. This means that proper storage design might be required, but we’ll leave this consideration for another time.    Additional benefits   With the ability to run CAS based process on SAS programming run-time (Compute) servers there are new possibilities to leverage GPUs. Prior to February 2025, PROC IML was the only procedure that could leverage GPUs when executing in a SAS programming run-time server. To benefit from the multiple analytical procedures that can run faster on a GPU, programmers had to run code on a CAS server (For example, see https://go.documentation.sas.com/doc/en/pgmsascdc/default/casdlpg/p0hj75sw11grh2n1fg225n1uj61d.htm). Starting with the SAS Viya 2025.02 stable release, all these GPU-enabled procedures are available and can be executed in the SAS programming run-time server. Obviously, training a deep neural network on a very large data set is a time-consuming task, even with the advantages of GPU acceleration. Distributing the workload across multiple nodes in a cluster environment effectively reduces the runtime over a single node setup. That’s an example where you can still leverage the horizontal scalability provided by CAS MPP servers.   The need to configure a personal CAS server is now limited to a few specific cases. A personal CAS server, which is an ephemeral CAS server, is created on demand for a single user. For development purposes data scientists want the ability to work with advanced analytic procedures that, as we have seen, until now were only available in SAS Workbench or, in SAS Viya enterprise environments, through a CAS server. Imagine a few data scientists submitting resource-intensive tasks to central CAS server shared with the rest of the enterprise. They could negatively affect all of the other clients of the shared server. Until now, the only solution was to configure personal CAS servers, but this required additional considerations for proper design and setup. Starting with the SAS Viya 2025.02 stable release, data scientists can simply submit their advanced analytics to a SAS programming run-time server running on a dedicated, scalable node pool, and no one else is impacted.   Closing comments   The enhancements to the SAS programming run-time (Compute) servers in the February 2025.02 release of SAS Viya are hugely significant. They will provide customer organizations with the potential to run their SAS Viya workloads even quicker and at the same time potentially reduce the total cost of ownership. And from an architecture, administration and deployment perspective the new enhancements do not increase the complexity in most scenarios. So for customer organizations who have both SAS and Python programmers looking to speed up the completion of their workloads, SAS Viya 2025.02 will be a great benefit to them.     Find more articles from SAS Global Enablement and Learning here. ... View more

SAS Redis Server is a SAS Viya platform component based on the open-source Redis project. Redis provides a distributed cache technology to microservices in the SAS Viya platform. SAS Redis was first introduced with the 2022.10 SAS Viya platform release; starting with the 2024.12 release, its architecture has been simplified and the overall resources utilization reduced.   Streamlining the architecture   The SAS Viya platform introduced SAS Redis as a brand-new server component, used to provide distributed caching capabilities, and replacing the previously used SAS Cache Server and SAS Cache Locator. The initial architecture was designed with good engineering principles, including flexibility, scalability and management though a custom-built Kubernetes operator. Real-world experience showed that a more focused approach would better serve the current requirements, aligning with actual usage patterns and maintenance needs, while reducing complexity.   Starting with the 2024.12 release, the following changes have been introduced:     Previously, a SAS Viya platform deployment created a Redis Cluster with 3 Kubernetes Statefulsets, each one including a controller and a replica node. The cluster was described by a custom resource, called “distributedredisclusters.redis.kun.sas-redis-server”, and managed by the SAS Redis operator, which run in a stateless pod.   Now, Redis is still started in cluster mode, but it only has a single Statefulset with two pods: one controller and one replica node. There are no more custom resources to describe it, and it is not managed by the SAS Redis operator anymore.   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Redis pods as seen in OpenLens on a SAS Viya platform release 2024.12   Streamlining the management   You might have noticed that the Redis Operator is still deployed and running in the Kubernetes cluster, yet it’s not managing SAS Redis server anymore. Why is it there? We have decided to keep it to simplify updates from previous releases: as soon as it starts, it will delete any resource that is not required anymore, such as any instance of the “distributedredisclusters.redis.kun” custom resource, or the previously-defined Statefulsets. No need to manually delete any unnecessary Kubernetes objects!   The operator log confirms this at startup, printing this message:   {"level":"info","ts":"2024-12-23T21:05:51Z","logger":"cmd","msg":"The redis operator is now deprecated. Resources previously created by the operator will be removed if they exist."}   Once this is done at the first run, you can scale the operator down and stop it, if you want to. SAS will remove the SAS Redis operator in a subsequent release.   Events that were handled by the operator will now be performed directly by the SAS Redis server pods. As an example, the command to shutdown the Redis server changes from:   kubectl -n name-of-namespace patch distributedredisclusters.redis.kun sas-redis-server --type json --patch '[{"op":"replace", "path": "/spec/shutdown", "value":true}]'   to:   kubectl -n name-of-namespace scale statefulset sas-redis-server --replicas 0   Nothing else has changed. SAS Redis provides the same functionality as before. It has the same requirements: for example, each server pod is configured to request 1 Gi of persistent storage with RWO accessMode. And we still recommend using a block storage device for Redis persistent storage. For more information, see https://go.documentation.sas.com/doc/en/sasadmincdc/default/itopssr/n0ampbltwqgkjkn1j3qogztsbbu0.htm#n1egh9hqndi6lin13w58nozc7vco.   Conclusion   SAS Viya, in its continuous evolution, keeps optimizing its components to lower resource consumption and simplify platform management. In this article, we have seen how SAS Redis has changed starting with the 2024.12 release. For more information about the management of SAS Redis Server, consult the updated SAS Viya Administration: Infrastructure Servers page.     Find more articles from SAS Global Enablement and Learning here. ... View more

SAS/CONNECT, first released with SAS 6 in 1990, keeps evolving with the SAS Viya platform to adhere to modern enterprise requirements, especially in the cloud.   With the 2024.09 release there have been multiple changes:   Updated SAS/CONNECT spawner container image to remove unneeded elements New availability feature for the SAS/CONNECT spawner Data transfer services performance improvements   And we are not counting the continuously released fixes that remove bugs and improve the general user experience.   Let’s review each of these changes.   Updated Connect Spawner container images   The SAS/CONNECT spawner has been enhanced to optimize cloud resource utilization, minimizing the infrastructure footprint required to deploy and increasing performance.   Up to the 2024.08 Stable release, the SAS/CONNECT spawner container image includes binaries and additional files to support locally spawning a SAS/CONNECT server inside the SAS/CONNECT spawner pod. In details, it includes a full copy of SAS executable files.   The ability to locally spawn a SAS/CONNECT server inside the SAS/CONNECT spawner pod was previously deprecated in the 2024.03 release and has finally been removed with the 2024.09 release.   For this reason, it’s now been possible to remove those additional files and trim down the SAS/CONNECT spanner container image.   The results are impressive. The image size has been reduced by 80%, from ~10GB to ~2GB. Here is a comparison between the image sizes for the 2023.10 release (similar to the 2024.08 one) and the 2024.09 releases:   #LTS 2024.09 sh-5.1# podman image list cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-connect-spawner REPOSITORY                                                   TAG                           ... CREATED     SIZE cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-connect-spawner 1.0.33-20241126.1732636028288 ... 3 weeks ago 2.03 GB #LTS 2023.10 sh-5.1# podman image list cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-connect-spawner REPOSITORY                                                   TAG                            ... CREATED      SIZE cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-connect-spawner 0.68.28-20240723.1721760979402 ... 4 months ago 10.6 GB   This size reduction not only saves disk storage on Kubernetes nodes, but it speeds up SAS Viya deployment and maintenance, because the container can be downloaded from source repositories up to 5 times faster.   Internal testing with an on-premises cluster and a local mirror registry has seen image download times reduced from 1m18s to 12.5s (>80% reduction); testing on a cloud environment with a remote registry went from 5m38s down to 1m18s (>75% reduction).   A smaller image also helps with pod scalability, as we’ll see in the next point.   Other than decreasing the container image size, removing the capability to locally spawn a SAS/CONNECT server inside the SAS/CONNECT spawner pod brings additional benefits:   SAS/CONNECT Spawners no longer require any security context constraint when deployed on Red Hat OpenShift. There is no need to configure a workload class or node pool dedicated to SAS/CONNECT. SAS/CONNECT workloads will be associated with the compute workload class in all cases.   These and additional considerations are better described in the 2022 article Moving to the cloud with SAS/CONNECT, which covers in great detail the differences between the legacy behavior (locally spawn a SAS/CONNECT server inside the SAS/CONNECT spawner pod) and the current one (launching SAS/CONNECT servers in their own pod). There is one additional consequence that the article highlights and is worth reiterating: connections using the current SAS/CONNECT behavior require modern clients, such as SAS 9.4M7 or later or Viya 3.5. Removing the legacy capability means that SAS 9.4M6 and earlier clients cannot connect anymore to SAS Viya 2024.09 and later. As a result, you should migrate to SAS® 9.4M7 or later. This is explained in the SAS Note https://support.sas.com/kb/68/611.html.   New availability feature for the SAS/CONNECT spawner   Starting with the SAS Viya 2024.09 release, you can configure high availability (HA) for the SAS/CONNECT Spawner by applying the enable-ha-transformer.yaml.   A default SAS Viya deployment configures stateless microservices, such as the SAS/CONNECT spawner, without any high availability.   It is easy to enhance the platform resilience to failure: to enable HA, just add a reference to the enable-ha-transformer.yaml file to the base kustomization.yaml file:   ... transformers: ... - sas-bases/overlays/scaling/ha/enable-ha-transformer.yaml ...   This increases the number of replicas of almost all stateless pods to two, making the SAS Viya platform more resilient to pod or node failure and increasing its availability. Up until the 2024.08 stable release, the SAS/CONNECT spawner was not included in this transformer, and this service always run with a single pod. Starting with the 2024.09 release, the SAS/CONNECT spawner leverages the HA transformer, and two pods are now started when HA is enabled for the SAS Viya platform.   Note that while this increases service availability, it does not provide full protection from failures. When a client uses the SAS/CONNECT spawner to open a server session, the TCP connection is proxied through the spawner’s pod. If a spawner pod crashes, all the connections that it was proxying are severed and the backend server sessions die. The good news is that with the new HA capability, only half of the connections would be impacted, and not all of them, as with the earlier versions. As always, after a pod failure, Kubernetes will try to restart it, possibly on a new node, and clients will be able to re-connect and restart their backend sessions.   This new capability also increases SAS Viya platform availability during cluster maintenance. If the SAS/CONNECT spawner is started with a single instance, when an administrator wants to perform maintenance on the Kubernetes node where that pod is running, they would have to cause a service disruption to users, to move the single pod to another node. With the new capability, as long as the two SAS/CONNECT spawner pods are started on different nodes, the administrator can perform a graceful shutdown of one of the two pods and move it to another node, all while the remaining pod keeps servicing users’ requests.   Data transfer services performance improvements   Every SAS Viya release contains fixes and improvements to the internal working of the system, but many are hidden from our users, and often not mentioned. I’d like to share this one, since it can have a huge impact on data transfer performance. One of the many features that SAS/CONNECT provides is the capability to transfer data between clients and servers. Data Transfer Services are implemented by using the UPLOAD and DOWNLOAD procedures. One of the key advantages of SAS/CONNECT data transfer services, compared to generic external data movement tools, is the capability to automatically translate the internal data representations between the client and the server computers. As an example, the client and the server could run on different operating environments on incompatible computer architectures or different SAS software releases (such as SAS Viya platform and SAS 9). While migrating existing programs from SAS 9 to SAS Viya, some customers observed that the new version could be considerably slower in handling data transfer. Starting with the SAS Viya 2024.09 release, internal processing has been reviewed and optimized and is now as fast as, if not faster than SAS 9.     Find more articles from SAS Global Enablement and Learning here. ... View more

This is the third and final article in the series that investigates the new batch capabilities introduced with SAS Viya 2024.04 stable release. These new features address customers who engage in extensive batch processing, involving time sensitive, high volumes of SAS jobs of very short duration.   [ Part 1 | Part 2 ]   We close the topic by presenting some performance tests that compare the throughput and total run time of a burst of short-running jobs when run on default batch servers versus high-throughput batch servers.   The test environment   The results presented in this article do not come from a comprehensive benchmark. We are simply sharing the results we measured and some considerations noted while exploring these new capabilities in our small internal environment, based on SAS Viya 2024.06 stable release.   The Kubernetes cluster is hosted on-prem, with 4 worker nodes and Kubernetes 1.29. Each node has 8 CPU cores and 64GB of memory. It is not built for performance, rather the opposite. It is the just enough to test the deployment and configuration of this new capability. Yet, by limiting the concurrency of the submitted jobs to 1 per core (with 1 core left for Kubernetes services) we verified that the results are not impacted by a suffering or overloaded environment. To obtain this objective, we bound the compute server to only run on one of the worker nodes, and we made sure that the node was almost completely dedicated to that. Then we configured SAS Workload Orchestrator to limit 7 concurrent jobs on the default queue, and 7 pre-started batch servers on the high-throughput queue.   The test jobs   We used the very basic short CPU-bound job introduced in the previous article. The batch job reads a small input csv file, outputs a table report and artificially consumes 100% of a cpu core for 5 seconds. The whole program runs in a bit less than 5.5 seconds:   filename csv "!BATCHJOBDIR/iris.csv"; /* Import the CSV file  */ proc import datafile=csv out=work.myiris dbms=csv; getnames=yes; run; /* Print the first 10 observations **/ proc print data=work.myiris(obs=10); run; /* load the CPU for a specified amount of time */ %let maxtime=5;    ** max # of seconds **; data _null_; stt= time(); i=1; do until (i = 0); i=i+1; j=i**2; b=j-9+i; now= time(); elap=(now-stt); *put stt now elap; if elap > &maxtime then do; put "looped " i " times in " elap " seconds"; i=0; end; end; run;   We tested 2 different batch use cases:   the default case, which is the only one available in older SAS Viya versions. It uses on-demand batch servers and uploads/downloads files between the client and the server via the Files service. the high-throughput case, which leverages the new capabilities introduced with SAS Viya 2024.04: pre-started batch servers and optimized I/O sharing.   For the former case the batch jobs were submitted to the default SAS batch context, while the latter leveraged the high-throughput context, configured as described in the first article of the series.   The submission commands were similar to the ones used in the previous article:   sas-viya batch jobs submit-pgm -c default --name "Default-job" \   --pgm "importIris.sas" \   --job-file "iris.csv" sas-viya batch jobs submit-pgm -c high_throughput --name "HT-job" \   --rem-pgm-path "/mnt/viya-share/data/importIris.sas" \   --rem-output-dir "/mnt/viya-share/data/@FILESETID@"   For each case, we used 2 client machines to submit a total of 200 batch jobs, to be able to sustain a submission rate in terms of jobs/minute enough to saturate the backend for a few minutes.   The results: execution times and overall throughput   Each set of tests run for a few minutes, after which we collected and consolidated multiple job runtime statistics. We learned in the previous article that you can get job metrics with the following commands (using json format to avoid rounding of the reported times):   sas-viya --output json batch jobs list --details sas-viya --output json batch filesets list   From these results, we focused on the following time stamps:   Time Stamp Description File Set Creation This marks the beginning of each submission from the client. Job Submitted This is when the client sends the job to the midtier (batch service), after eventually uploading input files into the file set. Job Started This is when the backend receives a processing request from the midtier. For on-demand batch servers, it’s the request to start a new pod, while for pre-started batch server it is directly the request to start the job. Job Ended This is when the the job ends on the backend server. For on-demand batch servers, this is after shutting down the server pod. Job Modified This marks the end of the submission, when the midtier marks the job as completed.   Then, from these time stamps, we calculated the following metrics:   Metric Formula Description Before Job Overhead Job Submitted - File Set Creation Includes time to upload the input files. After Job Overhead Job Modified - Job Ended Difference between when job ends on the midtier versus when it ends on the backend. Includes time to upload the output files. Midtier Overhead After Job Overhead + Before Job Overhead The time the midtier spends managing the job. Job Pending Time Job Started - Job Submitted How long the job was kept pending because the backend was already full processing other jobs. Job Backend Runtime Job Ended - Job Started The total job runtime on the backend. For on-demand batch servers, this includes starting and stopping a new pod and server process. Total Runtime Job Modified - File Set Creation – Job Pending Time The total job runtime as seen from the client, excluding the pending time.   The results show significative gains when using high-throughput batch servers:   Metric Default Batch Server High-Throughput Batch Server Min Max Average Min Max Average Before Job Overhead 0.58 7.36 0.99 0.14 0.75 0.19 After Job Overhead 0.42 8.61 3.76 0.01 1 0.52 Midtier Overhead 1.19 15.97 4.74 0.16 1.33 0.71 Job Pending Time 0 5 min 41 s 2 min 47 s 0 2 min 31 s 1 min 15 s Job Backend Runtime 14 24 17.2 6.31 8.95 7.41 Total Runtime 16.78 33.30 21.35 7.19 9.28 8.12   Looking at the average measures, we can see how handling the file set adds about 4 seconds to the midtier overhead. Starting and stopping pods more than doubles the job backend runtime and the total runtime, while the overall increase in server utilization doubles the time jobs spend waiting (pending time). Looking at the min and max values we can also understand that the variability between best and worst cases is much higher for default batch servers, while in the high-throughput case times are more consistent.   Considering that the SAS code in each job runs in about 5.2 seconds, we can see that the high-throughput batch only adds a total overhead of less than 3 seconds, while the default server adds on average more than 16 seconds!   After analyzing these “per job” metrics, we also looked at overall times and system throughput.   Running 200 jobs on the default batch server took in total 8 minutes and 7 seconds, while the same test on the high-throughput batch ended in 3 minutes and 47 seconds: less than half. This corresponds to a measured throughput of ~25 jobs/minute in the default case, versus ~53 jobs/minute in the high-throughput case. I’d say the name fits! Again, for comparison, the theoretical max throughput without any overhead would be ~80 jobs/minute (7 cores running 5.2-seconds-jobs for 60 seconds).   Monitoring the system   As we have discussed in the initial article, the new batch capabilities not only lower execution times of short batch jobs, and increase their throughput, but also help lowering resource utilization, including the number of API calls between SAS Viya services and to the Kubernetes API. In turn, this lowers the risk of potential timeouts and failures, and the environment becomes more stable. The metrics above already show how the system is more loaded in the default case, which leads to less consistent execution times with greater variability. We also checked a few Grafana dashboards to verify if this is also visible in other metrics.   The following dashboard shows the spikes in the Files service in the default case (left) versus the optimized case (right): the latter uses less resources for a shorter time.     This is true across other services. For example, the Authorization service dashboard shows almost identical spikes at the same times. This can be understood as each call to the Files services has to be authorized, requiring a call to the Authorization service.   Looking at the Kubernetes API read/write metrics, we can see that the high-throughput case is barely indistinguishable from the background noise. The default case instead clearly sends a lot of API requests to Kubernetes while creating and destroying batch pods:     The pod churn activity is highlighted in the following screenshot taken during the execution of the default case, where you can see batch server pods in different states of starting, running, terminating:     Conclusion   In this series of three articles, we have described the new capabilities that support high-throughput batch server environments, including pre-started batch server and sharing storage between batch clients and servers. We have presented some metrics to compare default batch servers versus reusable batch servers with optimized I/O. The results clearly highlight the improved processing times, better jobs throughput and lowered resources utilization.     Find more articles from SAS Global Enablement and Learning here. ... View more

This post is the second in a series where we investigate the new batch capabilities introduced with SAS Viya 2024.04 stable release. Continuing from where we left off, we’ll now look at the options that provide optimized input and output file storage.   [ Part 1 | Part 3 ]   In short, SAS introduced new batch capabilities that benefit customers who engage in extensive batch processing, involving time sensitive, high volumes of SAS jobs of very short duration.   Default Input and Output File Storage   By default, batch clients do not interact directly with batch servers. File transfers are mediated by the Files service, that uses the SAS Infrastructure Data Server (PostgreSQL) to store them. This additional step that can add significant latency to short, high-throughput batch submissions.   Let’s assume you have an input csv file, and you want to run a batch SAS program that reads it, performs some quick ETL and sends back some output. Here is what would happen by default.   1.1 The user prepares the input csv file and program on a local directory on the client.   1.2 The batch client sends the input csv file and the SAS program file to the files service as an input file set.   1.3 The files service writes the content of the file set in PostgreSQL.   2.1 When it’s ready to run the job, the batch server queries the files service to get the input file set.   2.2 The files service retrieves the file set from PostgreSQL.   2.3 The files service sends the file set to the batch server.   2.4 The batch server writes the content of the input file set locally in a temporary directory.    SAS Batch input file set    Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Similarly, to get back the results:   3.1 At the end of the process, the batch server writes all output files locally in a temporary directory. This includes a copy of the input SAS program, the LOG and LIST output files, and any output that the client requested.   3.2 The server sends all output files to the files service as an output file set.   3.3 The files service writes the content of the file set in PostgreSQL.   4.1 When the user wants to retrieve the results, the batch client queries the files service to get the output file set.   4.2 The files service retrieves the file set from PostgreSQL.   4.3 The files service sends the file set to the batch client.   4.4 The batch client saves the content of the output file set to a local directory.  SAS Batch output file set    Only at the end of this process can the user open the resulting output or consult the execution logs.   Just going through this list takes time, doesn’t it?     Optimized Input and Output File Storage   Starting with version 1.9 of the batch CLI, available since April 2024, you can use new options to optimize input and output file sharing between the client and the server. In short, if there is a shared storage between the client and the server, you can use it to store both input and output files, and the files service (with its PostgreSQL backend) will not get used at all. The 14-steps process listed above is shortened to 2 simple steps:   The user prepares the input csv file and program on a shared directory that both client and server can use. At the end of the process, the batch server saves all output files to the shared directory, immediately available to be used.   Easy, short, effective.   SAS Batch optimized I/O     A note about the shared storage architecture.   Although the focus of this article is not to discuss how to best architect a storage solution that can satisfy the use case at hand, I don’t want to leave you wandering in the dark. Here are some links to get you started on the topic:   File System and Shared Storage Recommendations: https://go.documentation.sas.com/doc/en/sasadmincdc/default/itopssr/n0ampbltwqgkjkn1j3qogztsbbu0.htm#p0u8ihdebannnxn1oe7fh89kavwj   SAS on Azure Architecture: https://learn.microsoft.com/en-us/azure/architecture/guide/sas/sas-overview#permanent-remote-storage-for-sas-data   Shared file system to use SAS Grid Manager or SAS Viya: https://docs.aws.amazon.com/whitepapers/latest/best-practices-for-deploying-sas-server/shared-file-system-to-use-sas-grid-manager.html   Azure Storage for SAS Architects: Do You Want to Share? https://communities.sas.com/t5/SAS-Communities-Library/Azure-Storage-for-SAS-Architects-Do-You-Want-to-Share/ta-p/687864   Keep in mind that most of the online literature on the topic correctly focuses on architecting a storage solution that can satisfy the high throughput demands of SAS Viya computing processes. Our use case presents 2 key differences:   As the picture above shows, the storage area should be shared between a submitting client and the Kubernetes nodes where a batch server could start, not only between the Kubernetes nodes. This area is only used to host input and output files, not the whole computing process data I/O. For this reason, the I/O requirements may be much lower than those usually required i.e. for SASWORK folders.   The default NFS storage optionally created by the Infrastructure as Code (IaC) GitHub projects is a perfect starting point to use as a practical example.     Configuring SAS Viya with the Shared File System   At this point, let’s assume you have a shared file system that can be used for the batch clients and servers. In our test environment, we used the SAS Viya 4 Infrastructure as Code (IaC) for Open Source Kubernetes to create a Jump server and an NFS server while deploying the infrastructure for the Kubernetes cluster. The Jump server will be our client, so we deployed there the sas-viya CLI with the batch plugin. We used the SAS Viya 4 Deployment project to deploy SAS Viya and its pre-requisites, including configuring the shared storage for us. That includes mounting the NFS storage on the Jump host:     It also creates the patch transformer to mount the proper subfolder of that path into all SAS server pods:     In summary, the path /mnt/viya-share/data (as seen by server processes) and the path /viya-share/dac/data (as seen by the client) both map to the shared NFS folder /export/dac/data.   Now, it’s time to use it!     Let’s use the Shared Storage   To use the shared storage, we must properly reference input and output files so that both the client and the backend process know where to find them, avoiding the file sets managed by the Files service. We can follow the examples and references available in Using the batch Plug-In for the SAS Viya Platform Command-Line Interface   For the input files, it’s just a matter of refencing them correctly. If we look the Example: Add Input Data to the File Set, we see the filename statement is using the BATCHJOBDIR path:   filename csv "!BATCHJOBDIR/mydata.csv";   BATCHJOBDIR always points to the default temporary server directory where the batch server reads/writes files, including those read from /written to the file set. We can simply change the statement to use the backend path where the input file is stored. In our test environment that’s the /mnt/viya-share/data/ path discussed in the previous section, so that statement becomes:   filename csv "/mnt/viya-share/data/mydata.csv";   For the output files, we can use the new options introduced with version 1.9 of the batch CLI described in Output Specifications and File Locations, as shown in the Example: Specify a Location That Is Mounted in a Pod: --rem-output-dir, --rem-list-path, --rem-log-path.   Let’s not forget the SAS program with the code to execute. The same Example: Specify a Location That Is Mounted in a Pod shows how to use the --rem-pgm-path option instead of --pgm when the program is directly accessible by the batch server in the Kubernetes pod. This instructs the batch CLI to avoid uploading the program to the file set, and similarly tells the batch server to avoid downloading it.   To summarize with an example, compare the following batch client commands:   sas-viya batch jobs submit-pgm -c high_throughput --name "Import-fileset" \ --pgm "importIris.sas" \ --job-file "iris.csv"   sas-viya batch jobs submit-pgm -c high_throughput --name "Import-filesystem" \ --rem-pgm-path "/mnt/viya-share/data/importIris.sas" \ --rem-output-dir "/mnt/viya-share/data/@FILESETID@"   The first command leverages the file set facility: --pgm and --job-file point to files available locally on the client, and there are no “output options”.   The second command leverages shared storage: there are no input files to upload, --rem-pgm-path tells the batch server where to find the SAS program inside its pod, and --rem-output-dir tells the batch server where to save the output log and listing, again inside its pod and skipping the file set. The input program is not listed in this second command, since it is directly referenced in the revised code as explained above.   Note the @FILESETID@@ placeholder in the output directory specification. This will be replaced at runtime with the value of the file set ID (every batch job gets a file set ID even if it’s not storing any files in the file set), and the output directory will be automatically created by the batch server.   Also note that, while these examples use the high-throughput context, leveraging shared storage does not require pre-started batch servers. But we can use both capabilities to build a high-throughput batch facility!     What is the measurable impact?   SAS Viya documentation explains that, by default, sas batch clients and servers only store selected files in the job's file set: the SAS program file, LOG and LIST output files. You can optionally use the --job-file option to specify input files that are also stored in the file set to be copied to the batch server pod and used by the SAS program. This latter option should be used for small files. Large files should always be handled differently, i.e. data should be stored in a database or files stored in a volume mounted to the batch server pod.   In summary, a job’s file set should be small both in size and number of files. Does it really make any difference, in terms of total runtime, whether you use it or not?   In our test environment we measured the runtime difference between the sample commands listed in the previous section.   We extracted job statics with the following command (using json format to avoid rounding of the reported times):   sas-viya --output json batch jobs list --details   Here are the results:   { "items": [ { "contextId": "99dddb38-c821-463e-89c3-f81fd9f9d656", "createdBy": "viya_admin", "creationTimeStamp": "2024-10-08T21:00:17.680645Z", "endedTimeStamp": "2024-10-08T21:00:25Z", "id": "9b3dbeb7-e4b3-4fa8-b5c3-553b2d9e9503", "modifiedBy": "user1", "modifiedTimeStamp": "2024-10-08T21:00:25.043386Z", "name": "Import-fileset", "processId": "6d28ba2e-d6b1-4255-9025-f57943795d8d", "returnCode": 0, "startedTimeStamp": "2024-10-08T21:00:17.763363Z", "state": "completed", "submittedTimeStamp": "2024-10-08T21:00:17.680645Z" }, { "contextId": "99dddb38-c821-463e-89c3-f81fd9f9d656", "createdBy": "viya_admin", "creationTimeStamp": "2024-10-08T21:00:32.707924Z", "endedTimeStamp": "2024-10-08T21:00:39Z", "id": "c075e008-3b97-4266-95c9-f685266b9d90", "modifiedBy": "user1", "modifiedTimeStamp": "2024-10-08T21:00:39.675398Z", "name": "Import-filesystem", "processId": "88ebaa58-92e8-4518-ae6c-b8688ac5989e", "returnCode": 0, "startedTimeStamp": "2024-10-08T21:00:32.847071Z", "state": "completed", "submittedTimeStamp": "2024-10-08T21:00:32.707924Z" } ] }   There are a lot of timestamps! Let’s take the earliest and latest for both: modifiedTimestamp - creationTimesStamp. This gives about 7.4s (using the file set) versus 7s (using the storage). This result proved consistent with a few repetitions of the test. It seems the I/O optimization can save up to 5% of the runtime. But there is a catch: the file set is created, and files are uploaded, BEFORE the job gets submitted. When these job timestamps are recorded, that time has already passed, and it has not been captured. We could increase the logging levels to record more detailed transaction times, but there is a simpler way. We have seen that a file set ID is always created, even when it’s not used. We can check the file set creation times with the following command:   sas-viya --output json batch filesets list   Here is the log for our two sample jobs:   { "items": [ { "contextId": "99dddb38-c821-463e-89c3-f81fd9f9d656", "createdBy": "viya_admin", "creationTimeStamp": "2024-10-08T21:00:16.621873Z", "id": "JOB_20241008_210016_621_1", "modifiedBy": "viya_admin", "modifiedTimeStamp": "2024-10-08T21:00:16.621873Z" }, { "contextId": "99dddb38-c821-463e-89c3-f81fd9f9d656", "createdBy": "viya_admin", "creationTimeStamp": "2024-10-08T21:00:32.554613Z", "id": "JOB_20241008_210032_554_1", "modifiedBy": "viya_admin", "modifiedTimeStamp": "2024-10-08T21:00:32.554613Z" } ] }   The difference between file set creation time and job creation time is about 1 second when the file set is used, and only about 0.15 seconds when not. We can see that this optimization actually saves more than 10% of the runtime.   Repeating this test multiple times, we measured that uploading the files into the file set creation took between 0.5s and 1s, while the time without file uploading was consistent between 0.14s and 0.16s.   In summary, in our little experiment the optimized I/O resulted in a 10% runtime optimization with better runtime consistency.   Is it worth? With a single job like our simple test, maybe not. In a high-throughput environment with hundreds of short-lived batch jobs, the savings can compound and become noticeable.     Coming up next   We have seen how to configure shared storage, usable from batch clients and servers, and what options are required to leverage it to avoid uploading/downloading files to a file set managed by the Files service.   In the next post we’ll present a longer performance study that compares default batch servers versus reusable batch servers with optimized I/O.     Find more articles from SAS Global Enablement and Learning here. ... View more

SAS Innovate is an amazing venue where customers can deep dive into SAS’ latest breakthroughs. This April I had the privilege to attend and present at SAS Innovate 2024 in Las Vegas. Amongst many energizing sessions, I particularly enjoyed the Super Demo “Need for Speed: Fast-Tracking Batch Workloads with SAS Viya Workload Management”, presented by Wes Gelpi, Director R&D, SAS Viya Compute, and Prasad Poozhikalayil, Sr Product Manager, SAS Viya Platform. The topic caught my attention, and I decided to look deeper into some details and test the new capabilities.   In short, starting with SAS Viya 2024.04 stable release, SAS introduced new batch capabilities that benefit our customers who engage in extensive batch processing, involving time sensitive, high volumes of SAS jobs of very short duration. The new features address some inefficiencies noted by customers in their previous experience, such as longer turn-around time, excessive resource utilization, potential timeouts and job failures due to sudden surge in number of pod creation.   This article is the first in a series where I share the result of my reading and researching. It describes reusable batch servers.   The second article will focus on optimized input and output file storage.   Finally, the third and last will present some performance tests that compare the throughput and total run time of a bust of short-running jobs when run on default batch servers versus high-throughput batch servers.   SAS Viya Batch Servers   The SAS batch processing infrastructure in the SAS Viya platform is designed to provide an ad-hoc set of servers and services that, by default, are tailored to processing long-running, scheduled batch jobs. It also supports interactive execution to simplify development and testing, but at its core the focus are long-running, fire-and-forget batch jobs.   In this default configuration, each job submission is processed by a batch server (a dedicated SAS process) running in a Kubernetes pod that is started on-demand, executes that one job and then shuts down.   What if your use case centers on managing a significant volume of small, time-sensitive data jobs? With the default batch server, you might face a critical performance bottleneck in the latency from when a pod starts to when it actually begins running each job. Starting a new pod can add an overhead of up to 10/15 seconds to the total execution time. Even worse, if your Kubernetes cluster is configured for dynamic auto-scaling and there are no compute nodes available, starting and initializing a new node might add an additional overhead of a few minutes. All these overheads do not impact too much a set of scheduled jobs when each job runs for a few hours – or even a few minutes. Executing a job in 20 minutes or 20 minutes and 15 seconds is a small difference of about 1%.  But when your use case is submitting quick burst of hundreds of 15-seconds batch jobs, adding 15 seconds to start up a pod each time means doubling the total execution time, or, if want to look at it from a different angle, the total throughput in terms of jobs per minute halves. That’s a significant performance hit! The high number of pod creation and destruction also stresses the Kubernetes API services, with the risk of excessive resource utilization leading to potential timeouts and failures.    Animation showing default batch servers processing 3 short jobs.   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Reusable Batch Servers   Starting with SAS Viya 2024.04 stable release, the SAS Viya platform provides a new feature: “Reusable Batch Servers”. A reusable batch server is a batch server pod that can process multiple jobs sequentially, without stopping/restarting the pod. These servers decouple jobs from pod creation and initialization.   SAS developed this feature to benefit most of our customers who engage in extensive batch processing, involving time sensitive, high volumes of SAS jobs of very short duration. It addresses the inefficiencies noted above, such as longer total execution time, excessive resource utilization due to sudden surge in number of pod creation, potential timeouts and job failures.  Animation showing reusable batch servers processing 3 short jobs.   How to Configure and Enable Reusable Batch Servers   To use reusable batch servers, you must enable the capability in the batch context, configure a few settings – such as the minimum number of servers that run all the time – and specify a shared account under which they should run.   Let’s describe these steps in more detail, including considerations about creating an efficient workload management configuration.   Configure the shared account credential   To configure the batch server to run under a shared account, you must use the SAS Viya CLI to create a new credential for this shared account. With the CLI, login with an administrative account and use the batch plugin to store the shared account userid and password with the following command:   sas-viya batch credentials create   The CLI will prompt for a username and password, then save them in the credentials microservice.     If you do not want to store a password, you also have the option to use an OAuth token. This is useful, for example, to support SCIM authentication providers. See the documentation for detailed steps.   (Optional) Created a dedicated workload queue.   While not strictly required, creating a dedicated workload queue to be used for high-throughput batch servers can simplify further tuning of the SAS Viya platform. For example, you can segregate high throughput batch jobs from other kinds of compute workloads. An administrator can use the Workload Orchestrator page of SAS Environment Manager to create and configure a dedicated queue.     Create and configure a new batch context   You may want to create a new batch context for high-throughput batch, and leave the default context configured as is, to support long running jobs. An administrator can use the SAS Viya CLI:   sas-viya batch contexts create \     --name "high_throughput" \     --launcher-context-name "SAS Batch service launcher context" \     --queue "batch_high_throughput" \     --run-as "user1" \     --reusable \     --min-servers "7" \     --max-servers "7"   The same result can be accomplished with the Contexts page of SAS Environment Manager.     As soon as the new context is created, the batch service starts the preset number of minimum servers.   An administrator can use to Workload Orchestrator page of SAS Environment Manager to verify the newly started servers. Notice that they are running under the shared account set in the context configuration.     About Server Sizing   You may have noticed in the screenshots above that both the new workload queue and the new batch context use the same value for the number of servers, in the examples above 7. How did we come to that number?   In this case, we only had one compute node with 8 cores, and we decided to fully dedicate it to this test. A common practice with batch jobs is to dedicate 1 core to each job. We wanted to leave 1 core to the OS and Kubernetes services, which means that our batch service can use 7 cores, leading to the above setting of 7 reusable servers. The workload queue parameter was set accordingly.   Also, for this environment we use a sustained flow of batch jobs, so we set the minimum and maximum number of servers to the same value. This optimizes server allocation, because the maximum number of servers is immediately available at startup, but it comes with the associated cost of full infrastructure utilization. If you know that your batch jobs come in burst of variable throughput, you can set a lower minimum - even down to zero. As usage falls off and the system is increasingly idle, the unused batch servers will terminate automatically, leaving the minimum number up and running and saving on infrastructure cost. The default server idle timeout is 300 seconds, and a custom value can be set when defining the new batch context.   Each customer environment is unique, and these values should be set after proper system sizing and architecture design. But do not feel compelled to find the perfect tuning at first try. SAS Workload management gives you the tools to put into practice the mantra “Plan. Monitor. Refine”.   For further reading, a very detailed discussion about the factors that influence the tuning of these setting is presented by Rob Collum in his article Determine how many SAS Viya analytics pods can run on a Kubernetes node – part 3.   Coming up next   When discussing this new feature with some colleagues, I noticed we were using many different names. We call it high-performance batch, high-throughput batch, reusable batch… In the end, call it as you want: what matters is that your demands for high-volume batch processing can be satisfied in a shorter time. Who does not like that?   In the next article of this series, we’ll explore another change that improves batch performance by eliminating the need to download and upload files exchanged between batch clients and servers. Stay tuned!   Edoardo Riva     Find more articles from SAS Global Enablement and Learning here. ... View more

The SAS extension for Visual Studio Code supports a special file type called “SAS Notebooks.” These are just like Python notebooks that you might have used in Jupyter or other environments. Notebooks allow you to mix code with documentation and results, encapsulating a work product that is easy to understand and demonstrate. What if you want to use SAS Studio to work with SAS notebooks as well? Read on to find out how!   I was privileged to teach a Hands-on Workshop at SAS Innovate in Las Vegas, Nevada this year. Titled “Interacting with SAS® Viya® Using SAS® Enterprise Guide® and Other Clients”, the session included an exercise dedicated to exploring the capabilities of  the SAS Extension for Visual Studio Code. One of these functionalities is the support for SAS Notebooks, which, just like Python notebooks that you might have used in Jupyter or other environments, allow you to mix code with documentation and results in an interactive online page. SAS Notebooks can integrate SAS, SQL, Python code and inline documentation in a seamless execution environment.   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   While I was demonstrating this capability, one of the attendees asked “I often work on the same SAS programs both in VSCode and SAS Studio. Can I open a SAS notebook with SAS Studio as well?”. I’m glad they asked: I was just going to show how to close the circle from VSCode to SAS Studio, so my answer was a cryptical “Well, no, but almost yes”. What does that mean?   While at the moment you cannot use SAS Studio to directly open a “.sasnb” file (the format of SAS notebooks), it’s really easy to export a SAS notebook from VSCode as a “.flw” file (the format of SAS flows). Every cell in the SAS notebook becomes a node in the SAS Studio flow. SAS Studio can then open the file and let you work with the flow. Voila'.   Since this is a file conversion, changes made in the flow would need to be synched back to the notebook, but that is beyond the scope of this post. Always use good version management processes to synch such changes.   Step by step   Here is a guided step-by-step guide directly taken from the SAS Innovate session that shows you how to do it.   After creating a SAS notebook in Visual Studio Code, save it by pressing CTRL+S. In this example, it is saved under “My Folder” with the name "Innovate24.sasnb". In the explorer pane, locate the file by expanding "My Folder", then right click on the "Innovate24.sasnb" file and select "Convert to Flow..." (this is a custom action that the SAS Extension to VS Code adds to the right-click context menu).     Press "Enter" to accept the default filename. VS Code will save the notebook as a SAS Studio flow, located in the same folder as the original file. Open the browser and sign into SAS Studio. In SAS Studio explorer pane, navigate to "Folder Shortcuts" > "Shortcut to My Folder" and double-click on the "Innovate24.flw" file to open it.     The SAS Studio flow that is automatically generated by VS Code includes each cell in a dedicated swimlane to preserve the sequential execution nature of notebooks, both visually and in the generated code:     You can verify the code in each node. As an example, select the Python Program node in SAS Studio to look at the code:     you should recognize the same code entered in VS Code in the original notebook:     Now that the flow is open in SAS Studio, you can submit it using the “Run” button to get the same results as the original notebook.   Conclusion   One of the objectives I had with this SAS Innovate session was to show how you can leverage your knowledge of a SAS client to jump into a different one, without having to re-learn how to use it from scratch. This can open the doors to benefiting from additional capabilities. As an example, SAS notebooks are inherently interactive and cannot be scheduled or submitted in batch from VS Code. Once you transform a notebook into a SAS Studio flow, that becomes possible:     If you want to see another example of how to use VS Code, including this integration with SAS Studio, see Ryan King’s post Using The SAS VS Code Extension to Check Prime Distribution.     Find more articles from SAS Global Enablement and Learning here. ... View more

The SAS Infrastructure Data Server is a fundamental component of the SAS Viya architecture, that can be deployed both in the same Kubernetes cluster of the other components or outside the cluster.   Are you looking to choose an external PostgreSQL instance for use with the SAS Viya platform? In this post, we’ll highlight useful considerations that can help choose the right instance when opting for an external, managed cloud instance. ​ A Quick Summary   Before diving into the topic at hand, let’s recap what are we talking about. If you’ve been following this space, you may recognize there have been a few posts on the subject of external (or internal) PostgreSQL:   Choosing Between Internal or External PostgreSQL for SAS Viya Deploying SAS Viya with an External PostgreSQL Configure an external PostgreSQL for SAS Viya in your Open-Source Kubernetes deployments – part 1 Configure an external PostgreSQL for SAS Viya in your Open-Source Kubernetes deployments – part 2   You can refer to these articles for additional details, but, in short, we can say that the SAS Infrastructure Data Server stores SAS Viya user and system content, such as reports, authorization rules, attachments, user preferences, and so on. The SAS Infrastructure Data Server requires an instance of PostgreSQL. The SAS Viya platform deployment can install and configure its own open-source PostgreSQL database inside the SAS Viya Kubernetes cluster; this is referred to as an internal instance. If you want, you can choose instead to provide another PostgreSQL instance, which is called an external instance.   External PostgreSQL choices   For an external PostgreSQL server, the SAS Viya platform support has changed over time and with the different releases. For this reason, you are always encouraged to refer to the official documentation corresponding to your SAS Viya platform release. You can find the latest one here: https://go.documentation.sas.com/doc/en/itopscdc/default/itopssr/p05lfgkwib3zxbn1t6nyihexp12n.htm#n0yp4jk7n9mcl1n1vu0oakq63bfc   As a general rule, with the SAS Viya platform, you can use managed PostgreSQL instances hosted by any supported cloud providers, or the Open Source version of PostgreSQL – for self-managed deployments:   Customer-managed PostgreSQL instance​   PostgreSQL (Open Source)​   Managed PostgreSQL instances hosted by cloud providers:​   Microsoft Azure Database for PostgreSQL - Single Server Microsoft Azure Database for PostgreSQL - Flexible Server​ Amazon RDS for PostgreSQL​ Google Cloud SQL for PostgreSQL   Two of the communities.sas.com articles previously referenced focus on the provisioning and configuration of self-managed Open Source PostgreSQL. The rest of this post will focus on the cloud-hosted managed options.   What drives the choice of a specific managed instance?   The official documentation highlights all the requirements that should be met when using an external PostgreSQL instance. For example, starting with SAS Viya platform 2023.11, PostgreSQL 11 is not supported anymore: this automatically excludes any instance of “Microsoft Azure Database for PostgreSQL - Single Server”, which only supports PostgreSQL up to version 11 (and, by the way, has been put on the retirement path by Azure).   Cloud architects, when designing the database to use for each project, must balance business drivers that sometimes drive in opposite directions. For example, it’s quite difficult to satisfy scalable performance, high availability, and low cost, all at the same time. However, it’s possible to get good performance and minimal failure rates for a given cost when considering the specific software requirements.   While other solutions may require high I/O throughput or fast storage, the most important factor that drives proper instance sizing with the SAS Viya platform is that PostgreSQL should support at least 1024 “maximum number of connections”.  How can you satisfy this on cloud-managed instances? Let’s see what it means for each of the supported cloud vendors.   Azure Database for PostgreSQL   Azure Database for PostgreSQL powered by the PostgreSQL community edition is available in two deployment modes: Single Server Flexible Server   “Azure Database for PostgreSQL - Single Server” is on the retirement path and is scheduled for retirement by March 2025. We do not recommend that you choose this edition for any new deployment.   “Azure Database for PostgreSQL – Flexible Server” provides different instances that meet the 1024 concurrent connections requirements. For production use, Burstable instances are usually not recommended because of their lack of predictable performance.   Highlights of the minimum “Azure Database for PostgreSQL – Flexible Server”  instance sizes that support at least 1024 max user connections Screenshot as of January 2024 of https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-limits#maximum-connections.   The viya4-iac-azure utility can be used to provision a managed PostgreSQL server on Azure. By default, it creates an instance type GP_Standard_D16s_v3 (16vCPU, 64GiB RAM) that exceeds the SAS Viya requirement for 1,024 concurrent connections and is suited for baseline production use.   You can choose to further customize the maximum number of connections independently of the chosen instance size. Microsoft recommends against this custom setting, for the risk of exhausting the memory of the PostgreSQL instance, risking crashes or high latency.   Amazon RDS for PostgreSQL​   Amazon RDS for PostgreSQL lets you choose from a variety of instance types with varying combinations of CPU, memory, storage, and networking. The maximum number of concurrent connections is calculated based on the DB Instance memory, with a maximum of 5000.   Highlights of the “Amazon RDS for PostgreSQL”  rules to calculate the max user connections parameter Screenshot as of January 2024 of https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Limits.html#RDS_Limits.MaxConnections   The viya4-iac-aws utility can be used to provision a managed PostgreSQL server on Amazon. It defaults to an instance type db.m5.xlarge (4vCPU, 16GiB RAM) that meets SAS requirement for 1,024 connections: 16 GiB = 17,179,869,184 bytes, which, divided by 9,531,392 is equal to 1,802 connections allowed.   Google Cloud SQL for PostgreSQL   Google provides Cloud SQL for PostgreSQL as a managed service. The most important factor impacting image sizing is the server memory. The underlying instance needs more than 120GB of RAM to meet the 1024 connections requirements.      Highlights of the “Google Cloud DQL for PostgreSQL”  rules to calculate the max user connections parameter Screenshot as of January 2024 of https://cloud.google.com/sql/docs/postgres/flags#postgres-m   The viya4-iac-gcp utility can be used to provision a managed PostgreSQL server on Google Cloud. By default, it creates an instance type db-custom-8-30720 (8 vCPU and 30 GB of RAM): you can see in the above table that this supports 600 connections by default. You can choose to further customize the maximum number of connections independently of the selected instance size.   Additional Considerations   If you are deploying multiple SAS Viya instances, you can use multiple PostgreSQL servers with a single database in each (one server per deployment), or you can share a single PostgreSQL server with multiple databases inside (one database per deployment).   Some SAS offerings require an additional PostgreSQL cluster called SAS Common Data Store, or CDS PostgreSQL. It is configured separately from the required platform PostgreSQL cluster that supports SAS Infrastructure Data Server. You can find a list of offerings that require this separate PostgreSQL cluster at https://go.documentation.sas.com/doc/en/itopscdc/default/itopssr/p05lfgkwib3zxbn1t6nyihexp12n.htm#n03wzanutmc6gon1val5fykas9aa. All of the considerations written in the previous sections apply to this instance, as well.   Conclusion   The SAS Viya platform requires an instance of PostgreSQL, and you can choose an external, cloud-based PostgreSQL instance for use with the platform. We have seen in this article different options and minimum instance sizes that support at least 1024 concurrent connections, which is the requirement for the SAS Viya platform. ... View more

The SAS Viya platform includes by default (and for free!) SAS/CONNECT with all its capabilities. Previous articles, including Moving to the cloud with SAS/CONNECT, have shown SAS/CONNECT evolution to adapt to cloud environments and support our customers in their journey to modern architectures. Innovations keep coming! Starting with SAS Viya version 2023.10, SASCMD sign-ons (a.k.a. MP Connect sign-ons) always start new SAS sessions in new pods by default.   Let's start by stepping back a moment to see what MP Connect is, and why this is a welcome change.   Distributed computing, with a single machine?   Before cloud computing, before any kind of multi-machine distributed computing, SAS customers had no choice but to run their code on a single machine (yes, you can tell I’ve been at SAS for quite some time).   SAS capabilities have always been disruptive, even in those limited environments. Multi-process CONNECT (or MP Connect) was conceived to divide time-consuming tasks into multiple units of work and to execute these units of work in parallel. It did that – and still does – by providing a framework that lets you start and coordinate multiple child SAS processes from a controlling parent SAS session. This way you can parallelize code that otherwise runs sequentially, for the purpose of reducing the total elapsed time necessary to execute a particular application.    An example of a process split into multiple parallel subprocess.   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   From machine to pod   MP Connect capabilities are automatically available in SAS Viya platforms, because they are provided by the SAS/CONNECT product, which is included by default in every SAS Viya environment. What happens when you move your existing MP Connect code to SAS Viya? Apparently, it's business as usual: child processes are spawned from the parent session to execute part of your code in parallel. But, under the hood, the execution environment is completely different. What used to be running on a physical machine, now runs in Kubernetes, inside a pod.   MP Connect spawning child sessions in a compute pod.   Maybe your original machine had 8 cores and 64GB of RAM (not too much, in modern terms) and your code is written to take full advantage of that computing power by spawning 7 additional sessions that use, in total, almost 100% of all CPUs. Kubernetes is built to control the execution environment and prevent a single pod from exhausting the resources of the node where it's running. As soon as your code starts requesting all that CPU power, Kubernetes will throttle down your pod - in a default configuration, down to 2 CPUs maximum. As a result, your existing code will run 4 times slower! It's obvious that this setup does not scale as expected. You could argue that the issue can be easily fixed by configuring the Kubernetes cluster to give more resources to SAS compute and connect pods. Yes, this solution could work, but it suffers of two problems:   It simply moves the bar a bit further. Today you want back your 8 CPUs, what will happen when you'll have more data and you will need 16, 32, or more CPUs? A single pod can only scale up to the size of the node where it's running. Scaling up an individual gigantic pod is an anti-pattern in the cloud. Cloud environments are designed to scale out by launching multiple, smaller pods. The Kubernetes cluster hosting your SAS Viya platform is probably designed to support multiple users with multiple applications. You cannot hog all resources for yourself!   What could be a better solution? Obviously, embracing a cloud-native design and scaling out to multiple pods!   Scaling out to multiple pods   If you've been following so far, you can now understand how the new default behavior is welcome in cloud environments. Starting with SAS Viya 2023.10, every new MP Connect sign-on always starts a child SAS process in its own dedicated pod, embracing cloud-native scalability and elasticity. Kubernetes can spread out the pods on multiple nodes, and, if the cluster is configured for auto-scaling, your limit is the sky... or better, your budget!    MP Connect launching child sessions in dedicated connect pods.   It's a matter of choices   Every time a new default is introduced in existing environments, it's important to give SAS Administrators to option to embrace this new capability, or to reset the SAS Viya platform to behave just like before. In this case, you can use a new environment variable, SAS_LOCAL_MPCONNECT. When set to true, it re-enables local MP Connect Sign-Ons, i.e. the original functionality of spawning a child session in the same pod where the parent process is running.   A SAS Administrator can use SAS Environment Manager to set the SAS_LOCAL_MPCONNECT environment variable to true in sas.compute.server: startup_commands and sas.connect.server: startup_commands configuration instances. In this case, the setting is configured for every SAS compute and connect server running in the environment.    Setting the SAS_LOCAL_MPCONNECT option for all compute server sessions.   A more limited scope could be achieved by setting the option case-by-case, as needed. As an end-user, you can add the following line in your code just before submitting the SIGNON statement:   options set=SAS_LOCAL_MPCONNECT=true;   In this case, only the current execution reverts back to the previous functionality.   Why would I revert back?   This new capability seems the obvious choice when you are writing code that uses MP Connect in SAS Viya. So why would you revert back to the previous functionality? The most obvious answer is when you are migrating existing code that could be broken by the new behavior. It's easy to understand that if the existing code uses local resources to share data between MP Connect sessions, it cannot work as-is when these sessions are launched in different pods. Here are a couple of examples:   The code saves datasets in the SASWORK library of one session to retrieve them from another session. SASWORK locations can be shared between SAS processes with the INHERITLIB= option of the SIGNON statement. You are using SASESOCK libraries to stream data between sessions through local pipes.   Those issues can be solved by re-architecting your application - for example, by sharing data through Kubernetes volumes mounted on all pod sessions, instead of using local directories. Yet, reverting back to the previous behavior can be an interim step to keep the code running during your migration.   Conclusion   I am always excited when I see how SAS Viya keeps evolving by embracing existing functionality and integrating it into modern cloud-native architectures. We have seen in this post how traditional MP connect code can be used without sacrificing Kubernetes capabilities to manage resource utilization, provide scalability when required, and ensure user-level process isolation.     Find more articles from SAS Global Enablement and Learning here. ... View more

The SAS/CONNECT Programming with SAS Viya: SIGNON article presents multiple ways to use the SIGNON statement in a SAS client to start and connect to a remote server session executing in the SAS Viya platform. One of the options covers the use case of external clients, with a brief list of preliminary configuration steps required to enable the capability. This article dives into the detailed steps behind that summary list, including a step-by-step recorded demo.   What are we trying to accomplish?   In a default deployment, the SAS/CONNECT spawner service is only reachable from internal clients. An example of an internal client is a SAS Studio session that submits the SIGNON command.    An internal client connecting to the SAS/CONNECT spawner to launch a server. Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   External clients need additional configuration to reach the SAS/CONNECT spawner from outside the Kubernetes cluster. You have to create either a LoadBalancer or NodePort service. Examples of external clients are a SAS 9.4 Display Manager session executing on a laptop, or a SAS Studio session running in a different SAS Viya cluster or namespace.    External clients connecting to the SAS/CONNECT spawner via a Kubernetes LoadBalancer or Nodeport to launch server sessions.   The configuration steps Preliminary: verify that internal connections work as expected   Before even starting any configuration change, it’s a good practice to test that the default functionality is working as expected. You can verify that internal SAS/CONNECT connections are OK, in your SAS Viya environment, by logging into SAS Studio and submitting this simple test code – no changes required:   /* launch a SAS/CONNECT session on the same cluster */ %let session1=sas-connect-spawner 17551; /* authenticate re-using the already logged-in credentials */ SIGNON session1 user="_OAUTH_"; %put The client session is running on host: &SYSHOSTNAME; rsubmit; %put The server session is running on host: &SYSHOSTNAME; endrsubmit; SIGNOFF;   This code will do the following: Sign on to the SAS Connect Spawner using the Kubernetes service name sas-connect-spawner , which is only available inside the namespace, re-using the same credentials you entered when logging into SAS Studio Locally output the hostname of the pod where the client SAS session is running Use rsubmit to return the hostname of the pod where the SAS/CONNECT session is running Sign off from the remote session If this all works, you are ready to perform the steps to enable external access. 1.      Enable external access to the SAS/CONNECT spawner.   You can follow the instructions from the README file at <deploy>/sas-bases/docs/configure_sasconnect_spawner_in_the_sas_viya_platform.htm , in the section “Provide External Access to sas-connect-spawner via a Load Balancer”.   As you can read there, SAS provides a sample file ( sas-bases/examples/sas-connect-spawner/enable-external-access/sas-connect-spawner-enable-loadbalancer.yaml ) that you can copy in your sas-config directory, customize as needed, and include in the kustomization.yaml used for the original deployment. Then it’s just a matter of re-applying it using the same method as the original deployment (manual kubectl commands, sas-orchestration CLI, etc.)   An effective customization I always like to do is to reuse the load balancer frontend, which was already provisioned for the SAS Viya Ingress. This can bring a couple of benefits: Lower infrastructure costs, by avoiding the creation or provisioning of another load balancer. May simplify the TLS configuration, as detailed in the next step, by re-using the same external IP address and DNS alias as the Ingress. If you want to do it, you have to find the IP of the ingress load balancer with a command such as:   NS=ingress-nginx kubectl -n ${NS} get service ingress-nginx-controller -o json | jq -r ".spec.loadBalancerIP"   Then add it to the sas-connect-spawner-enable-loadbalancer.yaml as the last line, as in this example:    Example configuration of sas-connect-spawner-enable-loadbalancer.yaml   After the new configuration has been applied and submitted to Kubernetes, it's always a good practice to verify that the load balancer service gets created as expected, listening on an IP address external to the cluster:   NS=sasviya kubectl -n ${NS} get service sas-connect-spawner-loadbalancer Sample result: NAME                               TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)           AGE sas-connect-spawner-loadbalancer   LoadBalancer   10.42.230.223   10.XXX.XX.X    17551:31883/TCP   15m   2.      Verify the DNS name presented by the SAS/CONNECT spawner TLS certificate.   In the previous step, we added the new Kubernetes LoadBalancer service to an existing external load balancer that had already been configured in the corporate DNS and whose DNS alias was already specified in SAS Viya kustomization.yaml. For that reason, the certificates generated by SAS Viya already include that DNS name. The simplest way to verify that is to find the DNS name used for the ingress, and confirm that it is included in the SAN field of the SAS/CONNECT spawner TLS certificate.   To find the DNS name of the ingress, you can look into the kustomization.yaml file used to deploy SAS Viya:   grep INGRESS_HOST kustomization.yaml   Sample result:   - INGRESS_HOST=sasviya.gelenable.sas.com   Now list all the aliases in the SAN field of the SAS/CONNECT spawner TLS certificate:   NS=sasviya CONNECT_SPAWNER_SECRET=$(kubectl -n ${NS} get secrets | grep sas-connect-spawner | awk '{print $1}') kubectl -n ${NS} get secret ${CONNECT_SPAWNER_SECRET} -o jsonpath="{.data['tls\.crt']}" | base64 -d | openssl x509 -text -noout | grep -A 1 "Subject Alternative Name"   With these commands, we first get the name of the Kubernetes secret used by the SAS/CONNECT spawner. Then, we extract the certificate from tls.crt field of that secret, base-64 decode it, and pass the result to an OpenSSL command that prints the details of the certificate in a human-readable format. Finally, we filter the results to only show the "Subject Alternative Name" details.   The result should be similar to the following. Note that it includes the full DNS name found in the previous step.   X509v3 Subject Alternative Name:     DNS:localhost, DNS:sas-connect-spawner, DNS:sas-connect-spawner-8b9cd797b-5wswp, DNS:sasnode07, DNS: sasviya.gelenable.sas.com, IP Address:10.42.126.166, IP Address:10.43.17.98, IP Address:10.XXX.XX.X, IP Address:127.0.0.1   If you did not reuse the existing load balancer, and you created a new load balancer with either a new frontend IP address or a different DNS alias for the SAS/CONNECT spawner, then you’ll have to follow the additional steps described in the Encryption in SAS® Viya®: Data in Motion guide in the section Add the External Host Name or IP Address to the SAN in the Signed Certificate.   3.      Configure the client to trust the SAS Viya root CA certificate.   It is quite common to deploy the SAS Viya platform using a self-signed SAS Viya root CA certificate. This CA certificate is unknown to clients external to the cluster, so they cannot trust certificates presented by any SAS Viya server - including SAS/CONNECT. You can tell you are in this situation when, the first time you log into a web application, such as SAS Studio, your browser presents you with a security warning that you have to accept before you can get in. In this case, to be able to use SAS/CONNECT from an external Windows client, you have to obtain the SAS Viya root CA certificate, and then import it into the Windows client truststore. After that, the client will trust the certificates presented by SAS Viya servers. The official documentation lists a simple command to save the root certificate into a temporary file:   NS=sasviya kubectl -n ${NS} get secret sas-viya-ca-certificate-secret -o go-template='{{(index .data "ca.crt")}}' | base64 -d > /tmp/SASViya_ca.crt   Then you have to copy the resulting SASViya_ca.crt file to your Windows client (you can email, ftp, network copy… ), and import it in the Windows truststore, for example following the instructions in the Encryption in SAS® Viya®: Data in Motion guide in the section Import CA Certificates into the Windows Trusted Root Certificate Authorities Store.   Finally: Connect!   After you get this far, it’s time to enjoy the fruit of your work. From the client machine, you can start a SAS session and submit some code to verify if all is fine and SAS/CONNECT is configured for external connections. You can use code like the following, after substituting the DNS alias or IP address of the load balancer in the session1 macro variable:   /* launch a SAS/CONNECT session from outside the cluster */ /* set TLS connection options */ OPTIONS NETENCRYPTALGORITHM=SSL; /* connect using a fully-qualifies hostname or DNS alias */ %let session1=sasviya.gelenable.sas.com 17551; /* authenticate with an authinfo file */ SIGNON session1 password="_AUTHINFO_"; rsubmit; %put &SYSHOSTNAME; endrsubmit; SIGNOFF;   The resulting submission should print, in the SAS log, the name of the SAS/CONNECT server pod that gets started in the SAS Viya cluster, as in the following screenshot:       Recorded Demo   A recording of the configuration of a demo environment, showing all the steps listed above, is available on the SAS YouTube channel: SAS Demo | Configuring SAS/CONNECT for external access to SAS Viya Summary   When you have an external SAS client that you want to connect to a SAS Viya environment, you have to perform some configuration steps before you can connect them. Enable the SAS/CONNECT spawner to support external clients. Add the external host name or IP address to the SAN in the SAS/CONNECT spawner certificate. Obtain the SAS Viya root CA certificate and add it to the truststore of the external client. Links Official documentation to configure SAS/CONNECT for external access: The README file available in your deployment directory at <deploy>/sas-bases/docs/configure_sasconnect_spawner_in_the_sas_viya_platform.htm , in the section "Provide External Access to sas-connect-spawner via a Load Balancer" The "External Client Sign-On to TLS-Enabled SAS Viya SAS/CONNECT Spawner" section of the SAS® Viya® Platform Administration documentation  Sample SAS/CONNECT code to sign-on from internal or external Clients Ensure the TLS certificate presented by SAS/CONNECT spawner includes the external hostname used to connect to the server and any desired DNS alias Ensure the client accepts as valid the TSL certificate presented by the SAS/CONNECT spawner: Obtain the SAS Viya Platform Generated Root CA Certificate and Import CA Certificates into the Windows Trusted Root Certificate Authorities Store SAS 9.4 and SAS Viya 3.x client requirements   Find more articles from SAS Global Enablement and Learning here. ... View more

Starting with the 2023.03 SAS Viya release, the SAS Viya platform supports using an external instance of OpenSearch to provide searching and indexing capabilities. What are the requirements, and how to enable this new capability? If you are interested, read on!   OpenSearch OpenSearch provides search and indexing features for software that runs on the SAS Viya platform. Starting with the 2020.1.3 Stable release, the SAS Viya platform includes an Apache 2.0-licensed distribution of OpenSearch with enhanced security and a SAS-provided Kubernetes operator to handle its operation and management.   Starting with the 2023.03 release, new configuration options are added to the platform to support either: an internal instance of OpenSearch automatically included in the deployment (just as before) an external OpenSearch instance that you administer and maintain OpenSearch is required by every SAS Viya offering except SAS Viya Programming.   Why? The OpenSearch instance deployed by default inside the SAS Viya platform includes opinionated deployment settings to configure it according to the platform requirements. For example, the pods must run under a particular user ID, and require specific kernel settings on the nodes underlying the Kubernetes cluster. Some customers may have security policies that prevent these settings in their Kubernetes clusters. And, let’s be honest, a production instance of OpenSearch does not fit typical cloud-native environments: for example, every update to the SAS Viya platform requires a full stop and restart of the OpenSearch cluster.   An external OpenSearch instance is managed outside of SAS Viya and can alleviate these considerations. Supporting an external OpenSearch removes some requirements from the Kubernetes cluster dedicated to the SAS Viya platform. As an added benefit, an external OpenSearch reduces the footprint of the SAS Viya platform deployment.   In the end, adding support for an external option provides you with more flexibility in your architectural choices!   Internal VS External A simple table can easily show the main comparison points between using an internal OpenSearch instance or an external one. Internal External Dedicated instance deployed by the same process as the rest of the SAS Viya platform OpenSearch instance deployed by you before starting the SAS Viya platform deployment Deployed in the same namespace as the SAS Viya platform Deployed outside the SAS Viya platform namespace SAS Viya platform cluster sizing and design must account for OpenSearch (including kernel requirements, and storage requirements) OpenSearch requirements and sizing do not impact the SAS Viya platform cluster Contains all required plugins (including enhanced security) Must include required plugins Automatically configured as required by the SAS Viya platform Must be configured according to SAS specifications Managed by a SAS-provided Kubernetes operator Managed as you choose to Supported by SAS Supported by you   Requirements Whether you decide to use an internal or an external OpenSearch, once your deployment is done, there is no turning back. It is not possible to reconfigure an existing SAS Viya platform to use a different OpenSearch. If you change your mind, you will have to un-deploy and re-deploy.   The official requirements for an external deployment are described in the readme file at <$deploy>/sas-bases/examples/configure-Elasticsearch/external/README.md . Here is a summarized listing, valid at the time of writing (always refer to the official documentation to have up-to-date requirements): You can only use the opensource OpenSearch edition, not Elasticsearch. The supported OpenSearch version is 2.5. It must include a specific list of plugins used by the SAS Viya platform. The OpenSearch server should be set up and running before you start the SAS Viya platform deployment. Managed cloud subscriptions to Elasticsearch and OpenSearch are not supported; only instances that you deploy and configure. TLS between the SAS Viya platform and OpenSearch is strongly recommended, but not required. SAS Visual Investigator requires additional configuration settings. You can find the listing of SAS Visual Investigator specific requirements in the README file at <$deploy>/sas-bases/examples/configure-Elasticsearch/external/config/README.md , and a sample config.yaml is provided in the same directory.   How to deploy the SAS Viya platform with an external OpenSearch The deployment instructions are provided in the same README file as the requirements.   To use an external OpenSearch instance, you need to gather the following parameters from the OpenSearch administrator: Username and Password of an account with administrative privileges for your external OpenSearch instance URL (and port) of your external OpenSearch instance NOTE: If the connection uses TLS and OpenSearch is using custom-signed certificates, you also need the server CA certificate so that you can import it into SAS Viya. This step is the same as with any other external CA certificate, whether it is used to protect a connection to OpenSearch, a database, an LDAP server, etc. You can follow the official instruction in the document Incorporate CA certificates into SAS Viya   After you have those connection details, follow these steps: Copy the provided external-opensearch-transformer.yaml , secret.yaml ,and  client-config-tls.yaml  files from: $deploy/sas-bases/examples/configure-elasticsearch/external/ to $deploy/site-config/external-opensearch/ . No need to change anything in its content. Adjust the username, password, and URL values in the latter two files as instructed by the in-line comments. Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.     Edit the base kustomization.yaml to reference these files: add external-opensearch-transformer.yaml to the transformers block transformers: ... - site-config/external-opensearch/external-opensearch-transformer.yaml add the secret.yaml and client-config-tls.yaml to the resources block resources: ... - site-config/external-opensearch/client-config-tls.yaml - site-config/external-opensearch/secret.yaml Start the SAS Viya platform deployment with your preferred method. And finally, a reminder: if you copied your base kustomization.yaml from an environment that uses an internal OpenSearch instance, be sure to remove any existing lines referencing the internal instance, otherwise you will get an unsuccessful deployment!   Conclusion   In this article we have described the benefits and considerations of the new option that you have when deploying the SAS Viya platform starting with the 2023.03 release, the support of an external instance of OpenSearch to provide searching and indexing capabilities. We have also listed the steps required to perform a deployment connecting to an external OpenSearch instance.   Find more articles from SAS Global Enablement and Learning here. ... View more