As you probably know, a new version of SAS Viya was shipped in late November of 2019.
As a reminder, SAS Viya is on its fifth major release.
It is worth noting that the successive SAS Viya 3.4 releases (May and August) also brought many changes. So users who have only been using the initial release of SAS Viya 3.4, will only discover them when starting to use SAS Viya 3.5.
19w47 is a major ship event as it brings new versions of many Viya products and solutions whether for the Data management (SAS Data Preparation 2.5), in the Models/Decisions Management area (SAS Model Manager 15.3, SAS Intelligent Decisioning 5.4) or in the analytic space (SAS Visual Analytics 8.5, SAS Visual Statistics 8.5, SAS Visual Data Mining and Machine Learning 8.5, SAS Visual Text Analytics 8.5, SAS Visual Forecasting 8.5, SAS Optimization 8.5, SAS Econometrics 8.5 and SAS IML 8.5). Each of the new versions or products brings its own whole set of exciting new features.
But this article will focus on Architecture and Deployment topics and has two purposes:
However, it is far from being an exhaustive list of all the changes coming with Viya 3.5. If you want to have more information on Viya 3.5 new features, a lot of resources have been made available for you: See the General What's new page, and the products specific what's new information.
SAS Viya 3.5 brings IBM POWER9 processor chip architecture support running Linux. This often referred to as the PowerLinux platform (PLX). SAS now supports Viya on Linux for Intel x64, AMD and Power 9 architectures.
SAS Viya can now take advantage of the new highly performant IBM POWER9 servers (including GPU utilization for Deep learning algorithms).
However, there are a some caveats at this point in time:
In terms of deployment, the PLX Platform needs to be chosen for the order and the orchestration CLI must use the new flag "–architecture" to specify the Power Linux architecture. Finally, a specific Red Hat Linux operating system, taking advantage of the Power Linux processors, must be installed.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
SAS Viya 3.5 introduces the Job Flow Scheduler that you can use to create a chain of actions composed of: code (SAS or other languages), scripts (bash, shell, etc.), user interface actions (import files and tables with SAS Data Explorer), run data plans (SAS Data Studio). You can add logic gates (OR, AND), create time-based triggers or schedule a flow. See Bogdan Teleuca’s recent article for a detailed explanations about this new product. SAS Job Flow Scheduler is a licensed offering like other offerings from SAS.
Until Viya 3.5, it was only possible to create flows through the command line, but SAS Job Flow Scheduler at Viya 3.5, a graphical user interface makes this feature much easier to use.
In previous releases of Viya, there have been some deployment challenges with Viya in scenarios where there are more complex or specific network configurations (hosts with multiple IP address/hostnames, IP address in Viya services auto-generated certificates, need to override default hostname, etc...).
A scenario where this was observed was when customer environments had host machines with 2 or multiple network interface cards (NICs). With Viya 3.5, SAS provides:
hostname –Aad-hoc commands (which may or may not report the wanted value), network configuration settings can be explicitly set for each machine of the deployment.
There are 3 categories of settings:
Before the deployment you must define your network settings in one file per host machine in the deployment playbook “host_var” folder (refer to the documentation for detailed instructions).
This is very flexible, but as with all environments where there are multiple configurable components, the choice of the proper configuration option will require careful planning. We also recommend customers test their environments using their own use cases to ensure their configured Viya 3.5 environment behaves in the required manner. Read carefully the official documentation to fully understand what you are doing.
Java Spring Microservices are used in Viya 3.5. In the development of Viya 3.5, consolidation of several services were completed to improve efficiency.This has helped reduce the memory footprint and for some services improved execution times.
The Viya 3.4 version of the SAS Infrastructure Data Server is based on PostgreSQL version 9.4 that is reaching End-of-Life on February 13, 2020. By itself it could already be a good reason to consider an upgrade to Viya 3.5 that is shipping with Postgres 11.
A new Viya 3.5 deployment will install Postgres 11. A Postgres upgrade utility (from Postgres 9.4 to 11) for those customers upgrading from Viya 3.4 to Viya 3.5 is provided.
The utility called sas-pgupgrade-cli automatically scans a SAS Viya deployment for all running PostgreSQL databases and migrates them to the latest version of PostgreSQL that was installed by SAS on the physical machines.
Also, with the official support of Pgpool in HA mode, the availability of the Viya 3.5 platform is increased compared to previous versions of Viya.
As a reminder Pgpool is the load-balancer in front of the PostgreSQL cluster. You can expect to see additional articles from my colleagues talking about Pgpool in the context of (higher) availability during 2020.
With Viya 3.5 there is only one version of SAS Studio (for Viya): 5.2. However it is important to note there are 2 ways which SAS Studio can leveraged.
The first is the "Basic Edition" approach (which relies on the object spawner and workspace server technology). The second is the "Enterprise Edition" (which works with the Viya Microservices and the Compute and Launcher servers).
Programming Only deployments come with the "Basic Edition" while a full-deployment of Viya gets the both versions (Basic and Enterprise).
It is recommended that in a full deployment of Viya 3.5, the end-users should only use the "Enterprise Edition". A good practice is to disable SAS Studio "Basic Edition" in a full Viya deployment.
This new SAS Studio 5.2 release brings many cool improvements such as Job definition authoring with prompting, local file system navigation and access, Query builder, Background submit, Scheduling, DATA Step Debugger, Git integration with UI. For more details, see this recent article by Edoardo Riva.
Current Hadoop distributions (such as Cloudera 6.x, Hortonworks HDP 3.x, or MapR 6.x) are now using the Apache Hadoop 3 base code and as the end-of-support (EOL) date draws nearer, customers running Hadoop 2 based environments, might start to look at a migration project to version 3.
Hortonworks Support policy (REVISED October 15, 2019)
Hadoop 3 brought many changes and started to be supported with the May release (19w21) of Viya 3.4.
There is, now a Hadoop section in the "Support for database" documentation and it’s great to see that both CAS Data Connectors (for serial and multi-node loading) and CAS Data Connect Accelerators (for parallel loading) as well as SAS Plug-ins for Hadoop and SAS Scoring Accelerator for Hadoop are now officially supported with Hadoop 3 based distribution.
However, there are 2 remaining caveats that I’ll remind here:
There are several great enhancements in terms of data integration for the CAS Server.
With SAS Viya 3.5, Azure Data Lake Storage is a new data source: CAS supports adding a caslib to Azure Data Lake Storage Gen2 for data access.
The ADLS caslib type supports loading and saving delimited files (CSV files) and Optimized Row Columnar (ORC) files.
CAS can now assign CASLibs to multiple remote HDFS clusters on a "caslib-by-caslib" basis (see the documentation).
It requires SAS plug-ins for Hadoop to be deployed on each remote HDFS cluster but it means that the same CAS instance can use HDFS caslibs with both co-located and remote HDFS cluster.
As a nice picture is usually a better way to explain than a long text, I'll reproduce here a table created by my colleague Nicolas Robert that gives a good summary of the changes coming with Viya 3.5.
For more details see Nicolas Robert's article.
The orange triangle signals what's new with SAS Viya 3.5.
The main changes in Viya 3.5 regarding the authentication are:
Aside from the major change which is the network settings, there were a few changes that I found in my experience, to be worthy of a mention in this article Let’s share!
SAS Viya ARK provides automated tools and utilities to help SAS customers prepare for a SAS Viya deployment.
But the most useful that every SAS Viya Installation engineer/administrator should know are the Pre-installation of SAS Viya System Requirements and the SAS Viya Multi-Machine Services Utilities – aka "MMSU" (recommended way to stop and start Viya services in Multi-Machines deployments).
While it is not supported directly through the SAS Technical Support, SAS Viya ARK is a GitHub project and the GitHub model can be used for tracking bugs and feature requests through GitHub issue or pull request for support.
There is a new Viya ARK release for Viya 3.5 with several changes.
Several issues have been fixed but there are also changes in the pre-installation playbooks behavior and consultants well accustomed to the 3.4 version could be taken off guards.
With Viya 3.5-ARK, if the mode is set to "enforcing", the pre-install playbook will fail and a message directing the user to take the necessary actions will be given.
The rational behind this change is that the requirement to disable SELinux was "relaxed" (since the Viya 3.4 August 2019 version). There are ways to cope with an environment which cannot disable SELinux or even turn it off during the installation, and they are documented in the deployment guide. So if Viya ARK detect enforced SELinux, then instead of disabling it, it will stop and print a message for the user to look at the documentation on how to configure SELinux to allow the deployment.
If there is no Java installed, Viya ARK will install it.
BUT if Java is already installed in previous version, the check mode fails but the normal mode does not update the Java package (a debug message will show the version found to be installed).
So, if Java is already installed in the target environment check mode is highly recommended.
With Viya 3.4, generally as soon as the deployment was completed, it was possible to logon and start the validation process.
However, in some of our Viya 3.5 deployment tests we noticed that the environment was not always immediately ready just after a successful deployment (failure=0 reported by each host of the deployment in the site.yml playbook run).
When connecting to the Visual interfaces too quickly after the deployment, a user could see missing GUI elements or dialog like:
In such case, just be patient and give it another try a few minutes later. What happens is that the jobs that populate Environment Manager view behind the scenes, run on a schedule, and those jobs must complete before a user could see anything.
Of course this behavior was observed in our own virtualized environment and might not occur with a more performant infrastructure (in terms of CPU and RAM).
While merging microservices had a positive impact in reducing the overall memory footprint needed for a Viya deployment, the change has broad impact for upgrades and for administrators who are familiar with the existing services.
As explained in the usage note in the Administration guide, some services have been merged into logical groups, in Environment manager, there is now a new concept of “Host service”. If a service has been merged with other services, there is a way to see it in Environment Manager (follow the official documentation instructions).
But when starting or stopping a “merged” service, you need to use the “Host service” name.
For example, the CAS Proxy service and the CAS Management service have been combined into a CAS Administration service. If the CAS Proxy service is down, then the CAS Administration service must be restarted.
Another side effect is for upgrade situations, because the log files of the old services are not cleaned up (discussions are underway to determine the possibility of developing a playbook in Viya ARK to do this post-upgrade clean up).
While SAS Viya 3.5 has just been released with a lot of improvements and new features, both functional and from a technical architecture and deployment perspective.
Thanks for reading!