SAS Viya 3.5 highlights for the technical architects, installation engineers and administrators
As you probably know, a new version of SAS Viya was shipped in late November of 2019.
As a reminder, SAS Viya is on its fifth major release.
- Version 3.1 Fall 2016
- Version 3.2 April 2017 (announcement at SGF)
- Version 3.3 December 2017
- Version 3.4 July 2018 (18w30), 2019 May Version (19w21), 2019 August Version (19w34)
- Version 3.5 November 2019 (19w47)
It is worth noting that the successive SAS Viya 3.4 releases (May and August) also brought many changes. So users who have only been using the initial release of SAS Viya 3.4, will only discover them when starting to use SAS Viya 3.5.
19w47 is a major ship event as it brings new versions of many Viya products and solutions whether for the Data management (SAS Data Preparation 2.5), in the Models/Decisions Management area (SAS Model Manager 15.3, SAS Intelligent Decisioning 5.4) or in the analytic space (SAS Visual Analytics 8.5, SAS Visual Statistics 8.5, SAS Visual Data Mining and Machine Learning 8.5, SAS Visual Text Analytics 8.5, SAS Visual Forecasting 8.5, SAS Optimization 8.5, SAS Econometrics 8.5 and SAS IML 8.5). Each of the new versions or products brings its own whole set of exciting new features.
But this article will focus on Architecture and Deployment topics and has two purposes:
-
- Firstly, to list and highlight some of the key changes from an Architecture perspective in this new release and provide some pointers to additional resources for further information.
- Secondly to zoom in on some specific technical details for Viya 3.5 which will likely impact the work of the SAS Administrator/Installation Engineer.
However, it is far from being an exhaustive list of all the changes coming with Viya 3.5. If you want to have more information on Viya 3.5 new features, a lot of resources have been made available for you: See the General What's new page, and the products specific what's new information.
Support of Power-9 Architecture
SAS Viya 3.5 brings IBM POWER9 processor chip architecture support running Linux. This often referred to as the PowerLinux platform (PLX). SAS now supports Viya on Linux for Intel x64, AMD and Power 9 architectures.
SAS Viya can now take advantage of the new highly performant IBM POWER9 servers (including GPU utilization for Deep learning algorithms).
However, there are a some caveats at this point in time:
- Customers wishing to license Viya 3.5 for this platform will be required to discuss with the SAS sales team their specific needs to ensure they can be met, as not all Viya based offerings are currently available on the IBM POWER 9 platform as of January 20th, 2020
- The current supported offerings as of January 20th, 2020 are: VA / VS / VDMML / Model Studio / VTA / Data Prep / Data Agent / Econometrics / VF / Optimization / Intelligent Decisioning / Model Manager / CONNECT / SPRE - QC / SPRE - IML / Risk Modeling Nodes for Visual Data Mining / Access Support: PC Files, ODBC and Postgres
- Accessing certain data structures like Parquet are currently not supported
- SUSE is not supported for use with Viya 3.5 on Power 9 servers
In terms of deployment, the PLX Platform needs to be chosen for the order and the orchestration CLI must use the new flag "–architecture" to specify the Power Linux architecture. Finally, a specific Red Hat Linux operating system, taking advantage of the Power Linux processors, must be installed.
SAS Job Flow scheduler
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
SAS Viya 3.5 introduces the Job Flow Scheduler that you can use to create a chain of actions composed of: code (SAS or other languages), scripts (bash, shell, etc.), user interface actions (import files and tables with SAS Data Explorer), run data plans (SAS Data Studio). You can add logic gates (OR, AND), create time-based triggers or schedule a flow. See Bogdan Teleuca’s recent article for a detailed explanations about this new product. SAS Job Flow Scheduler is a licensed offering like other offerings from SAS.
Until Viya 3.5, it was only possible to create flows through the command line, but SAS Job Flow Scheduler at Viya 3.5, a graphical user interface makes this feature much easier to use.
Support for additional network configurations
A scenario where this was observed was when customer environments had host machines with 2 or multiple network interface cards (NICs). With Viya 3.5, SAS provides:
- A better way to handle more sophisticated network configurations like multiple IP Addresses, multiple Hostnames
- For example, when there is one network for internal communications and another for external access, we can force the Viya internal services to bind only on the internal network).
- A consistent way across all the Viya services to use a set of defined values for the IP addresses and hostnames in the internal, external communications and generated TLS certs.
- Instead of having the Viya Services relying on potentially inconsistent results from
hostname –forhostname –Aad-hoc commands (which may or may not report the wanted value), network configuration settings can be explicitly set for each machine of the deployment.
- Instead of having the Viya Services relying on potentially inconsistent results from
There are 3 categories of settings:
- IP addresses used with SAS Viya
- TLS Certificate Parameters (to affect TLS certificates “Subject Alternative Name” value)
- Binding variables for external ports (variables for Public facing services such as HTTPD, CASL, etc…)
Before the deployment you must define your network settings in one file per host machine in the deployment playbook “host_var” folder (refer to the documentation for detailed instructions).
This is very flexible, but as with all environments where there are multiple configurable components, the choice of the proper configuration option will require careful planning. We also recommend customers test their environments using their own use cases to ensure their configured Viya 3.5 environment behaves in the required manner. Read carefully the official documentation to fully understand what you are doing.
Reduced Memory footprint
Java Spring Microservices are used in Viya 3.5. In the development of Viya 3.5, consolidation of several services were completed to improve efficiency.This has helped reduce the memory footprint and for some services improved execution times.
Postgres 11 and HA Pgpool
A new Viya 3.5 deployment will install Postgres 11. A Postgres upgrade utility (from Postgres 9.4 to 11) for those customers upgrading from Viya 3.4 to Viya 3.5 is provided.
The utility called sas-pgupgrade-cli automatically scans a SAS Viya deployment for all running PostgreSQL databases and migrates them to the latest version of PostgreSQL that was installed by SAS on the physical machines.
Also, with the official support of Pgpool in HA mode, the availability of the Viya 3.5 platform is increased compared to previous versions of Viya.
As a reminder Pgpool is the load-balancer in front of the PostgreSQL cluster. You can expect to see additional articles from my colleagues talking about Pgpool in the context of (higher) availability during 2020.
Using SAS Studio? Then read on.
With Viya 3.5 there is only one version of SAS Studio (for Viya): 5.2. However it is important to note there are 2 ways which SAS Studio can leveraged.
The first is the "Basic Edition" approach (which relies on the object spawner and workspace server technology). The second is the "Enterprise Edition" (which works with the Viya Microservices and the Compute and Launcher servers).
Programming Only deployments come with the "Basic Edition" while a full-deployment of Viya gets the both versions (Basic and Enterprise).
It is recommended that in a full deployment of Viya 3.5, the end-users should only use the "Enterprise Edition". A good practice is to disable SAS Studio "Basic Edition" in a full Viya deployment.
This new SAS Studio 5.2 release brings many cool improvements such as Job definition authoring with prompting, local file system navigation and access, Query builder, Background submit, Scheduling, DATA Step Debugger, Git integration with UI. For more details, see this recent article by Edoardo Riva.
Hadoop 3 support
Current Hadoop distributions (such as Cloudera 6.x, Hortonworks HDP 3.x, or MapR 6.x) are now using the Apache Hadoop 3 base code and as the end-of-support (EOL) date draws nearer, customers running Hadoop 2 based environments, might start to look at a migration project to version 3.
Hortonworks Support policy (REVISED October 15, 2019)
Hadoop 3 brought many changes and started to be supported with the May release (19w21) of Viya 3.4.
There is, now a Hadoop section in the "Support for database" documentation and it’s great to see that both CAS Data Connectors (for serial and multi-node loading) and CAS Data Connect Accelerators (for parallel loading) as well as SAS Plug-ins for Hadoop and SAS Scoring Accelerator for Hadoop are now officially supported with Hadoop 3 based distribution.
However, there are 2 remaining caveats that I’ll remind here:
- The Embedded process have always used the HCatalog component to manage the hadoop structures.
As this component was dropped by Hortonworks in HDP 3.x, working with the SAS Embedded Process and Hive Managed tables in HDP 3.x requires LLAP (Hive Live Long and Process) to be turned “on” in the Hadoop cluster. It is critical that SAS administrators are aware of this when migrating to Hadoop 3. - The SAS plug-ins for Hadoop incudes a security fix using HDFS block tokens. It worked well with Hadoop 2.x distributions but with Hadoop 3 there was a breaking change in the HDFS block tokens API.
While it is in the process of being fixed, a deployment of the SAS Plug-ins in a secured "Hadoop 3 based" distribution (such as Cloudera 6.3.2 or Apache Hadoop 3.2.1) is likely to require HDFS block token to be disabled for CAS access.
CASLibs Data source enhancements
There are several great enhancements in terms of data integration for the CAS Server.
New ADLS CASLib
The ADLS caslib type supports loading and saving delimited files (CSV files) and Optimized Row Columnar (ORC) files.
Multiple HDFS cluster support
CAS can now assign CASLibs to multiple remote HDFS clusters on a "caslib-by-caslib" basis (see the documentation).
It requires SAS plug-ins for Hadoop to be deployed on each remote HDFS cluster but it means that the same CAS instance can use HDFS caslibs with both co-located and remote HDFS cluster.
New file types support
As a nice picture is usually a better way to explain than a long text, I'll reproduce here a table created by my colleague Nicolas Robert that gives a good summary of the changes coming with Viya 3.5.
For more details see Nicolas Robert's article.
The orange triangle signals what's new with SAS Viya 3.5.
Security
The main changes in Viya 3.5 regarding the authentication are:
- Support of Kerberos constrained delegation (allows to support environments with Windows Defender
Credential Guard enabled) - Support of service accounts with the Compute Server (equivalent of the default sassrv account in SAS 9);
full support of Kerberos in Multi-tenant environments.
Zoom in on the deployment experience
A new Viya ARK version with new behaviors
SAS Viya ARK provides automated tools and utilities to help SAS customers prepare for a SAS Viya deployment.
But the most useful that every SAS Viya Installation engineer/administrator should know are the Pre-installation of SAS Viya System Requirements and the SAS Viya Multi-Machine Services Utilities – aka "MMSU" (recommended way to stop and start Viya services in Multi-Machines deployments).
While it is not supported directly through the SAS Technical Support, SAS Viya ARK is a GitHub project and the GitHub model can be used for tracking bugs and feature requests through GitHub issue or pull request for support.
There is a new Viya ARK release for Viya 3.5 with several changes.
Several issues have been fixed but there are also changes in the pre-installation playbooks behavior and consultants well accustomed to the 3.4 version could be taken off guards.
- SELinux Configuration
With Viya 3.4-ARK SELinux was automatically set to permissive to allow the deployment
With Viya 3.5-ARK, if the mode is set to "enforcing", the pre-install playbook will fail and a message directing the user to take the necessary actions will be given.
The rational behind this change is that the requirement to disable SELinux was "relaxed" (since the Viya 3.4 August 2019 version). There are ways to cope with an environment which cannot disable SELinux or even turn it off during the installation, and they are documented in the deployment guide. So if Viya ARK detect enforced SELinux, then instead of disabling it, it will stop and print a message for the user to look at the documentation on how to configure SELinux to allow the deployment. - Supported Java Runtime Environment (JRE) Versions
A minimum version of Java is now required (1.8.0_121 at the time of the writing) .
If there is no Java installed, Viya ARK will install it.
BUT if Java is already installed in previous version, the check mode fails but the normal mode does not update the Java package (a debug message will show the version found to be installed).
So, if Java is already installed in the target environment check mode is highly recommended.
Don't rush the validation !
With Viya 3.4, generally as soon as the deployment was completed, it was possible to logon and start the validation process.
However, in some of our Viya 3.5 deployment tests we noticed that the environment was not always immediately ready just after a successful deployment (failure=0 reported by each host of the deployment in the site.yml playbook run).
When connecting to the Visual interfaces too quickly after the deployment, a user could see missing GUI elements or dialog like:
In such case, just be patient and give it another try a few minutes later. What happens is that the jobs that populate Environment Manager view behind the scenes, run on a schedule, and those jobs must complete before a user could see anything.
Of course this behavior was observed in our own virtualized environment and might not occur with a more performant infrastructure (in terms of CPU and RAM).
Merged services
While merging microservices had a positive impact in reducing the overall memory footprint needed for a Viya deployment, the change has broad impact for upgrades and for administrators who are familiar with the existing services.
As explained in the usage note in the Administration guide, some services have been merged into logical groups, in Environment manager, there is now a new concept of “Host service”. If a service has been merged with other services, there is a way to see it in Environment Manager (follow the official documentation instructions).
But when starting or stopping a “merged” service, you need to use the “Host service” name.
For example, the CAS Proxy service and the CAS Management service have been combined into a CAS Administration service. If the CAS Proxy service is down, then the CAS Administration service must be restarted.
Another side effect is for upgrade situations, because the log files of the old services are not cleaned up (discussions are underway to determine the possibility of developing a playbook in Viya ARK to do this post-upgrade clean up).
Conclusion
While SAS Viya 3.5 has just been released with a lot of improvements and new features, both functional and from a technical architecture and deployment perspective.
Thanks for reading!
