BookmarkSubscribeRSS Feed

Get ready! SAS Viya 3.4 highlights for the Technical Architect

Started ‎01-21-2019 by
Modified ‎01-22-2019 by
Views 3,643

The SAS Viya 3.4 platform was released in the middle of last year (24th of July 2018). Less than 2 month after (18w38), the support of Microsoft Windows for Viya was made available. However many customers still have a Viya 3.3 platform in production or are still assessing the deployment of Viya (either in addition or as a transition target for their existing SAS 9 platform).

 

So now is a good time to come back on the key new architecture and deployment changes that came with the SAS Viya 3.4 release.

 

New supported OS (Windows, SUSE)

 

With SAS Viya 3.4, we introduce the support of two additional Operating System: SUSE Linux and Windows.

 

However, today the Windows flavor of SAS Viya is only be available for single machine deployments. All the SAS Viya components (Microservices, Web Application, Infrastructure Servers and CAS) need to be deployed on the same host (which obviously implies that only CAS SMP is supported in Windows deployments).

 

In terms of deployment (binaries installation and configuration), instead of Ansible and YUM packaging, Command/PowerShell scripts and MSI packages is used. A dedicated deployment guide of SAS Viya for Windows is available in the SAS documentation.

 

Regarding SUSE Linux, we won't have the same "Single Machine" limitation, but the deployment technology is different. We'll continue to use Ansible to orchestrate the deployment tasks, but it will interact with a different package manager called "Zypper" (instead of YUM) in order to lay down the Software binaries on the target machines.

 

In addition, for SUSE Linux deployments, it is mandatory to create and use a mirror repository for the software packages (instructions on how to use a mirror repository are provided in the official deployment Guide). There are also a few additional system requirements, as the default shell (bash) for the target machine(s) as SUSE and RHEL have sometimes slightly different ways to configure the Linux system or simply use different tools.

 

Finally, a noticeable change in the deployment process (introduced by the support of the new OS) is that the orchestration CLI is used to generate different "deployment artifact" depending on the target Operating System (ansible playbook or Powershell script).

 

Mirror Manager

 

Using a local mirror repository for your SAS Viya deployment is a general best practice for many reasons (remove the internet connection requirement for the target servers, avoid the maximum download trap, keep a static version of the SW packages for consistency across environments or tenants deployments, etc). In addition, in case of a deployment with SUSE Linux, it is mandatory.

 

However, the process to build and use a mirror for the deployment was not so simple with Viya 3.3. The efforts to improve this process have resulted in a completely new process and a new tool called "SAS Mirror Manager."

 

1mirrormgr.png

 

It is a "command line" tool that you can run anywhere (on any Linux, or on your local PC with Macintosh or Windows) which allows you to create AND refresh a local mirror repository from the SAS Hosted Repository.

 

By default, the mirror content will be placed in the "sas_repos" folder of the user's home directory; however, you can point to any directory available from the machine running the tool (such as a web server directory or a shared NFS mount).

 

Depending on your target Operating System, the tool downloads the appropriate native's packages (RPM files for Linux RedHat and SUSE, MSI files for Windows, APT files for Debian-based Linux, etc).

 

The key point is to ensure that the default location or the location that you select has enough free space because in the current version of the tool, you will not be able to know in advance how much storage space will be used.

 

From our experience, around 15 GB would be required for a very simple order (with VA for example), whereas a full stack order repository takes around 60 GB of Disk space.

 

Once downloaded, it is possible to move the files either in a web server directory (for example: /var/www/html/pulp/) or a shared NFS mount (for example: using secure copy or "rsync" commands).

 

Once the mirror repository has been made available and can be reached from all your target servers, all you will have to do before running your deployment playbook is to provide the mirror location in the vars.yml file, for example:

 

REPOSITORY_WAREHOUSE: "http://sascas01.race.sas.com:8123/09XXXX/"

 

Finally, it is worth noting that with SAS Viya 3.4, the Orchestration CLI can also point to a pre-built mirror repository to generate the playbook from the SAS Deployment Data file (attached in the Software Order Confirmation email).

 

/opt/sas/viya/home/bin/sas-orchestration build --platform redhat --input ./SAS_Viya_deployment_data.zip --repository-warehouse https://gelweb.race.sas.com/mirror/$ORDERNUM/sas_repos --deployment-type programming --output SAS_Viya_playbook_prog.tgz

 

New Host Groups, SAS Studio 5.1

 

As new components have been added with SAS Viya 3.4, new "Host Groups" appear in the deployment playbook inventory.

 

As an example, for the simplest possible order (in terms of product): Visual Analytics, there are 4 new Host Groups.

 

2newHG.png

  • The "CognitiveComputingServices" Hostgroup was available with specific products such as Model Manager or VDMML, it is now part of the minimum set of SAS Viya Microservices to perform common text analytics tasks.
  • The "ComputeServer" Host Group is also now systematically provided, not for Visual Analytics (which is exclusively using the CAS engine) but to support the new "SAS Studio Viya" which needs to run SAS Code in SPRE on behalf of the Compute Service.
  • The "GraphBuilderServices" host group contains services that provide tools to create and edit custom graphs.
  • The "StudioViya" Host Group contains the services providing the new SAS Studio 5.1.

SAS Studio 5.1 (aka  "SAS Studio Viya") is the new SAS Programming tool generation embracing the SAS Viya architecture paradigm (Microservice based, using REST APIs instead of IOM protocol). It is available by default in every "full" deployment of SAS Viya.

 

Host groups are very important for the SAS Technical Architect as he needs to decide on which host(s) they will go and whether they are subject to specific placement rules, support multiple instances distributed across machines, fail-over, etc.

 

Tip: Very useful information is often provided directly in the Host Groups comment of your inventory.ini default file.

 

Multiple CAS Servers

 

3multipleCAS.png

 

In SAS Viya 3.3, the only way to have multiple CAS Servers within the same SAS Viya deployment was to have restrictive requirements on the LDAP structure and to deploy the SW in a Multi-tenancy mode.

 

In the 3.4 release, it becomes possible to have multiple CAS servers, even with a standard single-tenant deployment.

 

In terms of deployment process, you always start with a single CAS Server (SMP or MPP), "cas-shared-default", but then you can add new empty machines where you can install additional CAS Server (SMP or MPP).

 

CAS resource management

4cakeparts.png

 

In the new SAS Viya version, new options are added for a better repartition of the CPU resources used by the CAS processing. In addition, it is now possible with the priority-levels policies to define disk quotas for the loaded tables and allocate a share of the CPU capacity to different groups of users (up to 5 groups).

 

Those options leverage cgroups which is a Linux Kernel feature, hence they are only available in the Linux version of CAS. As a consequence having the cgroups feature installed on the CAS machines is a new requirement if we want to use those new capabilities.

 

The resources used by CAS can be managed:

  • Either broadly with cas.MEMORYSIZE (limit the memory usage on the CAS node to a specific amount of RAM) and cas.CPUSHARES (limit the CPU usage on the CAS node to a share of the overall CPU capacity) server configuration options.
  • or more specifically through policies associated to group of users.

There are 2 types of policies:

  • Global caslibs: one policy per CAS Server that places a disk cache space quota on global caslibs.
  • Priority-level: one to five policies per CAS Server that place disk cache space quota on personal caslibs and a CPU consumption limit on CAS sessions that can be mapped to group of users.

How does it work?

At startup time, the CAS server reads the policies defined in Consul and dynamically create the cgroups to limit resources usage (which means that the resource management policies are only working in full deployment of SAS Viya - as the SAS Configuration server is needed to store their values).

 

When the CAS server is stopped then the cgroups are removed.

 

How to configure it?

The administrator can assign users to these "priority level policies" based on their identity group memberships, or can explicitly assign priority level policies to individual users.

 

The way to do it is described in details in the SAS Viya 3.4 administration guide but basically the SAS Administrator has to define policies in the Consul store through the SAS Viya command line interface (sas-cli) and create corresponding Custom groups in Environment Manager to associate the end-users to the resource management groups.

 

Typically, you can define 1 to 5 "priority" groups and allocate quotas for global and session tables but also a share of the CPU capacity per priority group.

 

For example, you could create 3 priority groups: one for the VA users, one for the ETL users and one for the Data Scientists - with a CPU repartition of 20, 30, and 50% and define different quotas to limit the space used for global, session, or HDFS loaded tables for each group.

 

Why?

It is a very interesting enhancement as you can limit the amount of space used in the CASlibs for your end users, but you can also avoid that a "crazy" data scientist, running a single but extremely heavy set of Machine-learning/advanced analytics actions takes all of the CPU power of your CAS cluster at the expense of the other end-user (maybe consuming simple VA reports or running Data processing jobs).

 

Overall, those options provide an even better control of the CAS server utilization and allow to divide the CAS resources between multiple groups of users of the same CAS environment.

 

Finally, there are other very interesting new resource management options such as cas.MAXCORES or env.CAS_ADDITIONAL_YARN_OPTIONS (for a better interaction with YARN).

 

 

Multi Tenancy improvements (different LDAP for each tenant)

 

5laundry.png

SAS Viya 3.4 makes the Multi-tenancy model much more appealing because the very unique requirement on the LDAP structure is removed (1 Organization Unit (OU) and 1 Organization Unit called "provider).

 

With the new SAS Viya release, each tenant can have its own LDAP configuration and connection. Each tenant can have its users and groups in his own LDAP Directory or they can share the same one - but it is possible to define a specific LDAP connection query for each tenant.

Another new feature of the Multi-tenancy is that you can choose, for the SAS Data Infrastructure Server, to use either different PostgreSQL schema or different PostgreSQL database for each tenant's user content.

 

Finally, with the capability to limit CPU consumption par CAS instances (cas.CPUSHARE), it is also easier to manage concurrent tenant's CAS servers sharing the same physical machine.

 

Conclusion

 

There are many other things with SAS Viya 3.4, such as new data integration capabilities (AWS S3, Spark, EMR support), improvement of the Compute and launcher servers (Kerberos support and launcher shared account).

But this new release of SAS Viya brings great benefits :

  • New features required by the business and the data science community
  • Response to the field feedback and experience: answers to customers requests and fix for identified shortages in the features or additional Enterprise-ready capabilities.

Thanks for reading!

Comments

Hello @RPoumarede,

 

thank you for your summary! A lot of new things indeed! 

 

As I know it so far, the Windows flavour of SAS Viya is having some challenges, specialy because Keberos is "mandatory" for its configuration. From your point of view, what kind of things should be expected of a SAS Viya installation on Windows? PROs? CONs?

 

Thank you in advance!


Best regards,

Juan

Hi @JuanS_OCS

 

Thanks for your comment.

Yes it is correct, for Viya on Windows, Kerberos is the only supported authentication mechanism for the Visual interfaces (although it is not mandatory if you deploy in "programming only" mode).

 

Today you have more capabilities and architecture options when you deploy Viya on Linux than on Windows - see this other blog : https://communities.sas.com/t5/SAS-Communities-Library/Getting-ready-to-install-SAS-Viya-3-4-on-Wind... for architecture and installation considerations.

 

However, there are customers who really prefer (for various reasons) to run their infrastructure on Windows. So it is more a PRO/CONS discussion between running enterprise applications in Windows or Linux environments.

My understanding is that SAS plans are to improve the Windows support (more products) and reduce the gap between Linux and Windows (multi-machine deployment, ...), but I suppose that it also depends on the customer feedback and appetite to use Windows platforms to run their High Performance Analytics.

 

Hope that helps

Thanks

Raphael

Hi Raphael @RPoumarede ,

 

is there any roadmap available regarding the products that will be made available and when? Specially, regarding Windows.

 

My other thought is that the problem is on Kerberos itself. meaning, if you implement SSO Kerberized authentication, on Linux, the problems are still the same. Right? And the SAS Mobile app does not support Kerberos authentication.

 

This brings me to share another question: is there any plan to support the SAS mobile apps, when Kerberos SSO authentication is set up? And, if so, when?

 

Thank you in advance,

Kind regards,

Juan

Hi Juan

 

As far as I know our roadmaps are not publicly available (although specific furture plans can be mentionned here and there in SGF papers or SAS user groups presentations).

If you have a customer who needs a specific feature or want to have the current roadmap for a specific product the best for him is surely to get in touch with his SAS representative.

Regarding Windows support, as said before I think one of the main goal is to be able to split Viya components (except CAS) across multiple machines (but it is not an official statement and still needs to be confirmed).

Good question on Kerberos Authentication with SAS Mobile apps. Regarding the associated roadmap, but again it is something your customer will have to check with his SAS representative.

Sorry for not being more precise on your questions, but as soon as public information will be available, I'm sure it will be communicated in communities and happy to answer any question on the current version of Viya 3.4 🙂

Thanks

Raphael

Also for anyone who wants to know more about SAS Visual Analytics App (new name for SAS BI mobile) with SAS Viya configured for Kerberos authentication, please see this page : https://communities.sas.com/t5/SAS-Communities-Library/SAS-Viya-3-4-Kerberos-with-CAS/tac-p/527372#M...

Thanks

 

Raphael

 

Hi @RPoumarede

Thanks for your sharing. In the section  "Multi Tenancy improvements (different LDAP for each tenant)". Does that mean we can let the users in different forest to log in Viya platform after appropriate setting? If so, are there any documents can show us how to config it step by step? In our situation, we are considering to upgrade our Viya 3.2 to 3.4 and we would like to allow our users in different AD forest (which means two different AD Server) to access the Viya. Is that possible to config it in the post-deployment if we can use multi-tenancy configuration in Viya 3.4?

Hi ROY30426

 

1- Yes users can be in different forest (authentication provider server: AD or LDAP) but they will be able to logon only against specific tenant(s). If a user has to be able to access multiple tenants, the Identities service configuration will be more challenging (and requires users and groups filtering at tenant level).

This is only possible with multi-tenancy deployment and from Viya 3.4.

See this section for the case with separate LDAP Servers per Tenant.

https://go.documentation.sas.com/?docsetId=dplyml0phy0lax&docsetTarget=n15hhewllr5ji2n1sxf96imqvtpj....

Please note that the multi-tenancy configuration is documented partly in the Viya deployment guide and partly in the Viya administration guide. 

 

2- If the deployment is a multi-tenancy deployment, yes this can be configured later. In the initial deployment, the Identities service can use a single authentication provider for all tenant. Then the Identities service can be reconfigure to use different authentication provider per tenant.(However it is impossible to move from a single tenant deployment to a multi-tenancy deployment without having to redeploy Viya.)

 

Finally, keep in mind that implementing Multi-tenancy is quite complex and has many implications - it answers to specific use cases, so please make sure you understand them well and ensure they correspond to your customer needs. If your goal is only to address the constraint of having Viya users in different LDAPs you might want to explore alternative scenarios such as implementing a LDAP proxy and modify your identities service configuration to use it - or separate the Viya environments (if it is possible in your context).

In any case if you plan to implement Multi-tenancy, I would recommend you to involve your local SAS representative and see how SAS Professional Services could help you.

 

Hope that helps.

Thanks

Raphael

Version history
Last update:
‎01-22-2019 02:38 PM
Updated by:

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

Free course: Data Literacy Essentials

Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning  and boost your career prospects.

Get Started

Article Tags