SAS Viya 3.4 Windows with Kerberos

2 Likes

In this post I would like to discuss the changes that the Windows release of SAS Viya 3.4 brings, with respect to Kerberos authentication. Remember Kerberos is the only supported authentication mechanism for SAS Viya 3.4 on Windows. We will review the impact to the major components; SAS Logon Manager, SAS Cloud Analytic Services, and SAS Compute Server.

The late 2018 ship event brings Windows support for SAS Viya 3.4. SAS Viya 3.4 on Windows is only supported on a single host. The services layer cannot be split apart from SAS Cloud Analytic Services, everything must run on a single Windows host. This single windows host must be a member of an Active Directory domain. For all three components only unconstrained delegation is supported.

SAS Logon Manager

Since SAS Logon Manager is a Java web application it is not directly impacted by the deployment on Windows. But, as all access to the visual applications is via SAS Logon Manager, configuring correctly for Kerberos authentication is critical. The prerequisites SAS Logon Manager with Kerberos are the same for either Windows or Linux, as we discussed before.

The main difference for Windows is how you specify the location of the Kerberos keytab. In SAS Environment Manager you enter a URI to point to the file. For Linux this was entered as:

file:///opt/sas/http.keytab

While for Windows we need to enter:

file:///d:/sas/HTTP.keytab

This points to a file d:\sas\HTTP.keytab, this file can be placed anywhere on the filesystem so long as the local service account has access to it. The majority of SAS services on the Windows host run as the local service account, more details on this account can be found here. Notice the slashes are the other way around and the colon for the drive letter is still required. Also, any spaces in the path name would need to be URI encoded as %20.

The other major change for SAS Logon Manager with this release is the ability to initialize Kerberos credentials. SAS Logon Manager, for either Windows or Linux, is now able to take a username & password and initialize the Kerberos credentials. For this to take place SAS Logon Manager must be configured for Kerberos. So, this will not impact end-users accessing SAS Logon Manager via a browser, they will still need to use Kerberos to authenticate to SAS Logon Manager. This impact the users of the command-line interfaces and the mobile applications. These users will now have a Kerberos credential stored with the Credentials microservice that can be used for out-bound connections to SAS Cloud Analytic Services or SAS Launcher Server.

If the browser is not configured for delegation the end-users will see a warning that parts of the environment will not function as expected, as shown here:

SAS Cloud Analytic Services

SAS Cloud Analytic Services on Windows changes more significantly than SAS Logon Manager. SAS Cloud Analytic Services now requires a domain service account. This domain service account will be used to run the SAS Cloud Analytic Services controller and will have the Service Principal Name registered against it. The format of the SPN is the same as for Linux and should be sascas/<fully qualified hostname>.

However, on Windows since the CAS Controller is running as the domain service account a Kerberos keytab is not required. Before running the deployment process, the credentials for the service account are encrypted using the following script:

<Orchestration Tool>/powershell-deployment/encryptCasUser.bat

This makes the domain service account credentials available to the deployment process.

With the introduction of Windows support the authentication processing for SAS Cloud Analytic Services changes slightly from what we reviewed before. The diagram below now covers both Windows and Linux.

Click image to see a larger version

Note: While the diagram shows the typical/expected case, that the session runs as the end-user when a stored username & password is returned, this could be any account a credential has been stored for.

The first change is very early in the flow; SAS Cloud Analytic Services assumes both that Kerberos is configured and the end-user is in CASHostAccountRequired for Windows deployments. Although, the actual custom group CASHostAccountRequired does not exist in a Windows deployment. This means that every SAS Cloud Analytic Services session will attempt to run as the end-user and that a Kerberos connection will always be attempted.

The other change in the flow is if the Kerberos connection fails and there are no stored credentials for the end-user. With the updates to SAS Logon Manager, being able to initialize a Kerberos credential, this should be an edge case. For example, where a CLI is used and the internal OAuth token is still valid, but the Kerberos credentials have expired. In this case SAS Cloud Analytic Services will examine if the end-user is able to assume the CAS Superuser role. If they are able to assume the role, then a CAS session will still be launched but running as the domain service account. If the end-user is not able to assume the CAS Superuser role then the session launch will fail.

Remember that SAS Cloud Analytic Services with SAS Viya 3.4 can use a stored username & password. So if the end-users expect to be able to run a session, when their Kerberos credentials are not valid they can store a username & password. This credential set, username & password, can be stored individually for each user or stored against a group within SAS Environment Manager. The credential set is stored in an Authentication Domain, the default name of the authentication domain is DefaultAuth. The name that SAS Cloud Analytic Services looks for can be changed, but the CAS controller will need to be restarted to pick up the change.

SAS Compute Server

The SAS Launcher Server on Windows also handles Kerberos differently to Linux. The SAS Launcher Server runs as the local system account on Windows. This means that the SAS Launcher Server is able to automatically register the Service Principal Name against the computer object during startup. This means that the only prerequisite for the SAS Launcher Server on Windows is that the Windows host is "trusted for delegation". Everything else will be handled automatically.

As with the authentication flow for SAS Cloud Analytic Services there is a slight change for SAS Launcher/Compute from before. The diagram below covers both Windows and Linux hosts.

Click image to see a larger version

Note: While the diagram shows the typical/expected case, that the session runs as the end-user when a stored username & password is returned, this could be any account a credential has been stored for.

Similarly, to SAS Cloud Analytic Services the SAS Launcher microservice on Windows will always assume that Kerberos will be used. So, in the early part of the flow diagram we loop around the check for whether Kerberos is configured for Windows hosts. This is the only change for launching the SAS Computer Server session on Windows.

Also, just like SAS Cloud Analytic Services the SAS Launcher microservice is able to fetch a username & password from an authentication domain. The same authentication domain name is used by the SAS Launcher microservice as is used by CAS. The default name being DefaultAuth. A potentially common scenario, where this stored username & password would be used, is for any scheduled or long-running offline jobs where the user is not presently using the system.

Summary

With SAS Viya 3.4 on Windows Kerberos authentication is the only supported mechanism. Unlike Linux deployments, the default setup will result in SAS Cloud Analytic Services sessions running as the end-user. The requirement of Kerberos places much more importance on the correct fulfillment of the prerequisites. To assist in completing the prerequisites SAS provides a PowerShell scripting tool that will both check the prerequisites and if given sufficient permissions complete those prerequisites as well. This tool is the SAS Viya Deployment Assistant for Windows and more details can be found in the official documentation.

bheinsius · ‎11-16-2019

Hi Stuart,

I successfully setup Kerberos for SAS Logon Manager but cannot get Kerberos for SAS Cloud Analytic Services to work.

From what I understand from your article and from what I read in the documentation there are no additional steps to

Configure Kerberos for SAS Cloud Analytic Services or to Configure Kerberos for SAS Launcher Server as it mentions for both:

Note: The configuration setting changes are not required in a Windows deployment.

So after following instructions to Configure Kerberos for SAS Logon Manager, Kerberos for SASDrive works but when navigating to SASStudioV I get this:

Unable to create compute server session.
Failed to launch process.
Failed to launch process: host=myserver.example.org port=33618 ssl=false
Failed to connect to server.
Kerberos handshake error.

The SAS Viya Deployment Assistant for Windows says all is OK:

PS C:\sas\install\powershell-deployment> .\sas-wvda.ps1 -validate all -keytabpath C:\sas\install\kerberos\http.keytab

INFO: sas-wvda version 1.1.08

INFO: Executing on host: myserver
INFO: JAVA_HOME (C:\Program Files\Zulu\zulu-8-jre) points to an installation of Java 8: OK
INFO: 64-bit version of Java 8 found in JAVA_HOME: OK
INFO: Running in a 64-bit environment: OK
INFO: Running PowerShell 5.1 or higher: OK
INFO: Running on Windows Server: OK
INFO: Server is part of a domain: OK
INFO: Running on Microsoft Windows Server 2016 Datacenter: OK
INFO: The version of the .NET Framework is 4.6 or higher
INFO: Microsoft Visual C++ 2013 Redistributable (x64) - 12.0.30501 is installed: OK
INFO: Microsoft Visual C++ 2015 Redistributable (x64) - 14.0.23026 is installed: OK
INFO: Validate Postgres User Credentials: OK
INFO: Validate CAS User Credentials: OK
INFO: Host Account for myserver trusted for delegation: OK
INFO: sascas/myserver.example.org is defined: OK
INFO: sascas SPN Account Name = EXAMPLE\cas
INFO: Stored CAS Username matches SPN User: OK
INFO: EXAMPLE\cas is trusted for delegation: OK
INFO: cas has Replace Process Level Token: OK
INFO: cas has Log on as a Service right: OK
INFO: EXAMPLE\cas is a member of the local Administrators group: OK
INFO: HTTP/myserver.example.org is defined: OK
INFO: HTTP SPN Account Name = EXAMPLE\http
INFO: KINIT using keytab: OK
      KINIT output:
      New ticket is stored in cache file C:\Users\sas\krb5cc_sas
      Picked up _JAVA_OPTIONS: -Dsun.security.krb5.debug=false -Djava.security.krb5.conf=C:\sas\install\powershell-deplo
yment\krb5.ini
WARNING: The Postgres service account was not found
INFO: Windows subsystem SharedSection tuning(SharedSection=1024,20480,20480) meets minimum: OK
INFO: TcpTimedWaitDelay is set to 30 : OK
INFO: Win32PrioritySeparation is set to 36 : OK
INFO: TCP ephemeral port range start value (32768) 32768 or less: OK
INFO: TCP ephemeral port quantity (32767) 32767 or greater: OK
INFO: SAS public code signing certs are installed: OK
PS C:\sas\install\powershell-deployment>

except for the Postgres account but that is an error in the sas-wvda.ps1 script as it only checks for a local Postgres account where I use a domain account.

The launcher log shows this:

2019-11-16 15:00:21.222 ERROR 48112 --- [o-auto-1-exec-4] com.sas.launcher.tklauncher.TKClient     : sasdemo(8accc580) [7c5c88af53ed7506] [CLIENT_CONNECT_ERROR] Failed to connect to server: Kerberos handshake error.
2019-11-16 15:00:21.334 ERROR 48112 --- [o-auto-1-exec-4] com.sas.launcher.tklauncher.TKClient     : sasdemo(8accc580) [7c5c88af53ed7506] [CLIENT_LAUNCH_ERROR] Failed to launch process: server=TKServer{id='sas-Viya-launcher-server-default:myserver.example.org', name='launcher-server', type=ServerType{name='launcher'}, state=ServerState{name='up'}, host=Host{address=myserver.example.org/10.0.0.10}, port=Port{port=33618}, operatingSystem=OperatingSystem{name='windows', pathSeparator='\', installBasePathPropertyName='sas.launcher.deployment.windows.install-base-path', configBasePathPropertyName='sas.launcher.deployment.windows.config-base-path', scriptExtensionPropertyName='sas.launcher.deployment.windows.script-extension'}, gridEnabled=false, environment=Environment{{}}, sasServicesUrl=SASServicesUrl{url=http://myserver.example.org:80/}, consulUrl=ConsulUrl{url=http://localhost:8500}, vaultUrl=VaultUrl{url=https://localhost:8200}, sslEnabled=false, sslCAList='null', sslCertificate='null', sslPrivateKey='null', spn=ServicePrincipalName{value='sas-launcher/myserver.example.org'}} cause=Failed to connect to server.
2019-11-16 15:00:21.341 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [7c5c88af53ed7506] com.sas.commons.rest.exceptions.ResourceException: Failed to launch process.
2019-11-16 15:00:21.341 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [7c5c88af53ed7506]   caused by: com.sas.launcher.tklauncher.ClientException: Failed to launch process: host=myserver.example.org port=33618 ssl=false
2019-11-16 15:00:21.341 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [7c5c88af53ed7506]   caused by: com.sas.launcher.tklauncher.ClientException: Failed to connect to server.
2019-11-16 15:00:21.341 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [7c5c88af53ed7506]   caused by: com.sas.launcher.error.LauncherRuntimeException: Kerberos handshake error.
2019-11-16 15:00:21.341 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [7c5c88af53ed7506]   caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Fail to create credential. (63) - No service creds)
2019-11-16 15:00:21.341 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [7c5c88af53ed7506]   caused by: sun.security.krb5.internal.KrbApErrException: Fail to create credential. (63) - No service creds
2019-11-16 15:00:22.788 ERROR 48112 --- [-auto-1-exec-10] com.sas.launcher.tklauncher.TKClient     : sasdemo(8accc580) [a2f2512d14059358] [CLIENT_CONNECT_ERROR] Failed to connect to server: Kerberos handshake error.
2019-11-16 15:00:22.899 ERROR 48112 --- [-auto-1-exec-10] com.sas.launcher.tklauncher.TKClient     : sasdemo(8accc580) [a2f2512d14059358] [CLIENT_LAUNCH_ERROR] Failed to launch process: server=TKServer{id='sas-Viya-launcher-server-default:myserver.example.org', name='launcher-server', type=ServerType{name='launcher'}, state=ServerState{name='up'}, host=Host{address=myserver.example.org/10.0.0.10}, port=Port{port=33618}, operatingSystem=OperatingSystem{name='windows', pathSeparator='\', installBasePathPropertyName='sas.launcher.deployment.windows.install-base-path', configBasePathPropertyName='sas.launcher.deployment.windows.config-base-path', scriptExtensionPropertyName='sas.launcher.deployment.windows.script-extension'}, gridEnabled=false, environment=Environment{{}}, sasServicesUrl=SASServicesUrl{url=http://myserver.example.org:80/}, consulUrl=ConsulUrl{url=http://localhost:8500}, vaultUrl=VaultUrl{url=https://localhost:8200}, sslEnabled=false, sslCAList='null', sslCertificate='null', sslPrivateKey='null', spn=ServicePrincipalName{value='sas-launcher/myserver.example.org'}} cause=Failed to connect to server.
2019-11-16 15:00:22.906 ERROR 48112 --- [-auto-1-exec-10] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [a2f2512d14059358] com.sas.commons.rest.exceptions.ResourceException: Failed to launch process.
2019-11-16 15:00:22.906 ERROR 48112 --- [-auto-1-exec-10] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [a2f2512d14059358]   caused by: com.sas.launcher.tklauncher.ClientException: Failed to launch process: host=myserver.example.org port=33618 ssl=false
2019-11-16 15:00:22.906 ERROR 48112 --- [-auto-1-exec-10] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [a2f2512d14059358]   caused by: com.sas.launcher.tklauncher.ClientException: Failed to connect to server.
2019-11-16 15:00:22.906 ERROR 48112 --- [-auto-1-exec-10] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [a2f2512d14059358]   caused by: com.sas.launcher.error.LauncherRuntimeException: Kerberos handshake error.
2019-11-16 15:00:22.906 ERROR 48112 --- [-auto-1-exec-10] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [a2f2512d14059358]   caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Fail to create credential. (63) - No service creds)
2019-11-16 15:00:22.906 ERROR 48112 --- [-auto-1-exec-10] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [a2f2512d14059358]   caused by: sun.security.krb5.internal.KrbApErrException: Fail to create credential. (63) - No service creds
2019-11-16 15:00:23.223 ERROR 48112 --- [o-auto-1-exec-4] com.sas.launcher.tklauncher.TKClient     : sasdemo(8accc580) [63fb96197eb751c9] [CLIENT_CONNECT_ERROR] Failed to connect to server: Kerberos handshake error.
2019-11-16 15:00:23.334 ERROR 48112 --- [o-auto-1-exec-4] com.sas.launcher.tklauncher.TKClient     : sasdemo(8accc580) [63fb96197eb751c9] [CLIENT_LAUNCH_ERROR] Failed to launch process: server=TKServer{id='sas-Viya-launcher-server-default:myserver.example.org', name='launcher-server', type=ServerType{name='launcher'}, state=ServerState{name='up'}, host=Host{address=myserver.example.org/10.0.0.10}, port=Port{port=33618}, operatingSystem=OperatingSystem{name='windows', pathSeparator='\', installBasePathPropertyName='sas.launcher.deployment.windows.install-base-path', configBasePathPropertyName='sas.launcher.deployment.windows.config-base-path', scriptExtensionPropertyName='sas.launcher.deployment.windows.script-extension'}, gridEnabled=false, environment=Environment{{}}, sasServicesUrl=SASServicesUrl{url=http://myserver.example.org:80/}, consulUrl=ConsulUrl{url=http://localhost:8500}, vaultUrl=VaultUrl{url=https://localhost:8200}, sslEnabled=false, sslCAList='null', sslCertificate='null', sslPrivateKey='null', spn=ServicePrincipalName{value='sas-launcher/myserver.example.org'}} cause=Failed to connect to server.
2019-11-16 15:00:23.340 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [63fb96197eb751c9] com.sas.commons.rest.exceptions.ResourceException: Failed to launch process.
2019-11-16 15:00:23.340 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [63fb96197eb751c9]   caused by: com.sas.launcher.tklauncher.ClientException: Failed to launch process: host=myserver.example.org port=33618 ssl=false
2019-11-16 15:00:23.340 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [63fb96197eb751c9]   caused by: com.sas.launcher.tklauncher.ClientException: Failed to connect to server.
2019-11-16 15:00:23.340 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [63fb96197eb751c9]   caused by: com.sas.launcher.error.LauncherRuntimeException: Kerberos handshake error.
2019-11-16 15:00:23.340 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [63fb96197eb751c9]   caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Fail to create credential. (63) - No service creds)
2019-11-16 15:00:23.340 ERROR 48112 --- [o-auto-1-exec-4] com.sas.commons.rest.ExceptionLog        : sasdemo(8accc580) [63fb96197eb751c9]   caused by: sun.security.krb5.internal.KrbApErrException: Fail to create credential. (63) - No service creds

Do you have an idea what is wrong?

Bart

StuartRogers · ‎11-18-2019

Hello Bart,

You should not use the comments on these posts as a substitute for opening a Technical Support track. I believe that you should be opening a Technical Support track to work through the issues with your configuration. When you open the track you should provide the details above, but also include the following:

1. Logging for SAS Logon Manager

2. The configuration properties you set for SAS Logon Manager - the easiest way to capture these is with the SAS Bootstrap Config tool.

3. More details on the HTTP principal, screenshot showing the delegation settings would help

4. Details of the host you are testing from, klist output for your test user to show the correct delegation settings on the Service Ticket

You state that access to SASDrive is successful so that Kerberos authentication to SAS Logon Manager is working. Your issues are with the delegation of the Kerberos tickets to SAS Logon Manager and then out to the downstream processes such as the SAS Compute Server or SAS Cloud Analytic Services. So Technical Support will work with you to obtain more information on this delegation stage.

Thank you for your time.

Stuart