New Support for External OpenSearch Instances in the SAS Viya Platform

[#1]

Starting with the 2023.03 SAS Viya release, the SAS Viya platform supports using an external instance of OpenSearch to provide searching and indexing capabilities. What are the requirements, and how to enable this new capability? If you are interested, read on!

OpenSearch

OpenSearch provides search and indexing features for software that runs on the SAS Viya platform. Starting with the 2020.1.3 Stable release, the SAS Viya platform includes an Apache 2.0-licensed distribution of OpenSearch with enhanced security and a SAS-provided Kubernetes operator to handle its operation and management.

Starting with the 2023.03 release, new configuration options are added to the platform to support either:

• an internal instance of OpenSearch automatically included in the deployment (just as before)
• an external OpenSearch instance that you administer and maintain

OpenSearch is required by every SAS Viya offering except SAS Viya Programming.

Why?

The OpenSearch instance deployed by default inside the SAS Viya platform includes opinionated deployment settings to configure it according to the platform requirements. For example, the pods must run under a particular user ID, and require specific kernel settings on the nodes underlying the Kubernetes cluster. Some customers may have security policies that prevent these settings in their Kubernetes clusters. And, let’s be honest, a production instance of OpenSearch does not fit typical cloud-native environments: for example, every update to the SAS Viya platform requires a full stop and restart of the OpenSearch cluster.

An external OpenSearch instance is managed outside of SAS Viya and can alleviate these considerations. Supporting an external OpenSearch removes some requirements from the Kubernetes cluster dedicated to the SAS Viya platform. As an added benefit, an external OpenSearch reduces the footprint of the SAS Viya platform deployment.

In the end, adding support for an external option provides you with more flexibility in your architectural choices!

Internal VS External

A simple table can easily show the main comparison points between using an internal OpenSearch instance or an external one.

Internal	External
Dedicated instance deployed by the same process as the rest of the SAS Viya platform	OpenSearch instance deployed by you before starting the SAS Viya platform deployment
Deployed in the same namespace as the SAS Viya platform	Deployed outside the SAS Viya platform namespace
SAS Viya platform cluster sizing and design must account for OpenSearch (including kernel requirements, and storage requirements)	OpenSearch requirements and sizing do not impact the SAS Viya platform cluster
Contains all required plugins (including enhanced security)	Must include required plugins
Automatically configured as required by the SAS Viya platform	Must be configured according to SAS specifications
Managed by a SAS-provided Kubernetes operator	Managed as you choose to
Supported by SAS	Supported by you

Requirements

Whether you decide to use an internal or an external OpenSearch, once your deployment is done, there is no turning back. It is not possible to reconfigure an existing SAS Viya platform to use a different OpenSearch. If you change your mind, you will have to un-deploy and re-deploy.

The official requirements for an external deployment are described in the readme file at <$deploy>/sas-bases/examples/configure-Elasticsearch/external/README.md. Here is a summarized listing, valid at the time of writing (always refer to the official documentation to have up-to-date requirements):

• You can only use the opensource OpenSearch edition, not Elasticsearch.
• The supported OpenSearch version is 2.5.
• It must include a specific list of plugins used by the SAS Viya platform.
• The OpenSearch server should be set up and running before you start the SAS Viya platform deployment.
• Managed cloud subscriptions to Elasticsearch and OpenSearch are not supported; only instances that you deploy and configure.
• TLS between the SAS Viya platform and OpenSearch is strongly recommended, but not required.

SAS Visual Investigator requires additional configuration settings. You can find the listing of SAS Visual Investigator specific requirements in the README file at <$deploy>/sas-bases/examples/configure-Elasticsearch/external/config/README.md, and a sample config.yaml is provided in the same directory.

How to deploy the SAS Viya platform with an external OpenSearch

The deployment instructions are provided in the same README file as the requirements.

To use an external OpenSearch instance, you need to gather the following parameters from the OpenSearch administrator:

• Username and Password of an account with administrative privileges for your external OpenSearch instance
• URL (and port) of your external OpenSearch instance

NOTE: If the connection uses TLS and OpenSearch is using custom-signed certificates, you also need the server CA certificate so that you can import it into SAS Viya. This step is the same as with any other external CA certificate, whether it is used to protect a connection to OpenSearch, a database, an LDAP server, etc. You can follow the official instruction in the document Incorporate CA certificates into SAS Viya

After you have those connection details, follow these steps:

1. Copy the provided external-opensearch-transformer.yaml, secret.yaml,and client-config-tls.yaml files from: $deploy/sas-bases/examples/configure-elasticsearch/external/ to $deploy/site-config/external-opensearch/. No need to change anything in its content.
2. Adjust the username, password, and URL values in the latter two files as instructed by the in-line comments.
   
   

   Select any image to see a larger version.
   Mobile users: To view the images, select the "Full" version at the bottom of the page.

   
3. Edit the base kustomization.yaml to reference these files:

• add external-opensearch-transformer.yaml to the transformers block
  
  

transformers:
...
  - site-config/external-opensearch/external-opensearch-transformer.yaml

• add the secret.yaml and client-config-tls.yaml to the resources block

resources:
...
  - site-config/external-opensearch/client-config-tls.yaml
  - site-config/external-opensearch/secret.yaml

4. Start the SAS Viya platform deployment with your preferred method.

And finally, a reminder: if you copied your base kustomization.yaml from an environment that uses an internal OpenSearch instance, be sure to remove any existing lines referencing the internal instance, otherwise you will get an unsuccessful deployment!

Conclusion

In this article we have described the benefits and considerations of the new option that you have when deploying the SAS Viya platform starting with the 2023.03 release, the support of an external instance of OpenSearch to provide searching and indexing capabilities. We have also listed the steps required to perform a deployment connecting to an external OpenSearch instance.

Find more articles from SAS Global Enablement and Learning here.