SAS Viya Cloud Data Exchange Deployment and Configuration (Part -1)
In my last post I talked about SAS Viya Cloud Data Exchange, it’s components and how it can help SAS users to access on-remises data from cloud hosted SAS Viya platform. As mentioned in last post, the configuration between Remote Data Agent and SAS Viya platform is a challenging task. So, let’s look at the details steps to integrate Remote Data Agent service with SAS Viya platform.
This post is in two parts, the first part describing the deployment and configuration of Co-located Data Agent service. The second part is describing the deployment and configuration of Remote Data Agent.
The SAS Viya Cloud Data Exchange software is available with most of the SAS Viya bundle license. There is no separate license for Cloud Data Exchange components like Co-located Data Agent service and Remote Data Agent services.
SAS Viya Cloud Data exchange have two types of Data Agents.
- Co-located Data Agent - Reside at SAS Viya platform environment.
- Remote Data Agent - Hosted at on-premises data center.
Co-located Data Agent Deployment and configuration
The Co-located Data Agent software and components are delivered with the standard SAS Viya deployment. The Co-located Data Agent (CDA) pods and services starts along with the rest of SAS Viya services. Configuration of Co-located Data Agents are described in the following steps.
Use example configuration file for Co-located Data Agent (CDA)
The CDA configuration example is provided under ../sas-bases/examples/sas-data-agent-server-colocated folder to configure the Co-located Data Agent and integrate with Remote Data Agent services. You can copy the example folder to site-config folder and update the configuration files.
cp -r …./sas-bases/examples/sas-data-agent-server-colocated …./site-config/
chmod 755 …./site-config/sas-data-agent-server-colocated/*.*
There are two configuration files sas-data-agent-server-colocated-secret.properties and sas-data-agent-server-colocated-config.properties.
The sas-data-agent-server-colocated-secret.properties file contains the SAS_DA_OAUTH_SECRET= variable. The value of SAS_DA_OAUTH_SECRET= is used by Remote Data Agent service to connect to SAS Viya platform for system level information like SAS Viya user identity. You must provide the same value while configuring Remote Data Agent service at on-premise server. The password could be in plain text or encrypted or hashed.
# hashed password in sas-data-agent-server-colocated-secret.properties
SAS_DA_OAUTH_SECRET=*3434F0D512A9E05E4EF36450FC676EF151D94B51
The sas-data-agent-server-colocated-config.properties file is meant for Data Agent level configuration. It contain variable like debugging trace flag, user data folder, SAS Access clients variables and so on.
# Variable examples in sas-data-agent-server-colocated-config.properties
SAS_DA_DEBUG_LOGTYPE=TRACEALL
SAS_DA_RESTRICT_CONTENT_ROOT=TRUE
SAS_DA_CONTENT_ROOT=/gelcontent/data/
Update kustomization.yaml file
Include the sas-data-agent-server-colocated-secret.properties and sas-data-agent-server-colocated-config.properties file in kustomization.yaml to build and start the Co-located Data agent with updated parameters.
printf "
- command: update
path: secretGenerator[+]
value:
name: sas-data-agent-server-colocated-secrets
behavior: merge
envs:
- site-config/sas-data-agent-server-colocated/sas-data-agent-server-colocated-secret.properties" | yq -I 4 w -i -s - …./kustomization.yaml
printf "
- command: update
path: configMapGenerator[+]
value:
name: sas-data-agent-server-colocated-config
behavior: merge
envs:
- site-config/sas-data-agent-server-colocated/sas-data-agent-server-colocated-config.properties" | yq -I 4 w -i -s - ….kustomization.yaml
Build and deploy the updated manifestation
cd ~/project/deploy/
kustomize build -o site.yaml
NS=myenv;
kubectl apply -n ${NS} -f ~/project/deploy/gelenv/site.yaml ;
## Recycle Data Agent Pods.
kubectl -n ${NS} delete pod --selector='app.kubernetes.io/name=sas-data-agent-server-colocated'
kubectl -n ${NS} delete pod --selector='app.kubernetes.io/name=sas-data-agent-services'
Add SAS Viya users to DataAgentAdmin and DataAgentPower user group
The administrative tasks of Data Agents are executed using sas-viya CLI. Make sure you have sas-viya CLI installed at your local client machine or at jump server. To execute Data Agent administrative task, user must be in a specific SAS Viya user group. The users from DataAgentAdminstrators group can configure and administer Data Agents, Data Source , Security Domain and Data Services. The users from DataAgentPowerUsers group can access the data sources by executing DQL, DML and DDL statement. As per user place them in the respective SAS Viya user’s group.
export SSL_CERT_FILE=~/.cerrts/my_trustedcerts.pem
sas-viya auth login --user sasboot –password
sas-viya identities add-member --group-id DataAgentAdministrators --user-member-id geldmui@gelenable.sas.com
sas-viya identities add-member --group-id DataAgentPowerUsers --user-member-id geldmui@gelenable.sas.com
sas-viya identities add-member --group-id DataAgentPowerUsers --user-member-id gatedemo001@gelenable.sas.com
sas-viya identities add-member --group-id DataAgentPowerUsers --user-member-id gatedemo002@gelenable.sas.com
With the above steps, SAS Viya environment is configured to integrate with Remote Data Agent services. You can also create data sources at Co-located Data Agent server which are directly accessible from Viya platform.
Stay tuned-in for second part of this series on Remote Data Agent Deployment and Configuration.
Many thanks to my colleague Brian Hess for sharing the expert knowledge and help on this post.
Important Links:
