Moving SAS® Viya® Contents: A RESTful Way
Abstract
A SAS customer wanted an application to be built around SAS Viya, which would help the IT organization manage its change management process. A solution was needed to help their developers promote SAS Viya contents from one environment to another, without logging into the SAS Viya user interface and having to learn the SAS export/import tools. Due to its current IT policy, the customer was reluctant to use the sas-admin CLI and wanted to use REST APIs to promote the contents from one environment to another. We explored the SAS Viya REST APIs and built an application that calls and orchestrates the promotion process in the backend to export and import SAS Viya contents from one SAS Viya environment to another. The application calls the SAS Viya promotion APIs to move the contents across the environment at the user-provided date and time to promote the contents to a higher environment. This presentation will demonstrate the three different ways (outlined below) to promote contents from one SAS Viya environment to another SAS Viya environment, including the steps involved in the process.
Watch the presentation
Watch Moving SAS® Viya® Contents: A RESTful Way by the author on the SAS Users YouTube channel.
Introduction
SAS® Viya® REST APIs are organized around REST principles. The APIs are based on resource-oriented URLs, use HTTP authentication and HTTP verbs, and return HTTP response codes to indicate API errors. All these features help you integrate the APIs into your applications and scripts in a standard way. With SAS® Viya® REST APIs, you can create and access SAS® resources using any client technology, making it easy to integrate the capabilities of SAS® Viya® into your business processes or to extend and customize SAS® Viya® to meet specific requirements. More details about SAS® Viya® API’s are available at https://developer.sas.com/apis/rest/.
Following three ways could be used to export-import contents in SAS® Viya®:
- Using wizard available in SAS Env Manager
- Using sas-admin CLI tool
- Using RESTful API
This document focuses on the third way, using RESTful API to export and import contents in SAS® Viya®. While exporting contents from the SAS® Viya® using RESTful API, we need to follow the same steps as for CLI. The component needs to be ‘exported’ and then ‘downloaded’ while exporting the content from SAS® Viya®, similarly component needs to be ‘uploaded’ and then ‘imported’ while importing the content from SAS® Viya®.
PROBLEM STATEMENT
We will talk about the promotion flow and the steps involved in the process; a developer raises a Release request on the application once his/her development is ready for promotion. An approver approves the changes. Once the changes are approved the contents are promoted to one environment to another.
Log in to the source environment
We have registered a client ID & client secret for our application, we use the client id and secret to generate access token to access SAS® Viya®.
If you haven’t registered a client ID and client secret please follow the steps outlined in the following SAS Global Forum 2019 paper https://blogs.sas.com/content/sgf/2019/01/25/authentication-to-sas-viya/
curl -k https://<<SourceEnvHostName>>/SASLogon/oauth/token \
-H "Content-Type: application/x-www-form-urlencoded" -u "<<myclientid>>:<<myclientsecret>>" \
-d "grant_type=password&username=<<UserName>>&password=<<MyPassword>>"EXPORT CONTENTS
To export the contents of the SAS Viya object, as a first step we need to send a post request as mentioned below after filling in the relevant details like hostname, OAuth token and object URI of the SAS Viya object that we want to export:
POST /transfer/exportJobs HTTP/1.1
Host: <<SourceEnvHostName>>
Accept: application/vnd.sas.transfer.export.job+json
Content-Type: application/vnd.sas.transfer.export.request+json
Accept-Encoding: application/gzip
Authorization: Bearer <<OAuth token>>
Payload:
{"version":0,"name":"exportcontent1","description":"<<Description>>","items":["<<component/resource URI>>"],"options":null}
The equivalent cURL command for the above POST call is:
curl -x post \
https://<<SourceEnvHostName>>/transfer/exportjobs \
-h 'accept: application/vnd.sas.transfer.export.job+json' \
-h 'accept-encoding: application/gzip' \
-h 'authorization: bearer <<oauth token>>' \
-h 'cache-control: no-cache' \
-h 'connection: keep-alive' \
-h 'content-type: application/vnd.sas.transfer.export.request+json' \
-h 'host: <<hostname>>' \
-d '{"version":1,"name":"exportcontent1","description":"<<Description>>","items":"<< object/resource URI >>","options":null}'
Sometimes the above export creation may take some time thus we need to see if the ‘extract’ has been created before we start to download the content package. For this, we can use the below REST API to determine if the ‘extract’ for the package has been completed or not.
curl -X GET \
https://<<SourceEnvHostName>>/transfer/exportJobs/<<id>>/state \
-H 'Accept: application/vnd.sas.transfer.export.job+json' \
-H 'Accept-Encoding: application/gzip' \
-H 'Authorization: Bearer <<OAuth Token>>'
Download ContentS
To completes the 'export' of the object from the source environment, we need to send a GET request as mentioned below after filling in the relevant details like hostname, OAuth token and id which was generated during the extract creation in the previous step:
GET /transfer/packages/<<id created in the export step>> HTTP/1.1
Host: <<SourceEnvHostName>>
Accept: multipart/form-data, application/json
Content-Type: application/json
Accept-Encoding: application/gzip
Authorization: Bearer <<oauth token>>
The equivalent cURL command for the above GET request is:
curl -x get \
https://<<SourceEnvHostName>>/transfer/<<id created in the export step>> \
-h 'accept: multipart/form-data, application/json' \
-h 'accept-encoding: application/gzip' \
-h 'authorization: bearer <<oauth token>>' \
-h 'content-type: application/json' \
-h 'host: <<SourceEnvHostName>>' > <<path_where_json_needs_to_be_downloaded_on_server>>/<<filename>>.json
If you are using tools such as Postman: you can need to save the response from the get call as shown below:
Login to the Target Environment
We have registered a client ID & client secret for our application, we use the client id and secret to generate access token to access target SAS Viya.
Command:
curl -k https://<<TargetEnvHostName>>/SASLogon/oauth/token \
-H "Content-Type: application/x-www-form-urlencoded" -u "<<myclientid>>:<<myclientsecret>>" \
-d "grant_type=password&username=<<UserName>>&password=<<MyPassword>>"
Upload ContentS
Now in order to upload the contents, we again need to follow a two-step process. Once we have authenticated ourselves in the target environment and generated the OAuth token we need to upload the contents in the target SAS Viya environment followed by importing them in the target environment.
To upload the exported SAS Viya object from the source environment, send a post request as mentioned below after filling the relevant details like hostname, OAuth token and json package which needs to be imported:
POST /transfer/packages
Host: <<TargetEnvHostName>>
Accept=application/vnd.sas.summary+json&
Content-Type=multipart/form-data&
Accept-Encoding=gzip HTTP/1.1
Content-Type: application/vnd.sas.transfer.package+json
Authorization: Bearer <<OAuth>>
Accept: */*
<<json which needs to be imported>>
The equivalent cURL command for the POST request is:
curl -x post \
'https://<<TargetEnvHostName>>/transfer/packages' \
-h 'accept: */*' \
-h 'accept-encoding: gzip, deflate' \
-h 'authorization: bearer <<oauth token>>' \
-h 'cache-control: no-cache' \
-h 'connection: keep-alive' \
-h 'content-type: application/vnd.sas.transfer.package+json' \
-h 'host: <<TargetEnvHostName>>' \
-d '@<<full path of json file to be imported>>'If you are using a tool such as Postman, you can use the form-data as a post body and select the JSON file from your local system as shown below:
The above POST request would return the response JSON. We need to grab the ‘id’ attribute from the response JSON which would be used to import the package in SAS® Viya®.
Import ContentS
The last step of the promotion process is importing the contents once the upload step is completed. To import the contents in SAS Viya, send a POST request as mentioned below after filling the relevant details like hostname, OAuth token and package id which was generated during the JSON upload in the previous step:
POST /transfer/importJobs HTTP/1.1
Host: <<TargetEnvHostName>>
Accept: application/vnd.sas.transfer.import.job+json
Content-Type: application/vnd.sas.transfer.import.request+json
Accept-Encoding: application/gzip
Authorization: Bearer <<OAuth>>
{"name":"/transfer/packages/<< package id generated in previous step>>
","packageUri":"/transfer/packages/<<package id generated in the import step>>","mapping":{}}
The equivalent cURL command of the POST request is:
curl -x post \
https://<<TargetEnvHostName>>/transfer/importjobs \
-h 'accept: application/vnd.sas.transfer.import.job+json' \
-h 'accept-encoding: application/gzip' \
-h 'authorization: bearer <<oauth token>>' \
-h 'content-type: application/vnd.sas.transfer.import.request+json' \
-d '{"name":"/transfer/packages/<< package id generated in previous step >>" ,"packageuri":"/transfer/packages/<<package id generated in the upload step>>","mapping":{}}'
If you are using a tool such as Postman, you can use the below set of configurations and settings for making a POST call
The above POST call would give the below return, in which it states that the import process is still running.
Sometimes the above import creation may take some time (10-30 seconds or up to a min depending on the size of the object being imported) thus we need to see if the import process has been completed or not. For this, we can use the below REST API to determine if the ‘import’ of the package has been completed or not.
curl -X GET \
https://<<TargetEnvHostName>>/transfer/exportJobs/<<id>>/state \
-H 'Accept: application/vnd.sas.transfer.export.job+json' \
-H 'Accept-Encoding: application/gzip' \
-H 'Authorization: Bearer <<OAuth Token>>'
Above GET request would return the state of the extract process as either ‘running’ or ‘completed’. Once the above request returns completed which indicates that the import process is also completed
Conclusion
Change and Release management (promotion) of development work can be easily managed and streamlines with above REST API solution for content promotion. With SAS® Viya® REST APIs, we can orchestrate, automate, and simplify a lot of ITID processes. And help SAS® customers build and incorporate their daily ITID processes with SAS® Viya seamlessly. SAS® Viya® REST APIs help to integrate SAS® analytics capabilities into simple applications and help the customers leverage the power of SAS®.
References
- SAS® Viya® 3.5 Administration: https://documentation.sas.com/?cdcId=calcdc&cdcVersion=3.5&docsetId=calchkcfg&docsetTarget=titlepage...
- Authentication to SAS® Viya®: a couple of approaches: https://blogs.sas.com/content/sgf/2019/01/25/authentication-to-sas-viya/
Contact Information
Your comments and questions are valued and encouraged. Contact the authors at:
Amit Choubey
Core Compete, part of Accenture
https://www.linkedin.com/in/amitc26/
Sounak Nag
Core Compete, part of Accenture
https://www.linkedin.com/in/sounak-nag-0014b1146/
