Promoting SAS Viya Job Flows

3 Likes

In my last post, I covered the Viya jobs object model as a precursor to promoting jobs and flows between Viya environments. In this post, I will look at the promotion process for jobs and flows. I will step through the process of promoting a flow from one Viya environment to another.

When promoting content in Viya, if a resource exists in a folder, you can use the SAS Environment Manager or the sas-admin transfer command-line interface. If a resource does not exist in a folder you must use the sas-admin transfer command-line interface (cli) to export, and either SAS Environment Manager or the command-line interface to import.

A reminder, in my previous post, we identified the following job-related objects.

job definitions
job requests
job flows
job actions
job instances

Of these resources only job definitions live in folders. As a result, job definitions can be promoted using either Environment Manager or the command-line interface. The other objects must be promoted on the command-line.

To export the job objects we will need to use the --request option of the transfer cli. To use the request option, we build a request file that includes a list of the Uniform Resource Identifiers (URI) of the objects we want to export and import. In Viya, a uniform resource identifier (URI) is used to identify unique resources.

In this post we will promote a job flow, viewed here in the Jobs and Flows area in SAS Environment Manager.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Identify the URI's of the Resources

A key step in exporting job objects is to find the URI of the objects we wish to export. For this key step we will use the job plug-in of the sas-admin command-line interface.

If you have not already done so, create a profile that references your source Viya environment as the endpoint, and sign in with an account that is a member of the Administrators group

Job definitions are the one object type in the process that may exist in a folder. In our case, they were created from SAS Code using "Create Job" in SAS Studio . As a result they exist in folders and can be exported simply by exporting the folder (we will not cover this in this post).

The job requests created from the job definitions are visible in the Environment Manager Jobs and Flows area. If you adjust the columns in the UI you can see the ID of the job request.

You can also use the sas-admin job cli to list the job requests and their id's.

/opt/sas/viya/home/bin/sas-admin -output text job requests list -limit 1000

22c6cc7d-def4-4cc4-84ac-f2b2034129db HRAnalysysProject_1_CreateFormatsInSAS 3 <nil>
3542cb95-60cc-4f61-a326-52b14fd0806c HRAnalysysProject_2_CreateDataInSAS 3 <nil>
7503d138-1ed7-4e99-a65f-020a7189a82d HRAnalysysProject_3_LoadFormatsInSAS 3 <nil>
568be1c9-a797-4d55-bce7-11fc906ad85c HRAnalysysProject_4_LoadDataInCAS 3 <nil>

At this point you have the ID of the job request. To build the URI you need the first part of the endpoint which, if you know it is /jobExecution/jobRequests. If you set output to full json you can get URI from the self attribute under links.

/opt/sas/viya/home/bin/sas-admin --output fulljson job requests list --limit 1000 --filter 'startsWith(name,"HR")'

"links": [
{
"href": "/jobExecution/jobRequests/22c6cc7d-def4-4cc4-84ac-f2b2034129db",
"method": "GET",
"rel": "self",
"type": "application/vnd.sas.job.execution.job.request",
"uri": "/jobExecution/jobRequests/22c6cc7d-def4-4cc4-84ac-f2b2034129db"

To get the URI of the Job flow, use the following to find the flow which starts with HR.

/opt/sas/viya/home/bin/sas-admin --output text job flows list --filter 'startsWith(name,"HR")'

Id Name Version Description
fa12415f-6931-4084-8d16-b85934248b88 HRAnalysysProject_Job_Flow <nil> Flow for HR Analysis

Let's use the job cli to look the details of the flow. This will identity flow URI, and the URI of the job actions within the flow. The job actions were created when the job requests were added to the flow. In the detailed output about the flow the job actions are in the "jobs" list.

/opt/sas/viya/home/bin/sas-admin --output json job flows show --id fa12415f-6931-4084-8d16-b85934248b88{

{
"defaultJobProperties": {},
"dependencies": [
{
"event": {
"condition": "all",
"events": [
{
"expression": "success('HRAnalysysProject_2_CreateDataInSAS')",
"type": "jobevent"
},
{
"expression": "success('HRAnalysysProject_3_LoadFormatsInSAS')",
"type": "jobevent"
}
],
"type": "gate"
},
"target": "HRAnalysysProject_4_LoadDataInCAS"
},
{
"event": {
"expression": "success('HRAnalysysProject_1_CreateFormatsInSAS')",
"type": "jobevent"
},
"target": "HRAnalysysProject_3_LoadFormatsInSAS"
},
{
"event": {
"expression": "success('HRAnalysysProject_1_CreateFormatsInSAS')",
"type": "jobevent"
},
"target": "HRAnalysysProject_2_CreateDataInSAS"
}
],
"description": "Flow for HR Analysis",
"flowProperties": {
"completionLevel": "default",
"endBehavior": "stop",
"exitcodeStrategy": "last"
},
"id": "fa12415f-6931-4084-8d16-b85934248b88",
"jobs": [
"/jobFlowScheduling/jobs/acdd1ca9-d463-44a0-b55b-38435fb7e66d",
"/jobFlowScheduling/jobs/ee261230-26a7-4c7a-88a5-40023226a175",
"/jobFlowScheduling/jobs/8cde1625-a943-4c08-9863-55abff6bb609",
"/jobFlowScheduling/jobs/f3aae428-20ac-4653-9cf5-9af8bad59e5a"
],
"name": "HRAnalysysProject_Job_Flow",
"schedulerId": "974d0934-e067-4b46-8da0-cecb8fa662fe",
"triggerType": "manual"
}

Build the Transfer Requests File and Export

At this point we have identified the URI's of the job requests, the job flow and the job actions within the flow. The next step is to build the requests file that we can pass to the export step in the transfer cli. Each URI you add to the request file will be exported from Viya.

{
"version": 1,
"name": "hrflow",
"description": "HR Flow and depedencies",
"items": [ "/jobExecution/jobRequests/22c6cc7d-def4-4cc4-84ac-f2b2034129db",
"/jobExecution/jobRequests/3542cb95-60cc-4f61-a326-52b14fd0806c",
"/jobExecution/jobRequests/7503d138-1ed7-4e99-a65f-020a7189a82d",
"/jobExecution/jobRequests/568be1c9-a797-4d55-bce7-11fc906ad85c",
"/jobFlowScheduling/flows/fa12415f-6931-4084-8d16-b85934248b88",
"/jobFlowScheduling/jobs/acdd1ca9-d463-44a0-b55b-38435fb7e66d",
"/jobFlowScheduling/jobs/ee261230-26a7-4c7a-88a5-40023226a175",
"/jobFlowScheduling/jobs/8cde1625-a943-4c08-9863-55abff6bb609",
"/jobFlowScheduling/jobs/f3aae428-20ac-4653-9cf5-9af8bad59e5a"
]
}

Use the transfer cli to export the content and download the resultant transfer package.

/opt/sas/viya/home/bin/sas-admin transfer --output text export --request @/tmp/exportjobs.json --name "HR job requests and flows"

/opt/sas/viya/home/bin/sas-admin transfer --output text download --id 7985128f-a2ef-4012-8199-f8f72dba9311 --file /tmp/jobflowpackage.json

On the target system you can use Environment Manger or the transfer cli to import the package. We will use the Environment Manager import Wizard. The results show the flow, 4 job actions and 4 job requests imported into the target system.

If the flow had been scheduled, the scheduling information is not included in the import and and export process. The flow would need to be rescheduled in SAS Environment Manager or via the jobs cli.

I hope you have found this post useful. Here is some additional information that may also be helpful.