SAS Viya jobs object model explained.
In my previous article, I covered how to promote SAS 9.4 Stored Processes to Viya jobs. In my next two posts, I want to cover the promotion of jobs, job flows and other job-related objects within Viya. Jobs can be a confusing topic in Viya. So, before I deal with promotion of jobs and flows, I want to cover the terminology around jobs and the relationship between the different Viya objects involved.
The term job is used in a lot of different ways in Viya. There are several places where you can create/use jobs, some of them are:
- CAS table state management : load/unload/refresh data in CASLIB
- SAS Data Explorer: load individual tables to memory and save
- SAS Data Studio: prepare and transform data
- SAS Visual Analytics: schedule the distribution of reports
- SAS Studio Jobs
- Model Studio
In addition to jobs, Viya now has job flows. Job flows are a new feature in SAS Viya 3.5 and their usage was covered by my colleague Bogdan Teleuca in a recent post.
The term "job" is used for all the items we list above. Under the covers the job architecture has a variety of different objects and these objects all relate to each other. As you use the different types of jobs, objects are created and surfaced in various Viya applications. The main objects are:
- job definitions
- job requests
- job flows
- job actions
- job instances
Let's look at what these objects are and how they relate to each other. For each object we will look at
- what is its endpoint that uniquely identifies the object
- where you can access the object in Viya
- how and where the object is created.
job definition: (/jobDefinitions/definitions)
Job definitions contain metadata about a job including the code, the type of job, any parameters and/or prompts and additional job metadata. The type of jobs are listed below (may not be an exhaustive list):
- AnalyticsFlow
- scheduleBackup
- casManager
- casl
- Compute
- ReportDistribition
Job definitions:
- some like SAS Studio Jobs reside in folders, others are not surfaced in folders
- created by:
- importing 9.4 stored process
- from SAS code in SAS Studio using "Create Job"
- from the Job Execution Web Application
- the deployment process to support implementing Viya functionality such as importing data, managing CAS table state etc.
This is a job definition in SAS Studio:
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
The following command shows a list of job definitions. This command was run on a fresh Viya deployment showing that many job definitions are created by the deployment process.
/opt/sas/viya/home/bin/sas-admin --output text job definitions list --limit 1000
job request (/jobExecution/jobRequests)
Job requests added runtime information to a job definition. The job request either includes the job definition to be executed or includes a link (via its URI) to the job definition that will be executed. The runtime arguments include items like the values provided at prompts, the compute context etc. Job requests make job definition's available for execution and scheduling.
Job requests:
- do not reside in folders, visible in the Environment Manager Jobs and Flows page
- created by selecting:
- Schedule Job from a Job Definition in SAS Studio
- New Job from a Data Plan in SAS Data Studio
- New Job from the scheduling pane in SAS Environment Manager Jobs and Flows
- New Job from Import in Data Explorer
- Schedule on a Report Distribution
- (likely in more ways that I have not listed).
This is a job request in the scheduling area of Jobs and Flows in SAS Environment Manager.
Unlike definitions there are only a few requests that initially exist in a Viya system and those relate to backup and CAS Table State Management. As the users use the system other requests are created and each request references(or includes) its job definition.
/opt/sas/viya/home/bin/sas-admin --output text job requests list --limit 1000
The output shows
- 3 default CAS table state management job requests (Sample)
- 2 default backup job requests (_BACKUP_SCHEDULE)
- 1 Report distribution (Report 1)
- 1 Customized CAS table state management (Sales)
- 4 Job requests created from SAS Studio Job Definitions (HRAnalysisProject)
Let's take a closer look at the Sales: Load cas-shared-default Sales data job.
/opt/sas/viya/home/bin/sas-admin --output text job requests show --id 5e88b440-fcb3-4300-9edd-229512f5e3f1
The JobDefinitionURI attribute of the request references the job definition /jobDefinitions/definitions/7b11627f-f68d-4d72-bb10-c34a2aa5b470. If we look at that job definition, we can see that it is the default Cas Management Load Job created when the software was deployed.
/opt/sas/viya/home/bin/sas-admin --output text job definitions show --id 7b11627f-f68d-4d72-bb10-c34a2aa5b470
Looking at the job request created by Scheduling a Job Definition in SAS Studio, we can see a different scenario. The request does not link to the URI of the job definition but instead includes the complete definition in the jobDefinition field of the job request.
/opt/sas/viya/home/bin/sas-admin --output text job requests show --id <id>
job flows (/jobFlowScheduling/flows)
A job flow is a group of jobs, job flows, dependencies, and conditions that are organized in a sequence and can be scheduled as a group.
- do not reside in folders, visible in the Environment Manager Jobs and Flows page
- created by the Job Editor in the Environment Manager Jobs and flows page.
job actions (/jobFlowScheduling/jobs/)
A job action is created when you add a job request to a flow. Job actions are only visible within flows. The job action references the job request and includes other information related to priority etc.
Job actions:
- do not reside in folders, visible inside flows in EV jobs and flows
- are created by adding a job request to a Job flow.
jobs (/jobExecution/jobs)
When you schedule or execute a job request it creates a job instance. These "jobs" are visible in the monitoring tab of the Jobs and Flows area. This is the actual object that runs, they are an instance of a job request, they therefor add information about the execution, for example, job state, job status, location of log etc.
Job instances:
- do not reside in folders, visible in the Environment Manager Monitoring Tab
- are created by scheduling or executing a job request.
I started this journey trying to determine how to promote a job flow. When you promote a job flow, you must be aware of all these relationships and include the dependent objects within the package. I hope this explanation of the job related objects and their relationships proves useful. In my next post I will show you how to promote a job flow and its related objects.
I would like to thank my colleague Gilles Chrzaszcz for his help in researching and understanding this topic. For more information:
