Migration of Stored Processes to SAS Viya Jobs

4 Likes

Two years ago wrote a post about the migration of SAS 9.4 Stored Processes to Viya Jobs in Viya 3.5. In the world of monthly Viya releases, things change fast. It is time for an update on this topic. Stored Processes are a very popular feature in SAS 9.4. They are used in reporting, analytics, and web application development. In Viya the equivalent is jobs. In this post, we will look at what a Viya job is, how you migrate your Stored Processes, and what happens to them as they migrate to Viya jobs.

SAS 9.4 Stored Processes are migrated to Jobs in Viya(Job Definition). A SAS Viya job consists of a SAS program and its definition(metadata related to the program). Just like Stored Processes jobs, have an execution URL that you can share with others at your site. The execution URL can be entered into a web browser to run the job. The job can also be run from within SAS Studio. When running a job from SAS Studio Viya Viya uses the SAS Studio Compute Context, when running a job from a URL Viya uses the Job Execution Compute Context.

Jobs can include task prompts(XML) or an HTML form to provide a user interface. When the user selects an option in the prompt and submits the job, the data that is specified in the form or task prompt is passed to the SAS session as global macro variables. The SAS program runs and the results are returned to the web browser. Sounds a lot like a stored process!

For more information on:

Job Definitions, Job Requests, Job Flows, etc see Viya Jobs object model
Authoring jobs in SAS Studio see SAS® Studio Developer’s Guide: Working with Jobs

Migrating Stored Processes to Jobs

Exporting

Stored processes can be exported from SAS 9.4 using the export wizard in SAS Management Console or the SAS 9.4 Platform Object Framework command-line tool ExportPackage.

In addition, SAS Content Assessment now provides tools that support the execution phase of a migration project. I introduced these tools in this post: Content Assessment delivers utility applications to support SAS Viya Content Migration

The createSASPackages application creates SAS packages (.spk files) for SAS objects that you intend to migrate from SAS 9 to SAS Viya. Currently, createSASPackages supports Data Integration Studio Jobs and Stored Processes.

In order to use createSASPackages, the SAS Content Assessment must be installed on the same machine as:

SAS 9.4 Foundation
SAS 9.4 Platform Object Framework

The createSASPackages application uses as input a CSV file downloaded from a SAS Content Assessment Report. After running the Content Assessment codeCheck application for your stored processes use the “SAS 9 Code Check for SAS Stored Processes” report, to download a CSV file of Compatible Stored Processes. This CSV filet can then be input to the createSASPackages application.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Run createSASPackages and input the CSV file. The createSASPackages application creates a SAS package for each object in the CSV. createSASPackages uses the settings established in the Content Assessment metaparms.sas to connect to the SAS 9.4 metadata server, and settings in the setenv.yaml(ASSESSMENT_PACKAGES_OUTPUT_LOCATION) to determine where packages are output.

createSASPackages.exe --source-file " Summary of compatible programs.csv"

If your Stored Process code is stored in physical files, in order to include the code, you must use a SAS Administrator identity associated with a host account that has access to all the stored processes and any physical code files (specify this account in Content Assessment metaparms.sas).

Importing

To import your stored processes, there are three options:

SAS Environment Manager Import Wizard
Transfer plug-in of the SAS Viya command-line interface(cli)
Content Assessment importSASPackages

In the first two methods, you can import a package file that contains many objects of different types. Using the Import Wizard in SAS Environment Manager, select your .spk file and navigate through the steps in the wizard.

Importing with the transfer plug-in of the sas-viya CLI is a multi-step process:

upload the package file to the Viya target environment
import the package

# upload to Viya and store id from the output
import_id=$(sas-viya  transfer upload --file "Finance-DataSupport.spk" | jq -r .id)
# import package with id 
sas-viya -q --output text transfer import --id "$import_id"

The importSASPackages application delivered with Content Assessment supports importing multiple package files at the same time. The application imports packages that include Stored Processes or DI Studio Jobs, it uses the transfer plug-in of the SAS Viya CLI to import the content of those files to SAS Viya. It is designed to be used with package files created by the createSASPackages application.

Requirements to use the importSASPackages application:

installed on the same machine as the SAS Viya CLI (minimum release 1.7.11)
the SAS Viya CLI must include at a minimum the 1.20.0 release of the transfer plug-in.
can access the Viya environment
can access the directory with the SAS Package files.

Authenticate to SAS Viya and run importSASPackages. Pass the application the location of the sas-viya CLI and the directory containing the packages.

importSASPackages.exe --sas-viya-path "D:\\ViyaHome\\CLI\\sas-viya.exe" --source-location "D:\workshop\SAS9ContentAssessment\assessment\SASPackages\Lev1\Foundation\gelcorp\financecontent\Reports"

What Happens?

A SAS 9 stored process is converted to a SAS Viya job definition. The job definition is created in the same folder and uses the same name as the source stored process. The Job Definitions can be run in SAS Studio or directly via a URL. The new job definition includes the:

Job metadata
SAS Code
Task Prompts.

Data

The data is not included in the promotion process. However, it must be made available to the Viya Compute server so that the code and any dynamic prompts can access it. There are two Viya server contexts that need to have access to the data:

Job Execution Context: used when running jobs via a URL
SAS Studio Context: used when running jobs from SAS Studio

To make the data available the compute server has to access it via a libname, and the table must exist in the library. One way to do is to use the Autoexec of the compute server. in SAS Viya 4 this is done using SAS Environment Manager. You can see how to do that in this post; Modifying Server Startup Configuration in Viya

Support for the import of Base libraries to Compute libraries is under development and will be available soon. Look for details in an upcoming post.

SAS Code

The SAS code that makes up the stored process is copied without modification to the Viya job definition. In many cases, the code will need to be edited so that it will run in the Viya environment. The SAS code in the definition can be modified using SAS Studio. Edit the code so that any:

paths to libnames, filenames, etc., in the new job point to the location of the data in Viya
other SAS 9 metadata-related code is removed from the stored process

As you test your new jobs in Viya there may be additional changes needed to get them running in Viya.

Prompts

There are two types of prompts in SAS Viya Task prompts(XML) and custom steps (JSON). Currently, prompts that were defined in metadata in the SAS 9 stored process are converted to the Task prompts and stored within the job definition. The XML for the prompt definition can be accessed and edited from SAS Studio or the SAS Job Execution Web App. For more information on working with task prompts in Viya see the Developers Guide to Writing Custom Tasks.

If you have shared prompts it is recommended that you select "Include dependent objects" when exporting from SAS 9.4. If you do not select this option any shared prompts will be omitted from the package. If the libnames and tables used in prompts are included in the package the SAS 9 library and table definitions referenced in dynamic prompts will be converted to use a library.table reference in Viya. When this happens the XML for the prompt will include the libname.tablename of the dataset used to populate the prompt values.

Issues encountered converting SAS 9 prompts to Viya SAS Prompt Interface will cause warning messages to be included at the beginning of the XML defining the prompt interface.

Running the Job

As I mentioned earlier you can run the job from SAS Studio or from the Job Execution Web Application. You can also run the job from a URL just like you could with Stored Processes in SAS 9.4. To get the URL for the job select the Job properties in SAS Studio, the URL is stored in the Job submit property.

I hope you found this post useful, here are some links to additional information and some related SAS 9.4 to Viya Migration content:

Find more articles from SAS Global Enablement and Learning here.

sridhar_m · ‎07-18-2022

Thanks a lot for sharing valuable stuff. Very helpful.