SAS 9 Content Assessment: the Profile Report for SAS Data Integration Studio Jobs (Update)

In this continuation of the series of posts on the SAS 9 Content Assessment reports, we look at the SAS 9 Profile for SAS Data Integration Jobs report and review what information it provides, what conclusions can be drawn from it, and what likely actions you would take after reviewing it.

As a reminder, this series of posts does not cover how to run SAS Content Assessment applications; instead, it focuses on using the results. For information about running the SAS Content Assessment, check out this post and this demo.

SAS Content Assessment Profile Application

The SAS Content Assessment Profile application provides detailed information on the attributes of SAS 9 content. Profile delivers a series of reports for each SAS 9.4 content type. The reports include details like:

Tasks used in Enterprise Guide projects
Transformations used in Data Integration Studio Jobs
SAS Code locations and use of prompts in Stored Processes
Encoding of SAS Data

The Content Assessment Profile application and reports help you to understand the details of SAS 9 Content and how those details impact migration to SAS Viya.

The SAS Profile for Data Integration Jobs report is the main report covering SAS Data Integration Studio Jobs.

SAS 9 Data Integration Studio jobs consist of transformations (the nodes in the jobs). Jobs migrate to SAS Studio flows, which include steps (the nodes in the SAS Studio flows).

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

The migration process will attempt to migrate a transformation to an equivalent step or steps in Studio. If a transformation cannot be migrated to equivalent step(s), it is often migrated to a SAS Studio Program step that includes SAS code to perform the function of the transformation. A transformation may be prevented from migrating to an equivalent step because there is no matching step in Studio, or some attribute of the transformation is not yet supported. The SAS 9 Profile for SAS Data Integration report helps you to understand what may happen to the SAS Data Integration Studio Jobs and their transformations during the migration process from SAS 9 to SAS Viya.

What can we learn?

The SAS Data Integration Studio report will analyze each SAS Data Integration Studio job and classify each job and each transformation within a job.

The Overview tab in the report summarizes the status of all the Jobs and Transformations in the deployment. In this deployment, there are 27 jobs with 53 Transformations. Of the 27 jobs, 24 are “Migration Candidates.” A migration candidate is a SAS object believed to have a high probability of success when migrating to SAS Viya. For SAS Jobs, there are varying degrees of success based on whether there is migration support for some or all of the transformations used in the job. The Overview tab also provides details of when Jobs and Transformations were created and last updated.

The Jobs tab provides details about the characteristics of the jobs in the environment, including:

transformations per job
whether jobs have pre-code and post-code
whether the jobs have been deployed for scheduling

The Transform Usage tab and the Jobs Using Transforms tab both focus on how many of each type of transformation is used and where they are used. The Transform Usage tab provides counts of the SAS Data Integration Studio transforms used in the jobs and whether those transformations have user-written code. Jobs Using Transforms lists the transformations for each job and whether they have user-written code.

The Data Usage tab documents the data used by the SAS Data Integration Jobs. Data is classified as:

Persisted Tables: library-based data sets that persist after the SAS session closes
Transient Tables: tables that typically use the SAS Work library are deleted when the SAS session closes
External Files: files that are read from the file system.

You can identify what tables are used in each job and whether they are source or target tables.

Transformation Migration Details is a key tab on the report. This page helps you understand what will happen to the transformations within a Job when the job migrates to a SAS Studio Flow. The report classifies transformation as Ideal Fit, Partial Code Fit, No Fit, or Undetermined Fit. A transformation is considered:

Ideal Fit: The migration of the transform is automatic. All features or functionality used are maintained.
Partial Fit: The transform's migration is automatic. All features or functionality used are maintained. After migration, an attribute might need to be set, or a relationship to another object might need to be verified.
Code Fit: The transform's migration is automatic. Most features or functionality are maintained using code in a Studio Program Step.
No Fit: The transform cannot be migrated, and there is no planned feature replacement for it in SAS Viya.
Undetermined Fit: if the transformation was unable to be profiled or evaluated for migration status.

Transformations within a Data Integration job that are profiled will fall into one of the categories mentioned above.

The Transformation Migration Details page allows you to explore the details of the transformation migration. It also allows you to select individual jobs and explore what will happen to the specific transformation in them. Let us look at a couple of jobs.

Ideal Fit Job: 01_Split_Finance_Data

On the Transformation Migration Details tab, we have selected 01_Split_Finance_Data (the job shown in the image at the start of this post). All of the transformations in the job migrate to an equivalent step in SAS Studio. The report shows the:

File Writer Transformations will migrate to an Export step in SAS Studio
Extract Transformation will migrate to a Query step
Splitter Transformation will migrate to a Branch Rows step.

This job is ready to migrate to SAS Viya and SAS. Studio.

Partial Fit: Supplier Load

On the Transformation Migration Details tab, we have selected Supplier Load. This job has two transformations one is ideal fit and one is partial fit. The partial fit transform is a SCD Type 2 loader in SAS Data Integration Studio. While it will migrate to an equivalent step in SAS Studio, as the conversion message indicates it may need some modifications to get it running.

Instead of approaching what will happen from the job perspective, Transform Migration Conversion Messages gives us a transformation-based view. The table on the left of the page shows the most frequently used SAS 9 transformations in the SAS 9 environment and the Steps they become in SAS Viya. In this report, the File Reader was the most frequently used transformation, and we can see that it will be migrated to a SAS Program Step. The table to the right shows the most common conversion method for these transformations.

Another key piece of information from this tab is the SAS Viya Studio licenses required for the equivalent steps in SAS Viya.

The Job Migration tab is where we can take the next step in the migration journey. Jobs are classified as migration candidates based on the percentage of transformations in the job that are an ideal fit.

The Job listing in this tab can be downloaded to a CSV file that will drive the migration process.

The CSV file is used as an input to the SAS Content Assessment CreateSASPackages application, which will automate the export of the SAS Data Integration Studio jobs that you select for migration. You can use the slider to investigate the impact of lowering the threshold for the "percentage of transforms with ideal fit". When you adjust the slider, Jobs higher than the ideal fit threshold are color-coded blue in the table listing. The buttons can be used to filter the jobs in the table, and the checkboxes can be used to select group(s) of jobs to export for package creation.

What do we do with what we learned?

The SAS 9 Profile for Data Integration report offers a detailed analysis of our SAS 9 Data Integration environment, highlighting job characteristics, transformation usage, and migration readiness. What can we do with this information?

Identify License Needed in SAS Viya

SAS Studio is currently available with three licenses. SAS Studio, SAS Studio Analyst, and SAS Studio Engineer. By documenting the license required for the transformations, the report will identify what level of SAS Studio is needed to run the resulting SAS Studio flows.

Prioritize Jobs for Migration

Using the report, you can prioritize the migration of jobs in the environment that are classified as migration candidates. These jobs should migrate easily, with minimal adjustments, and take full advantage of equivalent steps in SAS Studio. The reports also identify which jobs are being used and the frequency with which the jobs are changing. This information is used with other Content Assessment reporting and can help to develop a strategic plan for the migration of Data Integration jobs.

Remediate Partial Fit Transformations

The report identifies transformations within jobs classified as "Partial Fit." These transformations require targeted remediation. The report’s conversion messages help identify specific issues. Remediation can happen in the source environment or can be addressed in SAS Viya after the migration.

Evaluate No Fit and Undetermined Fit Transformations

Transformations labeled as No Fit or Undetermined Fit might require redesign or alternative implementation strategies in SAS Viya. These transformations should be reviewed to determine whether functionality can be replicated using supported features in SAS Viya or if custom development is needed.

Automate Job Export with CSV Integration

The Job Migration tab allows export of job listings to a CSV file. The CSV file is used as input to the createSASpackages application. This enables automated packaging of selected SAS Data Integration Studio jobs, streamlining the migration process and reducing manual effort.

Identify Key Data and its Location

The Data Usage section of the Data Integration Studio report reveals which datasets are actively used across jobs, including persisted tables, transient (temporary) tables, and external files. This enables us to identify critical data assets for migration. We can plan for the migration of the data so that it is available in SAS Viya for the SAS Studio Flows that are created during the migration. The report can also be used to flag unused or redundant data for potential cleanup.

Wrap Up

The SAS 9 Profile for SAS Data Integration report helps assess the migration status of  SAS Data Integration Studio jobs. It analyzes each job and its transformations, identifying whether the jobs are candidates for migration to SAS Studio flows. The report classifies transformations into categories (Ideal Fit, Partial Fit, Code Fit, No Fit, or Undetermined Fit) and provides detailed insights about job characteristics, data usage, and transformation frequency. It highlights licensing needs for SAS Studio and supports migration planning by generating a list of jobs ready for migration.

The report helps to plan and execute the migration of SAS Data Integration Studio jobs to SAS Studio flows on SAS Viya. It helps identify the appropriate SAS Studio license needed based on transformation requirements, it prioritizes jobs that are ready for migration, and it flags jobs needing remediation or redesign. The information in the report can be used to identify the usage of jobs, which can lead to cleaning up jobs before a migration.

The report supports creating a CSV file of candidate jobs that can be used with the createSASpackages application.

Here are some related resources, including the other posts in this series:

Find more articles from SAS Global Enablement and Learning here.