Migration- SAS 9.4 to SAS Viya - Spk file export

[Somu]

Hello Team, Need your immediate assistance.

We are currently in the process of migrating from SAS 9.4 to SAS Viya 4.0 and are evaluating the best approach to move our existing SAS content.

As part of our initial migration activities, we ran the SAS Viya Content Assessment and found the following:

• Approximately 24,000 .sas programs
  Around 400 Enterprise Guide (.egp) projects
  Nearly 160,000 datasets

Our primary focus for the migration is bringing the .sas programs into the SAS Viya environment, since most business logic resides in these programs.

Current Migration Approach Being Evaluated
Our plan is to use SAS Management Console (SMC) in SAS 9.4 to:
Export content using Metadata Packages (.spk files)
Extract the .sas programs from the SPK
Move the extracted code to the SAS Viya environment manually
(The two platforms are in separate environments and cannot directly communicate.)

During our research, we discovered the following important limitation:
.sas programs that reside as files on the SAS 9.4 server filesystem cannot be included while exporting the spk files.

Is the understanding correct that SMC/SPK exports cannot be used to migrate filesystem-based .sas programs?
For large-scale migrations (tens of thousands of .sas files), is the recommended approach to:

Directly copy .sas files from the SAS 9.4 filesystem? or are there any other tools that is recommended.

Appreciate your response.

[AhmedAl_Attar]

Hi @Somu 

Disclaimer: I have not been involved in such migration before, but what I know

- The SAS Contents Assessment tool has an option to examine all your existing SAS Programs (*.sas) and identify issues related to hard coded paths and other stuff that may not be applicable to your new Viya Platform. 

- Other consideration for your SAS programs would be to determine whether or not to CAS enable them, and which part?

- Your SAS data sets will need to have utf-8 encoding, and this may involve converting them if they have been using another encoding such as Latine1 or something else.

- Your current SAS Formats catalogs, would need to be rebuilt using utf-8 encoding, which may involve reverse engineering if these formats don't have associated scripts/programs to re-create them.

- 

Long Story Short -- Migration is a lengthy process and needs to be planned and phased out on several stages/phases. Such decisions should pay attention to business needs and priorities, resource availability and skills sets, plus many other factors.

Try to watch these recorded webinars to see what others had gone through in their migration journey

• https://communities.sas.com/t5/Ask-the-Expert/A-Modernization-Success-Story-How-One-Bank-s-Move-to-t...
• https://communities.sas.com/t5/SAS-Innovate-Presentations/A-Case-Story-of-Successfully-Modernizing-T...
• https://communities.sas.com/t5/Ask-the-Expert/Moving-to-SAS-Viya-From-SAS-9-Q-amp-A-Slides-and-On-De...
•  

Hope this helps

[Somu]

Thank you, Ahmed, for your response. The documentation shared as part of the webinar was very helpful in understanding the overall migration process.

However, my question was specifically related to the migration of standalone .sas programs that currently exist on the SAS server and are not packaged or embedded within any stored processes. Based on further clarification, I understand that SPK does not consider standalone  .sas files present on the server for migration. Only .sas code that is embedded within a Stored Process is included as part of the SPK package.

I wanted to understand if there are any recommended approaches or best practices to handle such standalone .sas programs during migration.

In our case, the two systems SAS 9.4 and SAS Viya are hosted in completely separate environments and do not have any direct connectivity between them. 

[SASKiwi]

We are going through a similar SAS upgrade and migration project ourselves currently. Ours is SAS 9.4M2 to SAS 9.4M8 however.

In my experience it is essential to have a full-time network connection between your source and target SAS installations so you can do incremental SAS content updates as well as being able to do parallel runs to confirm and source and target processes produce the same results.

We use SAS/CONNECT and SAS procedures like SAS DATASETS for SAS data migration as it can handle encoding differences. Third party tools can't do this.

We also use our version control tools for migrating SAS programs, namely MS Visual Studio and MS DevOps. We have automated pipelines in DevOps deploying to both source and target SAS installation file systems. This ensures that we keep code bases in sync and ensure we can track any necessary changes required to run the same code on both source and target installations. Given you have 24K programs I suspect there will be an extended period of parallel running while you complete and test your migration. Use of version control tools is the best way of managing this.

I highly recommend trying to keep your source and target installation code as similar as possible while migrating so you can easily test and confirm it produces the same results. Once your migration is complete, then you can start to make greater use of Viya functionality.

In my experience SPK exports should be just about the existing SAS 9.4 metadata migration. It isn't designed for moving 24K SAS programs! 

[Somu]

Really appreciate your response @SASKiwi 

In our case, the two systems SAS 9.4 and SAS Viya are hosted in completely separate environments and do not have any direct connectivity between them. Given this constraint, I wanted to understand whether SAS recommends any specific approach or best practices for migrating a large volume of standalone  .sas programs (approximately 24,000 files) that reside on the SAS 9.4 server.

Since these programs are not embedded within Stored Processes and SPK does not account for standalone .sas files, I am trying to understand the most efficient and supported way to handle their migration under such environmental limitations.

Any guidance or recommendations approach on this scenario would be greatly appreciated.

[SASKiwi]

@Somu  - My best practice recommendations are as I've stated above. IMHO not having direct connectivity between your source and target installations is problematic and seriously restricts your migration options. How will you transfer programs and data between your installations?

[Somu]

Thank you for the response and for taking the time to address my queries!
Wanted to share what we have in place so far.

Migrating standalone .sas files

We built a shell script that picks up .sas files from a specified folder and consolidates them into a single directory. We have already migrated the .sas files to the current environment. Going forward, we plan to handle the remaining migrations in batches, prioritizing based on business needs.

Handling code changes:

For updates to things like libname statements, include statements, datasets, and proc sql blocks, we put together a Python script that takes care of the necessary modifications to make the code compatible with our SAS Viya environment.

[SASKiwi]

@Somu - Thanks for sharing more details of your migration practices. Of course there are many ways of dealing with this depending on your business requirements and IT infrastructure.

In our case it is essential to be able to run the same applications in both our source and target installations in parallel and to check that we get the same results, right down to key datasets being identical or not materially different. That's why we have near real time data transfers and code synchronization.  

[LinusH]

.sas programs are just flat files, so from a movement perspective, it's "just" an copy operation of choice.
The tricky part is to make them compatible with your Viya environment. Most likely it comes down search paths to file and libraries. If you are migrating from 9.4 on Linux it is potentially easier than from a Windows Server. See Content Assessment Code Check for more details.
When it comes to EG, you may need to migrate, since EG is a supported client for Viya. If you migrate EG to SAS Studio you will lose some functionality, Content Assessment Profile should be able to tell you what.