SAS DM platform in a heavely devops inspired environment

[HrZiller]

hi there,

back in 2017 I posted a similar question: Change management with SAS platform - DI, metadata - any experience with "Continues Delivery"??  - but unfortunately didn't get any responses 😞

I am trying again now - hoping for change of times has created miracles... 😀

At our customer site we are increasingly moving towards a very devops style of working. Most of the other teams developing applikations with more code based technologies are using Github, Jenkins and everything related. Our SAS DI team is the only not using these tools.

Our environment: 

5 environments: Dev, Test, 2x Preprod and Prod

> 900 DI jobs, >1500 tables, many user written transformations, macros etc. - fairly large environment

we have potentiel of 15 developers using DI Studio - with check-out/in enabled

I have seen the latest material available regarding Github:

- Bogdant blog post: https://communities.sas.com/t5/SAS-Communities-Library/DevOps-Applied-to-SAS-9-SAS-DI-jobs-the-Git-P... 

- https://communities.sas.com/t5/SAS-Communities-Library/SAS-DI-Developers-Unite-The-new-GIT-plug-in-i...

and a number of the more SAS code supported entries

But.... neither of these seems to head in the direction of the terms branching and merging. Bogdant's post is very weel describing the proces of automating the moving/testing the DI job/code over what is called "automation valley" in the post.

We are seeing many customers with these requirements in Data Warehousing projects - and I am hoping that some of your projects might have looked into solutions to this 😀 - please let me know how you have looked at solutions for this challenge.

@ChrisHemedinger - would you by any chance know of any customer stories regarding this area? Thanks!

Thanks in advance

Jan Zilstorff

Knowit

[ChrisHemedinger]

Hi @HrZiller,

2017? Perhaps you were ahead of your time!

I've had the opportunity to speak with many SAS customers who are looking at Git-based workflows as part of their data ops and model ops, as well as some who have well-established Git-based processes in some areas but who are looking to bring SAS processes "into the fold."

When it comes to specific advice around project organization, branching, and such tactics -- I have not seen a single dominant pattern emerge.  Approaches vary by team and project, and since Git and adjacent devops tools are so flexible, there are many possibilities. The key seems to be in careful planning, communication, and buy-in from all stakeholders to adhere to/adopt a process.

I recently shared an article with some guidelines -- but not a prescriptive methodology. However, other users might be willing to share their specific approach and what works for them (and maybe their own experience with shortcomings, decisions they wish they had made differently).

[HrZiller]

hi Chris,
thanks for fast response! I do realize that this area probably will be very customer specific. With my many years in SAS and working with SAS and our customers, I have seen yes lots of approaches to promotion and handling "artifacts".
The recents years high focus on Continues Development/Integration seems to be pushing the use of point&click coding tools like DI in a direction with "the fit seems is a bit too narrow".
I have discussed versioning / metadata with Nancy/Stephen a number of times over the years - and can see the challenges ahead: keeping the right metadata available with the right versions. I would imagine similar challenges with Informatica (I believe their datajobs also are somewhat seudo code - XML/markup language ).

would be great to hear from a customer that has manage to find a way to work with branching/merging - AND use DI for creating the code.

Again thanks for your reply
regards
Jan