DataOpsWeek - Testing automation

[#1]

Introduction

In January 2024, we held a #DataOpsWeek event with the aims of showcasing our work, learning from our colleagues and building the SAS DataOps community. Each day had a theme, based on Alexey Vodilin' s 5 pillars for a successful DataOps strategy blog. Our fourth theme of the week was "Testing Automation".

Testing automation

Testing automation has always been a crucial part of CICD, in essence the continuous integration (CI) part to ensure that what we develop continuous to work together after we made changes. 

Alexey says:

"It's great to have individual functional assets and orchestration pipelines in place, but it's not enough to reach a desired state of operationalization. Why? There are multiple reasons. These pipelines and assets are typically dependent on other systems, it's not always possible to foresee all exceptional use cases. Additionally, a change in one artifact can cause a failure of a larger pipeline where this artifact is used. So, it's essential to have a strategy of automated testing and a respective testing framework.

Such a framework should allow for automated test runs, provide the ability to store and share results in standard formats and provide notification and alerting capabilities."

As SAS is an open environment, such frameworks works well with SAS, and as seen below - SAS as a platform and language is well suited on its own to support testing automation. A simple PROC Compare provides an efficient way of comparing expected results in data to actual results in data, and the SASUnit framework (see below) inspired by the xunit family of testing frameworks provides a familiar approach to test automation among SW developers. Great Expectation - designed to facilitate quality assurance also provides efficient ways of comparing the outcome of your dataops processing to what you expect. We have custom steps available in our public github repository available that enables efficient use of great expectations in your project - see links below.   

Top testing automation resources

Testing frameworks

• SAS Software Development with the V-Model (old, but gold!) - Andrew Ratcliffe
• SASUnit download | SourceForge.net - A test framework inspired by the xunit type of testing frameworks. 

 Great Expectations

• Predefined rules demo - Katarzyna Zuk
• Generate and run expectations - Katarzyna Zuk 

Move to Viya

• Juletip #12 - Snowflake data falling – how to ensure they fall right - Lars Arne Skår

Environment validation

• Validating SAS Viya Environments Has Never Been this Easy - Michael Erickson
• ValidateViya on pyviyatools ('Python command-line tools that call the SAS Viya REST APIs - for SAS administrators') - Michael Erickson

Summary

We saw a lot of good work across the day. I'd pick out two highlights - firstly Andrew's classic V-Model article which stands up many years after it was first written and presented at the SAS Global Forum. Coming right up to date, we saw how SAS Studio integration with Great Expectations can be applied to testing automation, as well as data validation and monitoring.

You can find resources from our other DataOpsWeek themes here:

1. Collaboration
2. Orchestration
3. Continuous delivery
4. Testing automation (this blog)
5. Environmental management