BookmarkSubscribeRSS Feed

DataOpsWeek - Testing automation

Started ‎03-01-2024 by
Modified ‎03-01-2024 by
Views 215

Introduction

 

In January 2024, we held a #DataOpsWeek event with the aims of showcasing our work, learning from our colleagues and building the SAS DataOps community. Each day had a theme, based on Alexey Vodilin' s 5 pillars for a successful DataOps strategy blog. Our fourth theme of the week was "Testing Automation".

 

Testing Automation.png

 

Testing automation

 

Testing automation has always been a crucial part of CICD, in essence the continuous integration (CI) part to ensure that what we develop continuous to work together after we made changes. 

Alexey says:

"It's great to have individual functional assets and orchestration pipelines in place, but it's not enough to reach a desired state of operationalization. Why? There are multiple reasons. These pipelines and assets are typically dependent on other systems, it's not always possible to foresee all exceptional use cases. Additionally, a change in one artifact can cause a failure of a larger pipeline where this artifact is used. So, it's essential to have a strategy of automated testing and a respective testing framework.

 

Such a framework should allow for automated test runs, provide the ability to store and share results in standard formats and provide notification and alerting capabilities."

 

As SAS is an open environment, such frameworks works well with SAS, and as seen below - SAS as a platform and language is well suited on its own to support testing automation. A simple PROC Compare provides an efficient way of comparing expected results in data to actual results in data, and the SASUnit framework (see below) inspired by the xunit family of testing frameworks provides a familiar approach to test automation among SW developers. Great Expectation - designed to facilitate quality assurance also provides efficient ways of comparing the outcome of your dataops processing to what you expect. We have custom steps available in our public github repository available that enables efficient use of great expectations in your project - see links below.   

 

Top testing automation resources

 

Testing frameworks

 Great Expectations

Move to Viya

Environment validation

 

Summary

 

We saw a lot of good work across the day. I'd pick out two highlights - firstly Andrew's classic V-Model article which stands up many years after it was first written and presented at the SAS Global Forum. Coming right up to date, we saw how SAS Studio integration with Great Expectations can be applied to testing automation, as well as data validation and monitoring.

 

You can find resources from our other DataOpsWeek themes here:

  1. Collaboration
  2. Orchestration
  3. Continuous delivery
  4. Testing automation (this blog)
  5. Environmental management
Version history
Last update:
‎03-01-2024 09:46 AM
Updated by:

sas-innovate-2024.png

Don't miss out on SAS Innovate - Register now for the FREE Livestream!

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

 

Register now!

Free course: Data Literacy Essentials

Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning  and boost your career prospects.

Get Started

Article Tags