One of the important steps in working with data is to ensure that the data is correct and follows the specific business rules that you need. And it’s always good to know how difficult it is to define those rules and to ensure that the data is in line with them. While I’ve been working on some Data Quality related cases, I’ve heard a lot about the Great Expectations package, so I decided to give it a try and just see what it can do for us. And how does this apply to SAS Viya? Let’s see.
I started with our great Custom Steps GitHub repository, just to see what’s already been done with Great Expectations – quite a lot! We’ll start with something nice and simple – do you need to validate your data in terms of min/max values, matching patterns, missing values, etc.? Then Great Expectations - Execute Business Rules custom step is for you. I’ve recorded a really simple video to show you how to use this custom step and what we can get using it:
