Early Picture of the Data Quality Status of Your Analysis Data? SAS® Visual Analytics Shows You How

Authors

Gerhard Svolba, SAS Institute Inc. Austria

Paper SAS1440-2015 - SAS Global Forum in Dallas, TX

Abstract

When you are analyzing your data and building your models, you often find out that the data cannot be
used in the intended way. Systematic patterns, incomplete data, and inconsistencies from a business
point of view are often the reason. You wish you could get a complete picture of the quality status of your
data much earlier in the analytic lifecycle. SAS® analytics tools like SAS® Visual Analytics help you to
profile and visualize the quality status of your data in an easy and powerful way. In this session, you learn
advanced methods for analytic data quality profiling. You see case studies based on real-life data, where
we look at time series data from a bird’s-eye view and interactively profile GPS trackpoint data from a sail
race.

Watch the presentation

This SAS Global Forum paper has been published as a number of presentations of in my Data Preparation for Data Science webinar series.

Missing Values in Machine Learning: Why my old aunt Susanne gives data scientists a hard time

Detecting Systematic Missing Values in Cross-Sectional Data

Detecting and Treating Missing Values in Longitudinal data

Profiling the Missing Value Structure of Your Timeseries Data

Verifying Data Quality Using Interactive Data Analysis of GPS-Data of a Sail Race

Simulation Studies to Quantify the Effect of Data Quality on Model Accuracy

DOWNLOAD THE FULL PAPER

https://support.sas.com/resources/papers/proceedings15/SAS1440-2015.pdf

DOWNLOAD THE SLIDES

Navigate to https://github.com/gerhard1050/DataScience-Presentations-By-Gerhard and download presentation #118.

SAMPLE GRAPHS

CONCLUSION

It is important to differentiate between regular data quality and data quality for analytics. Analytic methods
have additional requirements on data quality. But they also offer methods to profile and improve data
quality. SAS analytic procedures and SAS Visual Analytics offer a rich set of methods to get insight into
the quality status of your data.
SAS macros and SAS sample programs help you to profile your data in a very powerful way. It is very
important to find out whether there are systematic patterns in your data, such as the systematic patterns
that can occur in the case of missing values. Simulation studies give important insight into the
consequences of using poor data quality for predictive analytics.
SAS® Visual Analytics offers powerful methods to interactively profile your data. It allows you to get
closer to the data values to find out where inconsistencies or strange value combinations occur.

References

Svolba, Gerhard, 2021. Have a look at your TIMESERIES data from a bird's-eye view - Profile their missing value structure, SAS Communities

Svolba, Gerhard, 2021. Replace MISSING VALUES in TIMESERIES DATA using PROC EXPAND and PROC TIMESERIES, SAS Communities

Svolba, Gerhard, 2021. Using the TIMESERIES procedure to check the continuity of your timeseries data, SAS Communities

Svolba, Gerhard, 2021. The structure of MISSING VALUES in your data - get a clearer picture with the %MV_PROFILING macro, SAS Communities

Svolba, Gerhard. 2007. Data Preparation for Analytics Using SAS®. Cary, NC: SAS Institute Inc.
Svolba, Gerhard. 2012. Data Quality for Analytics Using SAS®. Cary, NC: SAS Institute Inc.
Svolba, Gerhard. 2013. “Is your data ready for analytics?” IT Briefcase.

Acknowledgements

Many people have helped and inspired me to write and to complete this paper: Udo Sglavo, Sascha
Schubert, Mike Gilliland, Anne Milley, Rainer Sternecker, Diane Hatcher, Anette Almer, Anne Baxter, and
Robin Langford.