topic Re: Data quality indicators report in SAS Data Management

Data quality indicators report

JinboZhao — Thu, 29 Jun 2017 10:59:35 GMT

Hi Guys,

I hope to get a data quality report which includes the following indicators:

var

mean

std

min

max

median

IQ_Range

n_low

n_low_percent

n_high

n_high_percent

n_far_low

n_far_low_percent

n_far_high

n_far_high_percent

null_rate

missing

missing_percent

Could anyone show me the respective code in SAS proc print, or proc freq, or other useful SAS procedure?

Many appreciation.

Re: Data quality indicators report

RW9 — Thu, 29 Jun 2017 11:00:51 GMT

Most of what you want can be gained from a proc means procedure:

http://support.sas.com/documentation/cdl/en/proc/70377/HTML/default/viewer.htm#p19dfq16fqt1t3n1eroiabnn6r3s.htm

http://www2.sas.com/proceedings/sugi29/240-29.pdf

Re: Data quality indicators report

JinboZhao — Thu, 29 Jun 2017 11:06:53 GMT

Thank you. I got most of them, but could not get the percentage ones. Could you help me?

Re: Data quality indicators report

RW9 — Thu, 29 Jun 2017 11:20:47 GMT

Without some test data (in the form of a datastep) I can only give generals. Percents are just count() / N, so you can do these in a datastep. You may need to proc freq your data to get counts, merge that on.

Re: Data quality indicators report

JinboZhao — Thu, 29 Jun 2017 11:32:44 GMT

Hi, my dataset is like this:

ID	OPENTIME	CLOSETIME	GENDER	GRADE	LOANS	FLAG
1	98	121	F	A	1200	Y
2	95	115	M	B	1300	Y
3	96	114	M	C	1500	N
4	99	120	F	D	1600	Y
5	98	107	F	E	1700	N

The following is the code I use:

proc means data=table n mean min max std q1 q3 qrange median nmiss ;
var _numeric_;
run;
proc freq data=table;
tables _character_;
run;

But the results for numeric variables do not include the percentages. Could you help me, as I want to get all those results in one report.

Re: Data quality indicators report

RW9 — Thu, 29 Jun 2017 12:06:27 GMT

Sorry, I don't have time to write a whole report for you. Use those procedures, then merge the required data together, and datastep to calculate any further numbers you need.

Re: Data quality indicators report

ballardw — Thu, 29 Jun 2017 14:18:01 GMT

One thing to consider for percentages is what is the numerator and denominator to be used. I don't believe you have specified that in any way clear enough. Likely the way will be to create the appropriate Sums in Proc means/summary and then in a data step calculate the percentages.

Or perhaps Proc Report or Tabulate using the data for a report will allow the percentage calculations.