topic Re: Data Quality Report in SAS in SAS Programming

Data Quality Report in SAS

AZIQ1 — Wed, 26 Apr 2023 00:46:42 GMT

Hi,

I want to run a report that gives me all the categories for all columns with percentages in one go. Like a single report. My tables could have >100 categorical (character) value fields. Issue is that it is taking very long and does not have enough memory to run for all fields - any ideas how to handle it

Have

Id	Gender	State	Grade
1	Male	IN	8
2	Female	MO	8
3	Female	TX
4	Null	TX
5	Null		6
6	Null	TX	7

Want a single output

Gender	n	%
M	1	17%
F	2	33%
Null	3	50%

State
IN	1	17%
MO	1	17%
TX	3	50%
Blank	1	17%

Grade
6	1	17%
7	1	17%
8	2	33%
Blank	2	33%

Re: Data Quality Report in SAS

rudfaden — Wed, 26 Apr 2023 10:30:40 GMT

Can you show your current solution?

Re: Data Quality Report in SAS

AhmedAl_Attar — Wed, 26 Apr 2023 11:18:09 GMT

Hi @AZIQ1

If you have access to SAS Enterprise Guide or SAS Studio, you can use the "Characterize Data" task to analyze your table and produce similar results to your desired output.

Hope this helps

Re: Data Quality Report in SAS

Ksharp — Wed, 26 Apr 2023 11:50:57 GMT

ods select none;
ods noresults;
ods output OneWayFreqs=OneWayFreqs;
proc freq data=sashelp.class ;
table _all_/ list;
run;
ods select all;

data temp;
set OneWayFreqs(drop=Table );
value=coalescec(of _character_);
keep value;
run;
data want;
 merge OneWayFreqs(keep=Table Frequency Percent) temp;
run;

Re: Data Quality Report in SAS

rudfaden — Wed, 26 Apr 2023 12:42:48 GMT

Maybe look into using proc report or tabulate instead. If that does not work, I would loop over the variable.

Re: Data Quality Report in SAS

ballardw — Wed, 26 Apr 2023 16:35:51 GMT

Suggest: Provide a bit better description of your data. You can do that with Proc Freq and NLEVELS to report on the number of levels of each variable. Example:

ods select nlevels;
proc freq data=sashelp.class nlevels;
run;

The ODS select means that only the Nlevels table is generated which should not run out of memory but will give us an idea just how much stuff you are dealing with.

"Blank" is not a SAS value you will see. Depending on which procedure/approach you are dealing with you can use the MISSING option to get a count.

You can run into memory issues when generated large tables because the table with all the border information and such are stored in memory before writing out the results. If you have the default HTML destination and are sending the data to another ods destination that can seriously increase the memory needed. So you might have better results by closing the HTML destination and writing to only one destination like ODS RTF of PDF.

One basic syntax would look like:

proc tabulate data=yourSetNameHere;
   class _character_/missing;
   table _character_, n pctn;
run;

but missing values will be at the top of each as the default order for character values is alphabetic and blanks come before any other printable character.