topic Re: Select Observations at Random for Data Checking in SAS Programming

Select Observations at Random for Data Checking

_maldini_ — Wed, 03 Aug 2016 20:46:19 GMT

Is there a simple way to select a number of observations at random when doing data checks?

I have been using PROC PRINT w/ the FIRSTOBS and OBS options, but there must be a way to select a number of observations at random.

Thanks!

Re: Select Observations at Random for Data Checking

Doc_Duke — Wed, 03 Aug 2016 21:01:34 GMT

The approach that I have used is to create a new variable in the data set based on the RANUNI funciton and then use that to make a pseudo-random selection of a prorportion of the records for audit. I'm sure you could set up something similar in PROC SQL so you didn't have to do multiple passes of the data.

Re: Select Observations at Random for Data Checking

ballardw — Wed, 03 Aug 2016 21:58:08 GMT

If you want a random sample another way would be to use proc surveyselect. The parameter samprate is very easy to use to select a percentage of records or sampsize to select a specific number of records.

Depending on the number of records and what conditions may be involved for checking you might use code looking for specific things.

Suppose you have a variable that should never have a value greater than 10:

Proc sql;

select count(*)

from dataset

where variablex > 10;

quit;

Would tell you how many records have an invalid value.

Conditions could be multiple such as variablex>5 and missing(variabley) if variabley should have a value whenever variablex is greater than 5.

If you have multiple variables that should have the same range of non-missing values then custom formats may help. Suppose I have some variables that should only have values of 1, 2, 3, 4, 5 ( 1 to 5 scale) or 9 to indicate no opinion in a typical survey question.

Proc format;

value validscale

1, 2, 3, 4, 5,9='Valid'

other='Invalid'

;

run;

proc freq data=have;

tables <list the variables with that code>;

format <same variables> validscale. ;

run;

would give you tables with counts of invalid values.

Unless the data is extremely large then you are validating all records not just a sample for those variables.