Is there a simple way to select a number of observations at random when doing data checks?
I have been using PROC PRINT w/ the FIRSTOBS and OBS options, but there must be a way to select a number of observations at random.
Thanks!
If you want a random sample another way would be to use proc surveyselect. The parameter samprate is very easy to use to select a percentage of records or sampsize to select a specific number of records.
Depending on the number of records and what conditions may be involved for checking you might use code looking for specific things.
Suppose you have a variable that should never have a value greater than 10:
Proc sql;
select count(*)
from dataset
where variablex > 10;
quit;
Would tell you how many records have an invalid value.
Conditions could be multiple such as variablex>5 and missing(variabley) if variabley should have a value whenever variablex is greater than 5.
If you have multiple variables that should have the same range of non-missing values then custom formats may help. Suppose I have some variables that should only have values of 1, 2, 3, 4, 5 ( 1 to 5 scale) or 9 to indicate no opinion in a typical survey question.
Proc format;
value validscale
1, 2, 3, 4, 5,9='Valid'
other='Invalid'
;
run;
proc freq data=have;
tables <list the variables with that code>;
format <same variables> validscale. ;
run;
would give you tables with counts of invalid values.
Unless the data is extremely large then you are validating all records not just a sample for those variables.
The approach that I have used is to create a new variable in the data set based on the RANUNI funciton and then use that to make a pseudo-random selection of a prorportion of the records for audit. I'm sure you could set up something similar in PROC SQL so you didn't have to do multiple passes of the data.
If you want a random sample another way would be to use proc surveyselect. The parameter samprate is very easy to use to select a percentage of records or sampsize to select a specific number of records.
Depending on the number of records and what conditions may be involved for checking you might use code looking for specific things.
Suppose you have a variable that should never have a value greater than 10:
Proc sql;
select count(*)
from dataset
where variablex > 10;
quit;
Would tell you how many records have an invalid value.
Conditions could be multiple such as variablex>5 and missing(variabley) if variabley should have a value whenever variablex is greater than 5.
If you have multiple variables that should have the same range of non-missing values then custom formats may help. Suppose I have some variables that should only have values of 1, 2, 3, 4, 5 ( 1 to 5 scale) or 9 to indicate no opinion in a typical survey question.
Proc format;
value validscale
1, 2, 3, 4, 5,9='Valid'
other='Invalid'
;
run;
proc freq data=have;
tables <list the variables with that code>;
format <same variables> validscale. ;
run;
would give you tables with counts of invalid values.
Unless the data is extremely large then you are validating all records not just a sample for those variables.
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.