topic Re: How to create random samples from dataset? in Statistical Procedures

How to create random samples from dataset?

lavernal — Thu, 07 Feb 2013 07:19:06 GMT

Hi,

I am trying to perform oversampling on my dataset (~200,000 observations), which consist of a flag variable of value 1 or 0. I want 100 samples with each sample to contain all the observations with flag=1 (~100 of them in total) and then randomly select ~750 of observations that have flag = 0. However, I seem to have some difficulty in getting what I want. I ended up with ~10,000 observations in total for each sample. And sometimes, my code takes forever to run. Can someone advice me on what is wrong?

My code is as follows:

data oversamples;

do sample=1 to 100;

set fun;

do i = 1 to _N_;

if flag= 1 or (flag=0 and ranuni(sample+7320) < 0.003) then output;

end;

run;

Thanks for any advice.

Re: How to create random samples from dataset?

VD — Thu, 07 Feb 2013 11:02:38 GMT

I think your set fun statement should be before the do loop.

Re: How to create random samples from dataset?

akberali67 — Thu, 07 Feb 2013 11:27:39 GMT

Why dont you try PROC SURVEYSELECT, it picks random samples.

proc surveyselect data=data_set

method=srs n=100 out=data_random;

run;

Re: How to create random samples from dataset?

lavernal — Thu, 07 Feb 2013 11:41:08 GMT

Hi akberali,

My output is now really random, I want it to include all elements with flag=1 and then randomly select ~800 elements with flag=0. Is it possible to create that using the proc surveyselect?

Thanks for the advice

Re: How to create random samples from dataset?

Rick_SAS — Thu, 07 Feb 2013 13:11:56 GMT

Use the STRATA option and set the sample rate for the "rare event category" to be 100%. See http://www.nesug.org/proceedings/nesug07/sa/sa02.pdf, beginning at the bottom of page 2.