Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Home
- /
- Programming
- /
- Programming
- /
- Re: How to randomly select X no of obs randonly from a dataset?

Options

- RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

🔒 This topic is **solved** and **locked**.
Need further help from the community? Please
sign in and ask a **new** question.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Posted 12-26-2012 11:57 AM
(15448 views)

Hi,

I have a dataset of X no of observations and I want to select Y no of records** randomly**. I know that firstobs,obs OR ranuni can be used but wondering if there is any other better way to do it.Because firstobs,obs doesn't give true random sample also ranuni requires manual input of cuts like random < 0.2 etc.,. In the below example I want to select 5125 records randomly from dataset sample.

%Let obs = 5125;

Data abc;

Set sample(firstobs = 1 obs = &obs.);

Run;

Thanks in advance for your help!

1 ACCEPTED SOLUTION

Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

: I like John's approach but, from my limited tests, John's code appears to either introduce a bias of some kind or at least doesn't select the first N records based on the random seed used.

I don't recommend the 3rd approach, below, but included it to compare with the other two methods. The 2nd and 3rd methods appear to consistently provide the same results regardless of seed or sample size specified. The first method, though, appears to always deviate slightly:

%let ssize=10;

%let seed=5;

data bigdata;

set sashelp.class;

do _n_=1 to 10;

recnum+1;

randnum=ranuni(&seed.);

output;

end;

run;

data sample (drop = k);

k = &ssize. ; /* specify sample size required */

if 0 then set bigdata nobs = n ; /* get nobs, without reading anything */

do i = 1 to n while (k > 0) ;

if ranuni(&seed.) < k/n then do;

k = k-1;

set bigdata point = i ;

output ;

end ;

n=n-1 ;

end ;

stop ;

run ;

proc sql OUTOBS=&ssize. ;

create table sample2 as

select *

from bigdata

order by RANUNI(&seed.)

;

quit;

proc sort data=bigdata

out=sample3;

by randnum;

run;

data sample3;

set sample3;

if _n_ le &ssize.;

run;

9 REPLIES 9

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

If you have SAS/STAT, you may find proc surveyselect coming in handy:

for your case, if non-replacement :

proc surveyselect data=sample method=srs n=5125

out=abc;

run;

Haikuo

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

As was suggested, if you have PROC SURVEYSELECT available, that's what it was built for. But if not, there are a few ways to go about it (all of which require entering no more than the number 5125). Here are a few questions that would become important:

1. Is your data set large so that speed becomes important?

2. Do you need exactly 5,125 observations, or would approximately 5,125 be acceptable?

3. Can the same observation be selected more than once?

4. What should happen if the data set contains fewer than 5,125 observations?

Even if you need exactly 5,125 randomly selected unique observations, using one pass through the data, this can be done. The programming would involve one short but somewhat complex DATA step.

Good luck.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

If you don't have sas/stat, you can also do it quite easily with proc sql. e.g.:

proc sql OUTOBS=5125 ;

create table abc as

select A.*

from sample as A

order by RANUNI(0)

;

quit;

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

This is a common question. The most efficient method I have seen is done using a data step and uses the POINT= option on a set statement so that only the selected observations are read from the source dataset. For an example of this code look at this posting from John Whittington in 1998.

http://listserv.uga.edu/cgi-bin/wa?A2=ind9810A&L=sas-l&D=0&P=2569

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

: I like John's approach but, from my limited tests, John's code appears to either introduce a bias of some kind or at least doesn't select the first N records based on the random seed used.

I don't recommend the 3rd approach, below, but included it to compare with the other two methods. The 2nd and 3rd methods appear to consistently provide the same results regardless of seed or sample size specified. The first method, though, appears to always deviate slightly:

%let ssize=10;

%let seed=5;

data bigdata;

set sashelp.class;

do _n_=1 to 10;

recnum+1;

randnum=ranuni(&seed.);

output;

end;

run;

data sample (drop = k);

k = &ssize. ; /* specify sample size required */

if 0 then set bigdata nobs = n ; /* get nobs, without reading anything */

do i = 1 to n while (k > 0) ;

if ranuni(&seed.) < k/n then do;

k = k-1;

set bigdata point = i ;

output ;

end ;

n=n-1 ;

end ;

stop ;

run ;

proc sql OUTOBS=&ssize. ;

create table sample2 as

select *

from bigdata

order by RANUNI(&seed.)

;

quit;

proc sort data=bigdata

out=sample3;

by randnum;

run;

data sample3;

set sample3;

if _n_ le &ssize.;

run;

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

You will get a different list of observations using the K/N method versus the assign a random number and then sort method. But it has been proven mathematically that each observation will have the same probability of selection.

The big advantage is when N is large relative to K as the POINT= operation will only load the needed observations. The other methods require reading all of the observations off of the disk at least once. I have used datasets with millions of observations that take hours to process sequentially, but I can take a sample of a couple of thousand observations in seconds.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. **Registration is now open through August 30th**. Visit the SAS Hackathon homepage.

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.