## split dataset into n folds

Regular Contributor
Posts: 210

# split dataset into n folds

I would like to split a given dataset into n stratified equal sized-ish folds by amending it with an additional column containing n. What is a common/simple way to achieve this? Thanks.

Super User
Posts: 24,026

## Re: split dataset into n folds

Generally, it's not a good idea.

That being said, here's two write ups on it.

Regular Contributor
Posts: 210

## Re: split dataset into n folds

I came across these references when I googled. They do not seem to split the original dataset randomly but rather based of column values. I would like to split randomly never mind stratification.
Super User
Posts: 24,026

## Re: split dataset into n folds

PROC SURVEYSELECT then? Choose N samples of X data?

Regular Contributor
Posts: 210

## Re: split dataset into n folds

Thanks. I thought about using PROC SURVEYSELECT. Can you please make an example for 3 folds? source dataset: Have, output datasets: wants1, wants2, wants3? Thanks
Super User
Posts: 24,026

## Re: split dataset into n folds

It won't create multiple data sets but will do the random selection. Then you can use the methods above to split.

Or the manual way of adding a random numbers, sort by random number and use any of the methods in the link above.

Regular Contributor
Posts: 210

## Re: split dataset into n folds

That's fair enough but can you please show code that creates computed column with 1, 2 and 3 in it to indicate fold (see original question).
Super User
Posts: 24,026

## Re: split dataset into n folds

You should have enough information and samples here to write the sample code yourself or at minimum provide sample data

Super User
Posts: 13,942

## Re: split dataset into n folds

No guarantee about randomish result but

data want;

set have;

split = mod(_n_, 9);

run;

will add a variable that will split the data set into in 9 parts and the size difference will be plus/minus 1 between any groups. Replace 9 with your desired number.

If randomization critical then add a variable the result of a random number function, sort by that variable and then use the method above.

Regular Contributor
Posts: 210

## Re: split dataset into n folds

Thanks randomness is important
Super User
Posts: 10,850

## Re: split dataset into n folds

``````data have;
set sashelp.heart;
call streaminit(12345678);
random=rand('uniform');
run;

proc rank data=have out=temp groups=3;
var random;
ranks group;
run;
data want1 want2 want3;
set temp;
if group=0 then output want1;
else  if group=1 then output want2;
else output want3;
run;``````
Posts: 1,399

## Re: split dataset into n folds

Using call rantbl, with regular updating of table probabilities will allow a single-step solution, creating the new variable subgroup.

``````data want (drop=_:);
set have nobs=nrecs;

array needed{10} _temporary_;
array needprob{10} _temporary_;

if _n_=1 then do;
do _I=1 to dim(needed);
needed{_I}=floor(nrecs/dim(needed));
end;
do _I=1 to dim(needed) while (sum(of needed{*})<nrecs);
needed{_I}=needed{_I}+1;
end;
end;

_nleft = nrecs-(_n_-1);
do _I=1 to dim(needed);
needprob{_I}=needed{_I}/_nleft;
end;

seed=1250666;
call rantbl(_seed,of needprob{*},subgroup);

needed{subgroup}=needed{subgroup}-1;
run;
``````

Notes:

1. Changing the dimension of arrays NEEDED and NEEDPROB is all that's required to change the number of randomly populated subgroups.
2. NEEDED tracks, for each subgroup, the number of observations yet to be added.  It's dynamically updated with every incoming observations.   The minimum and maximum starting values for NEEDED will differ by no more than one, and will start out summing to NRECS.
3. NEEDPROB array is required by the CALL RANTBL routine.  It uses elements of NEEDED divided by the number of observations remaining to be assigned.
Discussion stats
• 11 replies
• 312 views
• 0 likes
• 5 in conversation