random selection without repetition, but evenly distributed across users

[Anna_NZ]

Hi, 

I am looking at a data set of teachers who are combined teaching 129 different courses across  the internet. 

In an effort to guarantee the quality of the courses, the 12 teachers should be graded. 

Each month, 10 % of the courses offered should be randomly be chosen and graded, whereby the teachers teaching these courses should be evenly distributed over the sample taken. Additionally, the courses that were sampled last month, should not be sampled again. 

Proc Surveyselect data = teachers_&end_current-mth

out teacherevaluation_&end_current-mth

method = SRS

seed= 12345678

Samplerate= 0.10;

run;

This code, gives me 10% randomly chosen samples of teachercourses to be sampled -  without repetition. So far so good, but some teachers are more often sampled than others.

What condition can I add to this code to make sure, the samples randomly chosen are evenly distributed across all teachers? 

Your help is much appreciated. 

Many thanks, Anna

[Reeza]

---

@Anna_NZ wrote:

What condition can I add to this code to make sure, the samples randomly chosen are evenly distributed across all teachers? 

Your help is much appreciated. 

Many thanks, Anna

---

What do you mean by evenly distributed? 

Unfortunately random sampling means that occasionally sometimes it appears to follow a trend, but it really doesn't. Random isn't the same as evenly distributed and humans are really bad at doing random. 

See this example where  ~ 400 people were asked to choose a random number between 1 and 10, theory would say you should end up with a uniform distribution but .... https://www.reddit.com/r/dataisbeautiful/comments/7gtzt3/biases_when_asked_to_select_numbers_from_11...

[PGStats]

Here is a method that will yield a decent sample with the desired properties, as far as I undersrtand them...

/* Fake data, based on sashelp.class. Give a random number of courses 
   to each teacher. */
data courses;
call streaminit(84521);
set sashelp.class(rename=name=teacher) nobs=nobs;
retain course 0;
do i = 1 to rand('poisson',129/nobs);
    course + 1;
    output;
    end;
keep teacher course;
run;

/* Put the courses in random order for each teacher */

data rndCourses;
set courses;
rnd = rand('uniform');
run;

proc sort data=rndCourses; by teacher rnd; run;

data orderedCourses;
set rndCourses;
by teacher;
if first.teacher then sampleOrder = 0;
sampleOrder + 1;
drop rnd;
run;

/* Select the sampled teachers for each month during 6 months */

proc sql;
create table teachers as
select unique teacher from courses;
quit;

/* Monthly teacher sample: 12 of 19 teachers, repeat for 6 months*/
proc surveyselect data=teachers out=teacherSample method=srs sampsize=12 rep=6;
run;

/* Each time a teacher is chosen, take the next course on the 
   randomized list of courses */
proc sort data=teacherSample; by teacher replicate; run;

data teacherSampleOrder;
set teacherSample; 
by teacher;
if first.teacher then courseChoiceOrder = 0;
courseChoiceOrder + 1;
run;

proc sql;
create table monthlyCourseSample as
select 
    a.replicate as month "Month",
    a.teacher,
    b.course
from 
    teacherSampleOrder as a inner join
    orderedCourses as b on a.teacher=b.teacher and a.courseChoiceOrder=b.sampleOrder
order by month, course;
/* Check when each course will be evaluated */
select a.course, a.teacher, b.month
from
    courses as a left join
    monthlyCourseSample as b on a.course=b.course
order by course;
quit;

[ballardw]

You actually have two criteria that possibly creates an impossible solution: 12 months (per year) and 10% per month. 10% times 12 months = 120% so there will be duplicates if you are indeed sampling every month. A second complication is 129 courses with 12 teachers so as many as 9 of your 12 teachers teach at least one more class than others. You may need to decide which is more important "evenly distributed" or "not duplicated".

I would be very tempted to randomly order the data and create partitions numbered 1 to n, where n is the actual total number evaluations. Then for the first evaluation use group 1, second use group 2 and so forth.

This would insure no duplicates assuming no changes in the numbers of teachers or classes. But is your data actually that stable?