topic Re: Quartiling and then finding the average in each quartile in SAS Procedures

Quartiling and then finding the average in each quartile

Yeezus_ — Fri, 06 Jan 2017 15:32:13 GMT

Hi everyone,

I have a dataset with over 6000 observations. These observations are all rate ie 0.95, 0.83, 0.99 ext. I need to quartile the observations and then find the average rate in each quartile. Essentially I need the average rate in Q1, Q2, Q3, Q4. I tried various procedures and I couldn't do what I wanted. I thought I would ask this community!

Thanks

Re: Quartiling and then finding the average in each quartile

Rick_SAS — Fri, 06 Jan 2017 15:47:19 GMT

Use PROC RANK to form the quartiles, as shown in the article "Grouping observations based on quantiles."

Then use the CLASS statement in PROC MEANS to compute the statistics for each quantile, as shown in this example:

%let NumGroups = 4;
proc rank data=Sashelp.cars out=Want groups=&NumGroups ties=high;
  var MSRP;     /* variable on which to group */
  ranks Group;  /* name of variable to contain groups 0,1,...,k-1 */
run;

proc format;                  /* display 0-3 as Q1-Q3 */
   value Quartile 0="Q1" 1="Q2" 2="Q3" 3="Q4";
run;

proc means data=Want N MEAN STD;
var MSRP;
class Group;
format group Quartile.;
run;

Re: Quartiling and then finding the average in each quartile

Shmuel — Fri, 06 Jan 2017 15:54:53 GMT

I hope next link will help you:

http://support.sas.com/resources/papers/proceedings10/135-2010.pdf

Re: Quartiling and then finding the average in each quartile

Yeezus_ — Fri, 06 Jan 2017 16:08:47 GMT

Hey! Thanks for the quick reply. I originally used proc rank and and split the observations into 4 groups. The problem I run into is that there are so many 100% observations that it groups them all together in Q1. For example. If I have 6000 observations, approx 40 percent of them will fall into the first group because they are all 100% rates. That leaves much less observations in Q2.

What I need is to evenly split the observations into equal groups and then quartile it. Basically some of the 100% rates will then fall into Q2, creating equal groups.

Re: Quartiling and then finding the average in each quartile

Reeza — Fri, 06 Jan 2017 16:17:35 GMT

How do you know which ones go to which group? At that rate how is it different than random? You definitely can't call it quartiles...

EDIT: My mistake, it won't matter since thats the group you're summarizing. But don't call it quartiles, that would be misleading.

proc sort data=sashelp.cars out=cars;
by mpg_highway;
run;

data check; 
set cars nobs=num;

n_group=floor(_n_/(num/4));
if n_group=4 then n_group=3; *reassign last obs;
run;

proc means data=check;
class n_group;
var mpg_highway;
run;

Re: Quartiling and then finding the average in each quartile

Rick_SAS — Fri, 06 Jan 2017 16:34:58 GMT

Yes, unfortunately this problem can occur. See the article "Binning data by quantiles? Beware of rounded data." The article says that is there are N observations and k groups then "if there are more than N/k repeated values, the repeated value can occupy more than one quantile value. In fact, this will always happen if a particular value is repeated more than 2N/k times."

The article concludes with this warning: "Beware of using quantiles to bin rounded data into groups. Although the technique works great when almost all of the data values are distinct, you can run into problems if you ask for many bins and your data contain many repeated values."

Re: Quartiling and then finding the average in each quartile

Yeezus_ — Fri, 06 Jan 2017 16:50:36 GMT

Hey, I am going go try this now. Just to clarify, what does the _n_ refer to below?

n_group=floor(_n_/(num/4));
if n_group=4 then n_group=3; *reassign last obs;
run;

Re: Quartiling and then finding the average in each quartile

Reeza — Fri, 06 Jan 2017 16:55:35 GMT

_n_ is a pseudo row number, that I'm using as a row number. It's an automatic variable.

http://support.sas.com/documentation/cdl/en/lrcon/69852/HTML/default/viewer.htm#p0e0mk25gs9binn1s9jiu4otau29.htm

Re: Quartiling and then finding the average in each quartile

PGStats — Sat, 07 Jan 2017 02:58:43 GMT

If you want your ties to be randomly split between quartiles, use small random tie breakers:

data cars;
set sashelp.cars;
/* Add random noise smaller than data precision */
qtr_mpg_highway = mpg_highway + 0.00001*rand("uniform");
run;

proc rank data=cars out=check groups=4;
var qtr_mpg_highway; 
run;

proc means data=check;
class qtr_mpg_highway;
var mpg_highway;
run;

Re: Quartiling and then finding the average in each quartile

pmpradhan — Fri, 08 Jun 2018 16:47:13 GMT

Your code is helpful. why are you reassigning the last observation?