topic Re: using IQR and median to remove outliers in SAS Programming

using IQR and median to remove outliers

fengyuwuzu — Fri, 15 Jul 2016 14:45:50 GMT

I want to remove outliers using median +/- 1.5 IQR (Qrange in SAS).

The proc univariate can generate median and Qrange, but how do I use these values in another proc or data step?

Another way is to use proc sql, but it seems proc sql summary function does not have qrange

or proc boxplot can also generate median and qrange, but I still need to use these values in another step.

Re: using IQR and median to remove outliers

Reeza — Fri, 15 Jul 2016 15:15:28 GMT

Use proc means, capture the data in a dataset and merge that into the main data set.

If you have SAS 9.4 proc SQL supports the median function otherwise it doesn't.

Re: using IQR and median to remove outliers

rayIII — Fri, 15 Jul 2016 15:24:13 GMT

You could save the median and iqr as macro variables, apply them in a Data step:

proc univariate data = sashelp.iris;
var petallength;
output out=boxStats median=median qrange = iqr;
run; 

data _null_;
	set boxStats;
	call symput ('median',median);
	call symput ('iqr', iqr);
run; 

%put &median;
%put &iqr;

data trimmed;
set sashelp.iris;
    if (petallength le &median + 1.5 * &iqr) and (petallength ge &median - 1.5 * &iqr); 
run; 

proc print data = trimmed;
run;

Re: using IQR and median to remove outliers

stat_sas — Fri, 15 Jul 2016 15:24:38 GMT

Hi,

Please try this:

proc univariate data=sashelp.class;
var age;
output out=stats median=median Qrange=Qrange;
run;

data want(drop=median Qrange);
if _n_=1 then set stats;
set sashelp.class;
if age<median - 1.5*Qrange or age>median + 1.5*Qrange then delete;
run;

Re: using IQR and median to remove outliers

fengyuwuzu — Fri, 15 Jul 2016 18:03:29 GMT

Thank you, Reeza. I do have SAS 9.4; I will check if it supports qrange.

Re: using IQR and median to remove outliers

fengyuwuzu — Fri, 15 Jul 2016 18:04:15 GMT

Thank you, Stat_SAS. I thought I need to merge them, but your solution gave me new idea. I will test. Thank you.

Re: using IQR and median to remove outliers

fengyuwuzu — Fri, 15 Jul 2016 18:05:40 GMT

Thanks. This works.
What I did is: I used proc means to output median and qrange to a new data set, and use proc sql to query my data and this new dataset with median and qrange, and do comparisons with b.median+1.5*b.qrange ...