DATA Step, Macro, Functions and more

using IQR and median to remove outliers

Accepted Solution Solved
Reply
Super Contributor
Posts: 318
Accepted Solution

using IQR and median to remove outliers

I want to remove outliers using median +/- 1.5 IQR (Qrange in SAS).

The proc univariate can generate median and Qrange, but how do I use these values in another proc or data step?

 

Another way is to use proc sql, but it seems proc sql summary function does not have qrange

 

or proc boxplot can also generate median and qrange, but I still need to use these values in another step.

 


Accepted Solutions
Solution
‎07-15-2016 02:02 PM
SAS Employee
Posts: 106

Re: using IQR and median to remove outliers

Posted in reply to fengyuwuzu

You could save the median and iqr as macro variables, apply them in a Data step: 

 

proc univariate data = sashelp.iris;
var petallength;
output out=boxStats median=median qrange = iqr;
run; 

data _null_;
	set boxStats;
	call symput ('median',median);
	call symput ('iqr', iqr);
run; 

%put &median;
%put &iqr;

data trimmed;
set sashelp.iris;
    if (petallength le &median + 1.5 * &iqr) and (petallength ge &median - 1.5 * &iqr); 
run; 

proc print data = trimmed;
run; 

View solution in original post


All Replies
Super User
Posts: 19,768

Re: using IQR and median to remove outliers

Posted in reply to fengyuwuzu

Use proc means, capture the data in a dataset and merge that into the main data set. 

 

If you have SAS 9.4 proc SQL supports the median function otherwise it doesn't. 

Super Contributor
Posts: 318

Re: using IQR and median to remove outliers

Thank you, Reeza. I do have SAS 9.4; I will check if it supports qrange.
Solution
‎07-15-2016 02:02 PM
SAS Employee
Posts: 106

Re: using IQR and median to remove outliers

Posted in reply to fengyuwuzu

You could save the median and iqr as macro variables, apply them in a Data step: 

 

proc univariate data = sashelp.iris;
var petallength;
output out=boxStats median=median qrange = iqr;
run; 

data _null_;
	set boxStats;
	call symput ('median',median);
	call symput ('iqr', iqr);
run; 

%put &median;
%put &iqr;

data trimmed;
set sashelp.iris;
    if (petallength le &median + 1.5 * &iqr) and (petallength ge &median - 1.5 * &iqr); 
run; 

proc print data = trimmed;
run; 
Super Contributor
Posts: 318

Re: using IQR and median to remove outliers

Thanks. This works.
What I did is: I used proc means to output median and qrange to a new data set, and use proc sql to query my data and this new dataset with median and qrange, and do comparisons with b.median+1.5*b.qrange ...
Trusted Advisor
Posts: 1,228

Re: using IQR and median to remove outliers

Posted in reply to fengyuwuzu

Hi,

 

Please try this:

 

proc univariate data=sashelp.class;
var age;
output out=stats median=median Qrange=Qrange;
run;

 

data want(drop=median Qrange);
if _n_=1 then set stats;
set sashelp.class;
if age<median - 1.5*Qrange or age>median + 1.5*Qrange then delete;
run;

Super Contributor
Posts: 318

Re: using IQR and median to remove outliers

Thank you, Stat_SAS. I thought I need to merge them, but your solution gave me new idea. I will test. Thank you.
☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 6 replies
  • 2797 views
  • 2 likes
  • 4 in conversation