Create a subset data after removing outliers

[mayasak]

Hi,

I'm using the following code to print the outliers in my data. 

proc print data=MDRO_Report_2021 noobs;
where abs(susceptibility-ps_mean) > 2*ps_std ;
by drug_bug;
var hospital_DB drug bug isolates_hospital_n isolates_tested susceptibility isolates_susceptible;
run;

I still need to remove these data from the data source so that it won't mess up my analysis.

Is there a way to create a dataset that is free from those outliers?

Thank you

[mkeintz · Accepted solution]

Here's an analog using variable MSRP from sashelp.cars.  It has three steps (generate mean and std, generate upper and lower limits into macro variables, apply the WHERE filter:

proc univariate data=sashelp.cars noprint;
  var msrp;
  output out=need mean=msrp_mean  std=msrp_std ;
run;

proc sql noprint;
  select msrp_mean - 2*msrp_std ,msrp_mean + 2*msrp_std
    into :lower_limit           ,:upper_limit
  from need;
quit;

proc print data= sashelp.cars;
  where &lower_limit <= msrp <= &upper_limit;
run;

You might prefer to do a more non-symmetrical (no-paframetric?) outlier filter:

proc univariate data=sashelp.cars noprint;
  var msrp;
  output out=need p5=p5 p95=p95;
run;

proc sql noprint;
  select p5           , p95 
    into :lower_limit ,:upper_limit
  from need;
quit;

proc print data= sashelp.cars;
  where &lower_limit <= msrp <= &upper_limit;
run;

[mayasak]

Thank you mkeintz, how can I export it to an Excel worksheet?

[mayasak]

Thank you a lot.

[ballardw]

It may help to define what you mean by 'remove'. Do want to delete an entire observation? Set variables to missing? Something else?

Example deleting entire observations when an outlier, assuming that is what your Where actually finds:

Data want; 
   set MDRO_REport_2021;
   where not (abs(susceptibility-ps_mean) > 2*ps_std);
run;

Or to set variables missing:

Data want; 
   set MDRO_REport_2021;
    if abs(susceptibility-ps_mean) > 2*ps_std then call missing(<comma delimited list of variable names goes here>);
run;

If remove is something other than this, describe.