topic Re: Create a subset data after removing outliers in SAS Programming

Create a subset data after removing outliers

mayasak — Wed, 07 Jun 2023 22:50:45 GMT

Hi,

I'm using the following code to print the outliers in my data.

proc print data=MDRO_Report_2021 noobs;
where abs(susceptibility-ps_mean) > 2*ps_std ;
by drug_bug;
var hospital_DB drug bug isolates_hospital_n isolates_tested susceptibility isolates_susceptible;
run;

I still need to remove these data from the data source so that it won't mess up my analysis.

Is there a way to create a dataset that is free from those outliers?

Thank you

Re: Create a subset data after removing outliers

mkeintz — Thu, 08 Jun 2023 13:04:26 GMT

Here's an analog using variable MSRP from sashelp.cars. It has three steps (generate mean and std, generate upper and lower limits into macro variables, apply the WHERE filter:

proc univariate data=sashelp.cars noprint;
  var msrp;
  output out=need mean=msrp_mean  std=msrp_std ;
run;

proc sql noprint;
  select msrp_mean - 2*msrp_std ,msrp_mean + 2*msrp_std
    into :lower_limit           ,:upper_limit
  from need;
quit;

proc print data= sashelp.cars;
  where &lower_limit <= msrp <= &upper_limit;
run;

You might prefer to do a more non-symmetrical (no-paframetric?) outlier filter:

proc univariate data=sashelp.cars noprint;
  var msrp;
  output out=need p5=p5 p95=p95;
run;

proc sql noprint;
  select p5           , p95 
    into :lower_limit ,:upper_limit
  from need;
quit;

proc print data= sashelp.cars;
  where &lower_limit <= msrp <= &upper_limit;
run;

Re: Create a subset data after removing outliers

mayasak — Thu, 08 Jun 2023 00:23:01 GMT

Thank you mkeintz, how can I export it to an Excel worksheet?

Re: Create a subset data after removing outliers

ballardw — Thu, 08 Jun 2023 01:37:03 GMT

It may help to define what you mean by 'remove'. Do want to delete an entire observation? Set variables to missing? Something else?

Example deleting entire observations when an outlier, assuming that is what your Where actually finds:

Data want; 
   set MDRO_REport_2021;
   where not (abs(susceptibility-ps_mean) > 2*ps_std);
run;

Or to set variables missing:

Data want; 
   set MDRO_REport_2021;
    if abs(susceptibility-ps_mean) > 2*ps_std then call missing(<comma delimited list of variable names goes here>);
run;

If remove is something other than this, describe.

Re: Create a subset data after removing outliers

mayasak — Sun, 11 Jun 2023 23:49:41 GMT

Thank you a lot.