Macro to identify and output outliers for multiple variables in a dataset

[cbt2119]

Dear SAS Community

I am writing a program to flag and output outliers in a dataset for multiple variables. I want to output all outlier observations that are above or below 3x the interquartile range into a dataset to query. I would like to do this using a macro to avoid doing this for one by one, but I am not experienced macro writing. 

I'd like to produce a data set that is a list of all the observations that have an outlier with the following columns:

1) PID: The ID for the observation

2) Variable: The name of the variable that is an outlier

3) Value: The value of the outlier variable

4) Issue: The issue with the value i.e. "Too High" or "Too Low"

I was doing it manually by running Proc Univariate and looking at the distributions for all the variables and then outputting each one using the attached sample part of my code (there are many more variables), but it is taking too long for the number of variables I have.

data outlierqueries;
keep PID variable value issue;
	set fulldataset;
	if height > xxx then do;
		variable='height';
		value=put(height, 10.);
		issue='Seems too high';
		output;
		end;
	if height < xxx then do;
		variable='height';
		value=put(height, 10.);
		issue='Seems too low';
		output;
		end;
	if weight > xxx then do;
		variable='weight';
		value=put(weight, 10.);
		issue='Seems too high';
		output;
		end;
	if weight < xxx then do;
		variable='weight';
		value=put(weight, 10.);
		issue='Seems too low';
		output;
		end;
run;

Any suggestions/help you can offer would be much appreciated!

Thanks,

Cara

[Astounding]

Two suggestions ...

First, consider whether the 10. format is correct?  Perhaps you would want one position after the decimal point?

Second, what output data set(s) can you get from PROC UNIVARIATE?  The problem can be automated, but macros might not be the best approach.  Can you obtain an output data set with the equivalent of three variables:  VARIABLE_NAME, LOW_CUTOFF, HIGH_CUTOFF?

[PaigeMiller · Accepted solution]

You can run PROC SUMMARY on all of your variables, and have the interquartile range for each variable output to a data set. Then loop through the variables one by one to identify outliers. Here is an outline of the code:

%macro identify_outliers(variablenames=,datasetname=);
	proc summary data=&datasetname;
	    var &variablenames;
		output out=stats qrange= mean=/autoname;
	run;

	%do i=1 %to %sysfunc(countw(&variablenames));
	    %let thisname=%scan(&variablenames,&i,%str( ));
	    data outliers_&thisname;
		    if _n_=1 then set stats;
		    set &datasetname;
                    if &thisname>(&thisname._mean + 3*&thisname._qrange) then do;
			         /*** Do something here ***/
                    end;
             run;
	%end;
%mend;

Hashtag: #PROCSUMMARYRULEZ 

[cbt2119]

Thank you, this makes sense. I am just having a small issue when I call the macro and define the variables. I get this error, "ERROR: The keyword parameter INPUT was not defined with the macro.
ERROR: The keyword parameter VAR was not defined with the macro.
ERROR: The keyword parameter OUTPUT was not defined with the macro."

I think I'm likely calling it incorrectly. This is my modified code (not all variables are represented here, just a portion):

%let variablenames=weight height totalfat totalmass totallean;

 %macro identify_outliers(variablenames=,datasetname=);
	proc summary data=&datasetname;
	    var &variablenames;
		output out=stats qrange= mean=/autoname;
	run;

	%do i=1 %to %sysfunc(countw(&variablenames));
	    %let thisname=%scan(&variablenames,&i,%str( ));
	    data outliers_&thisname;
		    if _n_=1 then set stats;
		    set &datasetname;
                    if &thisname>(&thisname._mean + 3*&thisname._qrange) then do;
			        	variable=&thisname;
					value=put(&thisname, 10.);
					issue='Out of range';
                    end;
             run;
	%end;
%mend;

%identify_outliers(input=alldxa, var=&variablenames, output=outliers);

[PaigeMiller]

There's no OUTPUT parameter in the macro I wrote.

You'd have to add this in to the macro if you want to specify an OUTPUT data set name.

You might also want to add a KEEP statement in the output data set, which I have currently named OUTLIERS_&THISNAME.

[cbt2119]

Thanks, I've got it now. Works well!