topic Macro to identify and output outliers for multiple variables in a dataset in SAS Programming

Macro to identify and output outliers for multiple variables in a dataset

cbt2119 — Thu, 10 Jan 2019 19:15:50 GMT

Dear SAS Community

I am writing a program to flag and output outliers in a dataset for multiple variables. I want to output all outlier observations that are above or below 3x the interquartile range into a dataset to query. I would like to do this using a macro to avoid doing this for one by one, but I am not experienced macro writing.

I'd like to produce a data set that is a list of all the observations that have an outlier with the following columns:

1) PID: The ID for the observation

2) Variable: The name of the variable that is an outlier

3) Value: The value of the outlier variable

4) Issue: The issue with the value i.e. "Too High" or "Too Low"

I was doing it manually by running Proc Univariate and looking at the distributions for all the variables and then outputting each one using the attached sample part of my code (there are many more variables), but it is taking too long for the number of variables I have.

data outlierqueries;
keep PID variable value issue;
	set fulldataset;
	if height > xxx then do;
		variable='height';
		value=put(height, 10.);
		issue='Seems too high';
		output;
		end;
	if height < xxx then do;
		variable='height';
		value=put(height, 10.);
		issue='Seems too low';
		output;
		end;
	if weight > xxx then do;
		variable='weight';
		value=put(weight, 10.);
		issue='Seems too high';
		output;
		end;
	if weight < xxx then do;
		variable='weight';
		value=put(weight, 10.);
		issue='Seems too low';
		output;
		end;
run;

Any suggestions/help you can offer would be much appreciated!

Thanks,

Cara

Re: Macro to identify and output outliers for multiple variables in a dataset

Astounding — Thu, 10 Jan 2019 19:24:21 GMT

Two suggestions ...

First, consider whether the 10. format is correct? Perhaps you would want one position after the decimal point?

Second, what output data set(s) can you get from PROC UNIVARIATE? The problem can be automated, but macros might not be the best approach. Can you obtain an output data set with the equivalent of three variables: VARIABLE_NAME, LOW_CUTOFF, HIGH_CUTOFF?

Re: Macro to identify and output outliers for multiple variables in a dataset

PaigeMiller — Thu, 10 Jan 2019 19:36:22 GMT

You can run PROC SUMMARY on all of your variables, and have the interquartile range for each variable output to a data set. Then loop through the variables one by one to identify outliers. Here is an outline of the code:


%macro identify_outliers(variablenames=,datasetname=);
	proc summary data=&datasetname;
	    var &variablenames;
		output out=stats qrange= mean=/autoname;
	run;

	%do i=1 %to %sysfunc(countw(&variablenames));
	    %let thisname=%scan(&variablenames,&i,%str( ));
	    data outliers_&thisname;
		    if _n_=1 then set stats;
		    set &datasetname;
                    if &thisname>(&thisname._mean + 3*&thisname._qrange) then do;
			         /*** Do something here ***/
                    end;
             run;
	%end;
%mend;

Hashtag: #PROCSUMMARYRULEZ

Re: Macro to identify and output outliers for multiple variables in a dataset

cbt2119 — Thu, 10 Jan 2019 19:55:15 GMT

Thank you, this makes sense. I am just having a small issue when I call the macro and define the variables. I get this error, "ERROR: The keyword parameter INPUT was not defined with the macro.
ERROR: The keyword parameter VAR was not defined with the macro.
ERROR: The keyword parameter OUTPUT was not defined with the macro."

I think I'm likely calling it incorrectly. This is my modified code (not all variables are represented here, just a portion):

%let variablenames=weight height totalfat totalmass totallean;

 %macro identify_outliers(variablenames=,datasetname=);
	proc summary data=&datasetname;
	    var &variablenames;
		output out=stats qrange= mean=/autoname;
	run;

	%do i=1 %to %sysfunc(countw(&variablenames));
	    %let thisname=%scan(&variablenames,&i,%str( ));
	    data outliers_&thisname;
		    if _n_=1 then set stats;
		    set &datasetname;
                    if &thisname>(&thisname._mean + 3*&thisname._qrange) then do;
			        	variable=&thisname;
					value=put(&thisname, 10.);
					issue='Out of range';
                    end;
             run;
	%end;
%mend;

%identify_outliers(input=alldxa, var=&variablenames, output=outliers);

Re: Macro to identify and output outliers for multiple variables in a dataset

PaigeMiller — Thu, 10 Jan 2019 20:06:01 GMT

There's no OUTPUT parameter in the macro I wrote.

You'd have to add this in to the macro if you want to specify an OUTPUT data set name.

You might also want to add a KEEP statement in the output data set, which I have currently named OUTLIERS_&THISNAME.

Re: Macro to identify and output outliers for multiple variables in a dataset

cbt2119 — Thu, 10 Jan 2019 20:55:21 GMT

Thanks, I've got it now. Works well!