topic Re: How to 'print' the remaining data after remove the outliers? in Statistical Procedures

How to 'print' the remaining data after remove the outliers?

NURAO — Mon, 07 Aug 2017 15:49:50 GMT

Hi,

How can I 'print' the data that has been removed the outliers. Specially when it is univariate data.

Re: How to 'print' the remaining data after remove the outliers?

ballardw — Mon, 07 Aug 2017 16:50:22 GMT

You might want to show the code for what you are currently doing and even some example data so we can see some of your outliers. Example data really only would need one variable with outliers.

Default output for proc univariate shows the largest and smallest values by default. Do you need more than those?

Likely the approach is going to involve either manually writing a where clause for proc print or code to extract some key values to use as limits.

Re: How to 'print' the remaining data after remove the outliers?

NURAO — Tue, 08 Aug 2017 08:06:29 GMT

PROC IML;
RESET NONAME;

A={2,2,3,3,3,3,3,3,3,4,4,4,
4,4,5,5,5,6,6,6,7,7,7,8,
8,9,10,14,14,15,20,21,23};
mini=A[><,];
Q=quartile (A);
Q3=Q[4,1];
Q1=Q[2,1];
IQR=Q3-Q1;
L2=Q1-(1.5*IQR);
U2=Q3+(1.5*IQR);
PRINT "L2=" L2,
"U2=" U2;
maxA=max (A);
minA=min (A);
print "mini=" minA,
"max=" maxA;

Here, L2 and U2 are the fences to detect outliers. Based on data A, 20,21 and 23 are the outliers. So my intention is, i want to "print" the data set A by excluded those outliers.

Re: How to 'print' the remaining data after remove the outliers?

SAS_Rob — Tue, 08 Aug 2017 13:48:56 GMT

You should just have to use the LOC function to subset the matrix.

newa=a[loc(a>l2 & a<u2)];

Re: How to 'print' the remaining data after remove the outliers?

Ksharp — Tue, 08 Aug 2017 14:01:22 GMT

You really should post it at IML forum, since your code is IML code.

PROC IML;
RESET NONAME;
 
A={2,2,3,3,3,3,3,3,3,4,4,4,
4,4,5,5,5,6,6,6,7,7,7,8,
8,9,10,14,14,15,20,21,23};
mini=A[><,];
Q=quartile (A);
Q3=Q[4,1];
Q1=Q[2,1];
IQR=Q3-Q1;
L2=Q1-(1.5*IQR);
U2=Q3+(1.5*IQR);
PRINT "L2=" L2,
"U2=" U2;
maxA=max (A);
minA=min (A);
print "mini=" minA,
"max=" maxA;



left=A[loc(L2<a & a<U2)];
print "exclude outliers: " left;
quit;

Re: How to 'print' the remaining data after remove the outliers?

NURAO — Wed, 09 Aug 2017 02:42:51 GMT

Thank you

Re: How to 'print' the remaining data after remove the outliers?

NURAO — Wed, 09 Aug 2017 02:44:27 GMT

Got it. Thank you