Proc Discrim on Clustered data

[Fae]

Hi,

I need help on generating discriminant statistics to classify data generated by cluster analysis.

In the cluster analysis, I have done dimension reduction using proc factor (method=principal rotate=varimax) which give me 6 factors.  I then use proc cluster (Ward's method) and end up with 7 clusters.

Is what I have below correct?  What value of k should i use?

PROC DISCRIM data=train TESTDATA=test testout=newgroups method=npar k=6 OUTSTAT=newStat;
var Factors1-Factors6;
class clusterID;
id dataID;
run;

[PGStats]

Now that you have the clusters, why not perform the discriminant analysis on the original variables? And why not start with parametric methods?

When doing a non-parametric discriminant analysis on principal components you won't get reusable classification rules or any insight about classification logic.

[Fae]

Can discriminant analysis handle collinearity?  

Thanks for your suggestion about using parametric method, i will check what's their distribution, hopefully the variables or the log-transformed variables are normally distributed.

[PGStats · Accepted solution]

---

@Fae wrote:

Can discriminant analysis handle collinearity?  

---

Yes discriminant analysis can handle collinearity. When two variables are colinear, their multivariate distribution will look like an oblique ellisoid. Proc discrim is a multivariate procedure that handles such distributions, within each cluster. Parametric discrimination assumes that the multivariate distribution of each cluster is multinormal. If you look at data from a multinormal distribution, one variable at a time, you will see normal distributions, even if the variables are not completely independent.

Proc discrim gives you the choice between the hypothesis that every cluster has the same covariance matrix, or not, with option POOL=YES/NO..

[Fae]

One quick question, if I were just to stick with the principal factors, using non-Parametric method, how do i pick k=?

For Parametric method, do I use use all the variables (Standardized and log-transformed) that go into the original principal component analysis or should I screen them?

[PGStats]

I agree with the procedure documentation that says "In nearest-neighbor methods, the choice of k is usually relatively uncritical (Hand 1982). A practical approach is to try several different values of the smoothing parameters within the context of the particular application and to choose the one that gives the best cross validated estimate of the error rate."

Use only one version of each variable, the version that looks the most normal. Use only ordinal, preferably continuous, variables.