A Principal Components Analysis question

[aha123]

I just read this web page about PCA:

http://www.alglib.net/dataanalysis/principalcomponentsanalysis.php

It says, "...In this case, the PCA will give preference to the first (less informative) variable. This drawback is closely connected to the fact that the PCA does not perform linear separation of classes, linear regression or other similar operations, but it merely permits the input vector to be best restored on the basis of the partial information about it. All additional information pertaining to the vector (such as the identification of an image with one of the classes) is ignored..."

If you know any good books elaborating on this point, please let me know.

[Rick_SAS]

Jackson (1991), A user's guide to principal components, is an excellent book on PCA. There is a section on PCA vs linear discriminant analysis (LDA) on pp. 334-337, including references to the use of PCA in discriminant analysis. A more recent book is Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning, which is more advanced, but less explicit than Jackson. The issue raised in the web page you quote is covered on pp. 93-94.

In general, PCA and LDA are two different analyses with different objectives. PCA tries to find directions (=linear combinations fo variables) that explain the most variance. LDA tries to find a linear subspace that best separates groups.  Tjhey can be combined into canonical discriminant analysis, which in SAS is accomplished by using the CANDISC procedure. The overview section of the CANDISC doc discusses how CDA works. From a practical perspective, CDA works pretty well in many cases.