Regression Analysis and Dimension Reduction

[KafeelBasha]

Hello

1) In Multiple Linear Regression what is the relation between beta coefficients and errors. Can we say if errors are normal distributed then beta coeffients are also normally distributed?.

2) Why do we need to look for convergence of logistic regression model, when can we say the model doesn't converge. Please refers me a book or simple example.

3) Request you to suggest me books on Dimension Reduction Techniques which includes topics like Factor Analysis, Principal Component analysis and Descriminent analysis.

Thanks

[Ksharp · Accepted solution]

1) In Multiple Linear Regression what is the relation between beta coefficients and errors. Can we say if errors are normal distributed then beta coeffients are also normally distributed?.
 
beta coefficients and errors should not be correlation.
I don't think so.


2) Why do we need to look for convergence of logistic regression model, when can we say the model doesn't converge. Please refers me a book or simple example.
 
If it was convergence ,then the parameter estimator could be trusted , vice verse.
If there are many missing obs in data, or you have a level which occurred only once or few times.
E.X.
sex
F
F
..
F
M
M
...
M
X   <--- only have one obs .


3) Request you to suggest me books on Dimension Reduction Techniques which includes topics like Factor Analysis, Principal Component analysis and Descriminent analysis.
Check the documentation of PROC VARCLUS , PROC PRINCOMP ... at the end of them , you will see lots of reference books.

[mkeintz]

This is from way back in the reptialian part of my memory, but ...

First, Beta is the underlying coefficient.  It's not distributed.  But BetaHat (estimation of Beta) is distributed.

Now If

1. Y = Beta'X + e, and
2. error term e is normally distributed
3. BetaHat estimation is  BetaHat = Inverse(X'X) (X'Y)     [=Inverse(X'X)*Beta'X + Inverse(X'X)e]

then under closure of normal distributions under linear transformation, BetaHat must be normally distributed.