Hi,   I found that R and Python gives z statistics for the significance of coefficients and results match.   Hence the arguments that since the target variable is binary in Logistic regression and hence Chi-square distribution is used to test coefficients does not appear to be true. The coefficients can be tested by both z statistics as well as  Chi-Square statistics.   The question actually is why SAS gives Chi Square and R , Python gives z. I think it's a basis Statistical question not requiring any complicated mathematical derivation , very much in scope for people using SAS Stat.   Regards,   Vishal ... View more

"Because it is not a t-distribution when the response is binary.."   In real world, a variable never follows a t-distribution. The variable is standardized and then compared to a t-distribution. I think that's not the right justification.   " As stated by @StatDave_sas, if you take the square root of the statistic, then you have a t-distribution, if that's what you really want". This is not what I am asking. My question is , why are we using square of [(x-0)/Std error]. How is this derived?   "There are probably text books and web sites that go through the mathematics of this whole thing, those are better places to look for answers."    I have been trying to find the answer to this question since quite some time. What you have mentioned above is the answer I have received mostly. People generally answer by using complicated terms like asymptotic Chi-square distribution , Pseudo statistics, etc.   If you can forward refer some book/website which may help finding this answer, it would be very helpfull.   Thanks. ... View more

"The estimates and test of coefficients depend on the distribution of the response variable, not the distribution of the coefficients themselves. "   Can you elaborate on it,how the estimates depend on the distribution of response variables?   The difference between Linear and Logistic regression is response variable. So, indeed distribution of response variable is one of the reason for Chi square distribution being used for testing the coefficients. However, this needs to be supported with arguments.    " If the response variable is a binary variable, then after the logit transformation, the Wald Chi Square statistic of the coefficient follows an asymptotic chi-square distribution, then we use this inference to test the significance of the coefficient or its CI. "   Question is still the same. Why did we use Chi-Square statistics which is square of t-statistics and not t-statistics. ... View more

To make the question more objective.   In Logistic regression the estimates (coefficients) should be such that they maximize the likelihood of observing the data. Then how do we conclude that they will be coming from a Chi Square distribution and the Chi Square statistic would be ((Beta - 0 )/ Std Error)^2. ... View more

Thanks @JacobSimonsen  @koyelghosh  @Rick_SAS  @StatDave  @PaigeMiller    Agree to the point that in Linear regression the target variable is continuous and we use F statistics while in Logistic Regression the target variable is binary and we use Chi Square distribution. This is about testing the significance of the model, wherein we compare a Null model and a model with covariates.   However my question is about the significance of model coefficients. Let me put it this way,   Consider and Logistic regression mode:   Log[(1-p)/p] = Intercept + B1X1 + B2X2 + ERROR.   In this model p is the probability of an event (say Loan default). Event will have binary values. Now for testing the significance of the model we Chi- Square ratio , Likelihood ratio. The coefficient X1 can have any value from -infinity to +infinity, we can say it is coming from a continuous population, then why cannot I use a t-distribution for testing it. To make my question more clear, consider a Linear model:   Y = Intercept +B1X1 + B2X2 + ERROR   The difference between these two models is the target variable and method of finding the coefficients. For Linear it is OLS,while Logistic it is MLE. The values of B1, B2 have same distribution for Linear and Logistic, so why T distribution for Linear and Chi Square for Logistic.    I think the answer lies in the distribution of target variable which we are talking about, we need to frame it more objectively. Infact how the distribution of model coefficients will by impacted by target variable or approach(OLS/MLE) needs to be answered.   Regards,   Vishal ... View more

Thanks koyelghosh,   Agree to your points target is continuous in Linear and discrete in Logistics. I would like to add few points here:   1. Chi Square statistics = ((Beta - 0)/ Std error)^2, here beta is the coefficient which we are testing against the null hypothesis that it is 0. The part of formula (Beta - 0)/ Std error), is same as for t-statistics. I agree to the point that target variable is discrete , however Beta is coming from a population which is continuous (can be -ve/+ve) that's why it is standardized. Why don't we then compare to a t distribution, rather than squaring it and then comparing it to Chi square distribution (which is the square of a random number).   What you are saying in the point 'You use Chi-square statistics when the observations are coming from a Chi-square distribution while you use t-statistics when the observations are coming from a t-distribution.' is true for target variable , however we are checking the coefficients which not necessarily maybe from a Chi-square distribution.   2. Even in Logistic regression the target variable is transformed using Logit function in to a continuous variable (-infinity to infinity). It is actually a generalized linear model.So , why cannot a t-distribution be used for checking the significance of coefficient.    Thanks,   Vishal ... View more