Weights for each variable in Logistic

[jitb]

Hello,

Would it be correct to assign weights to variables based on the inverse of their variance? The weights could be normalized to total to 1. Say, I have a binary(0/1) response variable Y, and 2 independent variables X1 and X2. I assign weights to each as mentioned before, say W1 and W2.

Could I model as logOdds(Y) =  W1*X1 + W2*X2 ? Thanks in advance!

[StatDave]

The "weights" that you show (W1 and W2) in the model are the parameter estimates of X1 and X2 which are estimated by the procedure. They are not assigned in advance. There is no way to assign weights to the separate variables in the model. However, you could of course rescale each variable so that their variances are proportional to the weighting that you want. This could be done using the S= option in PROC STANDARD.

[jitb]

Thanks Dave.... actually, I meant multiplying X1 with W1 for each
observation in the data etc. Is that plausible?

[StatDave]

That would affect the variable's variance, so yes, but it will also affect its mean which would in turn affect the estimated parameter.

[jitb]

Yes, true. Have to think about it a bit more:). Basically, I would like to
reduce the importance of 2 variables based on business rules.

[StatDave]

Variable importance is usually assessed after fitting the model based on the parameter estimates (standardized in some way) or on a correlation measure like a partial R-square. See this note on assessing variable importance. 

[jitb]

Thank you, Dave. Yes, I am aware of using standardized estimates for importance. I was thinking if we could assign weights a priori. Maybe it's not a good idea. On an unrelated note, will you be attending SAS Innovate on May 6? Would like to meet if possible. Thanks.

[Rick_SAS · Accepted solution]

If I understand your question, the answer is that you can always rescale, but rescaling a variable does NOT change its significance (as measured by p-valuies) in the model.

If your original model is 
Y = X1  X2;

and then you define Z1=W1*X1 and Z2=W2*X2 for any nonzero values W1 and W2, the new model

Y = Z1  Z2;

will have different regression coefficient estimates, but the tests for significance (the p-values) will be the same. This is easily seen if you use standardized estimates. See https://blogs.sas.com/content/iml/2018/08/22/standardized-regression-coefficients.html 

For example:

data class;
set sashelp.class;
X1 = Height;
X2 = Weight;
Z1 = 0.0254*X1;      /* measure height in meters */
Z2 = 0.45359237*X2;  /* measure weight in kilos */
run;

title "Original Model: Inches and Pounds";
proc logistic data=class;
model Sex = X1 X2;
ods select ParameterEstimates;
run;
title "Rescaled Model: Meters and Kilos";
proc logistic data=class;
model Sex = Z1 Z2;
ods select ParameterEstimates;
run;

[jitb]

Thank you. Rick. Makes sense!

[Season]

Weighting is not necessarily needed in logistic regression, unless you are modeling complex survey data or dealing with rare events. See the documentation of PROC SURVEYLOGISTIC for more information of the former and Weighted logistic regression for large-scale imbalanced and rare events data - ScienceDirect and Improving performance of hurdle models using rare-event weighted logistic regression: an application... for more information of the latter.