topic Re: Weights for each variable in Logistic in Statistical Procedures

Weights for each variable in Logistic

jitb — Tue, 22 Apr 2025 20:04:33 GMT

Hello,

Would it be correct to assign weights to variables based on the inverse of their variance? The weights could be normalized to total to 1. Say, I have a binary(0/1) response variable Y, and 2 independent variables X1 and X2. I assign weights to each as mentioned before, say W1 and W2.

Could I model as logOdds(Y) = W1*X1 + W2*X2 ? Thanks in advance!

Re: Weights for each variable in Logistic

StatDave — Tue, 22 Apr 2025 20:09:38 GMT

The "weights" that you show (W1 and W2) in the model are the parameter estimates of X1 and X2 which are estimated by the procedure. They are not assigned in advance. There is no way to assign weights to the separate variables in the model. However, you could of course rescale each variable so that their variances are proportional to the weighting that you want. This could be done using the S= option in PROC STANDARD.

Re: Weights for each variable in Logistic

jitb — Tue, 22 Apr 2025 20:19:05 GMT

Thanks Dave.... actually, I meant multiplying X1 with W1 for each
observation in the data etc. Is that plausible?

Re: Weights for each variable in Logistic

StatDave — Tue, 22 Apr 2025 20:23:15 GMT

That would affect the variable's variance, so yes, but it will also affect its mean which would in turn affect the estimated parameter.

Re: Weights for each variable in Logistic

jitb — Tue, 22 Apr 2025 21:57:00 GMT

Yes, true. Have to think about it a bit more:). Basically, I would like to
reduce the importance of 2 variables based on business rules.

Re: Weights for each variable in Logistic

StatDave — Tue, 22 Apr 2025 22:03:30 GMT

Variable importance is usually assessed after fitting the model based on the parameter estimates (standardized in some way) or on a correlation measure like a partial R-square. See this note on assessing variable importance.

Re: Weights for each variable in Logistic

jitb — Tue, 22 Apr 2025 22:25:38 GMT

Thank you, Dave. Yes, I am aware of using standardized estimates for importance. I was thinking if we could assign weights a priori. Maybe it's not a good idea. On an unrelated note, will you be attending SAS Innovate on May 6? Would like to meet if possible. Thanks.

Re: Weights for each variable in Logistic

Rick_SAS — Wed, 23 Apr 2025 14:52:42 GMT

If I understand your question, the answer is that you can always rescale, but rescaling a variable does NOT change its significance (as measured by p-valuies) in the model.

If your original model is
Y = X1 X2;

and then you define Z1=W1*X1 and Z2=W2*X2 for any nonzero values W1 and W2, the new model

Y = Z1 Z2;

will have different regression coefficient estimates, but the tests for significance (the p-values) will be the same. This is easily seen if you use standardized estimates. See https://blogs.sas.com/content/iml/2018/08/22/standardized-regression-coefficients.html

For example:

data class;
set sashelp.class;
X1 = Height;
X2 = Weight;
Z1 = 0.0254*X1;      /* measure height in meters */
Z2 = 0.45359237*X2;  /* measure weight in kilos */
run;

title "Original Model: Inches and Pounds";
proc logistic data=class;
model Sex = X1 X2;
ods select ParameterEstimates;
run;
title "Rescaled Model: Meters and Kilos";
proc logistic data=class;
model Sex = Z1 Z2;
ods select ParameterEstimates;
run;

Re: Weights for each variable in Logistic

jitb — Wed, 23 Apr 2025 18:47:06 GMT

Thank you. Rick. Makes sense!

Re: Weights for each variable in Logistic

Season — Thu, 24 Apr 2025 14:32:22 GMT

Weighting is not necessarily needed in logistic regression, unless you are modeling complex survey data or dealing with rare events. See the documentation of PROC SURVEYLOGISTIC for more information of the former and Weighted logistic regression for large-scale imbalanced and rare events data - ScienceDirect and Improving performance of hurdle models using rare-event weighted logistic regression: an application to maternal mortality data - PMC for more information of the latter.