MI analyze for logistic regression with stepwise selection

[michellemabelle]

Hello:

I have run proc MI with 10 imputations, now I want to combine coefficients and odds ratios to make inferences using MI analyze. The problem I seem to run into is that parameters are not available for the non-selected variables in my var statement (I used the same var statement for imputation, proc logistic and mi analyze). How do I use mi analyze on logistic regression using stepwise selection to choose variables?

Cheers,

Michelle

proc logistic data=outfiles.miout; /*logistic regression*/
	class AGE_70plus(ref="1") GENDER_male(ref="1") BMI_status(ref="2") ASACLASS_above3(ref="1") FNSTATUS_NotIndependent(ref="1") PMHxSMOKING(ref="1") PMHxALCOHOL(ref="1") PMHxCAD(ref="1") PMHxCHF(ref="1") PMHxDIABETES(ref="1") PMHxHTN(ref="1") PMHxSTEROIDS(ref="1") PMHxCOPD(ref="1") PMHxDYSPNEA(ref="1") PMHxTIACVA(ref="1") PMHxNEUROIMP(ref="1") PMHxBLEEDINGDISORDER(ref="1") PMHxRENALFAIL PMHxPVD(ref="1") PMHxCANCER(ref="1");
	model COMPLICATIONS (descending)= AGE_40plus AGE_50plus AGE_60plus AGE_70plus AGE_80plus GENDER_male BMI_status ASACLASS_above3 FNSTATUS_NotIndependent PMHxSMOKING PMHxALCOHOL PMHxCAD PMHxCHF PMHxDIABETES PMHxHTN PMHxSTEROIDS PMHxCOPD PMHxDYSPNEA PMHxTIACVA PMHxNEUROIMP PMHxBLEEDINGDISORDER PMHxRENALFAIL PMHxPVD PMHxCANCER / selection=stepwise slentry=0.10 slstay=0.05 details lackfit;
	by _imputation_;
	ods output ParameterEstimates=outfiles.lgsparms OddsRatios=outfiles.lgsodds;
run;


proc mianalyze parms(classvar=classval)=outfiles.lgsparms; /*pooling coefficients*/
	class AGE_70plus GENDER_male BMI_status ASACLASS_above3 FNSTATUS_NotIndependent PMHxSMOKING PMHxALCOHOL PMHxCAD PMHxCHF PMHxDIABETES PMHxHTN PMHxSTEROIDS PMHxCOPD PMHxDYSPNEA PMHxTIACVA PMHxNEUROIMP PMHxBLEEDINGDISORDER PMHxRENALFAIL PMHxPVD PMHxCANCER;
	modeleffects AGE_70plus GENDER_male BMI_status ASACLASS_above3 FNSTATUS_NotIndependent PMHxSMOKING PMHxALCOHOL PMHxCAD PMHxCHF PMHxDIABETES PMHxHTN PMHxSTEROIDS PMHxCOPD PMHxDYSPNEA PMHxTIACVA PMHxNEUROIMP PMHxBLEEDINGDISORDER PMHxRENALFAIL PMHxPVD PMHxCANCER;
	ods output parameterestimates=outfiles.mianalyze_parms CovB=outfiles.mianalyze_covb;
run;

[PaigeMiller]

I don't think Stepwise logistic regression will work with repeated imputations from PROC MI. But there are many other reasons to avoid using Stepwise anyway, just go to your favorite internet search engine and type in "problems with stepwise regression".

[michellemabelle]

Yes, I understand the pitfalls of stepwise... but my supervisor seems to like it. What if I used LASSO or adaptive LASSO for variable selection instead? Would that be compatible with multiple imputation? Thanks for your advice!

[PaigeMiller]

---

@michellemabelle wrote:

Yes, I understand the pitfalls of stepwise... but my supervisor seems to like it. What if I used LASSO or adaptive LASSO for variable selection instead? Would that be compatible with multiple imputation? Thanks for your advice!

---

I don't see how the lasso avoids the problems you have encountered with multiple imputation.

[michellemabelle]

I see. In general, is it not possible to perform variable selection with multiple imputation? Or if its doable, how?

[PaigeMiller]

Variable selection is possible, it is the combining stepwise with multiple imputation that doesn't seem possible, as you have discovered.

I sometimes get into a rant about why you shouldn't do stepwise, instead you should do Partial Least Squares; and this is another benefit of Partial Least Squares, that this multiple imputation problem goes away ... but SAS doesn't provide a logistic Partial Least Squares, it only provides Partial Least Squares for continuous Y variables. This logistic PLS issue comes up several times a month here in the SAS communities, but I don't know if SAS will ever address this deficiency.

[michellemabelle]

Ok, I'll avoid stepwise then. Then my issue becomes doing variable selection with multiple imputation. Looking elsewhere on the boards, I've found someone suggest to perform variable selection on a single imputation, and then run that model on all the other imputations using mi analyze to pool coeffients and odds ratios, but I am confused as to how to implement this, or if this is statistically sound. Thank you very much!

[PaigeMiller]

---

@michellemabelle wrote:

Ok, I'll avoid stepwise then. Then my issue becomes doing variable selection with multiple imputation. Looking elsewhere on the boards, I've found someone suggest to perform variable selection on a single imputation, and then run that model on all the other imputations using mi analyze to pool coeffients and odds ratios, but I am confused as to how to implement this, or if this is statistically sound. Thank you very much!

---

I think doing variable selection on a single imputation and then using that model thereafter on the other imputations is questionable. That's my opinion. I note that I don't have any experience with that method.

But whatever you come up with, I'm probably not going to like if it involves variable selection; so you might as well come up with something that YOU like and YOU can feel good about.

By the way, how are you doing the imputation in PROC MI? If you are using the EM method, that would make everything feel better to me, as then the different imputations ought to produce somewhat similar models, although the horrors of stepwise could still cause trouble. You see, the EM method ought to (across multiple imputations) result in data that has roughly the same covariance matrix, and thus ought to produce roughly the same Logistic regression models, but no guarantees either. If you're not using EM here, then I don't like it.