topic Re: The impact of adding explanatory variables on significance (TSCSREG) in Statistical Procedures

The impact of adding explanatory variables on significance (TSCSREG)

deleted_user — Fri, 30 Apr 2010 06:47:32 GMT

Hi,

This question pertains perhaps more to panel regression in general than to the SAS software in particular. Anyway, I thought the question might be relevant also to other SAS users (than just me).

I'm doing exploratory analyses with a panel data set. I'm testing the ability of a large number of constructs to explain an outcome. My question is: if an independent variable is deemed insignificant without controls, will it be insignificant with controls.

In other words: Does statistical significance decrease monotonically with increasing number of control variables?

I would like to use the following procedure:
1. test a large number of interesting constructs: are they able to explain the outcome
2. if so, include control variables to rule out alternative explanations

Is it possible that I will drop constructs at phase 1 that would turn out significant at phase 2?

Thanks!

Re: The impact of adding explanatory variables on significance (TSCSREG)

Doc_Duke — Mon, 03 May 2010 13:09:58 GMT

In short, no.

In general, with linear models, that monotonic characteristic is displayed when the independent variables are all independent of one another. Nice in theory, but doesn't always happen in practice.

That behavior is part of the reason that there are so many approaches to model building.

So, it is possible that your phase II will miss some interesting variables. It is a variation on the Type II error in sample size computation.

Re: The impact of adding explanatory variables on significance (TSCSREG)

deleted_user — Mon, 03 May 2010 13:29:52 GMT

Thank you very much for your reply! This pretty much answers my question.