Panel proc: "Hausman statistic cannot be calculated"

[KLite]

Hi folks,

I tried running a panel procedure (RANTWO vcomp=fb), but the log shows the following error message:

"Hausman statistic cannot be calculated because different variables were dropped in the random effects model than in the fixed effects model."

The output shows:

Hausman Test for

Random Effects

DF m Value Pr > m

0        .         .

Could someone please advise on what went wrong and how to rectify this?

[stat_sas]

I am not sure, there may be other reasons as well but as a starting point check multicollinearity in the independant variables.

[KLite]

Thank you, I will check out if it is due to multicollineariy and report accordingly.

[SteveDenham]

Could you post your PROC PANEL code?  Besides multicollinearity, there may be some other surprises waiting in the wings that come about from the use of various options or statements.

Steve Denham

[KLite]

Steve,

The following is the PROC PANEL CODE for both Fuller-Battese and Wallace-Hussain. Both could not yield any result in the Hausman Test. I have replaced the variables with generic terms for easier reference. Thanks so much for your help in looking into this.

*Random effects estimator;

proc panel data=data;

id Country Monthyear;

model VARA = VARB VARC VARD VARE /RANTWO vcomp=fb;

label   VARA = 'Variable A'

        VARB = 'Variable B'

  VARC = 'Variable C'

        VARD = 'Variable D'

        VARE = 'Variable E';

run;

*Random effects estimator;

proc panel data=data;

id Country Monthyear;

model VARA = VARB VARC VARD VARE /RANTWO vcomp=wh;

label   VARA = 'Variable A'

        VARB = 'Variable B'

  VARC = 'Variable C'

        VARD = 'Variable D'

        VARE = 'Variable E';

run;

[SteveDenham]

Well. <insert several swears here>

And I assume that all the sorting is done, so that leaves the collinearity problem.  OR some of the variance components under OLS/GLS are negative and set to zero so that they dropped out.  I guess what I find curious is the part of the error statement about variables being dropped from the fixed effects model.

What happens when you run this as FIXTWO?

Steve Denham

[KLite]

Steve,

Good that you asked about FIXTWO because there is also a minor issue there which I am not sure if it is related to this Hausman un-calculability. No issues with FIXONE, Pooled OLS, Parks and DaSilva though as far as generating readable results.

The PROC PANEL codes for FIXTWO are:

*Fixed effects or within estimator;

proc panel data=data;

id Country Monthyear;

model VARA = VARB VARC VARD VARE /FIXTWO;

label   VARA = 'Variable A'

        VARB = 'Variable B'

  VARC = 'Variable C'

        VARD = 'Variable D'

        VARE = 'Variable E';

run;

The output is:

Intercept        1     1968.87       158.6      12.41      <.0001    Intercept

   VARB              0           0           .        .         .            Variable  B

   VARC              1    3.730877      0.9410       3.96      <.0001   Variable C

   VARD            1    -0.05416      0.0139      -3.88      0.0001    Variable D

   VARE               1    0.003923    0.000224      17.52      <.0001    Variable E

You see the similarity in the Hausman Test output and the FIXTWO output for VARB? The log says:

"The transformed regression does not have full rank. User should be aware of possible multicollinearity and/or estimability problems before using FixTwo method results."

[KLite]

Hi folks,

Looks like it is indeed a multicollinearity problem. I replaced my data with random dummy variables and I could run results.

Now, how do I deal with this multicollinearity problem...

[SteveDenham]

If it is two variables that are collinear, pick one based on substantive background knowledge.  If it is more than two, there are a couple ways you might go.  Variable clustering followed by selecting a variable near the cluster centroid.  Extract principal components and regress on those.  The first is kind of ad hoc, while the second presents interpretability problems.  In the end, I would say use substantive knowledge to frame questions that have estimable answers.

Umm, if you have more than four covariates though...

Steve Denham

[KLite]

Steve,

I tried using just VARA and VARB (and dropped VARC, VARD and VARE) for the PROC PANEL.

model VARA = VARB

But still the multicollinearity problem exist! How is that even possible?

[SteveDenham]

I would say that for at least one country, there is no variability in VARB by time, or conversely, that at some time point, all countries have identical values.  Thus, the matrix is singular, and everything falls apart.

I think a cross post to the Forecasting and Econometrics forum may be in order. or are great resources over there, and I am really starting to be out of my depth on this.

Steve Denham

[KLite]

Steve,

Thanks for all your advice. You have been extremely helpful and certainly had provided excellent insights to my problem. I'll try to sniff around some more and hope to report some good news soon.