Proc Countreg Negbin Model with Fixed Effects for Panel Data

[sg248]

Hello,

I want to estimate a Negative Binomial model with panel data (balanced panel) with fixed effects. My N is large (~ 12,000) and the dataset is large (~600,000). Introducing fixed effects as dummy variables in Proc Genmod tells me I don't have enough memory. So I tried Proc Countreg with the following specification. As best as I can tell, the specification of groupid and errorcomp is what tells Proc Countreg to do a fixed effects model.

proc countreg data = one
groupid = year_market;
class nu_week ;
model y = x
nu_week 
/ dist = negbin errorcomp = fixed;
output out = predicted_free mean = free_pred;
store out = results;

run;

From the results, as best as I can tell, year_market is not being used as a fixed effect. There are two reasons I say this. One, year_market does not appear anywhere in the output (I am not expecting to get estimates of the fixed effects, but at least something that says fixed effects are being used.) Second, I have a good idea of what to expect in terms of the coefficients of x when I include fixed effects, and the results suggest there are no fixed effects.

I would appreciate any suggestions on what I might be doing wrong here.

thanks

[SteveDenham]

I can't guarantee this, but I think you have to specify year_market in the MODEL statement and the CLASS statement, as well as in the GROUPID= option. .

Given the size of your data, you may want to consider HPCOUNTREG as an alternative.

Steve Denham

[sg248]

Thank you, Steve.

The suggestions here (bottom of the page) 

http://blogs.sas.com/content/sgf/2014/09/19/econometric-modeling-your-questions-answered/ 

indicate that the GROUPID variable does not need to be in the MODEL statement.

It is surprising to me that SAS documentation of PROC COUNTREG does not have any examples of the use of the ERRORCOMP or GROUPID options!

[SteveDenham]

I agree about the lack of an example being really frustrating--they are always my first choice of ways to approach a new analysis, and the basis of most of my posts here.

Maybe Ken can comment on what we need to be thinking, so pinging @ets_kps

Steve Denham

[Sadia_Sharmin]

Hello Steve,

I have used countreg statement for my negbin random and fixed effect modeling. I had some error and warning saying "Warning: Optimization cannot improve the function value". Can I use hpcountreg, would that help to find an output without error? 

My hpcountreg coding is not running. I have used

proc hpcountreg data=Panel groupid=id;
performance nthreads=2 nodes=1 details;
model PDO = ADT_Maj ADT_Min L_Maj L_Min LL_Maj LL_Min RL_Maj RL_Min Med_Maj Med_Min park skew Rain Snow ydr dui/dist=negbin errorcomp=fixed;
run;

Would you please check, what is wrong with my coding?