Programming the statistical procedures from SAS

Effect selection problem in proc NLMIXED

Reply
Contributor
Posts: 65

Effect selection problem in proc NLMIXED

Hi folks;

I am working with 9.3 version. I would like to do some sorts of effect selection in nlmixed since it does not have such tools. I encountered something strange in doing so. In one step, when I enter one effect, it seriously decreases the -2loglikelihood but the P-Value of that effect turns out to be insignificant. Seeing below, I want to know, out of "admsource" and "pheartind", which is better to enter the model:

For "admsource"

Fit Statistics

-2 Log Likelihood3038.0
AIC (smaller is better)3044.0
AICC (smaller is better)3044.0
BIC (smaller is better)3061.4

Parameter        Est.      StdE   DF   tValue Pr > |t| Alpha Lower   Upper    Gradient

b0-4.83240.37912448-12.75<.00010.05-5.5758-4.08912.64E-6
badmsource1.41990.359124483.95<.00010.050.71582.12405.435E-6
sd0.78930.112124487.04<.00010.050.56961.00900.000026

and for "pheartind"

Fit Statistics

-2 Log Likelihood2674.6
AIC (smaller is better)2680.6
AICC (smaller is better)2680.6
BIC (smaller is better)2697.4

Parameter    Est.      StdE   DF   tValue Pr > |t|  Alpha Lower   Upper  Gradient

b0-3.14690.18182025-17.31<.00010.05-3.5034-2.79030.000044
bpheartind-0.20950.16412025-1.280.20200.05-0.53140.11240.000016
sd0.67110.108120256.21<.00010.050.45910.88320.00004

Although "pheartind" can reduce much bigger amount in -2loglikelihood but its parameter turns out to be insignificant. So which effect should be included in the model?

Thanks!

Issac

Respected Advisor
Posts: 2,655

Re: Effect selection problem in proc NLMIXED

Hi Issac,

What is the value of -2 log likelihood for the null model in each case?  I have a sinking feeling that the data are not the same, so just comparing the log likelihoods of these two models is not the way to proceed.  My assumption is based on the difference in degrees of freedom for the tow models.  For bpheartind it is 2025, for badmsource it is 2448.  With so much more data for the badmsource, it is not surprising that the value of -2 log likelihood is substantially larger.

Steve Denham

Contributor
Posts: 65

Re: Effect selection problem in proc NLMIXED

Hi Steve;

For the null model containing only  b0 and e (random effect), I have the followings:

Fit Statistics

-2 Log Likelihood3082.5
AIC (smaller is better)3086.5
AICC (smaller is better)3086.5
BIC (smaller is better)3098.1

Par.  Est.    StdE      DF   tValue Pr > |t| Alpha Lower   Upper  Gradient

b0-3.38790.091362448-37.08<.00010.05-3.5671-3.20880.000033
sd0.64900.105524486.15<.00010.050.44210.85590.000028

sd is the standard deviation of e, since I have :    RANDOM e~NORMAL(0,sd*sd) SUBJECT=id;

This differences in DF happened, I think, because 'pheartind' has 551 missing values. So how can I proceed in choosing the best effect to be included at each step?

Really appreciate!

Issac

Respected Advisor
Posts: 2,655

Re: Effect selection problem in proc NLMIXED

This is like trying to compare apples and oranges.  One thing you might try is to subsample your data so that you have complete data for bpheartind and badmsource, then fit the models and look at the information criteria.  I would repeat this several (say 500 subsamples) and see which of the two more often gives the smaller IC.

Steve Denham

Contributor
Posts: 65

Re: Effect selection problem in proc NLMIXED

Steve;

Thanks for your hint. Kind of thinking that it's better to get more data points, cause the missing values are also existed in other variables in my data, leading to some instability of the model. BTW, that's really helpful point to consider.

Issac

Ask a Question
Discussion stats
  • 4 replies
  • 185 views
  • 6 likes
  • 2 in conversation