BookmarkSubscribeRSS Feed
followgrad
Calcite | Level 5

 

I am still working on an item response theory (IRT) analysis to find differential item functioning (DIF) in any of the items of a 17-item graded response likert scale. 

 
To briefly summarize what we've been doing, we are basically locating DIF on the items by creating three nested models and comparing their goodness of fit to each other. We then use a likelihood ratio test based on their goodness of fit values (in this case, -2loglikelihood value) and chi-square and df values to see whether the differences between their -2loglikelihood values is statistically significant. When model 1 and model 2 are compared, a significant difference indicates DIF. If this happens, model 2 and model 3 are compared using LRT and a significance indicates nonuniform DIF (as opposed to uniform DIF). Model 1 is our most restricted model, model 2 is our least restricted model and model 3 is between models 1 and 2 in restrictiveness. The problem we are having is that we are getting negative values for likelihood ratio statistic, and we can not determine whether or not there is significant difference between the fit of the models because the LR test static is negative. 
 
According to my understanding, likelihood ratio test statistics are not supposed to be negative. This may indicate that the sample size used is too small, among other explanations, but the LRT statistic should not be negative. 
 
Also according to my understand of LRT, the less restrictive model is theoretically supposed to fit the data better than a more constrained model. Therefore, the equation for the LRT statistic is: 2(loglikelihood(less restricted model)-loglikelihood(more restricted model)). However, the key difference here is that our output discloses -2loglikelihood values. So we have been basically subtracting -2ll of model 2(least restrictive) from -2llmodel3(more restrictive). However, if working with just regular log likelihood values, you'd subtract model 3(more restrctive) from model 2(least restrictive). I am not sure how the -2ll in our data effect the calculation of this LRT statistic, and whether this is responsible for the negative LRT statistic value's we've found. On previous items, we did not find negative LRT statistic values when comparing these models (using -2ll for all models, model 1 minus model 2, and model 3 minus model 2). 
 
If anyone has ANY idea at all, please throw it at me. Or any info regarding likelihood ratio testing. Can a chi-square value be negative (I don't think so)? So do we take the absoulte value of these negative numbers for our chi-square value?? please...HELP! 
2 REPLIES 2
SteveDenham
Jade | Level 19

My question is this: is your concern over a negative log likelihood (which happens a lot) or over a negative difference, such that the difference (more complex model minus less complex model) is uninterpretable?  Examples, with explanations, would help.

 

Steve Denham

lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12

Many SAS procedures print -2LL as a convenience, since one just has to subtract values without remembering to multiply by 2 or -2. The smaller the -2LL, the better the fit (ignoring the differneces in parameter numbers). The LRT is just the difference.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 3469 views
  • 0 likes
  • 3 in conversation