I am still working on an item response theory (IRT) analysis to find differential item functioning (DIF) in any of the items of a 17-item graded response likert scale.
To briefly summarize what we've been doing, we are basically locating DIF on the items by creating three nested models and comparing their goodness of fit to each other. We then use a likelihood ratio test based on their goodness of fit values (in this case, -2loglikelihood value) and chi-square and df values to see whether the differences between their -2loglikelihood values is statistically significant. When model 1 and model 2 are compared, a significant difference indicates DIF. If this happens, model 2 and model 3 are compared using LRT and a significance indicates nonuniform DIF (as opposed to uniform DIF). Model 1 is our most restricted model, model 2 is our least restricted model and model 3 is between models 1 and 2 in restrictiveness. The problem we are having is that we are getting negative values for likelihood ratio statistic, and we can not determine whether or not there is significant difference between the fit of the models because the LR test static is negative.
According to my understanding, likelihood ratio test statistics are not supposed to be negative. This may indicate that the sample size used is too small, among other explanations, but the LRT statistic should not be negative.
Also according to my understand of LRT, the less restrictive model is theoretically supposed to fit the data better than a more constrained model. Therefore, the equation for the LRT statistic is: 2(loglikelihood(less restricted model)-loglikelihood(more restricted model)). However, the key difference here is that our output discloses -2loglikelihood values. So we have been basically subtracting -2ll of model 2(least restrictive) from -2llmodel3(more restrictive). However, if working with just regular log likelihood values, you'd subtract model 3(more restrctive) from model 2(least restrictive). I am not sure how the -2ll in our data effect the calculation of this LRT statistic, and whether this is responsible for the negative LRT statistic value's we've found. On previous items, we did not find negative LRT statistic values when comparing these models (using -2ll for all models, model 1 minus model 2, and model 3 minus model 2).
If anyone has ANY idea at all, please throw it at me. Or any info regarding likelihood ratio testing. Can a chi-square value be negative (I don't think so)? So do we take the absoulte value of these negative numbers for our chi-square value?? please...HELP!
