topic PROC IRT Graded Response Model -- Interpreting Item Characteristic Curves in Statistical Procedures

PROC IRT Graded Response Model -- Interpreting Item Characteristic Curves

EricVanceMartin — Mon, 10 Aug 2020 18:03:46 GMT

I am as new to IRT as a freshly hatched chick. I recognize this is a more general IRT question, but maybe

I am exploring the results from an instrument in which all items have a five-point Likert response set. The responses reflect ordinal phases of policy adoption, in which 1 represents no consideration of the policy and 5 represents full implementation. IRT here models respondent policy responsiveness ("ability"), policy implementation "difficulty," and item discrimination with a graded response model.

My variables have two factors. After sussing out the factors, I have run PROC IRT for each factor separately in order to look at the ICCs and Item Information Curves. As I expect, the legend of the Item Characteristic Curves panels shows values 1-5, corresponding to the values of the responses, with colored lines.

I have noticed that for items in which the Item Information Curve peaks over a negative value of the latent trait, the peaks of the curves for the responses are generally ordered in a way that makes sense-- at a low level of the trait, the most frequent response is "1," or no consideration.

However, for items in which the IIC peaks over a positive value of the trait, the response curves are generally reversed--at a low level of the trait, the most frequent response is "5," or full implementation. On the one hand, this makes sense in that "easy" policies should be more frequently implemented by less "able" respondents. On the other hand, it does not make sense to me that "easy" policies should be less likely to be implemented by more "able" respondents.

There's clearly something I don't understand here or something very wrong with my conceptualization of what the data reflect. I'm hoping it's the former. Can you shed any light?

Re: PROC IRT Graded Response Model -- Interpreting Item Characteristic Curves

EricVanceMartin — Mon, 10 Aug 2020 20:41:08 GMT

I should clarify that it doesn't make sense for "less able" respondents to implement "easier" policies more frequently. In fact, it seems to me that the probability of full implementation should still still increase across the latent trait, just less less steeply.

In any case, a couple of thoughts. Some item do have negative slopes, but it did not look like this was associated with what I'm seeing in the graphs.... er, should it?

In addition, in regard to the item information curve peaking over a negative or positive value of the latent trait, the items in the factors have this characteristic in common (i.e., either all peaking over a positive value or all peaking over a negative value).

Re: PROC IRT Graded Response Model -- Interpreting Item Characteristic Curves

EricVanceMartin — Mon, 10 Aug 2020 23:39:58 GMT

OK, after more experimenting, I now see that when the items in the two factors are entered separately, their coefficients can be dramatically different from when they are entered together.