02-07-2013 11:24 PM
I have survey that is live right now with many 7-point Likert-type responses. Respondents will be categorized into anywhere from 3 to 9 groups, depending on the question, and these groups will not be equally-sized. Given that the data will be arranged such that the group is one variable and the responses each have their own variables, what is the best way to approach searching for differences across groups for each question? My instinct is to use the NPAR1WAY procedure to search for differences within one question across groups, but I'm drawing a blank on comparing the responses to one question with the responses to another question to see which had the higher scores. All advice will be appreciated.
02-11-2013 08:27 AM
Sounds a little like you need Cochran-Mantel-Haenszel stats, looking at row mean scores, except that group is one of the variables, so that is probably out. I am not at all comfortable with log-linear models, such as are implemented in PROC CATMOD, but I am relatively certain that you may have to use that tool to get what you are after. Check out some of the examples for the procedure to see if it looks like they may fit what you have. The repeated measures examples look like they may be the most helpful.
02-28-2013 11:18 AM
Thanks, Steve. I looked up the PROC CATMOD procedure and it led me through a few other suggestions to everything I needed to find. It turns out a brute force correllary to Occam's Razor applied, where I simply created new variables with the pariwise differences between the variables in question and PROC FREQ'ed and PROC NPAR1WAY'ed my way to the results I needed. I was thinking there would be a sleek solution that did it all at once, but sometimes I guess it pays to forge ahead like a bull in a china shop.
03-01-2013 08:18 AM
So long as the new variables are understandable/meaningful to your analysis objectives, this is probably the optimal approach. It's interesting that if you calculate differences in 1 to 7 Likert scores, you end up with a -6 to 6 range that should be unimodal. Appealing to the Law of Large Numbers (you do have large numbers, I hope), you could even get an approximate answer by considering the variable to be continuous and using generalized linear models methods. Probably no different than the PROC FREQ answers for Row Mean Scores.
03-01-2013 10:56 AM
Unfortunately, my n is only slightly over 100, and when I break down into a group-by-group analysis, my groiup sizes fall into the teens in some cases, keeping the CLT as a dangerous proposition to invoke. As such I performed tests under the traditional normality assumption (which the recipients of the analysis are familiar with) and non-parametric tests which actually made more sense to use. It actually made it easier to argue for the non-parametric results by showing both results. The ones with extremely high p-values or extremely low p-values were such under both tests, and the similarity of results at the tails made the group more accepting of the non-parametric results that differed in the p < 0.05 conclusion.