McNemar's test tests whether the discordant proportions are significantly different from each other.
In the context of your problem, let:
P[UpVote] = sum of all students who upvoted
P[DownVote] = sum of all students who downvoted
Students whose scores do not change are not factored into the chi-square statistic.
The null hypothesis is P[UpVote] = P[DownVote], meaning the total upvotes and downvotes are statistically equivalent
The alternative is P[UpVote] != P[DownVote].
McNemar's test is testing whether or not these values are significantly different from each other.
Your description " "did enough opinions change - whether up or down, doesn't matter - to be able to say that the new information changed opinions?", does not make a distinction between upvotes and downvotes.
If you're treating upvotes and downvotes the same, then no, McNemar's test is not the right choice.
The null is "the votes did not change": every relative vote (before vs. after) = 0
Alternative is "the votes changed": every relative vote (or a significant enough amount of them) = 1
The null is "the votes did not change": every relative vote (before vs. after) = 0
CORRECTION Alternative is "the votes changed": ABSOLUTE VALUE of every relative vote (or a significant enough amount of them) = 1
For each student, let:
D = After - Before
Here, D=0 corresponds to no change in score.
If D is normally distributed, use the t-test mentioned by others earlier in this thread. If it is not normally distributed, use the Wilcoxon Signed-Rank Test, which does not assume normality. Both tests assume the distribution is symmetric.
This blog describes both in more detail and how to apply in SAS: https://blogs.sas.com/content/iml/2023/07/19/wilcoxon-signed-rank.html
So there is no way to measure absolute change? Theoretically, by the t-test every score could change by 2 but as long as half of them are +2 and half are -2, the test will reflect no change at all. Is this correct, or am I missing something?
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.