topic Re: comparing pre and post data from multiple choice question in SAS Software for Learning Community

comparing pre and post data from multiple choice question

ecampos2 — Thu, 13 Jan 2022 19:14:16 GMT

Hello, I have a question

i have data from a survey... it was knowledge based... and there is pre and post data.

it is a multiple choice questions, with 4 answers and only one is correct.

how can i run a t test with this data?

Re: comparing pre and post data from multiple choice question

Reeza — Thu, 13 Jan 2022 19:19:39 GMT

What is the hypothesis?
What specifically do you need to know how to do?
Have you imported your data from Excel? Do you know how to structure your data for SAS to do a TTEST? Which procedure to use?

It seems like you would need a paired t-test so this example seems like it aligns with your question:
https://libguides.library.kent.edu/SAS/PairedSamplestTest

Re: comparing pre and post data from multiple choice question

Reeza — Thu, 13 Jan 2022 19:20:46 GMT

FYI - I looked at your data briefly, it appears to be categorical/quantitative so are you sure a T-Test is what you want? Again, what is the hypothesis?

Re: comparing pre and post data from multiple choice question

ecampos2 — Fri, 14 Jan 2022 00:29:57 GMT

the hypothesis changes for each question, for example question 1 was which food raised blood sugar levels the most... in that case answers were given (the right answer is 1 small apple) and we are comparing the increase in wright answers.

Question 2 is what non-starchy vegetables are how much of your plate? with the correct answer being 1/2.

Re: comparing pre and post data from multiple choice question

Reeza — Fri, 14 Jan 2022 00:56:47 GMT

Then first convert your data to numeric values that can be analyzed.

I'm assuming the overall question then is how many people get the answers correct before and after? So the hypothesis is actually the same across all questions if you generalize it a bit more with each question being different.

I would probably restructure my data to be something like:

User Question PreAnswerCorrect PostAnswerCorrect
1 1 1 1
1 2 0 1
1 3 0 1
1 4 1 1
2 1 1 1
2 2 1 0
2 3 0 0
2 4 1 0

Then you can use PROC TTEST as in the example above:

proc sort data = have;
by question;

proc ttest data = have;
by Question;
paired PreAnswerCorrect * PostAnswerCorrect;
run;

@ecampos2 wrote:

the hypothesis changes for each question, for example question 1 was which food raised blood sugar levels the most... in that case answers were given (the right answer is 1 small apple) and we are comparing the increase in wright answers.

Question 2 is what non-starchy vegetables are how much of your plate? with the correct answer being 1/2.

Re: comparing pre and post data from multiple choice question

ecampos2 — Fri, 14 Jan 2022 01:39:14 GMT

thank you much....the data you provided had 4 rows.... wouldn't it only have 2 one for pre and one for post?

I tried to do what you suggested and for wrong answers i put 0 and right answers i put 1.... did we need to analyze each possible answer that was given?

Re: comparing pre and post data from multiple choice question

PaigeMiller — Fri, 14 Jan 2022 11:52:16 GMT

@ecampos2 wrote:

thank you much....the data you provided had 4 rows.... wouldn't it only have 2 one for pre and one for post?

I tried to do what you suggested and for wrong answers i put 0 and right answers i put 1.... did we need to analyze each possible answer that was given?
pre	post 
0	1
1	1
1	1
1	1
1	1
0	1
1	1
0	1
0	0
1	1
1	1
1	1
0	0
1	1
1	1
1	1
1	1
1	1
0	0
1	1
0	0
1	1
0	0
1	1
0	0
0	0
1	0
0	1
0	0
1	1
0	0
0	1
0	0
0	1
1	1
1	1
1	1
1	1
1	0
1	1
0	0

This doesn't indicate which subject is being evaluated, and it doesn't indicate what question was correct, or not. So I like the data layout from @Reeza

Also, please don't tell us what you tried doesn't work and provide no other information, because we don't know what you tried. Show us the code you tried, and the result.

Re: comparing pre and post data from multiple choice question

Reeza — Fri, 14 Jan 2022 16:01:49 GMT

No, it would not have just two.

I assume you measured pre and post on the same person by the way. How would you identify which record goes with which person? That's why you need the ID.

Then adding Question in allows you to do all questions at once by adding in the BY QUESTION portion. Those are the two additional variables that make this easier to analyze.

Do you absolutely require them? No. Will they make your life easier and help make sure you don't mistakes? Yes. Do they make the work easier? Yes. Otherwise, how will you tell each data set apart for each question? You'll have 4 data sets (1 for each question instead) and 4 sets of code. If you need to change something one place you'll now have to change it in 4 places.

did we need to analyze each possible answer that was given? did we need to analyze each possible answer that was given?

Depends on if you want to include that data in your analysis. If you do, then yes. You can do that using a program though, not manually.

Re: comparing pre and post data from multiple choice question

ecampos2 — Fri, 14 Jan 2022 16:44:43 GMT

Thank everyone so much!

For some reason Im still not getting a t- test. I'm getting this error on SAS:

ERROR: Variable pre in list does not match type prescribed for this list.

I really don't understand how does sas knows what question is right or wrong?

I also changed the data format to what rezza suggested. I row patient id, 1 row question number, and pre and post response. I imported the doc i attached (question 1 pre) and then put the code you suggested

proc sort data = work.import;
by question;

proc ttest data = work.import;
by Question;
paired pre*post;
run;

Re: comparing pre and post data from multiple choice question

Reeza — Fri, 14 Jan 2022 16:55:59 GMT

@ecampos2 wrote:

Thank everyone so much!

For some reason Im still not getting a t- test. I'm getting this error on SAS:

ERROR: Variable pre in list does not match type prescribed for this list.

Usually this means a variable is character when it should be numeric.

From your file attached, I'm guessing PRE is a character variable when it should be numeric (based on alignment, just a guess).

I really don't understand how does sas knows what question is right or wrong?

Didn't you code the data as 1 being correct, 0 being wrong?

Otherwise how did you code your 0/1 in the data shown?

Re: comparing pre and post data from multiple choice question

ecampos2 — Fri, 14 Jan 2022 17:02:11 GMT

Yes so the question on the survey was a knowledge based question, "Which food raised blood sugar level the most?". There was 4 possible answers 1/3 cheddar cheese, 1 oz pork bacon, 3 oz hamburger patty, and 1 small apple. (The correct answer was 1 small apple)

I put 0 for people who answered 1/3 cheddar cheese, 1 oz pork bacon, or 3 oz hamburger patty, and 1 for people who answered the correct answer; 1 small apple.

Re: comparing pre and post data from multiple choice question

ecampos2 — Fri, 14 Jan 2022 17:17:38 GMT

Also how could i change my pre to be a character not numeric?

Re: comparing pre and post data from multiple choice question

Reeza — Fri, 14 Jan 2022 18:10:37 GMT

@ecampos2 wrote:

Also how could i change my pre to be a character not numeric?

You want numeric, not character.

Go back and fix how you calculated it. Show that code and we can help you fix it. If you put the 0/1 in quotes that's one way to have created them as character.

@ecampos2 wrote:

Also how could i change my pre to be a character not numeric?

Re: comparing pre and post data from multiple choice question

ecampos2 — Sat, 15 Jan 2022 16:38:36 GMT

okay i have attached the data in the format you recommended (question 1 pre.xlsx).

I first imported the doc in sas.

once downloaded my sas code was:

PROC TTEST DATA=work.import ALPHA=.05;
    PAIRED pre*post;
RUN;

Then it gives me the error: ERROR: Variable pre in list does not match type prescribed for this list.

Re: comparing pre and post data from multiple choice question

Reeza — Sat, 15 Jan 2022 18:42:18 GMT

Same as the previous error, so my answer is pretty much the same. Show all of your code. The import, where you import the data, where you recode the data to 0/1 or recode pre/post to 0/1.

Re: comparing pre and post data from multiple choice question

PaigeMiller — Sat, 15 Jan 2022 20:46:57 GMT

Most of us will not download XLSX (or other Microsoft Office) attachments, as they can be a security threat. The proper format to provide data is via SAS data step code (instructions).

Re: comparing pre and post data from multiple choice question

ecampos2 — Tue, 18 Jan 2022 01:05:29 GMT

okay the first image with the writing is the original from the survey. There were 4 possible answers, the right answer being:1 small apple.

I did what you recommended, and recoded it to what the second image is and if somebody answered wrong it would be 0 and if it was right, they got a 1.

In sas i imported the data. and then put the code listed above.

PROC TTEST DATA=work.import ALPHA=.05;
    PAIRED pre*post;
RUN;

Re: comparing pre and post data from multiple choice question

Tom — Tue, 18 Jan 2022 01:44:03 GMT

If that second photograph is how EXCEL shows your spreadsheet then the reason the variable is CHARACTER in SAS is because you made it character in EXCEL. That is what the little green triangles mean.

Re: comparing pre and post data from multiple choice question

Reeza — Tue, 18 Jan 2022 15:46:27 GMT

Typically what would happen is you import the raw data, you do not recode it manually. This is actually important as it allows for full scientific reproducibility and is an important part of the scientific process. Papers these days have been retracted because someone coded things as 1 or 0 and then someone else flipped that coding. Doing things manually is not the correct approach.

I would expect to see code that looks like the following:

*import raw data;
proc import datafile='myExcel.xlsx' out=rawData dbms=XLSX replace;run;

*clean up data;
data cleanData;
set rawData;
if Q3 in ('1 small apple') then Q3_Code = 1;
else Q3_Code=0;
/*rest of data cleaning required*/

run;

*Wrangle into format for pre-post, not sure because you have shown only the pre file;
data clean_long_data;
set cleanData;
/*merge cleanDatapre cleandatapost;*/
run;

*T-Test;
proc ttest data=clean_long_data;
....
run;

So far all you've shown is the first step. Which is the last step in what you need. Somewhere in that code above is where you would apply your fix for the data types.

Re: comparing pre and post data from multiple choice question

tom_grant — Wed, 19 Jan 2022 17:05:35 GMT

Can you state your null hypothesis & alternative hypothesis precisely? Do you really want to have a test for each question or do you want to examine whether the % of correct answers across all respondents & all questions increased in the post survey?

Even though you can code responses as numeric (0 = incorrect, 1 = correct) for pre & post survey, you are really working with a categorical variable at the individual & question level.

You may find that a chi-sq test is more appropriate if examining each question with the cohorts being correct responders & incorrect responders on pre-test.

If you are evaluating the effectiveness of the intervention across all questions, you could measure the difference in pre & post test scores.

Example:

ID Pre-score Post-score Diff

1 75% 85% +10%

2 50% 65% +15%

...

Again, clearly stating your hypothesis would be very helpful