topic TTEST results different between versions? in Statistical Procedures

TTEST results different between versions?

michael_at_work — Fri, 16 Feb 2024 14:46:06 GMT

Hi,
Has there been any change in the TTEST between SAS Version 9.2 and 9.4?
I was told that a value for a sample size determined using version 9.2 is now different if I calculate it using the same parameters in version 9.4.
I'd appreciate confirmation of a change or confirmation that the calculation has not changed and that my result should be the same given the same parameters.

If anyone is curious to calculate the sample size, here are the relevant parameters: Two-sided T Test for Mean Difference, normal distribution, exact method, null difference = 0, alpha = 0.01, mean difference = 4, nominal power = 0.9, number of sides = 2, and standard deviation = 5. Version 9.4 gives an answer of 49 whereas 9.2 supposedly gives an answer of 48. I've tried the calculation with other software and always get 49 so is version 9.2 buggy or does it also correctly give a value of 49? Appreciate any confirmation - thanks!

Re: TTEST results different between versions?

ballardw — Fri, 16 Feb 2024 17:52:38 GMT

Your subject line implies something about the TTEST Procedure but your question is apparently about Proc Power.

Please provide the EXACT code you run. It is very easy to say you are using XYZ parameters but a narrative may miss one or more other options that the code would clarify.

Since SAS 9.2 was replaced more than 8 years I sort of wonder why this is an important question.

Re: TTEST results different between versions?

michael_at_work — Fri, 16 Feb 2024 18:17:22 GMT

Thanks for the reply. I was given the result (n=96, 48 for each group) for a sample size calculation which was used in a clinical trial about 2 years ago. This was based on the parameters I mentioned and when I asked to see these parameters and the calculation to get the sample size to include in the paper, the result I got was now 98. I thought this was odd and asked why it is not 96 and was told that the software had changed from version 9.2 to 9.4 and now the result is just different. To be honest, I find this reason to be very suspect. I was hoping to verify that 96 (48 for each group) is not what version 9.2 provides as an answer and then I can figure out where the sample size came from.

Re: TTEST results different between versions?

FreelanceReinh — Fri, 16 Feb 2024 18:49:27 GMT

Hi @michael_at_work and welcome to the SAS Support Communities!

The relevant formulas in the v9.2 and v9.4 documentation are identical, as far as I see. The result, 49 per group, matches what I obtain manually with a (different, but apparently equivalent) textbook formula. I found documented bug fixes for PROC POWER between v9.2 and v9.4, e.g., Problem Note 42378, but not for the TWOSAMPLEMEANS statement. Also, your numeric example is not a borderline case where numeric precision could be an issue.

Did those who obtained 48 per group, by any chance, use the NFRACTIONAL option and then round the "Fractional N per Group" (which is 48.186103) to the nearest integer instead of using the "Ceiling N per Group"?

Re: TTEST results different between versions?

data_null__ — Fri, 16 Feb 2024 19:10:54 GMT

42378 - TWOSAMPLEFREQ in PROC POWER returns incorrect result for TEST=FISHER (sas.com)

49838 - TWOSAMPLESURVIVAL statement options EVENTSPERGROUP=, GROUPEVENTS=, and EVENTSTOTAL= produce incorrect results (sas.com)

32395 - Wrong numbers from ONECORR or MULTREG statement in PROC POWER (sas.com)

Re: TTEST results different between versions?

michael_at_work — Fri, 16 Feb 2024 21:33:54 GMT

Thanks for the confirmation. I'm not sure about the fractional option as it was not explicitly mentioned but I can follow up on that.