BookmarkSubscribeRSS Feed
cmhlavacek
Calcite | Level 5

I'm a SAS novice, so thanks in advance for bearing with me...

I have straightforward 2 sample t-tests for my variables using PROC TTEST for my continuous variables and PROC FREQ for my categorical variables. I get the 95% CI for the mean and SD in the output, but I want to report confidence intervals for my p-value. Likewise, I have multivariable logistic and linear regressions for some of these variables, but can't generate 95% confidence intervals for the p-values.

A sample of my code is attached.

Thanks!


SAS code examples.PNG
7 REPLIES 7
Reeza
Super User

I can honestly say I've never heard of a confidence interval for a p-value. 

Can you point to any statistical references for such values?

lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12

One does not calculate confidence intervals for p values.

SteveDenham
Jade | Level 19

Hmm--I'm going to disagree a bit with that assertion, The Edwards and Berry adjustment (ADJUST=SIMULATE) in the linear modeling procs allows you to specify EPS=, where epsilon is a confidence interval on the quantile.  This would establish an accuracy confidence of 100*(1 - epsilon)%.

So, the original poster could redo the analysis in PROC GLM, by changing the code to:

lsmeans illicits/cl diff=all adjust=simulate(EPS=0.05 report seed=1);

That should give some idea of a 95% CI.  Note that the default value for EPS is 0.01, which would give 99% confidence.

I hope I have not marched someone over the cliff with this, though.

Steve Denham

lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12

Steve, you are right that one is utilizing an aspect of the uncertainty in p with this procedure. Somehow, I doubt that this is what the OP was trying to do. But we haven't heard back....

PGStats
Opal | Level 21

I'm not sure my poor human brain is made to handle uncertainty about uncertainty measures... Smiley Happy

PG

PG
lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12

The p value is random variable, dependent on the data, model being used, and so on. Under the null hypothesis, p has a uniform distribution with support on the real line from 0 to 1. But the latter fact doesn't help when the null hypothesis is false, or when you don't know whether or not H0 is true (the usual situation). Under Ha, p can have a complex distribution.

SteveDenham
Jade | Level 19

That would explain why the resampled values obtained with the REPORT option are all over the damn place, and not symmetric...

Steve Denham

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 7 replies
  • 2964 views
  • 0 likes
  • 5 in conversation