turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Stat Procs
- /
- Confidence Intervals for a p-value

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

02-28-2015 07:12 PM

I'm a SAS novice, so thanks in advance for bearing with me...

I have straightforward 2 sample t-tests for my variables using PROC TTEST for my continuous variables and PROC FREQ for my categorical variables. I get the 95% CI for the mean and SD in the output, but I want to report confidence intervals for my p-value. Likewise, I have multivariable logistic and linear regressions for some of these variables, but can't generate 95% confidence intervals for the p-values.

A sample of my code is attached.

Thanks!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

02-28-2015 07:26 PM

I can honestly say I've never heard of a confidence interval for a p-value.

Can you point to any statistical references for such values?

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

03-01-2015 05:54 PM

One does not calculate confidence intervals for p values.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

03-02-2015 12:46 PM

Hmm--I'm going to disagree a bit with that assertion, The Edwards and Berry adjustment (ADJUST=SIMULATE) in the linear modeling procs allows you to specify EPS=, where epsilon is a confidence interval on the quantile. This would establish an *accuracy confidence* of 100*(1 - epsilon)%.

So, the original poster could redo the analysis in PROC GLM, by changing the code to:

lsmeans illicits/cl diff=all adjust=simulate(EPS=0.05 report seed=1);

That should give some idea of a 95% CI. Note that the default value for EPS is 0.01, which would give 99% confidence.

I hope I have not marched someone over the cliff with this, though.

Steve Denham

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

03-03-2015 02:02 PM

Steve, you are right that one is utilizing an aspect of the uncertainty in p with this procedure. Somehow, I doubt that this is what the OP was trying to do. But we haven't heard back....

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

03-03-2015 03:57 PM

I'm not sure my poor human brain is made to handle uncertainty about uncertainty measures...

PG

PG

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

03-03-2015 04:06 PM

The p value is random variable, dependent on the data, model being used, and so on. Under the null hypothesis, p has a uniform distribution with support on the real line from 0 to 1. But the latter fact doesn't help when the null hypothesis is false, or when you don't know whether or not H0 is true (the usual situation). Under Ha, p can have a complex distribution.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

03-04-2015 02:42 PM

That would explain why the resampled values obtained with the REPORT option are all over the damn place, and not symmetric...

Steve Denham