BookmarkSubscribeRSS Feed
larry2011
Calcite | Level 5
Right now in statistical analysis, we always want to present an actual p value if significant rather than, e.g., p<0.0001, for a statistic. I am wondering whether it is possible to directly output an actual p value in SAS analysis.

Thanks
5 REPLIES 5
lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12
Here is one way.
You can use ods output to create a sas file for each relevant part of the output from a procedure. For instance, you could use:

proc glimmix ;
ods output tests3=tests3;
class trt b ;
model y = trt|b;
run;

to store the Type 3 test results in a file called tests3 (these are the F tests, degrees of freedom, P values). The p value is called ProbF in this file (just print it to check this out). Then in a print procedure, you can choose a different format for ProbF. Below is one way, where I request 12 decimal places for P.

proc print data=tests3;
format ProbF pvalue15.12; *<--no space in pvalue15.12 ;
run;

The relevant ods file is different for each procedure. Check the documentation for your desired procedure.
larry2011
Calcite | Level 5
Thanks, lvm.
SteveDenham
Jade | Level 19
Now comes the question: Why? Why do you need the "actual" p value? Once you get to values this small, even trivial deviations from the assumptions in the analysis lead to changes that look like something is going on. For instance, a change from 0.0000012 to 0.000006 looks like a five-fold change in the p value, yet this amount of change in absolute value isn't at all uncommon with even slight changes in the underlying distribution of the residuals.

Wouldn't the effect size be more useful? You already know that it is unlikely that the deviation you see is due to chance alone. It strikes me (and that's just me and my personal opinion) that knowing the change observed is five sigma as opposed to three sigma would be more useful. The conversion to p value from effect size is really, really dependent on assumptions about the distribution.

Maybe I've been reading too many physics related things lately...
Or maybe I just went through the same kind of argument with a QA auditor.

SteveDenham
lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12
Actually, it is common to look at the "actual" small p values in some fields, such as in molecular biology. This is not my area, but in the analysis of microarrays, there are hundreds or thousands of genes tested for a signal in a single experiment, and -log(p) is determined for each, or for differences, etc., and these are then graphed in various ways. Check out the classic: Wolfinger et al. (2001; Journal of Computational Biology 8: 625-637). I know that a standardized effect size could show the same thing, but it appears to be the tradition in some fields to look at -log(p), where p is not truncated. Multiplicity adjustments are also done, depending on the study.
SteveDenham
Jade | Level 19
Thanks, lvm, for an answer that makes it clear why you might need "actual" p values. Microarray data is so information dense that even I can see that the CLT has made deviations from normality trivially small, and so the p values are meaningful.

Thanks again!

SteveDenham

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 5 replies
  • 3189 views
  • 0 likes
  • 3 in conversation