Right now in statistical analysis, we always want to present an actual p value if significant rather than, e.g., p<0.0001, for a statistic. I am wondering whether it is possible to directly output an actual p value in SAS analysis.
Here is one way.
You can use ods output to create a sas file for each relevant part of the output from a procedure. For instance, you could use:
proc glimmix ;
ods output tests3=tests3;
class trt b ;
model y = trt|b;
to store the Type 3 test results in a file called tests3 (these are the F tests, degrees of freedom, P values). The p value is called ProbF in this file (just print it to check this out). Then in a print procedure, you can choose a different format for ProbF. Below is one way, where I request 12 decimal places for P.
proc print data=tests3;
format ProbF pvalue15.12; *<--no space in pvalue15.12 ;
The relevant ods file is different for each procedure. Check the documentation for your desired procedure.
Now comes the question: Why? Why do you need the "actual" p value? Once you get to values this small, even trivial deviations from the assumptions in the analysis lead to changes that look like something is going on. For instance, a change from 0.0000012 to 0.000006 looks like a five-fold change in the p value, yet this amount of change in absolute value isn't at all uncommon with even slight changes in the underlying distribution of the residuals.
Wouldn't the effect size be more useful? You already know that it is unlikely that the deviation you see is due to chance alone. It strikes me (and that's just me and my personal opinion) that knowing the change observed is five sigma as opposed to three sigma would be more useful. The conversion to p value from effect size is really, really dependent on assumptions about the distribution.
Maybe I've been reading too many physics related things lately...
Or maybe I just went through the same kind of argument with a QA auditor.
Actually, it is common to look at the "actual" small p values in some fields, such as in molecular biology. This is not my area, but in the analysis of microarrays, there are hundreds or thousands of genes tested for a signal in a single experiment, and -log(p) is determined for each, or for differences, etc., and these are then graphed in various ways. Check out the classic: Wolfinger et al. (2001; Journal of Computational Biology 8: 625-637). I know that a standardized effect size could show the same thing, but it appears to be the tradition in some fields to look at -log(p), where p is not truncated. Multiplicity adjustments are also done, depending on the study.
Thanks, lvm, for an answer that makes it clear why you might need "actual" p values. Microarray data is so information dense that even I can see that the CLT has made deviations from normality trivially small, and so the p values are meaningful.