Confidence limits using the Miettinen-Nurminen stratified method as described in the 1985 paper

[vjaquet]

The FDA asked us to use a method other than Wald and strongly suggested that we use Miettinen-Nurminen 1985 (MN) for computing confidence limits (CL) at summary level. This method is becoming increasingly popular and is used by many pharmaceutical companies, including Merck, Abbvie, and Moderna (cf. simple Google search).

SAS has provided a partial implementation of MN scoring through COMMONRISKDIFF(CL=score) where the within-stratum calculation follows MN 1985 but inverse-variance stratum weights are used at the summary level. This method comes from the 2013 Agresti book. Has this method ever been published in a peer-reviewed article? Has it ever been used in a clinical trial submission? Is this method popular, i.e., number of citations? The lack of visibility in the use of this "SCORE" method prevent the use of this algorithm in a submission to health authorities.

The current alternative is to use R. Indeed, two packages have been developed to calculate CL based on MN 1985 method: metalite.ae (Merck) and scoreci. SAS users have already written about this 4 years ago. 

I am a biostatistical manager at Bristol Myers Squibb. This issue has risen to the VP level as we plan to adopt MN 1985 in all our submissions in all the therapeutic areas. Confidence limits play an important role because they influence key analyses such as futility analysis for the primary outcome.

Will you provide this algorithm and if so, when?

[Ksharp]

That is a good request. 

1) Maybe you could get one immediately if @Rick_SAS would like to write some IML code for this CI .

2) Or could consult with @StatDave  ,he might know some PROC or MACRO to get this CI ?

3) Or could you leave some resource, like Paper ,URL , Algorithm , Example ... If I have time ,I might wrote some code for this CI . 

[vjaquet]

Update:

This method is now available in SAS Viya >12.2022: COMMONRISKDIFF(CL=MN). An alternative using a fixed Cochran weight is also available: COMMONRISKDIFF(CL=MNMH).

An alternative is to use the Ratesci package in R by Peter Laud, 2017, and use PROC_R to compute R code within SAS. Note that there is a problem with the scoresci function from Ratesci, as it does not handle well when there are all responders or no responders among all strata in the same arm. The solution is to set p2ds = 10^-7 when p2ds = 0 and p2ds = 1-10^-7 when p2ds = 1 and not to correct the weight vector (wt) as originally coded.

[Ksharp]

BTW, Is this method available under SAS9 (PROC FREQ)?

[Ksharp]

"SAS has provided a partial implementation of MN scoring through COMMONRISKDIFF(CL=score) where the within-stratum calculation follows MN 1985 but inverse-variance stratum weights are used at the summary level."

Do you mean the following ?

[vjaquet]

no COMMONRISKDIFF(CL=score) is not Miettinen and Nurminen 1985. It uses score, but the weights of the strata are not fitted, but fixed to the inverse of the variance. We have evaluated this algorithm at BMS and we don't like it because the mean is also calculated with the inverse variance weight and not with CMH. This tends to underestimate the true rate and therefore this algorithm is different from the other methods.

Miettinen and Nurminen, like their 1985 paper, is now only available in SAS Viya >12.2022: COMMONRISKDIFF(CL=MN). It is not clear if SAS will ever release the MN algorithm for SAS9 aka SAS Studio and if so, when. Our evaluations at BMS have shown that this algorithm performs very well in most cases, even when there are no responders in all strata of the placebo arm. We tested it under different conditions and we always got the better coverage of all the algorithms available in SAS. The results from MN were closest to 2.5-95-2.5%, with 2.5% confidence intervals (CIs) lower than the true mean, 95% CIs including the true mean, and 2.5% CIs higher than the true mean. The score method tends to have a range of 4.5-95-0.5%, i.e., it underestimates the true rate.

[Ksharp]

I just tested the following code under my SAS9.4M7 . It is not running.

I don't know if SAS9.4M8 could run this code.

14   proc freq data=sashelp.heart;
15   table bp_status*status*sex/commonriskdiff(CL=MN test);
                                                  --
                                                  22
                                                  76
ERROR 22-322: 语法错误，期望下列之一: (, K, KLINGENBERG, MH, MINRISK, MR, NEWCOMBE, NEWCOMBEMH, NEWCOMBEMINRISK, NEWCOMBEMR,
              NO, NONE, SCORE, WILSON.
ERROR 76-322: 语法错误，语句将被忽略。
16   run;

NOTE: 由于出错，SAS 系统停止处理该步。
NOTE: “PROCEDURE FREQ”所用时间（总处理时间）:
      实际时间          0.01 秒
      CPU 时间          0.00 秒

[vjaquet]

As I wrote it is currently only available in SAS Viya >12.2022 which has a monthly update and not in SAS9.4. SAS support does not know if SAS will ever release the MN algorithm for SAS9.x aka SAS Studio and if so, when.

[yabwon]

@Ksharp it looks like someone already did it in 2008 using SAS 9.1.3. At least if I correctly understood this article: https://www.markstat.net/en/images/stories/lu_asa_2008.pdf

[Ksharp]

yabwon,

Yeah. But I didn't see any sas code in the paper yet.

[yabwon]

@Ksharp 

I know 😕 unfortunately I couldn't find any additional things/codes for that paper.