topic Random effects mixed model with 95% prediction band in Statistical Procedures

Random effects mixed model with 95% prediction band

MacRowJab — Tue, 16 Nov 2021 15:12:04 GMT

Hi there!

I'm trying to do a surrogate endpoint validation on aggregated data (trial level).

What I have is a bunch (N ~ 50) of studies reporting an estimate (hazard ratio [CI95%]) for the "true" endpoint (overall survival) and another estimate for the surrogate endpoint (progression-free survival).

My aim is to compute the surrogate threshold effect, that is the minimum amount of PFS effect able to predict a non-zero (significant) effect on OS. I'd like to do that in accordance with this publication, which says:

we fitted a random effects mixed-model to the data with moderator HRPFS and outcome variable hazard ratio of OS (HROS) weighted by standard error (SE) of OS using the restricted maximum likelihood (REML) estimator for the amount of heterogeneity. [...] Based on the regression fit, we calculated a prediction band to a significance level of α = 0.05 for HROS. Meta-regression model and prediction values were implemented with R [16] using functions rma.uni and predict.rma from metafor package [17]. The STE resulted from the intersection of the upper prediction limit curve and the horizontal where HROS = 1 (zero effect).

In SAS PROC MIXED, do I have the opportunity do plot 95% prediction bands from which I could obtain the STE? If yes, could you give me a sample code?

Thank you very much in advance!

Re: Random effects mixed model with 95% prediction band

sbxkoenk — Tue, 16 Nov 2021 20:02:09 GMT

You can start here:

proc mixed data=rc METHOD=REML COVTEST;
   class Batch;
   model Y = Month / s residual outpred=work.abc OUTPREDM=work.xyz;
   random Int Month / type=un sub=Batch s;
run;

... and tell us what else you need.

Thanks,

Koen

Re: Random effects mixed model with 95% prediction band

MacRowJab — Tue, 16 Nov 2021 20:30:53 GMT

Thank you a lot, Koen!

I tried the syntax

PROC MIXED data=surr METHOD=REML COVTEST;
   CLASS trial;
   MODEL oshr = pfshr / S RESIDUAL OUTPRED=pred OUTPREDM=predm;
   RANDOM INT pfshr / TYPE=un SUBJECT=trial S;
run;

with

oshr: hazard ratio on overall survival

pfshr: hazard ratio on progression-free survival

trial: study from which estimates are extracted.

In the pred and predm datasets, I get empty LCL and UCL columns for the predicted mean. The log says:

NOTE: Convergence criteria met but final Hessian is not
positive definite.
NOTE: Estimated G matrix is not positive definite.

When I apply a weight, it gets even worse, so that the predicted datasets are not being written at all:

WARNING: Stopped because of too many likelihood evaluations.
NOTE: The data set WORK.PRED has 0 observations and 0 variables.
WARNING: Data set WORK.PRED was not replaced because new file
is incomplete.
NOTE: The data set WORK.PREDM has 0 observations and 0
variables.
WARNING: Data set WORK.PREDM was not replaced because new file
is incomplete.

Is there an opportunity to get the prediction intervall anyway?

Re: Random effects mixed model with 95% prediction band

sbxkoenk — Tue, 16 Nov 2021 20:37:37 GMT

Hello,

Try to add

parms / ols;

statement to your proc mixed code.

Or have a look into below papers:

They help customers with various issues (including non-convergence) in MIXED, GLIMMIX, NLMIXED.

Good luck,
Koen

Re: Random effects mixed model with 95% prediction band

SteveDenham — Tue, 23 Nov 2021 17:52:18 GMT

NOTE: Convergence criteria met but final Hessian is not
positive definite.
NOTE: Estimated G matrix is not positive definite.

This is telling you that you have over-specified the random effects based on the data you have.

WARNING: Stopped because of too many likelihood evaluations.

This can be avoided in many cases by simply increasing the number of iterations using the MAXITER= option in the PROC MIXED statement. However, look through the iteration history and see if it is approaching convergence, or if it is jumping around. MAXITER will solve the first case, but the second will require more appropriate MODEL and RANDOM statements.

SteveDenham

Re: Random effects mixed model with 95% prediction band

jiltao — Wed, 24 Nov 2021 15:54:02 GMT

For the message --

WARNING: Stopped because of too many likelihood evaluations.

try adding singular=1e-8 in the MODEL statement in PROC MIXED to see if that helps.

Thanks,

Jill