topic Re: TOST Equivalence Test in SAS Programming

TOST Equivalence Test

Charlie — Thu, 06 Dec 2018 06:26:07 GMT

I use SAS program (Version 9.4) and Phoenix/Winnonlin to calculate two one-sided T test. The results of point estimate and 90% of confidence interval of SAS and Phoenix/Winnonlin are same.
But "TOST" have different value of upper and lower.
Here are the results of TOST value of SAS and Phoenix/Winnonlin.

SAS:

Result in TOST Level 0.1 Equivalence Analysis shows:

SAS code as follow

ods graphics off;
proc ttest data=temp1_1 dist=lognormal tost(1.0114488825,1.2793655184) alpha=0.1;
     class Treatment;
	 var  lnCmax;
run;

Phoenix/Winnonlin

There are questions:
1. Why the results are different?
2. How to calculate t1_TOST, t2_TOST, Prob_80_00 and Prob_125_00 in Phoenix by using SAS?

The data as attachment. Please help me answer these questions. Thank you.

Re: TOST Equivalence Test

Rick_SAS — Thu, 06 Dec 2018 14:21:06 GMT

I can't answer why some other software does not match SAS's results, but it is usually because they are using different formulas/methods/assumptions. For a nice exposition of the TOST tests in SAS, see Castello and Watts (2015).

Re: TOST Equivalence Test

FreelanceReinh — Thu, 06 Dec 2018 17:39:35 GMT

Hello @Charlie and welcome to the SAS Support Communities!

So, you started with a dataset created from test-2.csv:

data have;
infile 'C:\Temp\test-2.csv' dsd firstobs=2;
input Subject Treatment $ Period Sequence $ Cmax;
run;

Apparently you created an additional variable lnCmax. Like this?

data temp1_1;
set have;
lnCmax=log(Cmax);
run;

To replicate the results in your first screenshot I created a transposed dataset with a structure similar to Example 122.4 AB/BA Crossover Design in the PROC TTEST documentation (which I selected because obviously your data is from such a crossover trial):

proc sort data=have out=tmp;
by descending sequence subject;
run;

data trans;
do until(last.subject);
  set tmp(rename=(Cmax=c));
  by subject notsorted;
  array drug[2] $1;
  array Cmax[2];
  array lnCmax[2];
  drug[period]=treatment;
  Cmax[period]=c;
  lnCmax[period]=log(c);
end;
keep subject drug: Cmax: lnCmax:;
run;

proc print data=trans;
id subject;
run;

PROC PRINT output:

Subject    drug1    drug2    Cmax1    Cmax2    lnCmax1    lnCmax2

   1         T        R       23.7     19.2    3.16548    2.95491
   3         T        R       28.0     25.3    3.33220    3.23080
   5         T        R       24.0     17.4    3.17805    2.85647
   2         R        T       21.8     23.1    3.08191    3.13983
   4         R        T       16.6     14.8    2.80940    2.69463
   6         R        T       22.1     26.9    3.09558    3.29213

ods output statistics=stats0(where=(sequence=:'Both' & treatment=:'D' & method=:'P'));
proc ttest data=trans alpha=0.1;
var lnCmax1 lnCmax2 / crossover=(drug1 drug2);
run;

data stats;
set stats0;
point_esti=100*exp(-mean);
lower=100*exp(-upperCLmean); /* (!) */
upper=100*exp(-lowerCLmean);
run;

proc print data=stats noobs;
format mean lowerclmean upperclmean 9.6
       point_esti lower upper 12.8;
var lowerclmean mean upperclmean point_esti lower upper;
run;

Result:

    Lower                     Upper
   CLMean         Mean       CLMean      point_esti           lower           upper

-0.246364    -0.128874    -0.011384    113.75468447    101.14488825    127.93655184

Without the CROSSOVER option the confidence limits would be quite different. Note that the "Difference Between Means" (named "Mean" in my output) was calculated for R − T (referring to lnCmax), whereas the ratio (in percent, for Cmax) was calculated in the opposite direction: T / R.

The table "Equivalence Analysis" can be replicated with dataset TEMP1_1 as created above and your code, but, obviously, only after changing the TOST parameters to those in column "Null" of the output: (0.8, 1.25). However, I think this table is incorrect anyway because you specify the log-normal distribution for the logarithms of Cmax. And your PROC TTEST code doesn't reflect the crossover design.

Finally, the results from "Phoenix/Winnonlin" can be replicated (up to minor differences in the CI [see highlighted decimals below], also not converting ratios into percentages) with the code below:

ods output statistics=stats1(where=(sequence=:'Both' & treatment=:'Ratio'))
           EquivTests=eqtst(where=(test ne: 'O' & treatment=:'R' & method=:'P'));
proc ttest data=trans dist=lognormal tost(0.8, 1.25) alpha=0.1 order=data;
var Cmax1 Cmax2 / crossover=(drug1 drug2);
run;

proc print data=stats1 noobs;
format _numeric_ 12.9;
var GeomMean LowerCLGeomMean UpperCLGeomMean;
run;

proc print data=eqtst;
format _numeric_ 11.8;
var tValue Probt;
run;

Results:

                 LowerCLGeom     UpperCLGeom
    GeomMean            Mean            Mean

 1.137546845     1.011448883     1.279365518

     tValue          Probt

 6.38732009     0.00154175
-1.71050951     0.08117192

Does this answer your questions?

Re: TOST Equivalence Test

Charlie — Sat, 08 Dec 2018 21:10:07 GMT

Dear Freelance Reinhard

Thank you for the solution and it works fine with balanced dataset (excluding subject 17 from attachment file).

I used the code as follow:


proc import datafile="D:\unbalance.csv" out=raw dbms=csv replace;
 getnames=yes;
run; 

data raw;
 set raw;
 if subject=17 then delete;
run;

proc ttest data=raw dist=lognormal tost(0.8, 1.25) alpha=0.1;
 var Cmax2 Cmax1 / crossover=(drug1 drug2);
run;

The result of Equivalence Analysis shows as follow:

It matches the result of Phoenix/WinNonlin as follow:

However, after subject 17 is included in analysis, the Phoenix/WinNonlin shows as

The result from SAS shows still same as balanced dataset:

Tvalue Pvalue

4.43 <.0001

-4.08 0.0002

I have found that N in analysis of unbalanced dataset is same to balanced dataset as follow, which means subject with missing data is deleted before performing analysis like glm procedure?

In this situation, is there any solution by using ttest procedure? or I need to use mixed procedure (which analysis all subjects) to get data (such as SD and SE) and coding for calculation?

Re: TOST Equivalence Test

FreelanceReinh — Mon, 10 Dec 2018 09:55:04 GMT

Hello @Charlie,

First of all, I see several (potential) issues in your second post:

Most importantly, there are serious inconsistencies between your raw data files test-2.csv and unbalance.csv: For the odd-numbered subjects the Cmax values are assigned to different treatments (i.e., T and R have been interchanged).
Example: Subject 1 had Cmax=23.7 for treatment T and 19.2 for treatment R according to test-2.csv, but (seemingly) vice versa according to unbalance.csv.
The permutation of Cmax1 and Cmax2 in the VAR statement of PROC TTEST adds confusion and does not really compensate for the data issue because then the even-numbered subjects have their Cmax values incorrectly assigned.
The first table ("result of Equivalence Analysis") shows the tests for the comparison of periods. Is this your focus and not the comparison of treatments?
Are you sure that you want a=0.1 as the significance level? Note that the confidence intervals involved in the TOST are (1-2a)*100% CIs, so these would be 80% CIs.

It is true that PROC TTEST excludes the observation (subject 17) which has missing Cmax for one of the periods. This is because the analysis involves the square root of the "period ratios" (i.e., the ratios between period 1 and period 2 response values, see documentation). Hence, both results must be available.

One option for including the single available Cmax value of subject 17 is indeed to use PROC MIXED: I was able to replicate an arbitrarily selected CI for dataset RAW by means of a PROC MIXED step based on Program 8.1 from the book Pharmaceutical Statistics Using SAS®: A Practical Guide, p. 200. The program code is available from https://support.sas.com/downloads/package.htm?pid=1688 (after extracting 60622_example.zip it is in 60622_example\zipped_files\SAS code and data sets\Chapter08\Chapter 8 code.txt). If subject 17 is included, PROC MIXED uses its available Cmax value and produces a slightly different CI which could be used to perform the equivalence test. (As far as I see from the documentation, Winnonlin always uses linear models to compute the TOST t-statistics.)

However, the decision as to how to deal with the missing value of subject 17 should be based on the statistical analysis plan and (if applicable) relevant guidelines of regulatory authorities.

If you have further questions about the PROC MIXED code, I suggest you start a new thread in the appropriate subforum "Statistical Procedures."

Good luck with your analysis!

Re: TOST Equivalence Test

Charlie — Fri, 14 Dec 2018 17:20:00 GMT

Hi, Sir

Issue 1:

I apologized for issue 1 which making you were confused. Due to data came from my client, I was worried about legal issue if I posted full data. Therefore, I randomly selected some of data at beginning and used part of data in unbalanced.csv.

Issue 2:

Due to we use Phoenix/WinNonlin for data analysis which applying mixed effects modeling, I am trying using SAS to reproduce the result (T1_TOST and T2_TOST) and I am first time to use ttest for such data so I am not sure how the procedure will handle missing data.

Issue 3:

I also apologized for misunderstanding, I am sure that I am trying to compare treatment, not the periods, I just used period as variable to indicate what drug was administered in which period.

Issue 4:

Due to a=0.1 is used as significant level for bioequivalence analysis, therefore, I also used here.

I finally work out my analysis by using proc Mixed, thank you for your reply and suggestion, I've learned a lot here.