topic Re: PROC Mixed treatment effect percentage change in Statistical Procedures

PROC Mixed treatment effect percentage change

Naviava1973 — Thu, 25 Mar 2021 14:40:53 GMT

Hi All,

I am running a treatment effect longitudinal analysis with the outcome being a log transformed value collected over 5 days.

My covariates are time and treatment group (intervention vs. control) and their interaction (treatment*time).

The aim of the analysis is to determine if there is a treatment effect in favour of the intervention but expressed as a percentage change.

I have fitted my model assuming a fixed effect for time, treatment group and their interaction with a random intercept.

proc mixed; class treatment; model outcome= time time*treatment / s; random intercept/subject=ID; run;

Could someone point me to literature or example that illustrates how the treatment effect is determined as a percentage? Or an example if any?

Regards

Kennedy

Re: PROC Mixed treatment effect percentage change

SteveDenham — Fri, 26 Mar 2021 11:32:31 GMT

I can think of two ways, but neither is done entirely within PROC MIXED. I'll list them in order of probable appropriateness.

Method 1: Get the lsmeans and differences into datasets, merge them and post process it, dividing the difference from pretreatment for each post treatment time lsmean by the pretreatment time lsmean and multiply by 100.

Method 2: Shift to PROC GLIMMIX, use a log link. The differences will be ratios which are "fold" changes. From there convert to percentage changes by calculating pct_change = 100* (diff in lsmeans -1).

What you should NOT do is divide the values by each pretreatment value and analyze the ratios as if they were normal. This ignores all covariance within subject, plus ratios of variables with normal errors generally do not have normal errors.

SteveDenham

Re: PROC Mixed treatment effect percentage change

jiltao — Fri, 26 Mar 2021 14:00:23 GMT

It looks to me that your response variable is a log-transformed variable. The percent difference in lsmeans that you want would be on the logged scale, correct?

Like @SteveDenham pointed out, you can save the LSMEANS results (using an ODS OUTPUT statement) and use SAS data steps to compute the percent difference in lsmeans. However, this will not give you standard errors / p-values / confidence intervals. If you also wanted to make inferences on the percentage, you might consider using PROC NLMIXED to fit your model, and use the ESTIMATE statement in PROC NLMIXED to compute the percent difference. The ESTIMATE statement would produce the p-value and 95% confidence interval for you on the estimate (which is the percent difference).

For method 2 suggested by @SteveDenham, I assume he wanted you to use the Log link on your log-transformed variable? I just wanted to point out that these two methods will not give you the same model. Also, Steve might mean pct_change = 100* (exp(diff in lsmeans) -1) rather than pct_change = 100* (diff in lsmeans -1). Please correct me if I am wrong @SteveDenham.

Jill

Re: PROC Mixed treatment effect percentage change

Naviava1973 — Fri, 26 Mar 2021 19:35:39 GMT

Thank you very much Steve.
I will work to implement this approach and see what I get.

Re: PROC Mixed treatment effect percentage change

Naviava1973 — Fri, 26 Mar 2021 19:37:39 GMT

Thank you Jiltao.
I will look for PROC NLMIXED references that specifically address the percentage change.
This is definitely helpful.
I really thank you both.

Re: PROC Mixed treatment effect percentage change

Naviava1973 — Fri, 26 Mar 2021 19:39:21 GMT

Yes my response variable is log transformed.

Re: PROC Mixed treatment effect percentage change

Naviava1973 — Fri, 26 Mar 2021 21:07:30 GMT

Hi Jil,
I have run the approach proposed by Steve Denham. I was wondering if you could point me to a reference for NLMIXED that has done something similar so I can compare findings from both methods.
Naviava

Re: PROC Mixed treatment effect percentage change

jiltao — Fri, 26 Mar 2021 22:53:02 GMT

Here is the documentation for PROC NLMIXED, where you can find syntax and example code --

https://go.documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=v_009&docsetId=statug&docsetTarget=statug_nlmixed_toc.htm&locale=en

Suppose your treatment contains levels A and P, and your data is already sorted by ID. Then if this is your PROC MIXED statement --

proc mixed; 
class treatment; 
model outcome= time treatment time*treatment / s; 
random intercept/subject=ID; 
lsmeans treatment / e;
run;

You might write the following PROC NLMIXED program to fit the same model. Please note that the LSMEANS statement in PROC MIXED above was computed at mean value for TIME, which would be displayed in the E option output. For illustration purpose, I used 5 as the mean value for time for my sample code below. You would need to replace that with whatever mean value was used in PROC MIXED.

proc nlmixed;
  parms b0=2 b1=1.2 b2=0.5 b3=1.2 sb=2 se=1; ** you might want to specify more reasonable starting values;
  mu = b0 + b1*time + b2*(treatment="A") + b3*time*(treatment="A") + u;
  model outcome ~ normal(mu, se);
  random u ~ normal(0, sb) subject=ID;
  estimate 'percent difference in the A and P at the mean time of 5' 100*(b2+b3*5)/(b0+b1*5);
  run;

Please let me know if this does not work out.

Jill

Re: PROC Mixed treatment effect percentage change

SteveDenham — Mon, 29 Mar 2021 12:53:47 GMT

@Naviava1973 , I missed that your response was already log transformed. That means that for the first method I mentioned, the lsmeans need to be preprocessed by exponentiating. The same would apply to any confidence bounds. The second method could be applied directly to the lsmeans you currently obtain as well as any confidence bounds. As @jiltao said, you would have to use NLMIXED to get valid standard errors for the means on the original scale.

SteveDenham

Re: PROC Mixed treatment effect percentage change

Naviava1973 — Mon, 29 Mar 2021 16:14:37 GMT

Hi Steve,
Thanks for the advise.
I really appreciate.

Regards

KO

Re: PROC Mixed treatment effect percentage change

Naviava1973 — Mon, 29 Mar 2021 16:17:37 GMT

Dear Jill,
Thank you very much.
This has worked out perfectly and I can now see the outcome I was looking for though I noticed wide 95% CIs.
Once more, I really appreciate the support from you and the SAS users group.
Regards

KO

Re: PROC Mixed treatment effect percentage change

Rhea1234 — Wed, 25 Aug 2021 17:28:04 GMT

@jiltao How do we compare changes in a metric for two groups across two points T0 and T+2 in time using a similar approach ? Can you please help me arrive at the % change in the metric for the two groups ? I am using proc glm with a response variable as the value of the metric at T+2.

Re: PROC Mixed treatment effect percentage change

SteveDenham — Wed, 25 Aug 2021 17:57:57 GMT

Here is how I would approach this. There are two levels of group (say A and B) and two levels of time (t0 and t+2). I would use PROC MIXED as you have correlated responses. Follow this up with a post-analysis processing to get the percent change. Something like this:

proc mixed data=have;
class group time subject; /* subject is a member of one of the levels of group */
model response = group time group*time;
repeated time/type=un subject=subject;
lsmestimate group*time 'Baseline Group A' 1 0 0 0;
lsmestimate group*time 'Baseline Group B' 0 0 1 0;
lsmestimate group*time 'Difference over time for Group A' 1 -1 0 0;
lsmestimate group*time 'Difference over time for Group B' 0 0 1 -1;
lsmestimate group*time 'Difference between groups over time (B - A)' -1 1 1 -1;
ods output  lsmestimates=lsmestimates;
run;

Then you can calculate the percent change in each group by dividing the difference over time by the baseline for that group. The p value from the last lsmestimate would apply to the difference in group changes. No place along here would there be an analysis of percent change.

I strongly recommend against analyzing the percent changes as a response variable. See Harrell's Applied Regression Strategies or the website run by the Vanderbilt University biostats department https://discourse.datamethods.org/t/author-checklist/3407 . In particular, see this webpage: http://hbiostat.org/bbr/md/change.html#sec:changegen for the many caveats on analyzing change scores.

SteveDenham

Re: PROC Mixed treatment effect percentage change

SteveDenham — Wed, 25 Aug 2021 19:13:01 GMT

proc glm data=have;
class group covA covB; 
model response = covA covB covC baseline group  group*baseline;
lsmeans group/pdiff;
run;

@Rhea1234, this code should help you accommodate the things brought up in your private message. At this point, I would recommend responding here rather than be a private message - there are other folks who can help.

The code above will give the difference between groups, averaged over the categorical effects covA and covB,, and estimated at the mean values of the continuous effects covC and baseline. The F test for group*baseline will tell you whether the adjustment is the same in both groups. Now I'll recommend getting a good text on analysis of covariance to see how to proceed. If you have access to SAS for Mixed Models, there is a great chapter that goes through this step by step, and how to get the comparisons needed if the adjustments for baseline are not equal.

SteveDenham

Re: PROC Mixed treatment effect percentage change

Rhea1234 — Wed, 25 Aug 2021 19:28:58 GMT

Thank you Steve. I will try to get a copy of the book. I ultimately need to arrive at the % change (response - baseline) for the two groups and assess the impact other covariates have on the response. Hope it makes sense?

Re: PROC Mixed treatment effect percentage change

jiltao — Wed, 25 Aug 2021 22:13:15 GMT

My understanding is that, you wanted the percent change, also wanted the significance of the percent change (p-value, or 95%ci, or both), correct? If so, you would need to use PROC NLMIXED to fit your model and use the ESTIMATE statement to get what you want.

The syntax of the ESTIMATE statement depends on your model set up. For this new request, can you send me your current program (proc glm or proc mixed) and exactly what percent change you want? I will see if I can come up with the PROC NLMIXED program for you.

Thanks,

Jill

Re: PROC Mixed treatment effect percentage change

Rhea1234 — Thu, 26 Aug 2021 04:35:10 GMT

Thank you Jill. Here is what I am trying to solve for :

I have two groups with a metric that is measured across two time frames say T+2 and T0 , where T0 is the baseline. I need to compare the percentage change in the metric from T0 to T+2 across the two groups, with the baseline metric being one of the covariates. Additionally, I have been tasked to identify other covariates eg. Cov A Cov B and Cov C (mix of continuous and categorical) that are associated (have statistically significant relationship) with the change. I am aware and as Steve has rightly pointed out that we cannot regress percentage change or the difference T+2- T0 values. Hence, I tried to regress T+2 metric ~ T0baseline + group . Here is what I have put together :

proc glm data=have;

class group;

model T+2_metric= group T0_metric covA cov B covC /solution;

lsmeans group/pdiff=all ;

run;

I am struggling with how to arrive at the percentage change comparison from here.

With the proc mixed code, Steve had provided I was able to get the % Change in Group A vs Group B. However, I would need to control for the baseline value as well. Please let me know if this makes sense or if I have been approaching the problem incorrectly.

Thanks,

Rhea

Re: PROC Mixed treatment effect percentage change

SteveDenham — Thu, 26 Aug 2021 11:25:33 GMT

There is one issue that you should think about some more, in my opinion. It looks like you are "adjusting" for baseline twice - once by calculating the percent change for the given subject, and again by using the baseline as a covariate. If you have the percentage change for each subject, then I would remove the baseline covariate from the model.

SteveDenham

Re: PROC Mixed treatment effect percentage change

Rhea1234 — Thu, 26 Aug 2021 11:36:52 GMT

Hi Steve,

Since using percentage change as a response variable isn't recommended, I had to restate the equation as T+2_metric ~ T0_metric (baseline) + group.

As recommended, should I compare the means of t0_metric to get the average for each group and then calculate the difference between those values and the corresponding least squares means, dividing by the average t0_metric X100?

Will this lead to adjusting of baseline twice?

Thanks,

Rhea

Re: PROC Mixed treatment effect percentage change

jiltao — Thu, 26 Aug 2021 17:02:30 GMT

Your PROC GLM code treats the baseline measurements (T0) as a covariate, and I am not sure if a percent changes between the measurements at T2 and T0 can be computed based on this model specification.

Steve's PROC MIXED model specification computes the difference between the groups in T2, accounting for the baseline measurements. Is that what you wanted?

If you want to compute the difference between T2 and T0, whether it is a percent difference or the raw difference, you would probably need to arrange your data to have a TIME variable in there, just like what you sent in the original post.

I think my PROC NLMIXED code worked for your original question, right? That model has TIME and TREATMENT (or GROUP?) in there. How is your new question different from the previous one in terms of the data and model?

Thanks,

Jill