topic Re: Warning about number of covariates for Difference in Difference Analysis in SAS Programming

Warning about number of covariates for Difference in Difference Analysis

LL2023 — Wed, 03 Jan 2024 18:46:06 GMT

Hello,

I am currently working on some difference in difference analyses. I am comparing a number of outcomes between study sites to non study sites before the program started to after the program started (prepost). I was able to run these analyses without issues on most of my outcomes, but some of the outcomes gave the messages, "WARNING: More coefficients than levels specified for effect. Some coefficients will be ignored." I used proc mixed with the following code:

proc mixed data = data order = formatted;

class studysite prepost steid;

model outcome = prepost|studysite / solution;

random int / subject = siteid type = un;

Lsmeans prepost|studysite / diff;

estimate "D-I-D" studysite*prepost 1 -1 -1 1;

format futureintervention prepost;

run;

I did not have any issues running this code with categorical outcomes or a few of my continuous outcomes. The outcomes that are giving me difficulties are based on survey data where participants could give a response between 0 and 10. The survey data does have a lot of missing values, but I got the same warning message whether or not I used a dataset only with complete observations or not.

What could be causing this error? Do I need to rewrite this code to better accommodate continuous outcomes?

Thank you.

Re: Warning about number of covariates for Difference in Difference Analysis

PaigeMiller — Wed, 03 Jan 2024 19:11:00 GMT

Your ESTIMATE statement uses 4 levels of the interaction studysite*prepost, but SAS is saying that there are fewer than 4 levels of the interaction studysite*prepost in your data. Looking at the data to determine the number of levels in this interaction would be a good thing to do.

What could be causing this error? Do I need to rewrite this code to better accommodate continuous outcomes?

Also, this is not an ERROR, it is a WARNING indicating that perhaps you have something in the code that needs to be looked at more carefully. The WARNING has nothing to do with your continuous outcome. And despite the title of this post, it also has nothing to do with covariates.

Re: Warning about number of covariates for Difference in Difference Analysis

LL2023 — Wed, 03 Jan 2024 19:23:50 GMT

Both studysite and prepost are coded as binary variables, so I am confused how there would be more than 4 levels. Despite not having issues with the results with other continuous variables, the survey variables are unable to generate the overall estimates for the DID analysis or results for the Differences of Least Squares Means.

Re: Warning about number of covariates for Difference in Difference Analysis

PaigeMiller — Wed, 03 Jan 2024 19:29:30 GMT

@LL2023 wrote:

Both studysite and prepost are coded as binary variables, so I am confused how there would be more than 4 levels. Despite not having issues with the results with other continuous variables, the survey variables are unable to generate the overall estimates for the DID analysis or results for the Differences of Least Squares Means.

The message indicates that there are fewer than four levels. You need to look at your data to see what levels of this interaction exist, and if there is actual response data at each of those levels.

Re: Warning about number of covariates for Difference in Difference Analysis

LL2023 — Wed, 03 Jan 2024 20:03:36 GMT

Thank you so much! I hadn't seen this message before, so I thought it was indicating the opposite.