BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
RonLev
Calcite | Level 5

Steve,

This is a (rather delayed) followup to a question that Steve Denham answered in September 2011.   Please check my question "How to make LSM differences more useful in PROC GENMOD?"  

My problem at that time was to figure out how to create a Least Squares Means Difference result from procedures when the link was not linear;  in those cases (for example in PROC GLIMMIX, GENMOD, etc.) the ilink option result for the LS means is correct, but the LS means differences are wrong (ilink does not know that you cannot subtract 2 log quantites--the programming deficiency needs to be upgraded).

Your answer was quite clever (as usual, Steve!)  but I have found that the std. error and confidence intervals produced by this method are much too large.   To diagnose the problem, I ran a link=id model instead of a link=log model (in this case, the ilink gives the right LSM difference result,) so we can use that example to see where the problem lies.    As you can see in the upper table, your method to calculate the Std. Error of the individual LSM estimates is exactly the same as the SAS output, but the pooled Std Err estimates are much higher (middle table), so the resulting confidence intervals are much wider than what SAS computes (lower table).   The problem seems to be how to calculate the std err for the LSM difference--the pooled method adds the errors, but apparently we need some other method.   Do you have any ideas?

Ron Levine

Least Squares Means (Output from model and Steve's Std Err calculation)
cohortCombinedComplEstimate
StdErr
(SAS)
StdErr (Steve)DFtValueProbt
a_COMBO012.77950.77720.7772215376916.44<.0001
a_COMBO119.57820.77860.7786215376925.14<.0001
b_VALVE012.20070.77460.7746215376915.75<.0001
b_VALVE118.3750.77630.7763415376923.67<.0001
c_CABG010.39710.77190.7719115376913.47<.0001
c_CABG114.99210.77320.773215376919.39<.0001
Least Squares Means Differences (Steve method)
ObscohortCombinedComplmeandiffpoolstderrDF_95pctLCL_95pctUCLProbt
1a_COMBO06.798721.100141537694.642488.95496<.0001
2b_VALVE06.174311.09671537694.024818.32381<.0001
3c_CABG04.595021.092561537692.453636.73641<.0001
Least Squares Means Differences (SAS  method)
ObscohortCombinedComplEstimateStandard ErrorDF_95pctLCL_95pctUCLPr > |t|
1a_COMBO06.79870.14351537696.51757.08<.0001
2b_VALVE06.17430.11591537695.94716.4016<.0001
3c_CABG04.5950.069721537694.45844.7317<.0001

1 ACCEPTED SOLUTION

Accepted Solutions
RonLev
Calcite | Level 5

Thanks 1zmm,

I appreciate Steve and 1zmm working through this problem.   As many statisticians know, GENMOD, GLIMMIX etc., are great (potential) tools but if you can't extract a useful answer from the model, the technology is essentially useless.   Maybe ilink will be fixed some day, but in the meantime your fix is very insightfull.   I can get the difference in means by a simple subtraction;  and now, having a way to compute the std err makes it possible to create correct confidence intervals--everything you need for reporting a result.

Thanks everyone.

Ron Levine

View solution in original post

7 REPLIES 7
1zmm
Quartz | Level 8

The method that Steve uses for calculating the standard errors of the differences in the least-square mean estimates within each of the three cohorts assumes that these estimates are statistically independent (or are uncorrelated).  Apparently, however, these estimates are strongly negatively correlated to obtain such smaller estimates for the standard errors of these differences than that from Steve's method.

Therefore, you will have to specify the COV option on the LSMEANS statement to write to the output the variance-covariance matrix for the least-square mean estimates (I'm not sure whether this matrix can be written to a SAS data set for further manipulation using the ODS Table, LSMEANS).  The variance of the difference between two least-square estimates within each cohort is the sum of the variance estimates and the two (identical) covariance estimates for these within-cohort least-square mean estimates; I would again presume that these latter covariance estimates are negative to obtain the small standard errors for the differences that you've gotten.  The standard error of the difference between the two least-square estimates within each cohort is the square root of this variance of the difference.

RonLev
Calcite | Level 5

Hi 1zmm,

Adding COV to the LSM statement yields the following output:

cohortComplEstimateStdErrDFLowerUpperCov1Cov2Cov3Cov4Cov5Cov6
a_COMBO012.77950.777215376911.256114.30280.60410.59490.59450.59440.59420.5944
a_COMBO119.57820.778615376918.052121.10430.59490.60630.59490.59480.59470.5949
b_VALVE012.20070.774615376910.682413.7190.59450.59490.60010.59470.59450.5945
b_VALVE118.3750.776315376916.853419.89660.59440.59480.59470.60270.59420.5944
c_CABG010.39710.77191537698.884111.910.59420.59470.59450.59420.59590.5944
c_CABG114.99210.773215376913.476616.50760.59440.59490.59450.59440.59440.5979

I'm not sure whether this is what you are looking for:  I don't see a var/covar matrix here, and the covarianaces are all positive.   Where do we go from here?

Ron Levine

RonLev
Calcite | Level 5

Hi 1zmm,

I ran the CORR option to the LSM statement s well, and this does appear to show very high correlations among the LSM estimates;  is this more helpful?

cohortComplCorr1Corr2Corr3Corr4Corr5Corr6
a_COMBO010.9830.98750.9850.99040.989
a_COMBO10.98310.98630.98410.98940.9882
b_VALVE00.98750.986310.98880.99420.9926
b_VALVE10.9850.98410.988810.99160.9902
c_CABG00.99040.98940.99420.991610.9959
c_CABG10.9890.98820.99260.99020.99591

Ron Levine

RonLev
Calcite | Level 5

1zm,

From covariate matrix, according to your description, we have (for the COMBO cohort, for example):

sqrt(0.6041+0.6063+0.5949+0.5949) = 1.549.    This is way off the SAS compution of 0.1435, so we still have some work to do.

Ron Levine

1zmm
Quartz | Level 8

I was mistaken.about the formula for the variance of the DIFFERENCE between two least-square means (LSMs).

When you want to obtain the variance of the SUM of two LSMs, you should ADD the variances for each LSM AND the two (identical) covariances between the two LSMs:

   VAR(LSM1 + LSM2) = VAR(LSM1) + VAR(LSM2) + 2*COVAR(LSM1, LSM2).

The standard error of the SUM of two LSMs is the square root of VAR(LSM1 + LSM2).

When you want to obtain the variance of the DIFFERENCE between the two LSMs (as you wanted), you should add the variances for each LSM and, from this sum, SUBTRACT the two (identical) covariances between the two LSMs:

   VAR(LSM1 - LSM2) = VAR(LSM1) + VAR(LSM2) - 2*COVAR(LSM1, LSM2).

The standard error of the DIFFERENCE between two LSMs is the square root of VAR(LSM1 - LSM2).

For example, from the table of the variance-covariance matrix you show using the option, COV, in the LSMEANS statement,

    the variance of the LSM for a_COMBO=0 equals 0.6041,

    the variance of the LSM for a_COMBO=1 equals 0.6063, and

    the covariance between these two LSMs equals 0.5949.

Thus, the variance of the DIFFERENCE between these two LSMs equals

    0.6041  + 0.6063 - 2* 0.5949 = 0.0206.

The standard error of this DIFFERENCE equals the square root of this variance (0.0206) = 0.1435.

The variance of the DIFFERENCE between the LSM, b_VALVE=0, and the LSM, b_VALVE=1, equarls

     0.6001 + 0.6027 - 2*0.5947 = 0.0134.

The standard error of this DIFFERENCE equals the square root of this variance (0.0134) = 0.1158.

And so on.

SteveDenham
Jade | Level 19

Thanks for this discussion, guys.  I admit that my method was crude, and definitely was based on the assumption of independence.  I am incorporating 1zmm's methods in any future calculations of this type.

Steve Denham

RonLev
Calcite | Level 5

Thanks 1zmm,

I appreciate Steve and 1zmm working through this problem.   As many statisticians know, GENMOD, GLIMMIX etc., are great (potential) tools but if you can't extract a useful answer from the model, the technology is essentially useless.   Maybe ilink will be fixed some day, but in the meantime your fix is very insightfull.   I can get the difference in means by a simple subtraction;  and now, having a way to compute the std err makes it possible to create correct confidence intervals--everything you need for reporting a result.

Thanks everyone.

Ron Levine

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 7 replies
  • 3297 views
  • 0 likes
  • 3 in conversation