topic Checking assumption and writing syntax for a nested 3-level multilevel model in Statistical Procedures

Checking assumption and writing syntax for a nested 3-level multilevel model

CuriousMind — Tue, 21 Feb 2017 17:34:25 GMT

Hi Everyone, I am new to the community and having trouble about writing the syntax for my specific data in SAS. This is my first time trying doing a mixed model analysis so I apologize if the question is not enough brain-storming. I have a 3-level nested hierarchical model: Consider the famous example of students (level 1) nested in classes (level 2) where the classes are nested in school (level 3). I have no covariate in level 1; covariates X and B in my level 2; and covariate sC, D, E, F in level 3. Controlling covariates B, C, D, E, F, I want to see the fixed as well as random effects of X on the response Y.

I have added a sample data (in .xlsx). This is a cross-sectional study. For serving my analysis goal I think one of these two models will do (I will chose the better one based on AIC/BIC):

1. PROC MIXED data=SampleData covtest noclprint method = REML;
class level2 level3;
model Y= X B C D E F ddfm = SATTERTHWAITE;
random intercept/ sub=level3;
random intercept X/ sub=level2 (level3) type=UN;
run;

2. PROC MIXED data=SampleData covtest noclprint method = REML;
class level2 level3;
model Y= X B C D E F ddfm = SATTERTHWAITE;
random intercept X/ sub=level3 type=UN;
random intercept X/ sub=level2 (level3) type=UN;
run;

For testing assumption I am reading macro for testing 2-level mulitlevel model However, my model is 3-level model, is there any resources on how to test the assumptions for 3-level model? And how badly are we impacted if those assumptions are not met?

Re: How do I write syntex for a nested 3-level multilevel model

Rick_SAS — Thu, 19 Jan 2017 21:22:58 GMT

The keywords to search for are

SAS hierarchical model "proc mixed"

A paper that seems to present a step-by-step analysis with explanation is Suzuki and Sheu (1999). The UCLA site also has examples from Singer's seminar on multilevel models.

Re: How do I write syntex for a nested 3-level multilevel model

CuriousMind — Thu, 19 Jan 2017 22:03:20 GMT

Thank you, Rick for the references. The 2nd link only deals up to 2-level predictors. The first link, although considers a 3-level model, no random effect is considered for level 3, and thus I am not entirely sure how could I write mine. I am playing with different syntaxes and all seem to give me some output. My only concern is if I am writing it right.

Re: How do I write syntex for a nested 3-level multilevel model

Rick_SAS — Thu, 19 Jan 2017 22:18:44 GMT

I think you need to write out the statistical model and post the structure/design of your data. Then someone can help you translate it into PROC MIXED or GLIMMIX. For example, if there are repeated measurements (EX: the students took multiple tests) that will affect how you write the model.

A nice overview of the REPEATED and RANDOM statements is provided in Tao, Kiernan, and Gibbs (2015). Their earlier paper (2012) also is worth reading.

Re: How do I write syntex for a nested 3-level multilevel model

CuriousMind — Thu, 19 Jan 2017 22:41:52 GMT

Thanks, Rick, I have added a sample data. I hope somebody will be able to help me out with the code.

Re: How do I write syntex for a nested 3-level multilevel model

sld — Fri, 20 Jan 2017 00:21:23 GMT

You are more likely to get responses if you provide the sample data as a CSV file. Those of us without SPSS won't easily be able to use the file you've provided.

Re: How do I write syntex for a nested 3-level multilevel model

CuriousMind — Fri, 20 Jan 2017 15:24:13 GMT

Thanks, sld. Edited. CSV format is not supported so uploaded data n .xlsx

Re: How do I write syntex for a nested 3-level multilevel model

ballardw — Fri, 20 Jan 2017 15:42:04 GMT

If your data is in a SAS data set then instructions at https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-data-AKA-generate/ta-p/258712 show how to create datastep code that you can paste into the forum using the code box icon {i} or attach as a text file. Then we can run the code and generate a data set with the same characteristics as your data.

Anything where we have to read Excel has a very high probabiltity of something differing as either Proc Import will make guesses that don't actually match data after being filtered through Excel or 2) we have to pick options that may not match.

Re: How do I write syntex for a nested 3-level multilevel model

CuriousMind — Fri, 20 Jan 2017 18:46:47 GMT

Thanks ballardw, I tried to create a text file but it's taking me forever. I attached the data to give the reader an idea how my data look like, and I explained what the variables represent and what is my mixed model setting. All I need to know is how I should write the code here. Perhaps I can include my code and somebody can have a look to see if it's correct.

Re: How do I write syntex for a nested 3-level multilevel model

sld — Fri, 20 Jan 2017 21:31:12 GMT

To do justice to this analysis, you'll need to know more than just how to write the code.

My suggestion is to study the link that @Rick_SAS provided

http://www.ats.ucla.edu/stat/sas/seminars/sas_mlm/mlm_sas_seminar.htm

and the corresponding paper

https://www.ida.liu.se/~732G34/info/singer.pdf

until you understand the nature of this model. In particular, note the use of mean covariate values at level2 for covariates measured at level1. If you understand this paper (and the helpful UCLA website), you will be able to extend the concepts to your scenario and, even more importantly, to build the "right" model and interpret it appropriately. As you study, it might be useful and more intuitive to extract a subset of your study; for example, get rid of level1 by computing means, use one factor from level2 (now the new level1) and one factor from level3 (now the new level2).

In addition to all that, remember to pay attention to assumptions of linearity, normality and homogeneity of variance.

More in-depth resources are the texts by Snijders & Boskers (Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling) and Raudenbush and Bryk (Hierarchical Linear Models: Applications and Data Analysis Methods). This kind of model is complicated, and that's why there are whole books on the topic 🙂

Re: How do I write syntex for a nested 3-level multilevel model

CuriousMind — Mon, 23 Jan 2017 23:07:27 GMT

Yes @sld, I will go over these resources. Thank you 🙂

Re: How do I write syntex for a nested 3-level multilevel model

sld — Tue, 24 Jan 2017 00:48:09 GMT

Great! I'll look forward to seeing what you come up with.

Re: How do I write syntex for a nested 3-level multilevel model

CuriousMind — Tue, 21 Feb 2017 18:10:47 GMT

Hi @sld, I read some resources on multi-level model including the ones @Rick_SAS suggested. I think now I have a better understanding how the model should look like in my case. I have updated the question with a couple of new inquiries. Thanks.

Re: How do I write syntex for a nested 3-level multilevel model

sld — Thu, 23 Feb 2017 06:45:37 GMT

Either I'm not understanding your study design or it is not described correctly.

I'm looking at the sample dataset you attached. If variables C, D, E, and F are covariates measured at Level3, then I would expect to see the same value for each variable (C, D, E, or F) for all observations with the same value of Level3. For example, if Level3=1 then C=12 regardless of the values of Level2 and Level1. However, I see different values for C for Level3=1. Likewise, I would expect to see the same value for variable X or B for all observations with the same value of Level2. I'm perplexed.

IF your dataset had an appropriate multilevel structure (which I am not yet convinced of) and IF I correctly understand your design (which I also am not yet convinced of), then I would consider the following model:

PROC MIXED data=SampleData covtest;
  class level2 level3;
  model Y= C D E F Xmean Bmean X B / ddfm = SATTERTHWAITE s;
  random intercept X B / subject=level3 type=UN;
  random intercept / subject=level2;
run;

BUT I would think of this model as merely a first attempt, definitely not a final model. Even if this model is correct to some degree, there are many data characteristics and assumptions to be assessed (normality, homogeneity of variance, linearity, multicollinearity issues, which TYPE to use in the RANDOM statement, etc.).

If you want to respond to this message, please post a reply rather than edit your original message. It will be easier to track the discussion that way.

Re: How do I write syntex for a nested 3-level multilevel model

CuriousMind — Thu, 23 Feb 2017 17:31:07 GMT

@sld You are right, my sample data was not structured correctly. I am really embarrassed and apologize for that. Please have a look at this data file now where I have designed it exactly how my data look.

My objective is to see how the exposure (X) contributes to response (Y). Fixed effect of X is the primary interest, however, I would like to examine whether the random intercept (and slope) for X varies by level2 and/or level3 ( the parsimonious model will be selected by some AIC/BIC criteria). The other variables are some socio-economic variables I just want to control for.

Grand mean centering can be done for my variables, but not group mean centering I assume as the variable will have only 0 values (for example, Xgroupmean=0 for all children). In my data, the raw score with a 0 has a significant meaning. Do I still need to center? Why did you center only level2 variables but not level3? I read in the literature that centering usually helps with multicollinearity. Would it be a good practice then to center all my variables at grand mean?

Checking assumptions is another issue I am having. Not much is talked about it for 3-level data, and certainly, very few literature who even test the assumption. I am researching how I can extend the macro for 2-level data in my case to test assumptions, and which assumptions are not met. Or, can I consider my data as a 2-level model since there are no level-1 covariates? This will significantly make my life easier.

I really appreciate your input @sld. Thank you. Please feel free to ask me any questions that are still unclear.

Re: How do I write syntex for a nested 3-level multilevel model

sld — Thu, 23 Feb 2017 18:43:37 GMT

Thank you for clarifying your data set structure. I'm more sure now that we're on the same page.

Focusing on the covariate X: Because there are multiple (X,Y) observations (one for each level of Level2) within each level of Level3, we (or more properly, the statistical model) are able fit a regression of Y on X for each level of Level3; as you note, this set of regressions may have appreciable variance among intercepts, variance among slopes, and covariance between intercepts and slopes. These (co)variances are derived from the multiple Level3 regressions. Consequently, although you can assess whether there are random intercepts and random slopes, I'd say that assessment is "among" levels of Level3; there is no random intercept/slope among levels of Level2 because the model is using the different levels of Level2 (within each level of Level3) to define the regressions. I hope that make sense.

I failed to define "Xmean" and "Bmean". Xmean is the mean of the X values over the levels of Level2 for each level of Level3--it's like moving the X values up a tier, from Level2 to Level3, as if Xmean was measured at Level3. I hope that makes sense, too. This concept is addressed in the Singer paper (SES and MEANSES) I linked in an earlier response. Although I didn't intend them as centered variables, they certainly could be, and are in the Singer paper. If you center them correctly, both should be variable (i.e., not constant zero, although the mean would be zero). Should you center? Your call. If the model includes interactions (including polynomial terms, like X*X), then centering is very useful and potentially does reduce collinearity. In a model without interactions, it's less critical, I think. Centering doesn't hurt; you just have to rescale results to un-do centering if you want results on the original scales. Should you include Xmean and Bmean? Again, your call.

If it was me, because there are no covariates at Level1, I would compute the mean Y over the levels of Level1 for each level of Level2 within each level of Level3 and then use the mean Y as the response in the simpler, two-level model. Nothing wrong with an easier life 🙂 You would then be able to omit the second RANDOM statement. If the number of levels of Level1 are the same for all combinations of Level2 and Level3, then the statistical tests for fixed effects will be very similar, if not identical, to those from the three-level model. If the number of levels of Level1 varies dramatically among combinations of Level2 and Level3, then I might keep the three-level model.

I haven't looked in any detail at the paper you found with the macro for assessing assumptions. If you adequately understand how the macro is addressing assumptions, and know what the assumptions are and how to extract what you need from the MIXED procedure, you theoretically would be able to extend the methods to a three-level model. In a sense, your statistical model is a multiple regression in a mixed model, so you have all the assumptions associated with multiple regression plus the assumptions associated with a mixed model. A busy task, but not horribly difficult.

Good luck!

Re: How do I write syntex for a nested 3-level multilevel model

CuriousMind — Fri, 24 Feb 2017 18:11:25 GMT

Thank you so much for your well-put answer @sld. Everything makes a lot of sense to me now.

Yes, I understand now why the assessment of random intercept/slope is "among" levels of Level3, not "among" levels of Level2. Thank you for explaining it clearly.

I can see that 'Xmean' has a nice interpretation if it's significant. Not sure if there is any other advantage of including it in the model, but I guess, I might keep it.

Reducing this model to a 2-level model will be wonderful! Could you please have a look at the attached data to see if I did the reduction right? I assume if it works my model will look like this

PROC MIXED data=SampleData covtest;
  class Newlevel2;
  model NewY= C* D* E* F* Xmean*  X* B* / ddfm = SATTERTHWAITE s;
  random intercept X*/ subject=Newlevel2 type=UN;
run;

I am not interested in including Bmean* and B* in the model and random statement respectively, unless they improve the fit or have some distinct advantages. I am sorry, but I am not sure I understood what you meant by ' If the number of levels of Level1 are the same for all combinations of Level2 and Level3' and 'If the number of levels of Level1 varies dramatically among combinations of Level2 and Level3'. Can you explain it a little bit?

If the idea of model reduction works, then I assume I can use the macro for testing assumptions written for 2-level models. That is a good macro, elaborate with explanations.

Thank you again @sld. You were a big help! Cheers!

Re: How do I write syntex for a nested 3-level multilevel model

sld — Sat, 25 Feb 2017 07:16:07 GMT

I'm happy my response was coherent and, better yet, useful to you.

You get to choose how to build your model. If the number of observations is small relative to the number of predictor variables, then you may have to implement some form of variable selection to avoid overfitting. If the number of observations is large, then you might throw everything in (assuming no problems with multicollinearity, etc.) even if some predictor variables are not significant. Or you might want a parsimonious model.

If you are assessing the support for fixed effects factors, e.g., Xmean, and want to use an information criterion to decide, then remember that you must use a true maximum likelihood method (method=ml) rather than the default (method=reml) for the IC to be valid.

I'd try something like this (which I have not tested). It's close to what you have but includes everything apart from the kitchen sink, and the naming of things is different. Your new dataset looks fine.

/* Compute means over Level1 within Level2*Level3 */
proc sort data= SampleData;
 by level3 level2;
 run;
proc means data= SampleData noprint;
 by level3 level2;
 id c d e f x b;
 var y;
 output out=SampleData_means mean= ;
 run; 
/* Compute Xmean and Bmean (means of variables measured at Level2) */
proc means data=SampleData_means noprint;
 by level3;
 var x b;
 output out=level3_means mean= xmean bmean;
 run;
/* Merge means with data */
data SampleData_means_2;
 merge SampleData_means level3_means;
 by level3;
 run;
/* Fit really busy, very full model */
proc mixed data=SampleData_means_2 covtest;
    class level3 ;
    model y = c d e f xmean bmean x b / ddfm=satterth s;
    random intercept x b / subject=level3 type=un;
    run;

Regarding unequal numbers of levels of Level1: Paraphrasing and quoting from a chapter from Carl Schwarz's online text

http://people.stat.sfu.ca/~cschwarz/Stat-650/Notes/PDFbigbook-SAS/SAS-part017.pdf :

If the number of levels of Level1 varies among levels of Level2, then levels of Level2 with more levels of Level1 will have smaller sampling uncertainty than levels of Level2 with fewer levels of Level1. Consequently, "process + sampling error is not constant over the entire regression line. Estimates are still unbiased, but not fully efficient." (p 1104) The analysis using means is approximate to the extent that the Level1 data are unbalanced across the levels of Level2. If the unbalance is less, then the approximation is "less approximate". Hoping that makes sense.

If you don't have a lot of levels for Level3, you may have estimation problems with type=UN. You could try UN(1), which sets the covariance to zero, then you just have random intercepts and random slopes. (My datasets rarely allow estimation of the covariance, they are just too small.)

Re: How do I write syntex for a nested 3-level multilevel model

CuriousMind — Mon, 06 Mar 2017 22:22:09 GMT

Thanks, @sld for your further explanations and the sample code! My apology for getting back so late to you, life happened!

I have about 1000 level1 observations (Y) and 8 covariates. So I believe, I don't need to consider variable selection in my case.

Thank you for pointing out the estimation method (ML vs. REML). I will be assessing mainly the fixed effects, however, I am also interested to see if the random effects are significant. If I use ML instead of REML, do my significant random effects still make sense?

I incredibly appreciate the sample code you provided. I have tested it and it works! Thank you so much!

Yes, I understand now what you meant by unbalanced data.

Although I have a number of levels in level3, I often get my covariance between random intercept and slope as zero. I will try UN(1), thank you for pointing it out.

Finally a new question! How my interpretation would change if I reduce a 3-level model into a 2-level model? Specifically, we estimate the mean change in Y for a unit change in X in regression. Since my Y is already averaged over the levels of level2, do I interpret it as 'the mean change in Y at level2'?

Re: How do I write syntex for a nested 3-level multilevel model

sld — Tue, 07 Mar 2017 01:44:31 GMT

You're welcome. I'm happy it's making sense.

Using AIC based on ML works for comparing models that differ in fixed and/or random effects. (AIC based on REML is good only for models that differ in random effects.) Variance estimates under REML are thought to be less biased than those under ML (although extent of bias depends on sample size), so sometimes you'll see people select a model using ML, then refit with REML. But if your sample size is adequate and you are not aiming for a parsimonious model, this may not matter much to you this time.

Regarding your question about interpretation: in this case (with no covariates measured at Level1), I don't see that moving from a 3-level model to a 2-level model by averaging over Level1 alters your interpretation; it just makes the model simpler.