A Gentle Intro to SEM, Part 5: Useful Fit Statistics
The purpose of this post is to explain and illustrate the use of overall model fit measures other than the Chi-square test of overall fit. This is part 5 in a series about structural equation models. To catch up to where we are, please go read the earlier posts, listed at the bottom of this post.
Reminder- What Are We Doing?
Structural Equation Modeling (SEM) is a powerful statistical technique used to examine complex relationships between observed and latent variables. It integrates factor analysis and regression modeling, allowing you to test hypotheses about theoretical models that depict causal relationships among variables.
Why is SEM Special?
Unlike traditional regression models, SEM enables you to analyze multiple dependent relationships simultaneously, account for measurement errors, and include latent variables (unobserved constructs inferred from measured indicators).
The Role of Fit Statistics in SEM
Assessing the adequacy of a structural model is one of the main goals of SEM, and this is where fit statistics come into play. Fit indices help determine how well a model aligns with the observed data and whether its proposed structure makes theoretical and empirical sense.
Here is an example of a Fit Indices table from PROC CALIS for a model we are going to look at in just a minute:
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
In the previous post of this series, we tested the overall fit of a model using the Chi-square test of model fit. In that post, we rejected the null hypothesis of absolute model fit.
Researchers generally do not rely on the Chi-square test alone. It is often more helpful to compare fit between a full and reduced model using the difference of Chi-squares, and in the previous post, we did that and learned something useful about the model structure.
Absolute or Standalone Fit Indices
The Chi-square test, which measures the discrepancy between the model and the observed data, has already been discussed at length in an earlier post. A non-significant result suggests a good fit, but it is sensitive to sample size and might not represent a realistic hypothesis. Nonetheless, it is customary to publish this statistic with your model results, and it is one important way to assess the model.
The Standardized Root Mean Square Residual (SRMR) is the square root of the mean of the standardized squared residuals. This index evaluates the difference between the observed and predicted covariances, with values below 0.08 indicating good fit.
Parsimonious Fit Indices
Parsimonious fit indices penalize complex models in favor of simple models with good fit.
The Root mean square error of approximation (RMSEA) measures the overall variability that is not accounted for in the model. It is an estimates of how well the model fits the data, where smaller values are better. Values below 0.05 often indicate a good fit.
Schwarz’s Bayesian criterion (SBC) is used to select the best model among a number of candidate models. The model with the smallest value of SBC is considered the best.
Incremental Fit Indices
Incremental fit indices are constructed so that the model fit is assessed through the comparison with a baseline model. The baseline model is usually the independence model where all covariances among manifest variables are assumed to be zeros. The only parameters in the independence model are the diagonals of covariance matrix.
Bentler’s comparative fit index (CFI) compares the hypothesized model to the independence model. Values above 0.90 suggest adequate model fit.
Here is an example of fitting a pretty basic SEM in PROC CALIS. This example tests the relationships between three latent variables: Motivation, Performance, and Satisfaction. These three latent variables are measured by the observed indicators Q1 through Q9.
PROC CALIS DATA=satisfaction;
PATH
Motivation -> Q1 Q2 Q3,
Performance -> Q4 Q5 Q6,
Satisfaction-> Q7 Q8 Q9,
Motivation -> Performance ,
Performance -> Satisfaction ,
Motivation -> Satisfaction ;
PVAR
Motivation Performance Satisfaction = 1 1 1;
PATHDIAGRAM structural arrange=groupedflow fitindex=[chisq df probchi rmsea cfi];
RUN;
The PATH Statements define the relationships between the manifest and latent variables. The arrows indicate directional paths, and the parentheses contain the estimated or fixed parameters.
The PVAR Statement specifies that the variance of latent variables is fixed to 1 for identification.
The PATHDIAGRAM statement asks for diagrams to be arranged hierarchically in a flow grouped by latent variables (ARRANGE=GROUPEDFLOW) and for a customized table of fix statistics in the inset box (FITINDEX= ).
Here is the diagram:
And here is the structural diagram:
Conclusion: Which Fit Statistics to Report?
There is a great deal of discussion in the SEM literature about how to use fit statistics and which ones should be reported. One article I find especially informative and thoughtful is this one (Stone 2021), which addresses ethical use of fit statistics and recommends that journals publishing SEM results might require a standardized approach similar to Kline’s (2016) three-step process. The third step of this process condones reporting the CFI, RMSEA, and SRMR, and guides the researcher to treat the model as a plausible explanation of the data, rather than the One True Model. I like this approach because good scientific inquiry always has room for theories to be refined and improved based on new data. We will never say that we have the One True Model, particularly in social and behavioral sciences. You might not pick the exact same set of fit statistics as the ones I’ve shown here, but it’s important that whatever set of metrics you use, they be considered carefully, a priori, rather than selectively choosing fit indices after the fact that support the findings you hope to see.
In short, SEM fit statistics help you assess and validate complex theories by providing a comprehensive approach to analyzing relationships between variables. Understanding these fit measures gives you the power to develop models that are statistically sound as well as meaningful in their application. I hope you have found this series to be useful and interesting! I know that I have had fun writing it for you.
See you in class!
Resources:
https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2021.783226/full
https://books.google.com/books?id=t2CvEAAAQBAJ&lpg=PP1&ots=sWVHX3b2gH&lr&pg=PP1#v=onepage&q&f=false
Posts in this series:
To simulate the data used in this post, use this code:
DATA satisfaction;
DO ID = 1 TO 500; /* Simulating 500 observations */
/* Generating latent variables */
Motivation_ = RAND("NORMAL", 0, 1);
Performance_ = 0.7 * Motivation_ + RAND("NORMAL", 0, 1);
Satisfaction_ = 0.6 * Performance_ + 0.4 * Motivation_ + RAND("NORMAL", 0, 1);
/* Simulating measured indicators */
Q1 = Motivation_ + RAND("NORMAL", 0, 0.5);
Q2 = Motivation_ + RAND("NORMAL", 0, 0.5);
Q3 = Motivation_ + RAND("NORMAL", 0, 0.5);
Q4 = Performance_ + RAND("NORMAL", 0, 0.5);
Q5 = Performance_ + RAND("NORMAL", 0, 0.5);
Q6 = Performance_ + RAND("NORMAL", 0, 0.5);
Q7 = Satisfaction_ + RAND("NORMAL", 0, 0.5);
Q8 = Satisfaction_ + RAND("NORMAL", 0, 0.5);
Q9 = Satisfaction_ + RAND("NORMAL", 0, 0.5);
OUTPUT;
END;
RUN;
Find more articles from SAS Global Enablement and Learning here.
