Montreal Cognitive Assessment, Mini-Mental State Examination: Impairment & Dementia Cutoff Points

Paper 1198-2021

Authors

Feier Han, Zhen Wu, Jing Shang, Yutong Zhang, Emory University

Abstract

As the population ages with increased life expectancy, the prevalence and incidence of dementia and mild cognitive impairment (MCI) continue to increase. About 5-8% of Americans aged 60 or above are diagnosed with dementia (WHO, 2020), and 6.7-12.5% of them have MCI (Petersen et al., 2018). The increasing prevalence of MCI and dementia has resulted in increasing comorbidities and associated health care expenditures. Moreover, even with comprehensive treatments, MCI and dementia can significantly impact both patients and caregivers quality of life. As such, it is critical to diagnose cognitive impairment and prescribe interventions promptly.

WATCH THE PRESENTATION

Watch Montreal Cognitive Assessment, Mini-Mental State Examination: Impairment & Dementia Cutoff Points as presented by the author on the SAS Users YouTube channel.

INTRODUCTION

As the population ages with increased life expectancy, the prevalence and incidence of dementia and mild cognitive impairment (MCI) continue to increase. About 5-8% of Americans aged 60 or above are diagnosed with dementia (WHO, 2020), and 6.7-12.5% of them have MCI (Petersen et al., 2018). The increasing prevalence of MCI and dementia has resulted in increasing comorbidities and the associated healthcare expenditure. Moreover, even with comprehensive treatments, MCI and dementia can significantly impact both patients' and caregivers' quality of life. As such, it is critical to diagnose cognitive impairment and prescribe interventions promptly.

A good screening test can significantly improve the efficiency and accuracy of MCI and dementia diagnosis. The Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA) are two commonly used cognitive function screening tools specifically developed for screening MCI (Folstein et al., 1975; Nasreddine et al., 2005). Previous studies had investigated the optimal cutoff scores of MMSE and MoCA for MCI and mild Alzheimer’s disease among elderly Chinese veterans (Tan et al., 2015) and in a Singapore population (Ng A et al., 2013) according to age and education level. Additionally, Chapman’s work (2016) focused on a large Alzheimer’s cohort to explore MMSE cutoff thresholds for clinical trials and diagnostic use, also stratified by age and education level. However, these studies were not conducted in the general population and some had a relatively small sample size. It is essential to have a clinically useful diagnostic test for cognitive status in various subgroups.

PROBLEM

Our study objectives are 1) to examine MMSE and MoCA's performance in detecting cognitive impairment, and 2) to identify and recommend optimal cutoff points of MMSE and MoCA for self-diagnosis and screening of cognitive impairment status. This study utilized the National Alzheimer’s Coordinating Center (NACC) database, which is a large and inclusive database. Given the ample sample size, we investigated the two tests' optimal cutoff points in each subgroup stratified by age, sex, and education level. The findings from our study can be useful for clinical diagnosis and screening of MCI and dementia.

DATA

The data were obtained from the NACC website (https://naccdata.org/). The dataset used in this analysis was published in December 2020. Study subjects had to meet the following inclusion criteria to be included in the analysis: 1) had completed a valid MMSE or MoCA test, 2) age 61-90 years. We excluded subjects ≤60 or >90 years of age due to extremely small sample sizes.

DATA VALIDATION

In the NACC data, each subject’s cognitive status was defined as one of the four stages: Normal, Cognitively impaired but not MCI, MCI, and Dementia, which was determined by clinicians and the pre-specified criteria for MCI and all-cause dementia (Kukull, 2015).

The selected covariates should be easily accessible and have a strong association with impaired-cognitive impairment risks. Initially, we considered the following variables: age, sex, years of education, smoking history, alcohol abuse, diabetes, hypertension and hypercholesterolemia. The detailed description is provided in Appendix I. For the subsequent analysis, age was categorized into 3 groups based on 10 years interval (61-70, 71-80, 81-90); and years of education was categorized into 4 levels: high school degree or below, some college or bachelor’s degree, graduate school or master’s degree, and doctorate or above. Due to a large amount of missing data in other clinical variables, age, sex, and education were selected to form a total of 24 subgroups for cutoff point investigation.

ANALYSIS

Descriptive statistics were presented to summarize the characteristics of the study participants, which were used to guide our decision to include or exclude certain variables for further analysis. To assess the test performance of MMSE and MoCA using logistic regression and the receiver operating characteristic curve (ROC), three binary cognitive status variables were created: I: normal vs. cognitively impaired, MCI, or dementia; II: normal or cognitively impaired vs. MCI or dementia; III: normal, cognitively impaired, or MCI vs. dementia. Logistic regression (PROC LOGISTIC) was employed to examine the association between scores of MMSE and MoCA, and cognitive status (dichotomized) in different sub-groups based on age, sex and education level. We compared the change in the area under the ROC curve (AUC) between models with MMSE or MoCA, age, sex, and education and the model with age, sex and education only. The optimal cutoff points were estimated by maximizing the Youden Index (Youden, 1950; Goddard and Hinberg,1990) and minimizing the distance to (0,1) point (Perkins and Schisterman, 2005) of the ROC curve. In case the maximum Youden Index and minimum distance indicated different cutoff points, we chose the one with higher sensitivity. SAS macro was applied to efficiently repeat the analysis and calculate optimal cutoff points for different subgroups. All the analyses were performed using SAS version 9.4 (Cary, NC).

RESULTS AND VISUALIZATION

A total of 120,099 subjects (57.78% female) were included in the analysis: 58,886(49.03%) had normal cognitive function, 5,871(4.89%) had Cognitively impaired but not MCI, 22,256(18.53%) had MCI, and 33,086(27.55%) were diagnosed with dementia. The sample sizes for subjects who took MMSE or MoCA were 98,617 and 33,752, respectively. The mean and standard deviation (SD) of age and education years were 75.80 (SD=7.37) years and 15.42 (SD=3.31) years, respectively. The mean scores of both MMSE and MoCA in subjects with dementia appeared to be significantly different from those with other cognitive status. Based on Table 1, we excluded diabetes, smoking history, alcohol abuse, hypertension and hypercholesterolemia in our subgroup stratification due to small numbers of subject with available data and unbalanced distribution.

Table 1. Summary of study subject’s characteristics in the NACC database

Cognitive Status

Total

Frequency

120,099

Normal

58,886

(49.03%)

Impaired-not-MCI 5,871

(4.89%)

MCI

22,256

(18.53%)

Dementia

33,086

(27.55%)

MMSE 25.57(5.97)

86,348

28.91 (1.46)

28.03 (2.20)

27.07 (2.52)

19.27 (7.04)

MOCA 23.33(5.80)

33,752

26.37 (2.75)

24.73 (3.36)

22.57 (3.50)

14.92 (6.17)

Age 75.80(7.37)

120,099

75.20 (7.25)

75.26 (7.14)

76.33 (7.24)

76.61 (7.59)

Education Years

15.42(3.31)

119,721

15.89

(3.00)

15.20

(3.63)

15.36

(3.36)

14.69

(3.64)

Sex: Female

69,394 (57.78%)

120,099

20,299

(34.47%)

2,550

(43.43%)

11,045

(49.63%)

16,811

(50.81%)

Diabetes

5,418 (14.26%)

37,982

2,862

(13.42%)

276

(14.79%)

1,276

(17.65%)

1,004

(13.82%)

Smoking History

40,920 (46.63%)

87,762

18,922

(47.11%)

2,256

(52.39%)

7,883

(47.00%)

11,859

(44.72%)

Alcohol Abuse

5,104 (5.65%)

90,322

1,568

(3.82)

371

(8.47%)

1,025

(5.96%)

2,140

(7.73%)

Hypertension

19,620 (51.58%)

38,037

10,689

(49.33%)

1,096

(58.61%)

4096

(56.54%)

3739

(51.54%)

Hypercholesterolemia

21,083 (55.87%)

37,734

11,669

(54.28%)

1,036

(55.55%)

4,285

(59.68%)

4093

(56.90%)

*mean (standard deviation) and frequency count (percentage) are reported for continuous and categorical variables, respectively

Table 2 illustrates the AUC for each model of cognitive status (defined as three different binary outcome variables) with baseline covariates (including age, education, and sex) and with the additional MMSE/MoCA test (i.e., age, education, sex, and MMSE or MoCA). After adding MMSE, the AUC increased by 23-31%, which were considered as clinically significant (Fan & Worster, 2006). The most substantial increase in AUC after adding MMSE or MoCA was for the outcome: MCI vs. Dementia (31.06% and 31.82% respectively). The ROC curves corresponding to Table 2 are shown in Appendix III.

Table 2. Area under the ROC curves for comparisons between models with Age, Education and Sex, and models after adding MMSE/MoCA

AUC	Independent Variables	I	II	III
MMSE	Age, Education, Sex	63.77%	63.68%	62.92%
	Age, Education, Sex, MMSE	86.61%	88.19%	93.98%
	Difference in AUC	+22.84%	+24.51%	+31.06%
MoCA	Age, Education, Sex	62.33%	62.56%	62.28%
	Age, Education, Sex, MoCA	86.65%	88.74%	94.10%
	Difference in AUC	+24.32%	+26.18%	+31.82%

* I: normal vs. cognitively impaired, MCI, or dementia; II: normal or cognitively impaired vs. MCI or dementia; III: normal, cognitively impaired, or MCI vs. dementia

The overall sensitivity, specificity and AUC were 71.7%/75.9%/87.3%, 86.3%/84.8%/85.7% and 86.0%/87.63%/93.7% for MMSE (without covariate adjustment) and 78.2%/82.9%/85.1%, 79.2%/77.4%/86.9%, 79.2%/77.4%/86.9% and 86.2%/88.2%/93.7% for MoCA in detecting 1) cognitively impaired, MCI, or dementia, 2) MCI or dementia, and 3) dementia, respectively. These statistics were further examined within each subgroup of subjects, stratified by age, sex, and education level (a total of 24 subgroups, results not shown). Table 3 shows the averaged sensitivity, specificity and AUC: 78.2%, 83.9% and 88.1% for MMSE and 80.6%, 81.2% and 88.3% for MoCA. Importantly, focusing on dementia diagnosis, the sensitivity and specificity achieved 85.4%, 87.4% for MMSE and 86.3%, 85.5% for MoCA respectively.

Table 3 Sensitivity, specificity and AUC of MMSE and MoCA for different cognitive statuses in subgroups based on age, sex, education years

Test	MMSE				MoCA
Cognitive Status	I	II	III	Mean	I	II	III	Mean
Sensitivity	73.0%	76.1%	85.4%	78.2%	76.1%	79.4%	86.3%	80.6%
Specificity	82.2%	82.1%	87.4%	83.9%	79.1%	79.0%	85.5%	81.2%
AUC	84.6%	86.5%	93.3%	88.2%	84.9%	86.9%	93.1%	88.3%

* I: normal vs. cognitively impaired, MCI, or dementia; II: normal or cognitively impaired vs. MCI or dementia; III: normal, cognitively impaired, or MCI vs. dementia

Table 4 shows optimal cutoff points of MMSE and MoCA for each subgroup. The differences in optimal points between cognition impairment and MCI appeared to be negligible, whereas the differences were significant for dementia diagnosis. For MMSE, the optimal cutoff points were different between the education level below and above high-school, but the differences across age and sex were minimal. For MoCA, the optimal cutoff points exhibited little difference between females and males. On the other hand, age and education level appeared to affect the optimal cutoff points.

Table 4 The MMSE and MoCA optimal cutoff points in different subgroups defined by age, sex, and education level

MMSE	Male			Female
AGE (years)	61-70	71-80	81-90	61-70	71-80	81-90
High School or Below	26/26/24	26/25/24	25/25/24	27/27/25	26/26/24	25/25/24
Bachelor degree or equivalent	28/27/26	27/27/26	27/27/25	28/28/26	28/28/26	27/27/26
Master degree or equivalent	28/28/27	28/28/26	27/27/26	28/28/27	28/28/27	28/28/26
Doctoral degree or above	28/28/27	28/27/27	27/27/26	28/28/28	28/28/27	28/28/26

MMSE	Male			Female
AGE (years)	61-70	71-80	81-90	61-70	71-80	81-90
High School or Below	24/24/19	22/22/19	20/20/18	23/23/19	22/22/19	20/20/18
Bachelor degree or equivalent	24/23/22	23/23/21	22/22/20	24/24/22	24/23/21	23/23/20
Master degree or equivalent	25/25/23	24/24/22	24/24/21	25/25/23	25/25/23	24/24/21
Doctoral degree or above	26/25/23	25/25/22	24/24/22	26/25/24	25/25/22	25/25/23

* The numbers in each cell refers to the optimal cutoff point to differentiate I: normal vs. cognitively impaired, MCI, or dementia; II: normal or cognitively impaired vs. MCI or dementia; III: normal, cognitively impaired, or MCI vs. dementia

CONCLUSION AND GENERALIZATION

This study findings suggest the necessity of applying personalized cut-off points in MMSE and MoCA tests. On top of age, sex, and education, both MMSE and MoCA are extremely helpful in testing cognitive status based on the substantial change in AUC. Specifically, these tests demonstrated greatest increase in AUC for the outcome of dementia, indicating that they may be more useful for dementia than impaired cognitive function or MCI. The sensitivity and specificity shared the same trend with AUC, could be a more powerful proof of our conjecture. In addition, we also presented the optimal cutoff points in a population 61 to 90 years old, stratified by sex and education level. However, most of the cutoff-points are similar for diagnosing impaired and MCI, suggesting that these tests are not sensitive to these characteristics in cognition impairment or MCI screening and may serve as a more efficient diagnostic tool for dementia.

Although Tan et al. (2015) and Ng A et al. (2013) have studied the similar topic, the comparison between their works and ours may not be powerful enough due to the difference in sample size and study population. Even we used the same database as Chapman et al’s work (2016), our study considered age, education, and sex as covariates and created subgroups. In addition, the previous study treated MCI and dementia as two distinct diseases, while we considered impaired, MCI and dementia as an ordinal variable (though three separate binary variables were used in analysis). Given these differences in our analysis strategies, the sensitivity and specificity in Chapman’s work could be greater than ours. The exclusion of impaired cognitive function and the distinction of MCI and dementia may significantly enhance the positive predictive values of the test. Although our work failed to improve the previously reported sensitivity and specificity, the consideration of covariates and subgroups can make these tests more practical in real life.

Overall, our findings suggested that the sensitivity and specificity of MMSE and MoCA can vary depending on subject’s characteristics, thus implying that using a personalized test cutoff may be more effective in the diagnosis of cognitive function than a single universal cutoff.

SUGGESTIONS FOR FUTURE STUDIES

Our study has some limitations. First, given the fact that only one of the NACC study subjects had both MMSE and MoCA data, the increases in AUC after accounting for age, sex, and education are not directly comparable between MMSE and MoCA. Future studies may wish to further investigate the test performance of MoCA and MMSE in the detection of cognitive impairment and dementia across various subgroups in the general population. Second, depression was not considered in our analysis. Andersen, et al. (2020) suggested that depression could usually coexist with Alzheimer and may even accelerate the course of Alzheimer. The manifestation of depression could often be confused with early preference of dementia, which may cause difficulty in diagnosing dementia from depression. Since depression was not routinely screened in this dataset, we did not consider depression or exclude those with depression in our analysis. We believe that the findings could be more generalized to real world settings. Lastly, due to a relatively small group with impaired cognitive function, our results showed difficulty of MMSE/MoCA in diagnosing Impaired. Future studies may wish to recruit more participants with impaired cognitive function.

CONTACT INFORMATION

Feier Han feier.han@emory.edu Jing Shang jing.shang@emory.edu

Zhen Wu zhen.wu@emory.edu Yutong Zhang yutong.zhang@emory.edu

ACKNOWLEDGEMENT

We would like to acknowledge Dr Yi-An Ko for being our advisor. Her valuable and constructive suggestions have been very much appreciated.

APPENDIX

Appendix I Related Variable Description

Variables	Descriptions
NACCUDSD	The subject’s cognitive status with a clinical diagnosis at each visit.   NACCUDSD=1: Subjects with normal cognition; NACCUDSD=2: Subjects who are cognitively impaired but who do not meet the criteria for mild cognitive status; NACCUDSD=3: Subjects with amnestic or non-amnestic mild cognitive status; NACCUDSD=4: Subjects with dementia
NACCMOCA	MoCA Total Score corrected for  Education: 0 - 30 = Correct Test results  88 = Item(s) or whole test not administered   99 = Years of education missing/unknown   -4 = Not available: UDS form submitted did not collect data in this way, or a skip pattern precludes response to this question
NACCMMSE	Total MMSE score (using D-L-R-O-W)  0 – 30 = Correct Test results; 88 = Score not calculated; missing at least one MMSE item;  95 = Physical problem;   96 = Cognitive/behavior problem 97 = Other problem;   98 = Verbal refusal;   -4 = Not available: UDS form submitted did not collect data in this way, or a skip pattern precludes response to this question
EDUC	Subject’s years of education:  12 = high school or below,  16 = bachelor’s degree, 18 = master’s degree, 20 = doctorate or above.
SEX	Subject’s sex: 1 = Male; 2 = Female
AGE	Subject’s age derived from detracting “BIRTHYR” from “VISITYR”  BIRTHYR: Subject’s year of birth  VISITYR: Visit year comes from the date Form A1 was completed.
SMOKYRS	Total years smoked cigarettes
ALCOHOL	Subject with alcohol abuse occurring over a 12-month period with clinical diagnose.   0 = No; 1 = Yes
DIABET	Subject with diabetes present at visit:  0 = No; 1 = Yes
HYPERT	Subject with hypertension present:  0 = No, 1 = Yes
HYPERCHOL	Hypercholesterolemia present: 0 = No, 1 = Yes

Appendix II, The ROC Curves for comparisons between combined age, education level, sex and after adding MMSE/MoCA

	MMSE	MoCA
I: normal vs. cognitively impaired, MCI, or dementia
II: normal or cognitively impaired vs. MCI or dementia
III: normal, cognitively impaired, or MCI vs. dementia

Reference

World Health Organization. 2020. “Dementia.” Accessed March 10, 2021 https://www.who.int/news-room/fact-sheets/detail/dementia
Petersen, R. C., Lopez, O., Armstrong, M. J., Getchius, T., Ganguli, M., Gloss, D., Gronseth, G. S., Marson, D., Pringsheim, T., Day, G. S., Sager, M., Stevens, J., & Rae-Grant, A., 2018, "Practice guideline update summary: Mild cognitive impairment: Report of the Guideline Development, Dissemination, and Implementation Subcommittee of the American Academy of Neurology.” Neurology, 90(3), 126–135.
Folstein, M.E, Folstein, S.E., and McHugh, P.R. 1975 “’Mini-mental state’: A practical method for grading the cognitive state of patients for the clinician.” Journal of psychiatric research, 12(3):189-198.
Nasreddine, Z.S., Phillips, N.A., Bedirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J.L., Chertkow, H. 2005 “The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment.” Journal of the American Geriatrics Society 2005, 53(4):695-699.
Tan, J.P., Li, N., Gao, J., Wang, L.N., Zhao, Y.M., Yu, B.C., Du, W., Zhang, W.J., Cui, L.Q, Wang, Q.S., Li, J.J, Yang, J.S., Yu, J.M., Xia, X.N. and Zhou, P.Y. 2015. “Optimal cutoff scores for dementia and mild cognitive impairment of the Montreal Cognitive Assessment among elderly and oldest-old Chinese population.” Journal of Alzheimer's Disease, 43(4), 1403-1412.
Ng, A., Chew, I., Narasimhalu, K., and Kandiah, N. 2013. “Effectiveness of Montreal Cognitive Assessment for the diagnosis of mild cognitive impairment and mild Alzheimer’s disease in Singapore.” Singapore Med J, 54(11), 616-619.
Chapman, K. R., Bing-Canar, H., Alosco, M. L., Steinberg, E. G., Martin, B., Chaisson, C., Kowall, N., Tripodis, Y., and Stern, R. A. 2016. “Mini Mental State Examination and Logical Memory scores for entry into Alzheimer’s disease trials.” Alzheimer's research & therapy, 8(1), 1-11.
Kukull, Walter A., 2015. “NACC Uniform Data Set: Researchers Data Dictionary Version 3.0.” National Alzheimer’s Coordinating Center. Accessed March 14, 2021, https://naccdata.org/data-collection/forms-documentation/uds-3
Youden, W.J., 1950. “Index for rating diagnostic tests.” Cancer 3.1, 32-35.
Goddard, M.J., Hinberg, I., 1990. “Receiver operator characteristic (ROC) curves and non-normal data: an empirical study.” Statistics in Medicine 9(3), 325–337.
Perkins, N.J. and Schisterman, E.F., 2006. “The inconsistency of ‘optimal’ cut points obtained using two criteria based on the receiver operating characteristic curve.” American Journal of Epidemiology, 163(7), 670-675.
Fan, J., Upadhye, S., and Worster, A.,2006. “Understanding receiver operating characteristic (ROC) curves”. Canadian Journal of Emergency Medicine, 8(1), 19-20.
Andersen, K., Lolk, A., Kragh-Sørensen, P., Petersen, N. E., & Green, A. 2005. “Depression and the risk of Alzheimer disease.” Epidemiology, 233-238.