BookmarkSubscribeRSS Feed

Montreal Cognitive Assessment, Mini-Mental State Examination: Impairment & Dementia Cutoff Points

Started ‎04-19-2021 by
Modified ‎05-24-2021 by
Views 2,601
Paper 1198-2021
Authors

 Feier Han, Zhen Wu, Jing Shang, Yutong Zhang, Emory University

Abstract

As the population ages with increased life expectancy, the prevalence and incidence of dementia and mild cognitive impairment (MCI) continue to increase. About 5-8% of Americans aged 60 or above are diagnosed with dementia (WHO, 2020), and 6.7-12.5% of them have MCI (Petersen et al., 2018). The increasing prevalence of MCI and dementia has resulted in increasing comorbidities and associated health care expenditures. Moreover, even with comprehensive treatments, MCI and dementia can significantly impact both patients and caregivers quality of life. As such, it is critical to diagnose cognitive impairment and prescribe interventions promptly.

 

WATCH THE PRESENTATION

Watch Montreal Cognitive Assessment, Mini-Mental State Examination: Impairment & Dementia Cutoff Points as presented by the author on the SAS Users YouTube channel.

 

INTRODUCTION  

As the population ages with increased life expectancy, the prevalence and incidence of dementia and mild cognitive impairment (MCI) continue to increase. About 5-8% of Americans aged 60 or above are diagnosed with dementia (WHO, 2020), and 6.7-12.5% of them have MCI (Petersen et al., 2018). The increasing prevalence of MCI and dementia has resulted in increasing comorbidities and the associated healthcare expenditure. Moreover, even with comprehensive treatments, MCI and dementia can significantly impact both patients' and caregivers' quality of life. As such, it is critical to diagnose cognitive impairment and prescribe interventions promptly.

A good screening test can significantly improve the efficiency and accuracy of MCI and dementia diagnosis. The Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA) are two commonly used cognitive function screening tools specifically developed for screening MCI (Folstein et al., 1975; Nasreddine et al., 2005).  Previous studies had investigated the optimal cutoff scores of MMSE and MoCA for MCI and mild Alzheimer’s disease among elderly Chinese veterans (Tan et al., 2015) and in a Singapore population (Ng A et al., 2013) according to age and education level. Additionally, Chapman’s work (2016) focused on a large Alzheimer’s cohort to explore MMSE cutoff thresholds for clinical trials and diagnostic use, also stratified by age and education level. However, these studies were not conducted in the general population and some had a relatively small sample size. It is essential to have a clinically useful diagnostic test for cognitive status in various subgroups.

 

PROBLEM

Our study objectives are 1) to examine MMSE and MoCA's performance in detecting cognitive impairment, and 2) to identify and recommend optimal cutoff points of MMSE and MoCA for self-diagnosis and screening of cognitive impairment status. This study utilized the National Alzheimer’s Coordinating Center (NACC) database, which is a large and inclusive database. Given the ample sample size, we investigated the two tests' optimal cutoff points in each subgroup stratified by age, sex, and education level. The findings from our study can be useful for clinical diagnosis and screening of MCI and dementia. 

 

DATA

The data were obtained from the NACC website (https://naccdata.org/). The dataset used in this analysis was published in December 2020. Study subjects had to meet the following inclusion criteria to be included in the analysis: 1) had completed a valid MMSE or MoCA test, 2) age 61-90 years. We excluded subjects ≤60 or >90 years of age due to extremely small sample sizes.

 

DATA VALIDATION

In the NACC data, each subject’s cognitive status was defined as one of the four stages: Normal, Cognitively impaired but not MCI, MCI, and Dementia, which was determined by clinicians and the pre-specified criteria for MCI and all-cause dementia (Kukull, 2015).

The selected covariates should be easily accessible and have a strong association with impaired-cognitive impairment risks. Initially, we considered the following variables: age, sex, years of education, smoking history, alcohol abuse, diabetes, hypertension and hypercholesterolemia. The detailed description is provided in Appendix I. For the subsequent analysis, age was categorized into 3 groups based on 10 years interval (61-70, 71-80, 81-90); and years of education was categorized into 4 levels: high school degree or below, some college or bachelor’s degree, graduate school or master’s degree, and doctorate or above. Due to a large amount of missing data in other clinical variables, age, sex, and education were selected to form a total of 24 subgroups for cutoff point investigation.

 

ANALYSIS 

Descriptive statistics were presented to summarize the characteristics of the study participants, which were used to guide our decision to include or exclude certain variables for further analysis. To assess the test performance of MMSE and MoCA using logistic regression and the receiver operating characteristic curve (ROC), three binary cognitive status variables were created: I: normal vs. cognitively impaired, MCI, or dementia; II: normal or cognitively impaired vs. MCI or dementia; III: normal, cognitively impaired, or MCI vs. dementia. Logistic regression (PROC LOGISTIC) was employed to examine the association between scores of MMSE and MoCA, and cognitive status (dichotomized) in different sub-groups based on age, sex and education level. We compared the change in the area under the ROC curve (AUC) between models with MMSE or MoCA, age, sex, and education and the model with age, sex and education only. The optimal cutoff points were estimated by maximizing the Youden Index (Youden, 1950; Goddard and Hinberg,1990) and minimizing the distance to (0,1) point (Perkins and Schisterman, 2005) of the ROC curve. In case the maximum Youden Index and minimum distance indicated different cutoff points, we chose the one with higher sensitivity. SAS macro was applied to efficiently repeat the analysis and calculate optimal cutoff points for different subgroups. All the analyses were performed using SAS version 9.4 (Cary, NC).

 

RESULTS AND VISUALIZATION

A total of 120,099 subjects (57.78% female) were included in the analysis: 58,886(49.03%) had normal cognitive function, 5,871(4.89%) had Cognitively impaired but not MCI, 22,256(18.53%) had MCI, and 33,086(27.55%) were diagnosed with dementia. The sample sizes for subjects who took MMSE or MoCA were 98,617 and 33,752, respectively. The mean and standard deviation (SD) of age and education years were 75.80 (SD=7.37) years and 15.42 (SD=3.31) years, respectively. The mean scores of both MMSE and MoCA in subjects with dementia appeared to be significantly different from those with other cognitive status. Based on Table 1, we excluded diabetes, smoking history, alcohol abuse, hypertension and hypercholesterolemia in our subgroup stratification due to small numbers of subject with available data and unbalanced distribution.

 

Table 1. Summary of study subject’s characteristics in the NACC database 

Cognitive Status 

Total 

Frequency 

120,099

Normal 

58,886 

(49.03%)

Impaired-not-MCI 5,871 

(4.89%) 

MCI 

22,256 

(18.53%) 

Dementia 

33,086 

(27.55%) 

MMSE 25.57(5.97) 

86,348 

28.91 (1.46) 

28.03 (2.20) 

27.07 (2.52) 

19.27 (7.04) 

MOCA 23.33(5.80) 

33,752 

26.37 (2.75) 

24.73 (3.36) 

22.57 (3.50) 

14.92 (6.17) 

Age 75.80(7.37)

120,099

75.20 (7.25)

75.26 (7.14)

76.33 (7.24)

76.61 (7.59) 

Education Years 

15.42(3.31) 

119,721

15.89 

(3.00) 

15.20 

(3.63)

15.36 

(3.36)

14.69 

(3.64) 

Sex: Female

69,394 (57.78%)

120,099

20,299 

(34.47%) 

2,550 

(43.43%)

11,045 

(49.63%) 

16,811 

(50.81%) 

Diabetes

5,418 (14.26%)

37,982

2,862 

(13.42%) 

276 

(14.79%)

1,276 

(17.65%)

1,004 

(13.82%) 

Smoking History

40,920 (46.63%) 

87,762

18,922 

(47.11%) 

2,256 

(52.39%)

7,883 

(47.00%) 

11,859 

(44.72%) 

Alcohol Abuse

5,104 (5.65%) 

90,322

1,568 

(3.82) 

371 

(8.47%)

1,025 

(5.96%) 

2,140 

(7.73%) 

Hypertension

19,620 (51.58%) 

38,037

10,689 

(49.33%) 

1,096 

(58.61%)

4096 

(56.54%) 

3739 

(51.54%) 

Hypercholesterolemia

21,083 (55.87%) 

37,734

11,669

(54.28%)

1,036 

(55.55%)

4,285 

(59.68%) 

4093 

(56.90%) 

*mean (standard deviation) and frequency count (percentage) are reported for continuous and categorical variables, respectively  

 

Table 2 illustrates the AUC for each model of cognitive status (defined as three different binary outcome variables) with baseline covariates (including age, education, and sex) and with the additional MMSE/MoCA test (i.e., age, education, sex, and MMSE or MoCA). After adding MMSE, the AUC increased by 23-31%, which were considered as clinically significant (Fan & Worster, 2006). The most substantial increase in AUC after adding MMSE or MoCA was for the outcome: MCI vs. Dementia (31.06% and 31.82% respectively). The ROC curves corresponding to Table 2 are shown in Appendix III. 

 

Table 2. Area under the ROC curves for comparisons between models with Age, Education and Sex, and models after adding MMSE/MoCA  

AUC 

Independent Variables 

I

II

III

MMSE 

Age, Education, Sex

63.77% 

63.68% 

62.92% 

Age, Education, Sex, MMSE 

86.61% 

88.19% 

93.98% 

Difference in AUC

+22.84% 

+24.51% 

+31.06% 

MoCA 

Age, Education, Sex

62.33% 

62.56% 

62.28% 

Age, Education, Sex, MoCA 

86.65% 

88.74% 

94.10% 

Difference in AUC 

+24.32% 

+26.18% 

+31.82% 

* I: normal vs. cognitively impaired, MCI, or dementia; II: normal or cognitively impaired vs. MCI or dementia; III: normal, cognitively impaired, or MCI vs. dementia

 

The overall sensitivity, specificity and AUC were 71.7%/75.9%/87.3%, 86.3%/84.8%/85.7% and 86.0%/87.63%/93.7% for MMSE (without covariate adjustment) and 78.2%/82.9%/85.1%, 79.2%/77.4%/86.9%, 79.2%/77.4%/86.9% and 86.2%/88.2%/93.7% for MoCA in detecting 1) cognitively impaired, MCI, or dementia, 2) MCI or dementia, and 3) dementia, respectively. These statistics were further examined within each subgroup of subjects, stratified by age, sex, and education level (a total of 24 subgroups, results not shown). Table 3 shows the averaged sensitivity, specificity and AUC: 78.2%, 83.9% and 88.1% for MMSE and 80.6%, 81.2% and 88.3% for MoCA. Importantly, focusing on dementia diagnosis, the sensitivity and specificity achieved 85.4%, 87.4% for MMSE and 86.3%, 85.5% for MoCA respectively.

 

Table 3 Sensitivity, specificity and AUC of MMSE and MoCA for different cognitive statuses in subgroups based on age, sex, education years

Test

MMSE

MoCA

Cognitive Status

I

II

III

Mean

I

II

III

Mean

Sensitivity

73.0%

76.1%

85.4%

78.2%

76.1%

79.4%

86.3%

80.6%

Specificity

82.2%

82.1%

87.4%

83.9%

79.1%

79.0%

85.5%

81.2%

AUC

84.6%

86.5%

93.3%

88.2%

84.9%

86.9%

93.1%

88.3%

* I: normal vs. cognitively impaired, MCI, or dementia; II: normal or cognitively impaired vs. MCI or dementia; III: normal, cognitively impaired, or MCI vs. dementia

 

Table 4 shows optimal cutoff points of MMSE and MoCA for each subgroup. The differences in optimal points between cognition impairment and MCI appeared to be negligible, whereas the differences were significant for dementia diagnosis. For MMSE, the optimal cutoff points were different between the education level below and above high-school, but the differences across age and sex were minimal. For MoCA, the optimal cutoff points exhibited little difference between females and males. On the other hand, age and education level appeared to affect the optimal cutoff points.

 

Table 4 The MMSE and MoCA optimal cutoff points in different subgroups defined by age, sex, and education level 

MMSE 

 Male 

Female 

AGE (years) 

61-70 

71-80 

81-90 

61-70 

71-80 

81-90 

  High School or Below

26/26/24

26/25/24

25/25/24 

27/27/25

26/26/24

25/25/24

Bachelor degree or equivalent

28/27/26 

27/27/26 

27/27/25 

28/28/26 

28/28/26 

27/27/26 

  Master degree or equivalent

28/28/27 

28/28/26 

27/27/26 

28/28/27 

28/28/27 

28/28/26 

  Doctoral degree or above 

28/28/27 

28/27/27 

27/27/26 

28/28/28 

28/28/27 

28/28/26 

 

MMSE 

 Male 

Female 

AGE (years) 

61-70 

71-80 

81-90 

61-70 

71-80 

81-90 

  High School or Below

24/24/19 

22/22/19 

20/20/18 

23/23/19 

22/22/19 

20/20/18 

Bachelor degree or equivalent

24/23/22 

23/23/21 

22/22/20 

24/24/22 

24/23/21 

23/23/20 

  Master degree or equivalent

25/25/23 

24/24/22 

24/24/21 

25/25/23 

25/25/23 

24/24/21 

  Doctoral degree or above 

26/25/23 

25/25/22 

24/24/22 

26/25/24 

25/25/22 

25/25/23 

* The numbers in each cell refers to the optimal cutoff point to differentiate I: normal vs. cognitively impaired, MCI, or dementia; II: normal or cognitively impaired vs. MCI or dementia; III: normal, cognitively impaired, or MCI vs. dementia 

 

CONCLUSION AND GENERALIZATION

This study findings suggest the necessity of applying personalized cut-off points in MMSE and MoCA tests. On top of age, sex, and education, both MMSE and MoCA are extremely helpful in testing cognitive status based on the substantial change in AUC. Specifically, these tests demonstrated greatest increase in AUC for the outcome of dementia, indicating that they may be more useful for dementia than impaired cognitive function or MCI. The sensitivity and specificity shared the same trend with AUC, could be a more powerful proof of our conjecture. In addition, we also presented the optimal cutoff points in a population 61 to 90 years old, stratified by sex and education level. However, most of the cutoff-points are similar for diagnosing impaired and MCI, suggesting that these tests are not sensitive to these characteristics in cognition impairment or MCI screening and may serve as a more efficient diagnostic tool for dementia.  

Although Tan et al. (2015) and Ng A et al. (2013) have studied the similar topic, the comparison between their works and ours may not be powerful enough due to the difference in sample size and study population. Even we used the same database as Chapman et al’s work (2016), our study considered age, education, and sex as covariates and created subgroups. In addition, the previous study treated MCI and dementia as two distinct diseases, while we considered impaired, MCI and dementia as an ordinal variable (though three separate binary variables were used in analysis). Given these differences in our analysis strategies, the sensitivity and specificity in Chapman’s work could be greater than ours. The exclusion of impaired cognitive function and the distinction of MCI and dementia may significantly enhance the positive predictive values of the test. Although our work failed to improve the previously reported sensitivity and specificity, the consideration of covariates and subgroups can make these tests more practical in real life.   

Overall, our findings suggested that the sensitivity and specificity of MMSE and MoCA can vary depending on subject’s characteristics, thus implying that using a personalized test cutoff may be more effective in the diagnosis of cognitive function than a single universal cutoff.  

 

SUGGESTIONS FOR FUTURE STUDIES

Our study has some limitations. First, given the fact that only one of the NACC study subjects had both MMSE and MoCA data, the increases in AUC after accounting for age, sex, and education are not directly comparable between MMSE and MoCA. Future studies may wish to further investigate the test performance of MoCA and MMSE in the detection of cognitive impairment and dementia across various subgroups in the general population. Second, depression was not considered in our analysis. Andersen, et al. (2020) suggested that depression could usually coexist with Alzheimer and may even accelerate the course of Alzheimer. The manifestation of depression could often be confused with early preference of dementia, which may cause difficulty in diagnosing dementia from depression. Since depression was not routinely screened in this dataset, we did not consider depression or exclude those with depression in our analysis. We believe that the findings could be more generalized to real world settings. Lastly, due to a relatively small group with impaired cognitive function, our results showed difficulty of MMSE/MoCA in diagnosing Impaired. Future studies may wish to recruit more participants with impaired cognitive function.

 

CONTACT INFORMATION

Feier Han  feier.han@emory.edu                Jing Shang     jing.shang@emory.edu

Zhen Wu   zhen.wu@emory.edu                Yutong Zhang  yutong.zhang@emory.edu

 

ACKNOWLEDGEMENT

We would like to acknowledge Dr Yi-An Ko for being our advisor. Her valuable and constructive suggestions have been very much appreciated.

 

APPENDIX

Appendix I Related Variable Description 

Variables 

Descriptions  

NACCUDSD 

The subject’s cognitive status with a clinical diagnosis at each visit.   

NACCUDSD=1: Subjects with normal cognition; NACCUDSD=2: Subjects who are cognitively impaired but who do not meet the criteria for mild cognitive status; NACCUDSD=3: Subjects with amnestic or non-amnestic mild cognitive status; NACCUDSD=4: Subjects with dementia   

NACCMOCA 

MoCA Total Score corrected for  

Education: 0 - 30 = Correct Test results  

88 = Item(s) or whole test not administered   

99 = Years of education missing/unknown   

-4 = Not available: UDS form submitted did not collect data in this way, or a skip pattern precludes response to this question  

NACCMMSE 

Total MMSE score (using D-L-R-O-W)  

0 – 30 = Correct Test results; 88 = Score not calculated; missing at least one MMSE item;  95 = Physical problem;   96 = Cognitive/behavior problem 97 = Other problem;   98 = Verbal refusal;   -4 = Not available: UDS form submitted did not collect data in this way, or a skip pattern precludes response to this question  

EDUC 

Subject’s years of education:  12 = high school or below,  16 = bachelor’s degree, 18 = master’s degree, 20 = doctorate or above.  

SEX 

Subject’s sex: 1 = Male; 2 = Female  

AGE 

Subject’s age derived from detracting “BIRTHYR” from “VISITYR”  

BIRTHYR: Subject’s year of birth  

VISITYR: Visit year comes from the date Form A1 was completed.  

SMOKYRS 

Total years smoked cigarettes  

ALCOHOL  

Subject with alcohol abuse occurring over a 12-month period with clinical diagnose.   0 = No; 1 = Yes  

DIABET 

Subject with diabetes present at visit:  0 = No; 1 = Yes  

HYPERT  

Subject with hypertension present:  0 = No, 1 = Yes  

HYPERCHOL  

Hypercholesterolemia present: 0 = No, 1 = Yes  

 

Appendix II, The ROC Curves for comparisons between combined age, education level, sex and after adding MMSE/MoCA  

 

MMSE 

MoCA 

I: normal vs. cognitively impaired, MCI, or dementia

FeierHan_0-1618886874109.png

 

FeierHan_1-1618886874118.png

 

II: normal or cognitively impaired vs. MCI or dementia

FeierHan_2-1618886874124.png

 

FeierHan_3-1618886874129.png

 

III: normal, cognitively impaired, or MCI vs. dementia

FeierHan_4-1618886874132.png

 

FeierHan_5-1618886874136.png

 

 

Reference

  1. World Health Organization. 2020. “Dementia.” Accessed March 10, 2021 https://www.who.int/news-room/fact-sheets/detail/dementia 
  2. Petersen, R. C., Lopez, O., Armstrong, M. J., Getchius, T., Ganguli, M., Gloss, D., Gronseth, G. S., Marson, D., Pringsheim, T., Day, G. S., Sager, M., Stevens, J., & Rae-Grant, A., 2018, "Practice guideline update summary: Mild cognitive impairment: Report of the Guideline Development, Dissemination, and Implementation Subcommittee of the American Academy of Neurology.” Neurology, 90(3), 126–135.  
  3. Folstein, M.E, Folstein, S.E., and McHugh, P.R. 1975 “’Mini-mental state’: A practical method for grading the cognitive state of patients for the clinician.” Journal of psychiatric research, 12(3):189-198. 
  4. Nasreddine, Z.S., Phillips, N.A., Bedirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J.L., Chertkow, H. 2005 “The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment.” Journal of the American Geriatrics Society 2005, 53(4):695-699. 
  5. Tan, J.P., Li, N., Gao, J., Wang, L.N., Zhao, Y.M., Yu, B.C., Du, W., Zhang, W.J., Cui, L.Q, Wang, Q.S., Li, J.J, Yang, J.S., Yu, J.M., Xia, X.N. and Zhou, P.Y. 2015. “Optimal cutoff scores for dementia and mild cognitive impairment of the Montreal Cognitive Assessment among elderly and oldest-old Chinese population.” Journal of Alzheimer's Disease, 43(4), 1403-1412. 
  6. Ng, A., Chew, I., Narasimhalu, K., and Kandiah, N. 2013. “Effectiveness of Montreal Cognitive Assessment for the diagnosis of mild cognitive impairment and mild Alzheimer’s disease in Singapore.” Singapore Med J, 54(11), 616-619. 
  7. Chapman, K. R., Bing-Canar, H., Alosco, M. L., Steinberg, E. G., Martin, B., Chaisson, C., Kowall, N., Tripodis, Y., and Stern, R. A. 2016. “Mini Mental State Examination and Logical Memory scores for entry into Alzheimer’s disease trials.” Alzheimer's research & therapy, 8(1), 1-11.
  8. Kukull, Walter A., 2015. “NACC Uniform Data Set: Researchers Data Dictionary Version 3.0.” National Alzheimer’s Coordinating Center. Accessed March 14, 2021, https://naccdata.org/data-collection/forms-documentation/uds-3 
  9. Youden, W.J., 1950. “Index for rating diagnostic tests.” Cancer 3.1, 32-35.
  10.   Goddard, M.J., Hinberg, I., 1990. “Receiver operator characteristic (ROC) curves and non-normal data: an empirical study.” Statistics in Medicine 9(3), 325–337. 
  11.   Perkins, N.J. and Schisterman, E.F., 2006. “The inconsistency of ‘optimal’ cut points obtained using two criteria based on the receiver operating characteristic curve.” American Journal of Epidemiology, 163(7), 670-675.
  12.      Fan, J., Upadhye, S., and Worster, A.,2006. “Understanding receiver operating characteristic (ROC) curves”. Canadian Journal of Emergency Medicine, 8(1), 19-20.
  13.      Andersen, K., Lolk, A., Kragh-Sørensen, P., Petersen, N. E., & Green, A. 2005. “Depression and the risk of Alzheimer disease.” Epidemiology, 233-238.  

 

Version history
Last update:
‎05-24-2021 08:12 AM
Updated by:

SAS Innovate 2025: Register Now

Registration is now open for SAS Innovate 2025 , our biggest and most exciting global event of the year! Join us in Orlando, FL, May 6-9.
Sign up by Dec. 31 to get the 2024 rate of just $495.
Register now!

SAS Training: Just a Click Away

 Ready to level-up your skills? Choose your own adventure.

Browse our catalog!

Article Tags