Skip to main content

Construct validity and reliability of the Test Your Memory Chinese version in older neurology outpatient attendees

Abstract

Background

Early distinguishing the cognitive impairment from healthy population is crucial to delay the progression of mild cognitive impairment (MCI) and Alzheimer disease (AD). Test Your Memory (TYM) has been proved to be a valid and reliable screening instrument for AD and MCI. This study aimed to develop a culturally appropriate and functional Standard Mandarin Chinese translation of the TYM, and to evaluate its reliability and validity in detecting AD and MCI in Chinese.

Methods

182 subjects with AD/MCI and 55 healthy controls were recruited to participate in this study, and everyone undergo the test of Standard Mandarin Chinese version of the TYM (TYM-CN), Mini-mental State Examination (MMSE), Montreal cognitive assessment (MoCA-BJ), and Clinical Dementia Rating (CDR) Scale. Concurrently, all the subjects with AD/MCI received the general physical and neurologic examinations, extensive laboratory tests, and brain computed tomography/magnetic resonance imaging (MRI). Of which, 90 subjects were asked to complete the re-test of TYM-CN at 3 weeks after the initial visit. Intra-class correlation coefficient (ICC) and Cronbach’s alpha was used to assess the test–retest reliability and the internal consistency. The validity, sensitivity and specificity were also analyzed. One-way analysis of variance, χ2 test, correlation analysis, and receiver operating characteristic curve (ROC) analysis were employed, as needed.

Results

The total scores of TYM-CN was 43.89 ± 3.44, 40.88 ± 4.38, and 29.12 ± 7.44 (p < 0.01) for healthy controls group, MCI group, and AD group, respectively. The ICC for 11 items of TYM-CN ranged from 0.863 (copying) to 0.994 (anterograde), and that of the total scale was 0.993, suggesting an excellent reliability. Furthermore, the significant correlation was also found between TYM-CN and MMSE (r = 0.76), MoCA-BJ (r = 0.74), and CDR scores (r = 0.76), indicating a good validity. A TYM-CN scores ≤ 39.5 had 95% sensitivity and 95% specificity in differentiating AD from healthy controls, and that ≤ 43.5 had 75% sensitivity and 91% specificity in distinguishing MCI from healthy controls, respectively.

Conclusion

The reliability and validity of the TYM-CN are statistically acceptable for the evaluation of cognitive impairment, which may contribute to neuropsychological tests for the diagnosis of AD and MCI from healthy controls in China.

Background

Dementia and other cognitive problems are becoming an important public health concern with the increase of aging population worldwide. An estimated 44.4 million individuals in the world have dementia in 2013 and the number will be expected to reach an estimated 75.6 million by 2030 [1], and 115.4 million in 2050 [2]. In China, which has the largest population of people with dementia, the prevalence of dementia appears to have increased steadily between 1990 and 2010 with the aging intensification [1, 3, 4]. A recent article published in 2014 showed that the prevalence of dementia among individuals aged over 65 years was 5.14% in China [5]. Similarly, another recent review study, conducted by Prince and his colleagues, indicated that the age-standardized prevalence for individuals over 60 years varied in a narrow range of 5–7% in the most world regions, with a higher prevalence of 8.5% in Latin America and a distinctively lower prevalence of 2–4% in the four sub-Saharan African regions [2]. It was also noteworthy that 58% of all people with dementia lived in countries with low or middle incomes in 2010, and this proportion will continue to rise to 63% in 2030 and 71% in 2050 [2]. Dementia mainly includes several types: Alzheimer’s disease (AD), vascular dementia (VaD), dementia with Lewy bodies (DLB) and frontotemporal dementia (FTD), and AD is the commonest form of dementia, contributing to 50–75% of dementia cases [6]. In China, a notably higher prevalence of dementia and AD was found in rural areas than in urban ones [5]. Future projections should focus on the preventive interventions for lowering the incidence, the improvements in treatment and care for prolonging survival, and the disease-modifying interventions for preventing or slowing progression.

Mild cognitive impairment (MCI), which is associated with an increased risk of developing Alzheimer’s or other subtypes of dementia, affects many more people [2, 7], who always represents a transitional status between healthy aging and dementia. Its’ prevalence has been reported to be between 10 and 20% in people older than 65 years [8]. Therefore, the appraisal of a patient’s cognition is a crucial part of many medical consultations. Cognitive tests not only aid the diagnosis of dementia, but also contribute the medical and social management of patients [9]. The need for an early diagnosis of AD and other dementia has been widely recognized and supported by groups including the UK National Dementia Strategy [10] and the National Institute for Clinical Excellence (NICE) [11]. In reality, there was a long delay between symptom onset and diagnosis, varying from 8 to 32 months [12]. It’s well known that early diagnosis requires recognition of the first cognitive deficits seen in AD and MCI [13].

Once the effective treatments for Alzheimer’s disease are available, there will be an even greater need for a quick sensitive test that is suitable for use in primary care and by non-specialists. Therefore, a short standardized mental status examination, which meets the three critical requirements for widespread use by a non-specialist [9]: take minimal operator time to administer; test a reasonable range of cognitive functions; and be sensitive to mild Alzheimer’s disease, will be helpful for the assessment of cognitive function in subjects with memory impairment [14].

Currently, the Mini-Mental State Examination (MMSE) [15] and Montreal Cognitive Assessment (MoCA) [16] are the most commonly administered psychometric screening assessment of cognition in China and other countries. However, the former has serval disadvantages, such as the insensitivity to the earliest changes in highly educated individuals [17], the bias against visually impaired [18], and a lack of ability to measure frontal/executive function [14]. Although the latter had more sensitive at detection of mild dementia and a slightly better diagnostic accuracy than the former, it’s still has some bias against people with poor education [18], who have difficulty in completing a test. Therefore, these two tools don’t meet the three critical requirements above [9] for widespread use by a non-specialist. In the light of the above, Brown et al. [9] designed a brief test (Test Your Memory, TYM) for the detection of Alzheimer disease (AD) and amnestic mild cognitive impairment (aMCI) [13], which consists of a series of 10 self-administered tasks, and was reported to be a valid and reliable screening test for the detection of AD. Currently, TYM has been a powerful short cognitive test that examines verbal and visual recall and been a valuable addition to the assessment of patients with aMCI/AD [19]. Concurrently, TYM has been translated into different languages, such as Japanese [14], French [20], Spanish [21, 22], and Polish [23], and also presented a good psychometric properties and diagnostic capacity to identify case of dementia.

Primary care is a fundamental part of health care systems in both high and low income countries, and there was also ample evidence that primary care contributed to the improvement of health outcomes [24]. In rural China, primary care including township health center (THCs) and village clinics, have dramatically improved access to health care in the communities of rural China over the last few decades, and are still playing an important role in the rural health system. From 2009, Chinese government comprehensively implemented the national basic public health services projects. Village doctors and the medical workers in THCs are responsible for providing basic public health services, including the establishment of health records, chronic disease screening and management, severe mental disorders management, and health education, to rural residents. However, we could not ignore the fact that most of workers, especially village doctors, in primary cares of rural China did not received the professional training in medicine. Too complicated technology and screening tools were not easy to understand by them. Therefore, a simple, convenient, and powerful short cognitive assessment tool would better help them deliver public health services to residents, and further increase the accessibility and efficiency of public health services. Therefore, this study aimed to develop a culturally appropriate and functional Chinese version of the TYM (TYM-CN) and to evaluate its reliability and validity for measuring AD and mild cognitive impairment (MCI) in Chinese.

Methods

Instrument

The original version of TYM consists of 10 tasks on a double-sided sheet of card with spaces for the subjects to fill in response [9]. Specifically, the tasks include orientation (10 points), ability to copy a sentence (2 points), semantic knowledge (3 points), calculation (4 points), verbal fluency (4 points), similarities (4 points), naming (5 points), visuospatial abilities (2 tasks, total 7 points), and recall of the copied sentence (6 points) [9]. Additionally, the subjects’ ability to complete the test is also scored from 5 points for subjects requiring no help to 0 point for patients requiring major help as the 11th task [9, 20], this limitation of help is to ensure that the test is performed adequately in the process of filling the questionnaire. The total scores of TYM is 50 points.

Translation and procedures

To obtain a Chinese version of TYM, multiple translation procedures were performed according to Beaton’s guideline, which was used for cross-cultural adaptation of health-related questionnaires [25]. The original version of TYM was firstly translated into a standard Mandarin Chinese version by two translators, one was a psychologist and another is a physician majoring in neurology. These two translators produced two primary TYM Chinese versions, independently. Subsequently, the third reconciled version was accomplished based on a comparison of the former two versions. On this basis, the fourth version was a translation of the Chinese version back into English and was produced by two additional translators, who had no the knowledge of the original questionnaire in advance. Eventually, any discrepancy of the fourth version from two translators was also resolved in the fifth version (the TYM-CN used in the present study) by an expert committee of School of Social Development and Public Policy at Beijing Normal University and the Department of Outpatient, General Hospital of the People’s Liberation Army (301 Hospital).

Cross-cultural adaptation was necessary in the process of translation of original TYM, therefore, the minor modifications were made when it was translated into Chinese. Specifically, the section about the ability to copying a sentence “good citizens always wear stout shoes” was adjusted to “Chinese working people always wear Jiefang shoes”, because the words “Jiefang shoes” is more familiar to Chinese people, especially, those over the age of 40. Furthermore, the words “Jiefang shoes” is much more vividly and specifically. The questions of semantic knowledge are “who is the prime minister?” and “In what year did the 1st World War start?” those are not common knowledge in China. However, Chinese people commonly know “the current national chairman” and “in what year People’s Republic of China was founded”. Therefore, these two questions were changed into “Who is the current national chairman of China?” and “The people’s Republic of China is founded in which year?” In the verbal fluency test, words beginning with the letter “S” were replaced by words beginning with the Chinese character ‘Hong’, which means red in Chinese. Chinese was familiar with this word, which was always on behalf of the happy, lucky, morale, happiness, etc. The last modification exists at the first question of visuospatial abilities test, the letter “W” was replaced by the Chinese character “Shang”. Chinese people, especially those in rural, are unfamiliar with the English letter, however, the vast majority of Chinese people know the word “Shang”. The other items were translated directly into Chinese.

In order to determine the readability and understandability of the preliminary TYM-CN, it was eventually distributed in a pilot study to 15 subjects, including 6 patients with mild AD, 4 patients with MCI, and 5 family members of patients with AD, who were asked about any unclear words, phrases, or concepts. The results of the pilot study showed that the TYM-CN could been easily understood by the subjects without significant complaints.

Ethics approval and consent to participate

This study was approved by the Institutional Review Board (IRB) of School of Social Development and Public Policy (SSDPP) at Beijing Normal University (BNU). All subjects provided written informed consent.

Subjects

Convenience sampling method, a type of nonrandom sampling, was used in this study. The main assumption associated with this sampling method is that the members of the target population are homogeneous [26]. Meanwhile, computing reliability and validity for questionnaires/scales need the minimum sample size. Based on the previous studies [27, 28], the minimum sample size in reliability and validity test should be at least 100, or the minimum ratios of sample size to the number of variables should be at least five.

A recent review study also found that about 90% of articles had a sample size greater than or equal to 100 for validating a scale [29]. In this study, we think that the minimum sample size of 100 subjects should be included based on the number of tasks of TYM-CN. Eventually, 182 subjects with AD or MCI were recruited from the patients attending the neurology outpatients from the General Hospital of the People’s Liberation Army (301 Hospital) between June 2014 and July 2015. Simultaneously, 55 healthy controls, who are family member of the patients above, were also included to this study.

Inclusion criteria included the following: (1) an ability to speak and read Chinese, (2) willing to cooperate with the investigators, and (3) agreeing to sign an informed consent form. While Exclusion criteria included: (1) the impaired verbal communication, visual impairment, (2) hearing-impaired, and mental retardation that could interfere with neuropsychological assessment, (3) underlying medical or psychiatric illness that could affect cognition, and absence of a reliable proxy.

The diagnosis of AD was based on the criteria which was published by the National Institute of Neurological and Communicative Disorders and Stroke and Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA) [30] and the standard clinical diagnostic criteria for the diagnosis of dementia [31]. Additionally, the following criteria [32], were also used for MCI diagnosis of this study, including a CDR score (controls = 0, dementia ≥ 1, and MCI ≤ 0.5) [33,34,35], the absence of dementia, memory complaints by the patients or their family, normal global cognitive function, normal activities of daily living, and the objective impairment in memory as evident by scores more than 1.5 standard deviations (SD) below the age-appropriate mean. Beside above, all patients with AD/MCI received the detailed general physical, neurologic and psychiatric examinations, extensive laboratory tests, brain computed tomography (CT)/magnetic resonance imaging (MRI).

A neurologist at neurology department of 301 Hospital diagnosed dementia and MCI based on detailed neurological, neuropsychological, laboratory, and neuroimaging data for each subject. Eventually, one hundred and two subjects were diagnosed as having AD, eighty subjects were diagnosed as having MCI. Additionally, 55 healthy controls were recruited from family member of patients, based on the principle of voluntary participation.

Neuropsychological tests

All the subjects underwent the Mini-mental State Examination (MMSE) [36,37,38], the Beijing version of Montreal cognitive assessment (MoCA-BJ) which was translated by Wei Wang and Hengge Xie from 301 Hospital [39,40,41], and the Chinese version of Test Your Memory (TYM-CN) by a three Ph.D. Candidates in psychometrics, under the guidance of physician in neurology. Clinical Dementia Rating (CDR) Scale was also used to assess the disease severity by a neurologist at neurology department. Concurrently, MMSE, MoCA-BJ, and TYM-CN tests were also administrated to the subjects in healthy controls. Of which, ninety subjects, including 70 patients with AD, 14 patients with MCI, and six healthy control, were asked to complete the re-test of TYM-CN by three nurses who didn’t know the subjects’ condition, 3 weeks after the initial visit, when they returned visit in the neurology outpatients. The demographic data of all subjects including gender, age, and educational level were also gathered in this study.

Statistical analysis

The continuous data were described by using the means and standard deviation values, and the categorical data were presented by using frequencies and percentage. Differences in gender were analyzed by using the χ2 test. Between-group differences in age, years of education, and neuropsychological test scores were analyzed by using one-way analysis of variance (ANOVA) with a post hoc Bonferroni test. The test–retest reliability was quantified by using the intra-class correlation coefficient (ICC), Cronbach’s alpha was calculated to assess internal consistency, and the value above 0.70 was considered to be adequate [42]. Meanwhile, the validity of the scale was also assessed by calculating the correlations between scores of two tests, among TYM-CN, MMSE, MoCA-BJ, by using the Spearman rank correlation analysis. Furthermore, the correlations between the scores of TYM-CN and CDR was also evaluated. Eventually, the sensitivity and specificity of TYM-CN was also assessed by using the receiver operating characteristic curve (ROC) analysis. A statistical software SPSS v. 21 (IBM, Chicago, IL, USA) was used for analyses in this study.

Results

Demographic and clinical data

The total sample included 237 subjects including 158 males and 79 females. Table 1 summarized their demographic characteristics and clinical information based on the results of post hoc analysis. No significant differences (p > 0.05) were found among groups with respect to age [F(2, 234) = 2.147, p = 0.119]. However, there were significantly difference in years of education [F(2, 234) = 3.58, p = 0.029]. Additionally, three groups did differ significantly in global cognitive impairment of MMSE scores [F(2, 234) = 108.39, p < 0.001] and MoCA scores [F(2, 234) = 236.56, p < 0.001], and dementia severity [CDR: F(2, 234) = 131.61, p < 0.001]. Furthermore, the subjects with dementia performed significantly worse than that with MCI and healthy controls, whereas the subjects with MCI performed significantly worse than healthy controls on the evaluation of global cognitive impairment and disease severity.

Table 1 Demographic data and cognitive screening tests by group (all subjects)

Comparisons of TYM-CN scores among three groups

The results of comparisons between the total TYM-CN scores and subscale scores for each group were shown in Table 2. The average total scores on the TYM-CN were significantly lower in AD and MCI groups than in the healthy controls group, and also significantly lower in AD group than in MCI group. The group of subjects with AD showed significantly lower scores on all the subscales including orientation, the ability to copy a sentence, semantic knowledge, calculation, verbal fluency, similarities, naming, visuospatial abilities, anterograde memory, and executive function, than MCI and healthy controls group. Additionally, the group of subjects with MCI has also significantly lower scores in most of subscales, except for orientation, copying, and naming.

Table 2 Comparison of performance on TYM-CN (total and subscale scores) between three groups

Effect of age, education, and gender on TYM-CN scores

In this study, age showed a weak correlation with the TYM-CN score within the healthy controls group (Kendall’s tau = − 0.25, p = 0.016), but not within the AD groups (Kendall’s tau = − 0.05, p = 0.58) and MCI group (Kendall’s tau = − 0.02, p = 0.80). As for subscales, there was only a weak evidence that the scores of calculation and naming of the TYM-CN varied with age. The years of education showed a weak positive correlation with the TYM-CN score within the MCI group (Kendall’s tau = 0.23, p = 0.008), but not within the AD groups (Kendall’s tau = 0.05, p = 0.55) and healthy controls group (Kendall’s tau = 0.14, p = 0.20). Concurrently, there was also a weak correlation in between the years of education and the scores of semantic knowledge (Kendall’s tau = 0.15, p = 0.006), naming (Kendall’s tau = 0.24, p < 0.001), and Visuospatial 1 (Kendall’s tau = 0.20, p = 0.001). Additionally, a weak positive correlation was found between male and the total scores of TYM-CN in AD group (Kendall’s tau = 0.19, p = 0.02) and MCI group (Kendall’s tau = 0.19, p = 0.049). Furthermore, a weak correlation in between gender and the scores of semantic knowledge (Kendall’s tau = − 0.17, p = 0.006), verbal fluency (Kendall’s tau = − 0.12, p = 0.05), similarities (Kendall’s tau = − 0.21, p = 0.001), naming (Kendall’s tau = 0.17, p = 0.005), visuospatial 1 (Kendall’s tau = − 0.27, p < 0.001), and anterograde (Kendall’s tau = − 0.13, p = 0.027). However, these results may be attributed to the sex ratio imbalance, because the male female ratio was 2:1 in this study.

Reliability analysis of TYM-CN

The results of reliability analysis showed that the ICC for 11 subscales was highly correlated between the test–retest among ninety subjects, with a range from 0.863 (copying) to 0.994 (anterograde) (Table 3), indicating an excellent reliability between the test–retest. Furthermore, the ICC for the total TYM-CN also presented excellent reliability with a value of 0.993. Cronbach’s alpha coefficient was 0.994 for total scores and 0.843 above for each subscale, also suggesting an excellent internal consistency for the total scale and subscales of the TYM-CN at the primary and secondary visits, respectively. Additionally, for all subjects, Cronbach’s alpha coefficient was 0.739, suggesting a good internal consistency for the 11 items of TYM-CN.

Table 3 ICC between the test–retest of the TYM-CN

Validity analysis of TYM-CN

The total TYM-CN scores was significantly correlated with scores of MMSE (r = 0.76, p < 0.0001) and MoCA-BJ (r = 0.74, p < 0.0001). Furthermore, the total TYM-CN scores was also significantly correlated with CDR scores (r = 0.76, p < 0.0001), which assessed the disease severity.

Sensitivity and specificity of TYM-CN

The diagnostic utility of TYM-CN with that of both MMSE and MoCA-BJ, which examine several cognitive domains, were also compared in this study. We found there was a significant difference in the Area Under the Curve (AUC) in between TYM-CN and MMSE, and in between TYM-CN and MoCA-BJ, and the sensitivity of TYM-CN fell in between MoCA-BJ and MMSE. Figure 1 showed the ROC curves which discriminated between AD group and healthy control group (Fig. 1a) and between MCI group and healthy controls group (Fig. 1b), respectively. ROC analysis demonstrated that the AUC was 0.989 (95% CI 0.977–1.00) for the TYM-CN, 0.999 (95% CI 0.997–1.000) for MoCA-BJ, and 0.941 (95% CI 0.907–0.976) for MMSE in differentiating AD from healthy controls group (Fig. 1a). Table 4 illustrated the sensitivity and specificity for the diagnosis of AD with different cut-offs of the TYM-CN. TYM-CN achieved the best differentiation between AD group and healthy control group for a cut-off value of ≤ 39.5, with a sensitivity and specificity of 95% and 95%, respectively. The sensitivity and specificity of the MMSE were 81.8% and 90%, respectively, with the established cut off ≤ 24. It should be noted that MoCA-BJ also presented the excellent diagnostic utility for AD, the sensitivity and specificity of the mini-mental state examination were 98.2% and 90%, with the established cut off ≤ 23.5. Additionally, Fig. 1b showed the results of ROC analysis for MCI group and healthy control group, the AUC was 0.887 (95% CI 0.824–0.951) for the TYM-CN, 0.909 (95% CI 0.859–0.959) for MoCA-BJ, and 0.813 (95% CI 0.739–0.887) for MMSE in differentiating MCI group from healthy control group. Table 5 illustrated the sensitivity and specificity for the diagnosis of MCI with different cut-offs of the TYM-CN. The TYM-CN differentiated MCI group from healthy controls group for a cut-off value of ≤ 43.5, with a sensitivity and specificity of 75% and 91%, respectively. Similarly, MoCA-BJ also presented the excellent diagnostic utility for MCI, the sensitivity and specificity were 75% and 87% with the established cut off ≤ 25.5. However, MMSE didn’t present a good performance in distinguishing MCI from health controls, the sensitivity and specificity were 81.8% and 67.5%, respectively, with the established cut off ≤ 24.5. These findings suggested that the TYM-CN may be a powerful diagnostic instrument for AD and MCI in Chinese.

Fig. 1
figure 1

Receiver operating characteristic (ROC) curves for the TYM-CN, MMSE, and MoCA-BJ screening tests in differentiating AD (a) and MCI (b) from health controls. TYM-CN, TYM Chinese version; MMSE, the Mini-mental State Examination; MoCA-BJ, Beijing version of Montreal cognitive assessment

Table 4 Sensitivity and specificity of optimal cut-off scores for diagnosis of AD
Table 5 Sensitivity and specificity of optimal cut-off scores for diagnosis of MCI

Discussion

To the best of our knowledge, we most likely provided the first report of the validity and reliability of the TYM-CN for the evaluation of cognitive impairment in subjects with AD or MCI in a certain Chinese populations. The reliability and validity of the TYM-CN were confirmed by using an internal consistency analysis and correlation analysis, respectively. Concurrently, the diagnostic utility was also evaluated by using ROC analysis. The ICC, calculated by the test–retest method, indicated the excellent reliability with values of 0.863–0.994 (Table 3), was consistent with the results of the original study conducted by Brown et al. [9].

Cultural adaptation will be necessary in the process of cross-cultural translation of scales in some researches. In this study, TYM-CN was modified with the minor adjustments to the copying, the semantic knowledge question, fluency test, and visuospatial abilities, taking into account cultural differences. Specifically, the sentence of “good citizens always wear stout shoes” about the ability to copying a sentence was adjusted to “Chinese working people always wear Jiefang shoes”, considering that “Jiefang shoes” is more familiar to Chinese people. Similarly, most of Chinese people could not answer the questions of semantic knowledge “who is the prime minister?” and “In what year did the 1st World War start?” because those are not common knowledge in China. However, almost all Chinese people know “the current national chairman” and “in what year People’s Republic of China was founded”. Therefore, the original two questions were changed to the latter two questions. Additionally, in the verbal fluency test, words beginning with the letter “S” were replaced by the Chinese character words ‘Hong’, which means red color, and sometimes symbolizes the happy, lucky, morale, and happiness in Chinese. The first question of visuospatial abilities test was also modified, namely, the letter “W” was replaced by the Chinese character words “Shang”. Chinese old people, especially in rural, are unfamiliar with the English letter, however, the vast majority of Chinese people know the word “Shang”.

Being consistent with the English original, we found good correlations between the TYM-CN and other neuropsychological tests. Strong and statistically significant correlation, which was found between the TYM-CN and the other measures of global cognitive impairment (MMSE and MoCA-BJ) and dementia severity (CDR), supported the content validity of TYM-CN. These results were consistent with those reported previously by the scientists in Japan [14], Span [22], France [20], Poland [23], and South Africa [43] by using this scale in their respective native languages, which have also reported acceptable correlations between the TYM and other cognitive measures. These findings also suggested that the original English version of the TYM could be applied cross-culturally for the evaluation of cognitive impairment.

The important correlation between the TYM-CN and other global cognitive impairment measures, such as MMSE and MoCA-BJ used widely in China, suggested that the TYM should be sensitive to executive disorders. Furthermore, we also thought that an administration time of approximately 10 min should be acceptable for a screening test in outpatients or clinics in primary health centers, because the MMSE also takes an average of 8 to 10 min to administer. However, MMSE was insensitive to the earliest changes of subjects with high education level [17]. Furthermore, the sensitivity of TYM-CN was better than MMSE in this study. As Brown put forward earlier [9], a screening instrument, meeting the three critical requirements, including the minimal operator time, a reasonable range of cognitive functions, and the sensitivity to mild Alzheimer’s disease, for widespread use by a non-specialist would contribute to early diagnosis of certain dementias. Therefore, such tools should be included in the cognitive tools and should be encouraged to use widely in practices.

In this study, the average total TYM-CN scores were similar to those of the English original TYM, which were 43.9 and 46.6/44.1 [9, 19] in healthy controls group, 40.9 and 36.3 [19] in subjects with MCI, and 29.12 and 29.5/33.2 [9, 19] in subjects with AD, respectively. Besides the English original TYM, the results were in good agreement with those reported by other scientists [14, 21, 22, 44]. We also noted that there were slight differences in each items scores between the Chinese version and the English original in patients with healthy control and AD, however, these two tests indicated that subjects with AD had particularly impaired anterograde memory compared with healthy controls. Furthermore, the significant discrepancy were also found in the scores of each subscale in between AD and healthy controls, and in between AD and MCI. This also indicated that cognitive impairments deteriorated gradually from healthy controls to MCI and to AD. As for the results of comparison between healthy controls and MCI, most of items presented the significant difference except for the subscale of orientation, copying, and naming, this was most likely due to the differences in age and severity of cognitive dysfunction.

The diagnostic utility of the TYM-CN to differentiate the cases of AD and MCI from healthy controls was supported by the high AUC and its acceptable sensitivity and specificity. In this study, TYM-CN differentiate significantly better between the AD group and the healthy controls than the MMSE, with the optimal cut-off scores (39.5), both the sensitivity and specificity for diagnosis of AD were 95%. Although, the cut-off score were lower than that of English original TYM with the cut-off score (42/43), the sensitivity of 93%, and specificity of 86% [9], our results also supported that TYM-CN performed better than MMSE in distinguishing AD from healthy controls. Meanwhile, TYM-CN also distinguished significantly MCI from healthy controls with the optimal cut-off scores (43.5), the sensitivity and specificity for diagnosis of MCI were 75% and 91%, respectively.

As we expected that MMSE also discriminated AD from healthy controls with the sensitivity of 81.8% and specificity of 90%, at the established cut off of ≤ 24. However, it did not exhibit the good ability in distinguishing MCI from the healthy controls. An early study, conducted by Trzepacz et al., also indicated that MoCA and MMSE were more similar for dementia cases, however MMSE didn’t distinguish MCI cases [45]. This further illustrated that MMSE was insensitive to the earliest changes in highly educated individuals [17]. Additionally, the mean education level of subjects in this study was 12.28 years, which may also influence the performance of MMSE. It must be mentioned that the sensitivity of MoCA in the detection of mild dementia and early cognitive impairment has been well known [46, 47]. Indeed, MoCA-BJ also presented a good ability in discriminating AD and MCI from health controls, and the order of the AUC was the following: AUCMoCA-BJ > AUCTYM-CN > AUCMMSE for differentiating AD and MCI from health controls in this study. A previous study also showed that the MoCA was a better cognitive tool than the widely used MMSE for the screening and monitoring of MCI and AD in clinical settings [48]. However, the level of difficulty of the items of MoCA-BJ was more than that of TYM-CN, therefore, it also took a long time in completing the evaluation in this study. Previous studies also reported that MoCA has some bias against people with poor educations, who have difficulty in completing a test [18, 49]. Actually, most of individuals over 60 years old in rural China did not got a good educations, this may limit the use of MoCA in these population. Concurrently, this disadvantage may also limit its application by the non-specialists at the grass-root health care, especially, in rural area. In comparison, the advantage of TYM became apparent, it was a self-administered test with a brief but rigorous scoring system, excellent inter-rater agreement for scoring, short time to completion, and can also be used easily by non-specialists, besides the advantage of cognitive tests. From this perspective, TYM-CN test not only can be used by the professionals in hospital, but also can be used by non-specialists at the village clinics in rural China.

Limitations of this study

Several limitations existed in the present study. First, a small convenience sample from the outpatient of one hospital was employed, furthermore, the subjects were older with mean age of 79 years old, although, there’re no difference in age among three groups. Therefore, it could preclude a generalization of the results obtained to an unselected population. Second, sex ratio imbalance (male female ratio of 2:1) was another limitation of this study, there was a weak correlation in between gender and the scores of subscale in TYM-CN, which may be originated from the gender differences. Third, the number of healthy controls was small, furthermore, the subjects in healthy controls group not only were younger than that in AD group, but also has less educational level than that in AD group, these limitations may influence the confirmation of cut-off value. Further studies of large sample size are needed to determine the extent of susceptibility to cultural (such as different ethnic and regions), educational, and age bias in our research team in the future.

Of course, there were several strengths which should be highlighted. First, all the patients with AD and MCI were diagnosed by a neurologist at neurology department of 301 Hospital, which is one of the highest level of general hospital in China, according to the detailed clinical examination data of each subject, based on criteria of NINCDS-ADRDA). Second, CDR scores were also employed by the nurses at neurology department of 301 Hospital under the guidance of the neurologist. Third, to better perform the reliability analysis, 90 subjects were asked to complete the re-test of TYM-CN by other three nurses who didn’t know patients’ condition at 3 weeks after the initial visit. These strengths undoubtedly enhanced the credibility of the results.

Conclusion

To the best of our knowledge, this may be the first study on the validity and reliability of the TYM-CN in China. We found that the TYM-CN was a valid and reliable instrument, which could contribute to the diagnosis of AD and MCI from the healthy controls. Furthermore, it promises to be a screening test by a non-specialist in primary cares in China in the future.

References

  1. Junfang X, Wang J, Wimo A, Fratiglioni L, Fratiglioni L, Qiu C. The economic burden of dementia in China, 1990–2030: implications for health policy. Bull World Health Organ. 2017;95:18–26.

    Article  Google Scholar 

  2. Prince M, Bryce R, Albanese E, Wimo A, Ribeiro W, Ferri CP. The global prevalence of dementia: a systematic review and metaanalysis. Alzheimers Dement. 2013;9(63–75):e62.

    Google Scholar 

  3. Zhang Y, Xu Y, Nie H, Lei T, Wu Y, Zhang L, Zhang M. Prevalence of dementia and major dementia subtypes in the Chinese populations: a meta-analysis of dementia prevalence surveys, 1980-2010. J Clin Neurosci. 2012;19:1333–7.

    Article  Google Scholar 

  4. Chan KY, Wang W, Wu JJ, Liu L, Theodoratou E, Car J, Middleton L, Russ TC, Deary IJ, Campbell H, et al. Epidemiology of Alzheimer’s disease and other forms of dementia in China, 1990–2010: a systematic review and analysis. Lancet. 2013;381:2016–23.

    Article  Google Scholar 

  5. Jia J, Wang F, Wei C, Zhou A, Jia X, Li F, Tang M, Chu L, Zhou Y, Zhou C, et al. The prevalence of dementia in urban and rural areas of China. Alzheimers Dement. 2014;10:1–9.

    Article  Google Scholar 

  6. Alzheimer’s Disease International (ADI). World Alzheimer Report 2009: the Global Prevalence of Dementia. London: Alzheimer’s Disease International; 2009.

  7. Yanhong O, Chandra M, Venkatesh D. Mild cognitive impairment in adult: a neuropsychological review. Ann Indian Acad Neurol. 2013;16:310–8.

    Article  Google Scholar 

  8. Petersen RC, Roberts RO, Knopman DS, Geda YE, Cha RH, Pankratz VS, Boeve BF, Tangalos EG, Ivnik RJ, Rocca WA. Prevalence of mild cognitive impairment is higher in men. The Mayo Clinic Study of Aging. Neurology. 2010;75:889–97.

    Article  CAS  Google Scholar 

  9. Brown J, Pengas G, Dawson K, Brown LA, Clatworthy P. Self administered cognitive screening test (TYM) for detection of Alzheimer’s disease: cross sectional study. BMJ. 2009;338:b2030.

    Article  Google Scholar 

  10. Department of Health. Living well with dementia: a national dementia strategy. London: Department of Health; 2009.

    Google Scholar 

  11. (NICE) NIfHaCE. Dementia: supporting people with dementia and their carers in health and social care. London: National Institute for Health and Clinical Excellence (NICE); 2006.

    Google Scholar 

  12. Bond J, Stave C, Sganga A, O’Connell B, Stanley RL. Inequalities in dementia care across Europe: key findings of the facing dementia survey. Int J Clin Pract Suppl. 2005;59:8–14.

    Article  Google Scholar 

  13. Brown JM, Wiggins J, Dong H, Harvey R, Richardson F, Hunter K, Dawson K, Parker RA. The hard Test Your Memory. Evaluation of a short cognitive test to detect mild Alzheimer’s disease and amnestic mild cognitive impairment. Int J Geriatr Psychiatry. 2014;29:272–80.

    Article  Google Scholar 

  14. Hanyu H, Maezono M, Sakurai H, Kume K, Kanetaka H, Iwamoto T. Japanese version of the Test Your Memory as a screening test in a Japanese memory clinic. Psychiatry Res. 2011;190:145–8.

    Article  Google Scholar 

  15. Black LJ. Complimentary, sequential antifertility effects of chlormadinone and norethindrone in the rabbit: implications in progestin only fertility control. Contraception. 1975;12:189–97.

    Article  CAS  Google Scholar 

  16. Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, Cummings JL, Chertkow H. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53:695–9.

    Article  Google Scholar 

  17. O’Bryant SE, Humphreys JD, Smith GE, Ivnik RJ, Graff-Radford NR, Petersen RC, Lucas JA. Detecting dementia with the mini-mental state examination in highly educated individuals. Arch Neurol. 2008;65:963–7.

    PubMed  PubMed Central  Google Scholar 

  18. Oxford Medical Education. Cognitive function tests in dementia. Oxford: Oxford Medical Education; 2017.

    Google Scholar 

  19. Brown JM, Lansdall CJ, Wiggins J, Dawson KE, Hunter K, Rowe JB, Parker RA. The Test Your Memory for mild cognitive impairment (TYM-MCI). J Neurol Neurosurg Psychiatry. 2017;88:1045–51.

    Article  Google Scholar 

  20. Postel-Vinay N, Hanon O, Clerson P, Brown JM, Menard J, Paillaud E, Alonso E, Pasquier F, Pariel S, Belliard S, et al. Validation of the Test Your Memory (F-TYM Test) in a French memory clinic population. Clin Neuropsychol. 2014;28:994–1007.

    Article  Google Scholar 

  21. Ferrero-Arias J, Turrion-Rojo MA. Validation of a Spanish version of the Test Your Memory. Neurologia. 2016;31:33–42.

    Article  CAS  Google Scholar 

  22. Munoz-Neira C, Henriquez Chaparro F, Delgado C, Brown J, Slachevsky A. Test Your Memory-Spanish version (TYM-S): a validation study of a self-administered cognitive screening test. Int J Geriatr Psychiatry. 2014;29:730–40.

    Article  Google Scholar 

  23. Szczesniak D, Wojtynska R, Rymaszewska J. Test Your Memory (TYM) as a screening instrument in clinical practice—the Polish validation study. Aging Ment Health. 2013;17:863–8.

    Article  CAS  Google Scholar 

  24. Starfield B. Is primary care essential? Lancet. 1994;344:1129–33.

    Article  CAS  Google Scholar 

  25. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25:3186–91.

    Article  CAS  Google Scholar 

  26. Suen LJ, Huang HM, Lee HH. A comparison of convenience sampling and purposive sampling. Hu Li Za Zhi. 2014;61:105–11.

    PubMed  Google Scholar 

  27. MacCallum RC, Widaman KF, Zhang S, Hong S. Sample size in factor analysis. Psychol Methods. 1999;4:84–99.

    Article  Google Scholar 

  28. Gorsuch RL. Factor analysis. 2nd ed. Hillsdale: Lawrence Erlbaum; 1983.

    Google Scholar 

  29. Anthoine E, Moret L, Regnault A, Sebille V, Hardouin JB. Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures. Health Q Life Outcomes. 2014;12:176.

    Google Scholar 

  30. McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology. 1984;34:939–44.

    Article  CAS  Google Scholar 

  31. American Psychiatric Association. Diagnostic and statistical manual of mental disorders (4th Edition) (DSM-IV-TR). Washington: American Psychiatric; 2000.

    Google Scholar 

  32. Petersen RC, Doody R, Kurz A, Mohs RC, Morris JC, Rabins PV, Ritchie K, Rossor M, Thal L, Winblad B. Current concepts in mild cognitive impairment. Arch Neurol. 2001;58:1985–92.

    Article  CAS  Google Scholar 

  33. Hughes CP, Berg L, Danziger WL, Coben LA, Martin RL. A new clinical scale for the staging of dementia. Br J Psychiatry. 1982;140:566–72.

    Article  CAS  Google Scholar 

  34. Gao Z, Wang W, Shang Y, Bai X, Wu W. Exploration of the Chinese version montreal cognitive assessment in the diagnosis of mild cognitive impairment. Chin J Health Care Med. 2011;3:225–7 (in Chinese).

    Google Scholar 

  35. Morris JC. The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology. 1993;43:2412–4.

    Article  CAS  Google Scholar 

  36. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–98.

    Article  CAS  Google Scholar 

  37. Gao M, Yang M, Kuang W, Qiu P. Factors and validity analysis of Mini-Mental State Examination in Chinese elderly people. J Peking Univ (Health Sciences). 2015;47:443–9 (in Chinese).

    Google Scholar 

  38. Peng D, Xu X, Liu J, Jiao Y, Zhang H, Yin J, Meng X, Xie Y, Feng K. Discussion on application of MMSE for senile dementia patients. Chin J Neuroimmunol Neurol. 2005;4:187–90 (in Chinese).

    Google Scholar 

  39. Sun H, Xie Y, Zhang X, Xie H, Wu W. Items in montreal cognitive assessment. Chin J Geriatr Heart Brain Vessel Dis. 2014:387–90 (in Chinese).

  40. Xie H. Cognitive impairment and neuropsychological assessment in Alzheimer’s disease. Chin J Pract Intern Med. 2010;30(10):883–7 (in Chinese).

    Google Scholar 

  41. Huang F, Wang Y, Li J, Wang L, Jiang Y, Liao S. Diagnostic value of montreal cognitive assessment for mild cognitive impairment in Chinese middle-aged adults: a meta-analysis. Chin J Evid Based Med. 2017;17(4):450–7 (in Chinese).

    Google Scholar 

  42. Nunnally JC, Bernstein IH. Psychometric theory. 3rd ed. New York: McGraw-Hill; 1994.

    Google Scholar 

  43. van Schalkwyk G, Botha H, Seedat S. Comparison of 2 dementia screeners, the Test Your Memory Test and the Mini-Mental State Examination, in a primary care setting. J Geriatr Psychiatry Neurol. 2012;25:85–8.

    Article  Google Scholar 

  44. Papachristou E, Ramsay SE, Papacosta O, Lennon LT, Iliffe S, Whincup PH, Goya Wannamethee S. The Test Your Memory cognitive screening tool: sociodemographic and cardiometabolic risk correlates in a population-based study of older British men. Int J Geriatr Psychiatry. 2016;31:666–75.

    Article  Google Scholar 

  45. Trzepacz PT, Hochstetler H, Wang S, Walker B, Saykin AJ, Alzheimer’s Disease Neuroimaging I. Relationship between the Montreal Cognitive Assessment and Mini-mental State Examination for assessment of mild cognitive impairment in older adults. BMC Geriatr. 2015;15:107.

    Article  Google Scholar 

  46. O’Caoimh R, Timmons S, Molloy DW. Screening for mild cognitive impairment: comparison of “MCI Specific” Screening Instruments. J Alzheimers Dis. 2016;51:619–29.

    Article  Google Scholar 

  47. Hoops S, Nazem S, Siderowf AD, Duda JE, Xie SX, Stern MB, Weintraub D. Validity of the MoCA and MMSE in the detection of MCI and dementia in Parkinson disease. Neurology. 2009;73:1738–45.

    Article  CAS  Google Scholar 

  48. Freitas S, Simoes MR, Alves L, Santana I. Montreal cognitive assessment: validation study for mild cognitive impairment and Alzheimer disease. Alzheimer Dis Assoc Disord. 2013;27:37–43.

    Article  Google Scholar 

  49. Gomez F, Zunzunegui M, Lord C, Alvarado B, Garcia A. Applicability of the MoCA-S test in populations with little education in Colombia. Int J Geriatr Psychiatry. 2013;28:813–20.

    Article  CAS  Google Scholar 

Download references

Authors’ contributions

DHT and HH gave us the idea of this study. XML and WJZ participated in the design and conducted the research as well as drafted the manuscript. SFZ, JSZ, JRZ and YRZ collected the data and provided the data analysis help. All authors read and approved the final manuscript.

Acknowledgements

We thank Beijing Normal University and 301 Hospital for giving financial support. We also thank the hospital officials and the data collectors for providing the support for this study, and all the respondents for participating our study.

Competing interests

The authors declare that they have no competing interests.

Funding sources

This study was supported by the Fundamental Research Funds for the Central Universities (No: SKZZX2013053 in Beijing Normal University).

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Weijun Zhang or Donghua Tian.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Zhang, S., Zhang, J. et al. Construct validity and reliability of the Test Your Memory Chinese version in older neurology outpatient attendees. Int J Ment Health Syst 12, 64 (2018). https://doi.org/10.1186/s13033-018-0240-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13033-018-0240-0

Keywords