Development of neuropsychological evaluation screening tool: An education-free cognitive screening instrument
Correspondence Address: Source of Support: None, Conflict of Interest: None DOI: 10.4103/0028-3886.227304
Source of Support: None, Conflict of Interest: None
Keywords: Cognition, illiterates, India, neuropsychological assessment, neuropsychological screening
Cognitive impairment is often considered as a precursor to serious disorders such as dementia and depression. As life expectancy increases, prevalence of cognitive impairment and dementia continues to increase; the number of patients with cognitive complaints in primary care units and specialized outpatient clinics is expected to grow further. The prevalence of cognitive impairment in a north Indian elderly population was found to be 8.8%, which highlights the need to screen geriatric population for neuropsychological impairment. Different neurological, psychiatric, and other conditions/disorders/diseases such as multiple sclerosis (MS), stroke, schizophrenia and psychosis, mood disorders, diabetes mellitus, and human immunodeficiency virus (HIV) are associated with cognitive impairment.
As a detailed neuropsychological assessment is time-consuming and requires a stepwise organization for optimizing medical resources, cognitive screening tools are important at the primary stage of evaluation. The primary care professionals need simple tools to detect early cognitive impairment. Emphasis should be placed on tools that are easy to use, have a short application time, are reliable and valid, and are able to discriminate between those patients who are cognitively intact versus those who are cognitively impaired. To be used in India, the development of these tools either require adaptation and validation of the existing tests in the local language to improve the sensitivity, or to search for measures with adequate cross-cultural validity.
Although there are several screening tools available for clinicians to use, their merit has been a matter of debate. There are very few tools which have been developed for populations in which they are intended to be used. Despite being one of the most widely used screening tools, Mini Mental State Examination (MMSE) has many shortcomings such as insensitivity to mild and subcortical pattern of cognitive impairment, a prominent ceiling effect, and uneven sampling of cognitive domains; it also lacks sensitivity to frontal, linguistic, and early amnestic deficits. Despite many translations and adaptations for varied populations with different educational levels,,, the limitations of MMSE are well-known. However, MMSE, despite all its limitations, still remains widely used.
The need for an alternative method to screen for cognitive impairment has led to the development of various screening measures such as Demtect, Seven Minute Screen,,,, Montreal Cognitive Assessment,,, Addenbrooke's Cognitive Examination (ACE) and its revision,, Clock Drawing Test, Informant Questionnaire on Cognitive Decline in the Elderly- IQCODE, Test Your Memory, General Practitioner Assessment of Cognition- GPCOG, Six-Item Cognitive Impairment, Alzheimer's Disease Assessment Scale- ADAS-Cog, Cambridge Cognitive Examination (CAMCOG) and its revision, Cognitive Abilities Screening Instrument, Rowland Universal Dementia Scale, Mini-Cog, etc.
India is a land of lingual diversity where more than 450 languages are spoken, highlighting the need for a screening tool that is not impacted by cultural factors and which can be used by as many people as possible. Various Indian adaptations have been made for existing screening tools. The Indian modified adaptation of MMSE is the Hindi Mental State Examination (HMSE), which was adapted for the illiterate Hindi speaking population and provides a brief index of the subject's current level of functioning. HMSE was specifically adapted for screening dementia and not other neurological or psychiatric conditions. ACE, ADAS-Cog, and Cognistat (Neurobehavioral Cognitive Status Examination) were adapted specifically for patients with traumatic brain injury. Detailed lists of existing screening tools used in India, along with their limitations for use in India, have been highlighted in [Table 1]. Some of these tools are available for use in the Indian population, but are adapted and translated versions of their original tools, which is the biggest disadvantage of these tools.
Though most of these tools have demonstrated better sensitivity and specificity than MMSE, they have their own shortcomings. Some take a longer time to administer, some are more affected by education and some by culture, while some require complex test material and are not easily scored. Some limitations of test adaptations and translations are that the item modifications/translated items are usually not equivalent to the original item. Some examples are: in MMSE, “no ifs, and or buts” is not easily translated in other languages, and in translation, it may lose its “articulatory complexity” and the term might be unfamiliar in different contexts; orientation to time and place requires familiarity with western calendars and address styles; spelling “world” backward and counting back by 7s – requires education, which may be less relevant in some cultural groups; Some literal translations may have different meanings – e.g., in Italian, the word “memorize” translates to “learn,” and “close your eyes” can imply “death” in the Chinese culture.
Only a limited number of cognitive screening tools are available specifically for the Indian population. There are only adapted and translated cognitive tools and are not original, or are costly or not culture or education free. This led to the development of an original Indian cognitive screening instrument – the Neuropsychological Evaluation Screening Tool (NEST) which was developed in the Indian national language, Hindi, and in English.
After an extensive review of the literature, domains were selected based on a basic cognitive evaluation which involves evaluation of attention, memory, perception, verbal and language skills, constructional ability, and executive functioning, which are a part of the detailed assessment procedures in Neuropsychological Assessment, 5th Edition. The finalized specific domains were immediate memory, verbal fluency, abstraction, visuospatial ability, delayed memory, focussed attention, divided attention, and temporal sequencing. The major domains covered in the tool are memory, executive functioning, visuospatial ability, and working memory. All participants were informed and consented to participate in the study. This was a noninvasive procedure/assessment, where none of the participants were at any risk or discomfort.
NEST was developed with three major objectives:
The steps used for the development of NEST were:
The study was done in two phases. Phase 1 included the development of NEST; and, Phase 2 was the administration of NEST on patients. To form a representative group, both healthy controls and patients were included with varied ethnicity, geographic region, gender, age, education, occupation, work settings, etc.
Part 1 included the development of NEST to assess immediate memory, verbal fluency, abstraction, attention, visuospatial ability, long-term memory, and temporal sequencing. The test was developed in Hindi and English. Four clinical neuropsychologists independently worked on the areas which should be included in the tool. Items were selected based on the existing screening tools. The tool does not require the participant to read or write sentences as it was designed to be as education-free as possible. The items' instructions were formulated both in English and Hindi. The items were first developed in Hindi, and then were translated in English using forward and back translations. The numbers of correct and incorrect responses for each item were noted, histograms were compared for each item, and then items which were of medium difficulty were chosen among items which were too easy or too difficult. The entire study design is described in [Figure 1].
NEST is comprised of 6 items, namely
Item 1: Immediate recall: The patient needs to repeat and remember the address mentioned by the examiner/assessor; the patient is also told to remember it as he/she would be asked the same address later (Item 6).
Item 2: Verbal fluency: In 1 minute, the patient has to name all the things which we eat. This can include all fruits, vegetables, grains/cereals, etc.
Item 3: Abstraction: The patient is asked regarding the similarity between wood and paper.
Item 4: Temporal sequencing/Executive functioning: The patient is shown figures consisting of 5 hands and dots in variable numbers. The patient is asked to join the hands and dots in an increasing order, i.e., a hand with 1 finger up with 1 dot, similarly, 2 fingers up with 2 dots, and so on; this should be carried out without lifting the pencil/pen. All the hands and dots need to be joined to complete the item successfully.
Item 5: Visual-constructional ability: The patient is asked to copy a cuboid as shown to him/her.
Item 6: Delayed recall: The patient is asked the name and address he had been earlier asked to remember (Item 1).
Part 2 comprised administering NEST on patient groups along with HMSE and PGIMS to establish the cut-off scores and evaluate its efficacy as a screening tool for cognitive impairment.
NEST was administered to 84 healthy controls to analyze the normative data and revise and review items over a period of 1 year during the process of development. These participants were screened by a clinical neuropsychologist using a sociodemographic sheet, history, and a brief interview. Individuals were of both genders between the ages of 16 and 71 years. Participants with a history of any neurological disorder, major psychiatric illness, or serious life-threatening disease were not administered NEST. The participants were enrolled from the community after obtaining informed consent.
After an initial analysis and revisions on healthy controls, it was administered on 408 patients to establish the cut-off scores and calculate the sensitivity and specificity of NEST against PGIMS and the existing Hindi screening tool – HMSE. Participants belonged to both genders and were between 16 and 87 years of age. Patients were diagnosed by their respective consultants from where they were referred before sending them for a cognitive evaluation. The diagnosis was based on the International Classification of Diseases (ICD)-10 or the Diagnostic and Statistical Manual of Mental Disorders (DSM) for psychiatric disorders, National Institute of Neurological Disorders and Stroke (NINDS) Criteria for dementia and stroke, Glasgow Coma Scale (GCS) for traumatic brain injury, International League Against Epilepsy (ILAE) for epilepsy, etc.
The three scales which were independently administered were:
[Table 2] shows the sociodemographic details of the patients evaluated on NEST. [Table 3] and [Table 4] show the sample details and the descriptive analysis, respectively.
[Table 5] shows the per item errors committed by the patient group. As can be seen, item 5, which is for delayed recall has the maximum errors; followed by item 6, which is for visuospatial ability, temporal sequencing, and executive functioning; followed by item 2, which is for verbal fluency/language ability. The least number of errors were seen in item 1, which was for immediate recall.
As can be seen in [Table 6], a heterogeneous sample was taken for the study, which included both cognitively intact and impaired patients according to the “Gold Standard,” PGIMS, so that the sensitivity and specificity of NEST could be established as effectively as possible.
The cut-off score for NEST was based on the absolute standard, also called criterion- referenced standards, to establish a specific level of performance which must be attained by the subject. Pass/fail decisions were made based on whether or not this level of performance was met. Using receiver operating characteristic (ROC) analysis, three or more errors were identified as the optimum cut-off to label a screening test positive for cognitive impairment. As can be seen in [Table 7], three or more errors were taken as the cut-off for labelling a person as being an index case for a more detailed cognitive evaluation, highlighting the presence of a likely cognitive impairment in him/her. The sensitivity of NEST with this cut-off was 94.8% and its specificity was 60.3%. NEST was correctly able to classify and screen 82.2% of individuals with cognitive impairment when compared to a detailed evaluation using PGIMS.
According to [Table 8] and [Graph 1], the diagnostic characteristics of NEST with PGIMS were that it had a sensitivity of 94.78% and a specificity of 60.31%; it could positively predict 80.7% of the cognitively impaired cases; and, it had a negative predictive value of 86.8%. NEST, therefore, had an 82.5% agreement with PGIMS. The area under the curve was 87.9%. On the other hand, the diagnostic characteristics of HMSE with PGIMS as the reference were that it had a sensitivity of 73.8%, specificity of 82.4%, could positively predict value in 88.0% of the cases, and had a negative predictive value of 64.3%. HMSE had a 76.9% agreement with PGIMS.
As can be seen from [Table 9], NEST was sensitive enough to detect cognitive impairment among all educational levels, i.e., from individuals with no formal education to people with more than 20 years of education, where its sensitivity ranged from 72.5% to 95%, and specificity from 34% to 100%.
Epidemiological studies report a prevalence rate of 9.5 to 370/1000 for psychiatric disorders in India  where cognitive deficits are prevalent in patients with schizophrenia, mood disorder, obsessive compulsive disorder, somatoform disorder, and substance abuse, among others. As the incidence of neurological and psychiatric conditions increases, the need for screening these patients for cognitive impairment also becomes increasingly important. Moreover, due to the rising population of the elderly subjects, screening of the geriatric population for neuropsychological impairment needs to be included in priority care within the national primary healthcare framework.
Quick and brief screening tools for patients with dementia in primary care are yet to be validated in the non-Western settings where cultural factors and limited education need to be considered. The feasibility and validity of brief versions of tests need to be established in the routine primary care. Schooling can influence a subject's performance with regard to verbal fluency tasks. Neurologists, psychiatrists, neurosurgeons, emergency service personnel, and other clinicians report having little time to assess cognitive functioning of their patients despite recognizing its importance. Many screening tools either have subjective screening methods, judged to require too much time to administer, or are not available in the local/colloquial language.
A perusal through the literature of Neurology India itself shows various studies and reviews which focus on clinical situations where this tool would be relevant. Srinivas and Shah  addressed cognitive impairment, an underreported comorbidity of epilepsy, and Ungprasert et al., correlated the risk of dementia in patients with rheumatoid arthritis. In both the studies, a quick and easy-to-use cognitive screening tool like NEST would be useful for clinicians. In other studies, translated and adapted screening tools have been utilized to study cognitive functioning in different patient populations.,,,, Other studies utilizing formal neuropsychological testing for various diseases/disorders ,, would also benefit clinicians and researchers by screening for cognitive impairment in larger patient samples undergoing a detailed neuropsychological assessment.
Screening populations using tools which are free from the influence of ethnicity, gender and age, are important, but so far, these issues remain unresolved. Clinicians from developing countries use translated and adapted cognitive assessment tools to suit their culture and language. As schooling in such countries is relatively heterogeneous, it is necessary to establish normative and cut-off scores for these instruments locally, or to develop culture and education-appropriate tools for individual countries and for indigenous population groups.
This tool has been administered on a wide array of patient populations such as patients with traumatic brain injury, neurological diseases/disorders including Alzheimer's disease, frontotemporal degeneration, Lewy body disease, vascular disease, unspecified dementias, epilepsy, stroke, mild cognitive impairment, cerebral ataxia, encephalitis, multiple sclerosis, hypoxia, brain tumours, movement disorder, cerebral atrophy, meningitis, headache, migraine, psychiatric disorders including schizophrenia and related disorders, pseudodementia, mood disorders, alcohol abuse/dependence, depression and related disorders, somatoform disorders, substance abuse/dependence, anxiety and related disorders, and other medical conditions such as syncope, sleep disorders, diabetes, cardiovascular accidents, stiff person syndrome, gliosis, hydronephrosis, NERD's syndrome, as well as memory complaints by patients not being a part of any neurocognitive disorder. It can be used on diverse patient populations including various neurological, neurosurgical, psychiatric, and other medical conditions to screen cognitive impairment, which is as important to the patient's quality of life as his/her physical symptoms. It can be further standardized on other medical conditions and settings, such as cancer-related cognitive decline, and the effect of various surgical interventions such as bariatric surgery, among other conditions. The use of this tool can be explored not only in clinical conditions but can be used in screening for cognitive impairment or to track changes that occur in a healthy but aging population.
This study had its own limitations. First, the sample was male dominated, which was due to the readiness of male participants more than the female ones to participate in the study. Second, there are verbal items in the tool, which are affected by education. Despite this being as education-free as possible, it would be affected by low education levels of individuals. The sample was also heterogeneous, which is being examined by an ongoing study with a more homogeneous sample.
The study had a robust methodology, a good sample size, and a wide age range that included young adults, adults and elderly (16–87 years) patients. It included various neurological and psychiatric conditions, recruited patients with a broad range of educational level from being an illiterate to having more than 20 years of education. This tool could, therefore, correctly classify cognitive impairment in 82.2% of the cases. NEST, with its good sensitivity and specificity, can be used as a screening tool for various neurological, psychiatric, and other medical conditions to detect cognitive impairment among India's diverse population. Given the ease of administration, NEST can be the first-line method for identifying individuals who may subsequently require a formal and detailed neuropsychological assessment. NEST is a valid assessment of cognition that shows a good agreement with the existing screening tool (HMSE) and global measure of cognition (PGIMS), showing a good convergent validity. It was found to be superior to HMSE in this regard. NEST can be administered in approximately 5 minutes, is easily utilizable by various clinicians, and is cost-effective and free. In addition, preliminary evidence finds NEST to be psychometrically robust. NEST is only intended to screen individuals and populations for cognitive impairment and cannot be used for diagnostic purposes. Basic training in the administration, scoring, and interpretation should precede its use. This tool can be utilized by various clinicians to obviate the need for comprehensive neuropsychological evaluations.
NEST can be translated in various Indian languages such as Bengali, Tamil, Telugu, Odiya, Punjabi, and Marathi, and its efficacy can be tested in different patient populations in different states of India. NEST can also be used by other developing nations and can be translated in their own local language for cognitive deficit screening. It can also be used to screen for cognitive impairment among various neurological and psychiatric conditions, and a different cut-off point, if applicable, for each condition can be used.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5], [Table 6], [Table 7], [Table 8], [Table 9]