Publication performance and research output of Neurology and Neurosurgery training institutes in India: A 5-year analysis
Correspondence Address: Source of Support: None, Conflict of Interest: None DOI: 10.4103/0028-3886.158200
Source of Support: None, Conflict of Interest: None
Context: Scientific publications are a reflection of the quality of the clinical and academic work being carried out in an institute. Training in the process of research and scientific writing are important components of the residency curriculum.
Keywords: Citation; clinical neurosciences; India; neurology; neurosurgery; publication performance; rankings; research output; residency programs
The training curricula for neurology and neurosurgery residents have been constantly evolving over the past several decades. , Residency training now involves the acquisition of core competence across several domains. These domains include clinical patient care, procedural skills development, theoretical knowledge, communication and teaching skills, teamwork and management skills, exposure to evidence-informed medicine, training in research methodology, and scientific writing.  Since the quality of resident training directly affects the quality of clinical care and research, it becomes essential to monitor the skills acquired during the training period. ,, The most important of these domains would obviously be clinical care, procedural skills, and theoretical knowledge of the subject. The success of a training institute in imparting these skills, however, is difficult to evaluate; the required information is often not easily available.
One of these components, however, readily lends itself to quantitative evaluation and comparison. This is the exposure to research methodology and scientific writing, as measured by the publication and research output of a training institute. The number and quality of publications emanating from an institute can be construed to indicate (albeit indirectly) the quality of clinical work and research being undertaken in that institute. Thus, by extension, the degree of exposure of residents to these domains could be gauged. Publications and research output are factors that have consistently been used to rank institutes and training programs. , While the volume of clinical exposure in most training programs in India may be adequate, the adequacy of exposure to research and scientific writing is not known. , Although presentations at conferences also denote academic activity, the absence of a rigorous peer review and lack of a permanent record for peer usage make this activity inappropriate for evaluation and comparison.
Indian medical colleges have been ranked by the print media as well by academic organizations. , However, till date, there has been no evaluation of institutes conducting residency programs in neurology and neurosurgery on the basis of their publication performance and research output. In this study, the research output and publication performance of institutes that train residents in neurology and neurosurgery in India were evaluated.
The list of institutes recognized by the Medical Council of India (MCI) to conduct training programs in neurology (DM) and neurosurgery (MCh - 3 and 6 years programs) was obtained from the MCI website (http://www.mciindia.org/). However, similar information for the diplomate courses (DNB) was not readily available on the website of the National Board of Examinations (NBE, http://www.natboard.edu.in/). Therefore, the centers conducting DNB courses in neurology and neurosurgery were identified from the lists of seats offered in institutes as published in the admission prospectuses for super-specialty DNB courses for the years 2013-2015. The lists of papers published by departments (neurology/neurosurgery) in the listed institutes were then identified as described subsequently.
The website of each institute was accessed to determine the exact listed departmental name - for instance in some institutes, neurology and neurosurgery were combined and listed as "Department of Clinical Neurosciences" or "Department of Neurological sciences."
A PubMed search was built using the advanced search function to collate papers published by each institute. For all searches, 2 search fields were employed. In SEARCHSET-1, the first search string was the department/subject (neurology/neurosurgery) and this was searched as an "affiliation." If a different nomenclature (besides neurology/neurosurgery) had been displayed on the website, this was used in addition to neurology/neurosurgery, in a separate search. In SEARCHSET-2, neurology and neurosurgery were searched as Major Medical Subject Heading terms. In both search sets, the 2 nd search string was always the name of the institute; this was also searched as an affiliation. Every effort was made to incorporate as many spelling and abbreviation variations as possible in this search term. For instance, if the word "Shri" appeared in the name of an institute, irrespective of the spelling listed on the website, separate searches were carried out using all possible spelling variations (such as Sri, Shri, Shree, Sree, etc.,). At some instances, as many as 7 separate searches were carried out for a single institute in each search set. For example, publications by the Post Graduate Institute of Medical Education and Research (PGIMER), Dr. Ram Manohar Lohia Hospital were searched for using the entire name and various abbreviations thereof (such as PGIMER Dr. Ram Manohar Lohia hospital, PGIMER Dr RML Hospital, Dr. Ram Manohar Lohia hospital, RML Hospital, R M L Hospital, etc.,). The search limits were set to the past 5 years (2010-2014 inclusive).
For the next search, the institutional web site was accessed again to check if the names of the teaching faculty were listed. If faculty names were available, a PubMed search was performed using the name of each faculty member as an author.  The list of papers published by the faculty over the past 5 years were collated.
Lists of papers obtained by each of the several searches carried out were merged and duplicates eliminated. Separate databases were generated and maintained for each institute using EndNote® (vX7, Thomson Reuters).
Each publication was categorized as a case report, original paper, and review article or as belonging to a miscellaneous category (response to comments, opinion piece, editorial, etc.). Case series describing ≥4 cases were considered to be original articles. No distinction was made between basic laboratory research, clinical investigations or trials, translational research or case series - these were all categorized as original papers.
The following calculations were then made:
All results are current as of January 2015. A total of 223 neurology residency spots were available in 2014; these were distributed across 92 institutes. Of these, 173 (77.5%) were DM seats and 32.5% were DNB seats. 239 neurosurgery residency spots were available in 100 institutes as of 2014; 200 of these were MCh seats (83.7%) and the remainder, DNB seats. The distribution of these seats between the government and private sector is displayed in [Table 1].
Institutes running neurology residency programs
A total of 1911 papers had been published by neurology teaching institutes in PubMed indexed journals during the 5-year period 2010-2014 (both inclusive). The mean number of publications/institute/year was 4.2 (±10.11); the median was 0.8 papers/institute/year. The average number of papers published by government institutes each year (8.16 ± 14.5) was significantly higher than the number published by private institutes (1.22 ± 1.66) (t = 2.97, P < 0.001). The mean publications/institute/year for DM programs was 5.93 (±11.99) and for DNB neurology programs was 0.69 (±0.81); this difference was significant as well (t = 3.398, P = 0.0012). The mean faculty: annual resident intake ratio for institutes running DM programs was 2.09 (±1.45) and that for institutes running DNB programs was 3.76 (±2.27); this difference was significant on an independent samples t-test (t = −0.371, P = 0.002).
Six of the 39 government institutes (15.38%) and 16 of the 53 private institutions (30.02%) had not published any papers in the past 5 years (z = 1.6, P = 0.05). Assuming that the current intake of residents had remained stable over the past 5 years, 13% of all neurology residents have either graduated from or are currently pursuing residency in institutes that have not published any papers since 2010. The top 5 institutes with the highest overall number of publications (forming 5.5% of the 91 neurology teaching programs) had contributed 55.67% of all neurology papers and 69% of all original articles published over the past 5 years [Table 2]. The teaching institutes were ranked both on the basis of total publications over the past 5 years and the number of original papers published; National Institute of Mental Health and Neuro Sciences (NIMHANS) topped the ranking in both these rank lists [Table 2].
The ratio of the number of papers published per year by an institute to the faculty strength of the institute was computed. On this basis, the highest ranked institutes were PGIMER Chandigarh, SGPGI Lucknow, and AIIMS New Delhi. It should, however, be noted that this ratio does not exactly denote the publication/research output of an individual faculty member. This is because a single research project and thus, paper, may have involved more than one faculty member. However, in this analysis, each paper was only counted once. The annual publication output correlated with the number of faculty members for government institutes, but not for private institutions. The institute NPP was calculated by comparing the annual publication performance of each institute with the national average (vide supra). Based on this calculation, NIMHANS received an NPP score of 12.86 - implying that the publication performance of this institute was 1286% better than the national average.
The ranking of neurology training institutes based on the number of citations received by all papers published over the past 5 years is displayed in [Table 3]. Here AIIMS topped the list, followed by NIMHANS and SGPGI, Lucknow. The quality of original work was determined by assessing the cites per original paper for the institutes in the highest publishing quartile. In the 5 years studied, 902 original neurology papers had been published, receiving 6612 citations (median 2.2 cites/paper). On this basis, the top ranked institute was Jawaharlal Nehru Medical College, Belgaum (14.5 cites/paper, 8 papers), followed by NIMS, Hyderabad (64 papers, 14.47 cites/paper) [Table 3]. When ranked by the 5 year i10 index, NIMHANS topped this list, followed by AIIMS and SGPGI. When papers published over the past 10 years were considered and the 10 year h-index was computed, the highest ranked institute was AIIMS (h-index = 58), followed by SGPGI (h-index = 52).
Institutes running neurosurgery residency programs
The 100 neurosurgery training institutes had published 1639 papers over the past 5 years. The mean number of papers/institute/year was 3.28 ± 8.22; the median was 0.2. The mean publications/institute/year was significantly higher for MCh training institutes (4.16 ± 9.5) than for DNB training centers (1.22 ± 3.05) (t = 2.33, P = 0.021). The government institutes had a higher mean annual publication rate (5.28 ± 0.27) than the private institutions (1.43 ± 3.66) (t = 7.63, P < 0.0001). However, the faculty: resident intake ratio in private institutions (3.55 ± 2.38) was significantly higher than that in government institutes (2.08 ± 1.18) (t = −3.9, P < 0.001). Eighteen of 49 government institutes (36.7%) and 25 out of 52 private institutes (48.07%) had not published any papers in the past 5 years. Thus, approximately 30.9% of residents who have graduated in the past 5 years or are currently pursuing their residency are doing so in institutes that have not published any papers in the past 5 years. This is a much higher proportion than for neurology residency programs (z = 4.6, P < 0.001).
The 5 institutes with the highest number of publications in the past 5 years (forming 5% of 100 institutes) had contributed 53.3% of all papers published in the past 5 years and 70.1% of all original neurosurgical papers published by training institutes. This is an even greater degree of clustering than for neurology training programs [Table 4]. When ranked by the total number of publications over the past 5 years, the top 3 institutes were AIIMS, PGIMER (Chandigarh), and NIMHANS. When ranked by the number of original articles published over the past 5 years, the top 3 institutes were NIMHANS, AIIMS, and PGIMER (Chandigarh).
The mean annual number of publications per institute to faculty strength ratio in the neurosurgery group was 0.57 ± 1.12; the median was 0.067. This ratio does not reflect the actual productivity of each individual faculty member but attempts to depict the relationship between the faculty strength and output of an institute. The publications/year for each institute were normalized against the mean of the group as described. This data are presented in [Table 4]. AIIMS received an NPP score of 11.39 (i.e. 1139% more papers published than the average performance for India).
The ranking of neurosurgery training institutes based on the number of citations received by all papers published over the past 5 years is displayed in [Table 5]. NIMHANS, Seth GS Medical College, and AIIMS topped the list. The quality of original work was determined by assessing the cites per paper for the highest publishing quartile. On this basis, the top ranked institute was the Seth GS Medical College (10.11 cites/paper, 36 papers) [Table 5]. The 5 year i10 rank list was topped by NIMHANS (i10 = 22). On the basis of the 10 years h-index, the highest ranked institutes were AIIMS and SGPGI (h-index = 42 for both).
Publication of a paper is the final stage in the cycle of scientific research. Thus, the number of publications can act as a surrogate marker of the quantity of research being performed in an institute. The quality of research would be denoted by the number of citations that a paper receives. Citations received by a paper are dependent on how useful the paper is to other researchers in the field.  Publications and citation metrics have, therefore, routinely been used to evaluate and rank the performance of institutes. ,
Residency programs are aimed at helping the trainee acquire core competence across several domains. Indubitably, the most important of these would pertain to patient care and management. However, across medical specialties, and especially in the neurosciences, research and publication are imperatives for scientific progress. A recent study found that India ranked 21 st in the neuroscience research output globally.  Indian neuroscience papers were also less cited than those from other developing countries. Thus, there is a need to augment the quantity and quality of neuroscience research in India. This is the mandate of academic and teaching institutes. There is, thus, an imperative need to train neuroscience scholars and residents more rigorously in the domains of scientific research and writing.
Exposure to the process of writing and publishing a paper has several benefits for residency trainees. Publication of an original paper by a resident indicates exposure to the entire process of hypothesis formulation, data collection, statistical analysis, and publication. Even the publication of a case report or case series confers significant educational benefits. Publishing a clinical paper involves recognition of the rarity of a finding or the determination of a significant result, thus mandating a sound theoretical knowledge of the subject. Performing a literature search further augments the trainee's knowledge of the subject. The entire process thus creates and fosters the ability to effectively communicate technical data. Studies have shown that residents who have been exposed to scientific research and paper writing during their residency are more likely to opt for careers in academic institutes as well as more likely to do research on their own. , This would obviously lead to the evolution of clinician-scientists and have a salutary effect on the development of translational research in India.
It is interesting to note that although the faculty-to-annual resident intake ratio was statistically higher in the DNB programs than in DM Neurology and MCh Neurosurgery programs, the research and publication output were higher in the DM and MCh groups. Moreover, faculty strength correlated with publication output only in government institutes and not in the privately managed institutes. This situation needs to be pondered upon and measures to translate faculty numbers into adequate research and publication output need to be instituted.
Several separate evaluations are possible based on the data generated in this paper. The first is the total number of papers published over the past 5 years. This statistic would give a rough indicator of the quantity of clinical and research work being performed in that institute and by extension, reflect the exposure of residents to these domains. The second evaluation is the number of citations that all papers published from that institute have received over the past 5 years. This figure reflects the visibility and utility of the work done by the center to other institutes. Next, the ratio of papers published to faculty strength would indicate faculty productivity. However, several research projects and papers would have more than one faculty member involved - thus, this ratio merely denotes the relationship between faculty strength and output of an institute. The annual resident intake would obviously influence the publication and research output of an institute. However, the factors that determine resident intake in the government and private sectors vary and may not truly reflect the workload in that department. The faculty strength is more likely to reflect the workload of a department. A comparison of the annual output of each institute with the national average is rendered possible using the NPP [Table 2] and [Table 5]. The quality of original work is reflected by the cites/original paper. Standard metrics such as the 5 years i10 index and 10 years h-index reflect a combination of the quality and quantity of work emanating from an institute. The summary of the institute rankings on these bases is displayed in [Table 6].
Of significant concern is the fact that 13% of neurology residents and possibly 30.9% of neurosurgery residents have graduated from or are training in institutes that have published no papers in the past 5 years. The several benefits that accrue from such exposure during the period of residency remain unfulfilled for these residents. ,, Serious thought is required to determine the means to ensure that these residents acquire these skills. Most institutes conduct intra-mural training programs on scientific writing and statistical methods for residents. In order to cater to those residents training in institutes which do not conduct such training sessions, the Neurological Society of India could conduct periodic training programs on research methodology, statistics, critical appraisal of evidence, scientific writing, etc., These could also be held on the sidelines of all national conferences so that residents could attend the conference as well as the workshops. It could be made mandatory by the MCI and NBE for all neurology and neurosurgery residents to have published at least one paper before they are permitted to appear for their summative exam. Publication performance and research output could also be evaluated as mandatory components while granting recognition to institutes to run neurology and neurosurgery training programs. A mandatory long extra-mural posting for residents (lasting possibly 3-4 months) would offer the possibility for residents to work in institutes that have active research programs.
Limitations of the study
The major limitation of this study was that the data regarding the faculty strength was obtained from the websites of the individual institutes. This data were possibly incomplete and thus the publications: faculty ratios presented here are approximations at best. The other limitation was that owing to the rigorous nature of the search strategy, only the 5 years publication output could be analyzed. This analysis was based on the current faculty strength, resident intake, and recognition status. It was assumed that this had remained largely unchanged over the past 5 years. However, it was felt that it would be inappropriate to extrapolate this data to the past 10 years. However, the 5 years i10 index and 10 years h-index were calculated to reflect institutional performance over the past 5 and 10 years. These rank lists differ only slightly from each other; the top 10 institutes are identical [Table 6]. Another problem was the manner in which institutional affiliations were listed by authors in papers. Some institutes were listed in as many as 7 different ways - this would obviously lead to papers being missed when institutional performance metrics are calculated. It would be desirable, therefore, for each institute to impose a uniform style for listing the affiliation in all papers.
I would like to gratefully acknowledge all the logistic support rendered by Dr. Amrutha Bindu Nagella (Assistant Professor of Anesthesiology and Critical Care, Mahatma Gandhi Medical College and Research Institute, Pondicherry) and Mr. V Subeikshanan (4 th Semester MBBS student, JIPMER, Pondicherry).
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5], [Table 6]