Feasibility of Conducting a Virtual Exit Exam in Neurosurgery During the SARS-COV19 Pandemic
Correspondence Address: Source of Support: None, Conflict of Interest: None DOI: 10.4103/0028-3886.319207
Source of Support: None, Conflict of Interest: None
Keywords: SARS-COV19 pandemic, exit exam, neurosurgeryKey Message: Traditional neurosurgical exit exams are not feasible during this ongoing COVID 19 pandemic. Online meeting platforms with good audio-visual facility along with case scenarios, operative case modules, and radiology in power point format can be used for conducting such assessments.
The neurosurgery residency program is designed to impart clinical and surgical skills to trainees enabling them to manage patients safely. Traditionally, the residents are periodically evaluated for both clinical and surgical skills., This “internal” evaluation plays a vital role in ensuring that residents have the confidence to practice independently. At the end of the training, an exit examination allows for external experts, along with internal examiners to evaluate the trainees, thus avoiding possible bias and prejudice as well as ensuring a fairly standard level of neurosurgical competence across the country. However, the current COVID pandemic has put limitations on such an evaluation methodology where direct patient interview, clinical examination, and surgical skills assessment in the operation theatre are not possible. Here we discuss the preparations for and the conduction of an online clinical exit examination conducted at our institute during the lockdown period of the SARS-COV19 pandemic for six candidates who completed their 3-year neurosurgery training after 3 years of general surgery. Based on this experience, we propose an objective skills assessment module, using the online platform.
Six residents completed their neurosurgery training and appeared for the exit exam in June 2020. The first part of assessment included written examinations covering neuroanatomy, pathophysiology; operative management and recent advances The second part comprised of clinical and surgical assessments also known as the “practical examination.” Due to COVID pandemic, actual physical examination of patients by candidates was not carried out. However, all components of the assessment as stipulated by the Indian Medical Council were included.
Infrastructure at the examination center
Using an online meeting platform, the seminar room was prepared for clinical evaluation of the trainees. However, only the candidates and the internal examiners were allowed inside the examination room.
A laptop with an internet connection was used for the tele-videoconferencing using “ZOOM” (Zoom Video Communications, Inc. USA). The trainee sat in front of the laptop after the allotted case module was opened and a mouse was provided to navigate through the case module as needed. The projection onto the larger screen further provided the finer details of clinical images and radiology. The candidate used the laptop's in-built microphone. They were instructed regarding this new pattern, a day before the evaluation. We had additional laptops with uploaded case modules, electricity generators, and an alternate internet facility as back up.
Two neurosurgeons from two separate institutes elsewhere in the country along with two from our own institute conducted the assessment. The external examiners were provided with details of how to log in and to access the online platform and the pattern of the examination one week prior. They were requested to keep themselves free from 9 am to 5 pm for these two days and to use a device with audio-visual facilities for this purpose. Additionally, the examiners were asked to prepare interesting radiological images that could be shared at the time of assessment. Three candidates were evaluated on each day.
The case descriptions were prepared by faculty members based on the actual clinical history and examination findings of patients. The relevant examination findings, recorded in the form of clinical images (jpeg) and videos (mp4), were digitized and stored with the clinical details in a power point presentation as case modules. The long clinical cases included detailed clinical descriptions with videos demonstrating clinical signs. The short clinical cases were prepared with brief findings and relevant videos/images [Figure 1] and [Figure 2]. Radiology was added to the operative case modules. One long clinical case, two short clinical cases and one operative case were created for each candidate. Each folder had at least one cranial and one spinal case scenario. The aim was to evaluate the residents comprehensively for their clinical knowledge, decision making, and technical efficiency.
The external examiners were blinded to the clinical presentations and radiology so as to minimize the bias.
Residents were evaluated on the following parameters:
Apart from this, images of pathological specimens and radiology were digitized and prepared for discussion.
The examiners were instructed that 40 min were allotted for examining the candidates for the long case, 20 min each for short cases, 30 mins for operative, and 30 mins for radiology sections of the examination. Three candidates were evaluated per day in four sessions. Following the completion of assessment, feedback was obtained from all the examinees as well as the examiners. The feedback from examinee was based on a questionnaire [Table 1].
The audio-visual system arrangement in the examination hall was satisfactory and there were no technical interruptions during these two days. However, on one occasion the candidates had to be prompted to speak loudly into the microphone close to the laptop. As the larger screen was placed up on the wall, the candidates had to look up from the laptop and had some difficulty in manipulating the cursor with the mouse [Figure 3].
With regard to the external examiners (BKB) used a cell phone for the entire clinical interview that had very good audio and video quality but he had to switch to the desktop to show his radiology slides resulting in a few minutes disruption. The other examiner (AGC) had poor audio that was rectified to some extent after use of a headphone.
The candidates presented the history that was shared with the examiners so a continuous interaction was possible. The images and clinical videos helped to maintain the flow of interaction and broaden the scope of the discussion. The radiological images enabled detailed assessment of normal and abnormal anatomy., When the individual scores of trainees were collected from all the experts, there was no remarkable difference in the grades awarded by each examiner.
Feedback from the examinees was analyzed after the assessment. All residents were able to interpret the clinical signs from the patients' image or examination video shown in the case modules. Four residents were of the opinion that the clinical data provided was adequate to establish a localization of the lesion. However, two candidates were unable to correlate the clinical data accurately and on further questioning they felt that a direct interview and physical examination of the patient would have helped in establishing the clinical diagnosis better. Four residents rated this method of evaluation nearly equivalent to direct one on one interaction with the examiners. However, two were disappointed due to the lag effect inherent to the online audiovisual system. Overall, the online platform was rated 4 out of 5 by the candidates and all recommended this method of evaluation can be considered in future especially in an unprecedented situation like the current one.
Most neurosurgical training programs impart a core set of clinical knowledge and surgical skills in a time bound manner. Trainees are periodically appraised for the clinical knowledge and operative skills by faculty members of the training institute to enable progression to the next level in the program.,, The Medical Council of India mandates an exit examination at the end of the postgraduate training program where each trainee is comprehensively evaluated with theory and practical tests in the presence of both internal and external experts from other institutes.
At the time of exit examination, 2-4 cases of the hospital admissions, tested positive for COVID each day. Due to COVID restrictions postponing the exit exam was discussed considering the safety issues. However, the time line for resolution of the COVID pandemic and return to normal was uncertain. It was therefore decided to conduct an online examination. Given the uncertainty this might be the way examinations are to be conducted in future if the pandemic continues.
Setting up of an Online platform system
There are several platforms like Google meeting, Skype, Webex, Microsoft teams, and Zoom commercially available for use. Good audio-visual systems with internet facility are essential for the use of an online platform system. Certain platforms have the advantage of simultaneous access to the same screen, both for the examinee and the examiner so as to make the discussion more interactive. For smooth running of the online platform both the examinee and the examiners must be well informed in advance about the use of audio-visual system. Through this online platform examiners from distant part of the country able to interact with the examinee and continuous discussion can be carried out similar to direct objective evaluation.
A major issue with the online platform is “lag” in audiovisual system which can affect the flow of discussion. Prior knowledge of such “lag” minimizes the communication gap during the assessment.
The candidates should be appropriately dressed and should maintain the usual decorum. They should let the examiners finish with comments or question before speaking into the microphone.
History and examination
Use of original patient data (examination findings and clinical images) makes the case module more authentic and realistic for discussion and avoided classical text book case descriptions. The interpretation of clinical history require a particular level of understanding of the subject. All clinical, laboratory, and radiological data in a case module were presented in a sequential manner similar to what the residents would follow in their routine practice. The external experts too were blinded to the case scenarios so as to make the assessment more interesting and unbiased. Videos depicting a particular sign like pendular knee jerk, ankle clonus, spastic gait, Romberg sign, etc., were incorporated to add objectivity to the evaluation process. The videos of nystagmus, internuclear ophthalmoplegia, lobar signs, fasciculations, cerebellar signs, etc., can be incorporated in addition to pictures of fundus, muscle wasting to name a few. The chronological appearance of all these clinical details further helps the trainee to draw a mental picture of the patient and can correlate all the findings to reach up to a diagnosis. Four out of six residents successfully picked up the clinical clues depicted in the case modules and were able to make definite diagnosis.
Advantages of traditional methods – History taking is an art and this requires interacting with the patient. Similarly, listing the chronology of symptoms requires some sort of deliberation. Observing the candidate while directly examining and eliciting clinical signs has the advantage of evaluating his applied clinical knowledge which can be cross verified. Simultaneously the trainee can read the body language of the examiners and can judge whether their analysis is correct and can avoid unnecessary mistakes.
Testing all the patients, examinees, and internal examiners before the assessment may provide an opportunity for direct clinical evaluation which can be telecasted to the examiners by an overhead camera system. Needless to say that personal protective equipment is essential for conducting such evaluation. As suggested by one of the external experts, an “actor” can be included to assess the clinical acumen. The actor here could be one of the fellow residents or a faculty member.
Through the online platform the candidate and the examiner can view the radiology and imaging of specimen. This will allow a continuous interaction like that of the direct interview. The online platform with the sharing of the screens also enables testing the candidate's ability to identify verbally specific anatomic landmarks pointed out by the examiner or himself pointing out to landmarks as asked by the examiner.
Operating skills acquisition is the key step in neurosurgery training. Several tools have been devised and validated to evaluate them objectively.,,, Many centers of India and almost all in abroad have done away with surgical skills assessment in the presence of external experts due to high anxiety level of the candidates, unavailability of suitable cases, and ethical justifications.
Most centers have adopted operative skill assessment in the operating room where the trainee performs surgical procedures directly on patients. A real time scoring and feedback is considered as the best way of assessing residents for technical skills like instrument handling, tissue respect, dexterity, flow of surgery, and knowledge of a specific procedure. Here we have formulated case scenarios for tumor localization and provided the radiology with a skull bone for discussion of relevant surgical planning and peri-operative issues. For better objectivity, operative video scan be used where residents can be evaluated for knowledge of surgical anatomy, operative steps, intra-operative complications, and their management. Alternatively, video recordings of the surgeries performed by the trainees can be used for discussion. Then examiners have the opportunity to assess the operative skills of the trainee by observing that particular surgery step by step similar to the “procedure based assessment” in the operating theater., Neurosurgical centers with skills laboratory can also use operative modules (cadaveric/non-cadaveric) for such technical skills evaluation. Assessment directly in the skills laboratory provides a similar atmosphere like that of the operating theatre where the trainee can be evaluated for knowledge of instruments and operative procedures. This appears to be more practical and also feasible with the use of online platforms.
Online audiovisual system has some lag effect and may disturb the flow of discussion. Clinical case modules directly demonstrate some signs which cannot be cross verified either by the examinee or by the examiner. This methodology deprives the candidates from the advantage of reading the examiner's body language that gives important clue for avoiding mistakes.
Finally, one tends to draw an analogy from the fact that evaluation of an aviation pilot cannot be made on a simulator alone. It is difficult to assess a surgeon's skills based on these modules alone.
Conducting virtual assessment for neurosurgery residents using online platforms with an aim of objective evaluation of clinical and surgical skills is challenging. Case modules incorporated with actual patient data, clinical imaging, videos demonstrating clinical signs, actors, operative videos, and evaluation in the skills laboratory makes it more practical and realistic in such unprecedented times.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
[Figure 1], [Figure 2], [Figure 3]