Alarm criteria for motor evoked potentials
Correspondence Address: Source of Support: None, Conflict of Interest: None DOI: 10.4103/neuroindia.NI_1195_16
Source of Support: None, Conflict of Interest: None
Keywords: Class II statistical analysis, clinical neurophysiology, scoliosis, spinal cord
The utilization of transcranial electric motor evoked potential (TcMEP) monitoring during spinal fusion for scoliosis correction has been shown to reduce the incidence of paraplegia. Our meta-analysis showed that patients who have postoperative neurological deficits are 250 times more likely to experience perioperative TcMEP changes. Despite several studies reporting its high sensitivity and specificity,,, the clinical significance of this monitoring needs to be properly assessed in predicting iatrogenic injury in spinal surgery. The neurophysiological alarm criteria used to define a pathological TcMEP change is highly variable. The absence of approved guidelines for conducting the intraoperative neurophysiological monitoring (IONM) using TcMEPs and the absence of quantification of TcMEP waveforms has resulted in the inability to properly assess the diagnostic utility of intraoperative neurophysiological monitoring during spine surgery.
The commonly used criterion includes considering a decrease in the TcMEP amplitude of 50–80% to be significant.,, Additional alarm criteria include changes in the TcMEP morphology from polyphasic to biphasic to complete loss, the all-or-nothing criterion,, and the change in TcMEP stimulation threshold. The all-or-nothing approach, the complete loss of MEP signal indicating a clinically significant event, is the most widely used criterion in most spinal surgeries  despite studies suggesting that delaying until disappearance of the TcMEP waveform is often too late to allow for a timely intervention., In addition, TcMEP waveform variability and the choice of TcMEP alarm criteria may influence the rate of false positives observed in the operating room that may introduce hesitation in responding to the true positive TcMEP alerts.
Defining the TcMEP alarm criteria that predicts iatrogenic injury is critical as significant impairment can occur within minutes of a complete TcMEP loss. Iatrogenic injury can be reversible if interventional steps are immediately taken to address the issue in the operating room. Our primary aim was to evaluate the diagnostic accuracy of the three most commonly used alarm criterion for TcMEPs during spinal surgical procedures.
A systematic search of MEDLINE/PubMed, Web of Science, and EMBASE was conducted to identify eligible studies, published between 1945 and 2014, regarding the efficacy of TcMEP monitoring during spinal cord surgery. The following keywords were used to identify the relevant publications: spine, surgery, MEP, motor evoked potential, and spinal cord.
Studies were included based on the following inclusion criteria: (1) randomized controlled trial, a prospective study, or a retrospective cohort study, (2) conducted among patients undergoing surgery on the spine or spinal cord with TcMEP monitoring, (3) conducted among a group of ≥50 patients, (4) inclusive of postoperative neurological assessment within 24 h, and (5) published in English. Studies were not included if the procedures involved spinal tumor surgery. We defined a neurological deficit as any new motor deficit occurring within 24 h, to evaluate the value of the MEPs. A postoperative time frame of 24 h provides the most consistent and reliable period for these examinations.
All titles and abstracts were independently screened by authors (PDT, HLC, JH), against the inclusion criteria to identify the relevant studies. Studies that failed to meet the inclusion criteria were rejected. The reason for rejection was recorded on an Excel spreadsheet and indicated by the corresponding inclusion criteria (1–5). Disparities between the evaluators were resolved by discussion, and a final list of eligible publications was generated.
Data extraction and quality assessment
Data were extracted independently by the authors to ensure their consistency. The extracted information contained: first author's name, year of publication, study design, study data (total sample size, TcMEP changes, number of TcMEP changes that were classified according to different alarm criteria as either a complete loss of signal, amplitude reduction of 50%, amplitude reduction of 80%, or as reversible or irreversible changes to the TcMEP; and, as outcome data (reversible and irreversible neuromuscular deficits).
TcMEP changes corresponding to an amplitude reduction of 50% or more included patients with 80% or more amplitude loss. Amplitude reduction (50% or 80%) and total loss of signal were, therefore, considered mutually exclusive. A postoperative deficit was defined as any novel persistent neurological deficit (weakness or paraplegia) that was present at the 24-h postoperative examination. An irreversible TcMEP change was defined as any change that did not recover by the end of the procedure. A reversible TcMEP change was defined as any intraoperative significant change that resolved.
The number of true positive, false negative, false positive, and true negative outcomes were extracted and tabulated for each case: True positives (TP): Patients with TcMEP changes and with a new postoperative neurological deficit; false negatives (FN): Patients with no TcMEP changes and with a new postoperative neurological deficit; true negatives (TN): Patients with no TcMEP changes and no new postoperative neurological deficits; and, false positives (FP): Patients with TcMEP changes and without a new postoperative neurological deficit. The data was used to construct 2 × 2 tables, which were used to calculate the sensitivity and specificity of the study along with 95% confidence interval.
The Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 checklist for the quality assessment of the included study was used [Table 1]. The four domains assessed for the risk of bias by the QUADAS-2 tool were patient selection, index test, reference standard, and flow and timing. Patient selection, index test, and reference standard were also assessed for applicability. Signaling questions aided in assessing the potential risk of bias or applicability concerns. If the answers to all signaling questions in a domain were “yes,” then a “low” risk grade was given. If the answer to any signaling question was “no,” then a “high” risk grade was given. The “unclear” category was only used when the reported data was insufficient to permit a judgment.
Statistical analysis was performed with R (version 3.1.2) using the meta-analysis of diagnostic accuracy (MADA) package. Heterogeneity among the included studies was explored and quantified with the I2 quantity. For ease of interpretation, I2 values of 25%, 50%, and 75% were considered to roughly correspond to low, medium, and high heterogeneity, respectively. The diagnostic accuracies of the following alarm criteria were estimated individually: (1) all TcMEP changes (amplitude loss >50% including total signal loss); (2) >50% amplitude loss but not total signal loss; (3) >80% amplitude loss but not total signal loss; (4) total signal loss; and, (5) an irreversible signal loss. All TcMEP changes were classified based on review of the results from individual publications. The diagnostic odds ratio (OR) for each of these alarm criteria were estimated with a univariate random effects model. The summary sensitivity and specificity estimates in each case were obtained using a bivariate model. A linear mixed model was fitted to the logit transformed sensitivities and false positive rates, and summary estimates were obtained. Summary receiver operating characteristic (SROC) curves and forest plots of the log diagnostic OR, as well as the sensitivity and specificity data were constructed. The differences in sensitivities and false positive rates between the alarm criteria were examined for statistical significance using the chi-square test. This test was performed on unpaired data obtained from studies with complete information pertaining to the TcMEP changes. A funnel plot was constructed to evaluate the publication bias.
Literature search and study characteristics
The PRISMA flow chart [Figure 1]shows the literature search process. The initial database search identified 2042 articles, whose titles and abstracts were screened for their relevance in the assessment of TcMEP alarm criteria in patients undergoing spine and spinal cord surgery. After screening and removal of duplicates, a total of 135 articles were included for the full text review, of which 25 met the inclusion criteria [Table 1].
The analysis included data from 23 retrospective studies, 1 cohort study, and 1 prospective study. The included studies were published between 1996 and 2014. The procedures included scoliosis fusion, as well as cervical and lumbar spine procedures. Baseline TcMEPs were recorded either before or after induction of anesthesia. The various alarm criteria used by the included studies are listed in [Table 2]. Ten studies used a decrease in amplitude of 50% or greater as the alarm criteria, 8 studies used a decrease in amplitude of 80% or greater as the alarm criteria, 2 studies used a decrease in amplitude of 65%, 1 study used a decrease in amplitude of 50–80%, 1 study used a decrease in amplitude of 60%, 1 study used a decrease in amplitude of 75%, 1 study used a decrease in amplitude of 90%, and 1 study used complete loss of signal as the alarm criteria.
Data collection results and statistical analysis
[Table 3] and [Table 4] list the incidence of TcMEP alerts and neurological deficits corresponding to the three alarm criteria analyzed in our study –50% or greater amplitude loss, 80% or greater amplitude loss, and total signal loss. The 25 studies included in the quantitative analysis had a total cohort size of 9409 patients, 9081 (96.5%) of which had recordable MEPs. The overall incidence of significant alerts was 6.38% (579/9081). Of the 579 significant alerts, 303 were instances wherein there was an 80% or greater decrease in waveform amplitude, and 91 were instances wherein the signal had completely disappeared. Reversibility of TcMEP change, when data pertaining to such could be obtained, was achieved in 62.1% (197/317) of the patients who experienced a significant TcMEP change.
The overall incidence of neurological deficits was 1.82% (165/9081). TcMEP change preceded the occurrence of neurological deficits in 145 of the 165 cases. Of these 145 cases, 28 occurred with total loss of signal; 64 occurred with 80% or greater loss of amplitude. Overall, 117 of the total neurological deficits were associated with an amplitude reduction of 50% or greater. Data regarding reversibility of TcMEP change and the incidence of neurological deficits was obtained for 88 cases. TcMEP change preceding the neurological deficits was reversible in 23 instances and irreversible in 65 instances.
The following number of studies were used to determine summary measures of diagnostic accuracy for five categories:25 studies were used to determine summary measures for all TcMEP changes, 24 studies for ≥50% amplitude loss excluding total signal loss, 11 studies for ≥80% amplitude loss excluding total signal loss, 24 studies for total signal loss, and 16 studies with complete data regarding reversibility for irreversible signal loss [Table 5]. Unpaired data from studies with complete data were used for the chi-square test [Table 6].
The sensitivity and specificity of all TcMEP changes was 82.1% (95% confidence interval (CI): 73–88.6%; P< 0.001) and 95.7% (95% CI: 93.7–97.1%, P< 0.001), respectively. The pooled diagnostic odds ratio (DOR) was 139.730 (95% CI: 73.579–265.354), with positive likelihood ratio (PLR) 19.093 and negative likelihood ratio (NLR) 0.187. Heterogeneity testing yielded I2 values of 1.836 P = 0.4362 for sensitivity and 91.2 and P< 0.001 for specificity.
Pooled estimates for the sensitivity and specificity of ≥50% amplitude loss was 63.2% (95% CI: 47–76.8%; P = 0.108) and 96.7% (95% CI: 95.1–97.8%; P< 0.001), respectively. DOR was 61.027 (95% CI: 27.983 – 133.092), along with PLR 19.1515 and NLR 0.3806. The I2 value for sensitivity was 49.64 (P = 0.0033) and the value for specificity was 90.34 (P < 0.0001).
Random effects summary estimate of sensitivity of ≥80% amplitude loss was 71.7% (95% CI: 42–89.9%; P = 0.146) and the summary estimate of specificity was 98.3% (95% CI: 96.4–99.2%; P< 0.0001). The DOR was 194.021 (95% CI: 55.688–675.987), along with PLR 42.1765 and NLR 0.2879. The I2 value was 63.26, P = 0.0024, for sensitivity and 93.89, P< 0.001, for specificity.
The sensitivity and specificity of total signal loss was 30% (95% CI: 17.6–46.4%; P = 0.018) and 99.3% (95% CI: 98.6–99.7%; P< 0.001), respecitively. The DOR was 72.891 (95% CI: 39.262–135.325). The PLR and NLR were 42.8571 and 0.7049, respectively. Heterogeneity I2 value was 37.47, P = 0.0342 for sensitivity and 88.73, P< 0.0001.
The pooled summary measures for irreversible TcMEP change were: sensitivity 60% (95% CI: 46.9–71.9%; P = 0.134), specificity 98.5% (95% CI: 97.7–99.1%; P< 0.001), DOR 147.896 (95% CI: 61.799–353.939), PLR 40, NLR 0.4061, and I2 values of 25.32, P = 0.1687, and 57.11, P = 0.0025, for sensitivity and specificity, respectively.
Deciding on the TcMEP alarm criterion that best predicts an iatrogenic injury permits immediate intervention that may reduce the incidence of neurological deficits. Defining an appropriate alarm criterion can be difficult because of the challenge of clearly differentiating between a minor degree of TcMEP deterioration and its complete loss.,,, Additional concerns exist as this modality is rarely recorded at regular times or with high frequency, adding to the difficulty of differentiating between gradual amplitude decrements and abrupt decrements.
Our results affirm that overall TcMEP change has a strong sensitivity (82.1%) and specificity (95.7%), with a diagnostic OR of 139.730, indicating that patients with new postoperative neurological deficits after spine surgery are 139.730 times more likely to experience a significant TcMEP change than those without new postoperative deficits. The overall incidence of TcMEP change in our cohort was 6.38%, and the overall incidence of neurological deficit was 1.82%.
The sensitivities and specificies of the assessed alarm criteria, ≥50% reduction in amplitude, ≥80% reduction in amplitude, and total signal loss were 63.2% and 96.7%, 71.7% and 98.3%, and 30% and 99.3%, respectively. In order of highest-to-lowest sensitivity, ≥80% reduction in amplitude was the most sensitive, followed by ≥50% reduction in amplitude, and total signal loss. In order of highest-to-lowest specificity, total signal loss was the most specific, followed by ≥80% reduction in amplitude, and finally, ≥50% reduction in amplitude. The DOR was highest for ≥80% reduction in amplitude (194.021). These summary measures alone, as well as the high PLR (42.1765), suggest that using ≥80% reduction in amplitude as a significant change would lead to fewer false positives. However, the significantly wide 95% CI for the DOR of ≥80% reduction in amplitude indicates that the given value is an imprecise estimate of the population parameter. This decreases the reliability of the DOR, and as such, it is hard to state with confidence that an alarm criteria using ≥80% reduction in amplitude would lead to noticeably improved outcomes. The significance of irreversible signal loss was briefly explored; however, it did not yield particularly substantial results, as it had the second widest 95% CI for DOR.
The chi-square test was performed to determine the statistical significance of differences between the summary measures of sensitivity and specificity. The test indicated that there was a statistically significant difference between the specificity of overall TcMEP change compared to ≥80% reduction in amplitude (95.7% vs. 98.3%). The implication of this result is lessened by the fact that there was also a statistically significant difference between the specificity of overall TcMEP change and ≥50% reduction of amplitude, and that there was no statistically significant difference in the sensitivity and specificity between ≥80% reduction in ampitude and ≥50% reduction in amplitude, despite the slight differences present between the obtained values.
We used bivariate and univariate effects to assess sensitivity, specificity, and DOR to determine the most appropriate alarm criteria for TcMEP monitoring in spine and spinal surgery. The chi-square test also confirms that the differences between summary measures for all TcMEP changes were not statistically significant. The possible explanations may be that there were only 5 studies with complete data pertaining to ≥80% reduction in amplitude that could be used for the chi-square test. We recommend that further studies be carried out to determine the best alarm criteria.
It is important to note that our study was subject to limitations, and that while efforts were made to identify all relevant published data, some search bias may exist. Significant heterogeneity was observed in the sensitivity and specificity of the studies. No clear quantification of TcMEP response change in latency or amplitude was performed. There is no protocol for how to obtain real-time continuous TcMEP changes during the procedure. Postoperative neurophysiological examinations have not been systematically documented. Causes of heterogeneity were explored in the analyses; however, because of the nature of the meta-analysis, we were limited by the available data published by the individual studies. It is plausible that some of the heterogeneity can be attributed to the reversibility of TcMEP waveforms, which is desirable but not always achieved.
There was a statistically significant difference in the sensitivity and specificy between total signal loss and all other alarm criteria, including total TcMEP change. This is unsurprising as total signal loss is commonly the first spinal cord warning sign, and prolonged disappearance of TcMEP waveform is a strong predictor of a new neurological deficit. Patients with a new neurological deficit after spinal surgery are 139.730 times more likely to have had a significant TcMEP change than those without postoperative deficits; thus, TcMEP changes are a strong indicator of impending spinal injury .
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5], [Table 6]