Prospective validation study of an epilepsy seizure risk system for outpatient evaluation

被引:21
作者
Chiang, Sharon [1 ,2 ]
Goldenholz, Daniel M. [3 ]
Moss, Robert [4 ]
Rao, Vikram R. [1 ,2 ]
Haneef, Zulfi [5 ,6 ]
Theodore, William H. [7 ]
Kleen, Jonathan K. [1 ,2 ]
Gavvala, Jay [5 ]
Vannucci, Marina [8 ]
Stern, John M. [9 ]
机构
[1] Univ Calif San Francisco, Dept Neurol, 505 Parnassus Ave, San Francisco, CA 94143 USA
[2] Univ Calif San Francisco, Weill Inst Neurosci, 505 Parnassus Ave, San Francisco, CA 94143 USA
[3] Beth Israel Deaconess Med Ctr, Dept Neurol, Boston, MA 02215 USA
[4] Seizure Tracker TM LLC, Annandale, VA USA
[5] Baylor Coll Med, Dept Neurol, Houston, TX 77030 USA
[6] VA Med Ctr, Neurol Care Line, Houston, TX USA
[7] NINDS, Clin Epilepsy Sect, NIH, Bldg 36,Rm 4D04, Bethesda, MD 20892 USA
[8] Rice Univ, Dept Stat, Houston, TX 77251 USA
[9] Univ Calif Los Angeles, David Geffen Sch Med, Dept Neurol, Los Angeles, CA 90095 USA
关键词
clinical decision support system; interrater reliability; intrarater reliability; seizure risk; BIG DATA; NEUROLOGY; RELIABILITY; DIAGNOSES;
D O I
10.1111/epi.16397
中图分类号
R74 [神经病学与精神病学];
学科分类号
摘要
Objective We conducted clinical testing of an automated Bayesian machine learning algorithm (Epilepsy Seizure Assessment Tool [EpiSAT]) for outpatient seizure risk assessment using seizure counting data, and validated performance against specialized epilepsy clinician experts. Methods We conducted a prospective longitudinal study of EpiSAT performance against 24 specialized clinician experts at three tertiary referral epilepsy centers in the United States. Accuracy, interrater reliability, and intra-rater reliability of EpiSAT for correctly identifying changes in seizure risk (improvements, worsening, or no change) were evaluated using 120 seizures from four synthetic seizure diaries (seizure risk known) and 120 seizures from four real seizure diaries (seizure risk unknown). The proportion of observed agreement between EpiSAT and clinicians was evaluated to assess compatibility of EpiSAT with clinical decision patterns by epilepsy experts. Results EpiSAT exhibited substantial observed agreement (75.4%) with clinicians for assessing seizure risk. The mean accuracy of epilepsy providers for correctly assessing seizure risk was 74.7%. EpiSAT accurately identified seizure risk in 87.5% of seizure diary entries, corresponding to a significant improvement of 17.4% (P = .002). Clinicians exhibited low-to-moderate interrater reliability for seizure risk assessment (Krippendorff's alpha = 0.46) with good intrarater reliability across a 4- to 12-week evaluation period (Scott's pi = 0.89). Significance These results validate the ability of EpiSAT to yield objective clinical recommendations on seizure risk which follow decision patterns similar to those from specialized epilepsy providers, but with improved accuracy and reproducibility. This algorithm may serve as a useful clinical decision support system for quantitative analysis of clinical seizure frequency in clinical epilepsy practice.
引用
收藏
页码:29 / 38
页数:10
相关论文
共 19 条
[1]  
Baud M, FORECASTING SEIZURES
[2]   Multi-day rhythms modulate seizure risk in epilepsy [J].
Baud, Maxime O. ;
Kleen, Jonathan K. ;
Mirro, Emily A. ;
Andrechak, Jason C. ;
King-Stephens, David ;
Chang, Edward F. ;
Rao, Vikram R. .
NATURE COMMUNICATIONS, 2018, 9
[3]   CONTROLLING THE FALSE DISCOVERY RATE - A PRACTICAL AND POWERFUL APPROACH TO MULTIPLE TESTING [J].
BENJAMINI, Y ;
HOCHBERG, Y .
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY, 1995, 57 (01) :289-300
[4]  
Bone I, 2001, J NEUROL NEUROSUR PS, V70, P1
[5]  
Chiang Sharon, 2018, Epilepsia Open, V3, P236, DOI 10.1002/epi4.12112
[6]   Does my posterior look big in this?-Bayesian solutions to seizure counting problems [J].
Cook, Blake ;
Cook, Mark .
EPILEPSIA OPEN, 2019, 4 (02) :235-236
[7]   Seizure Prediction: Science Fiction or Soon to Become Reality? [J].
Freestone, Dean R. ;
Karoly, Philippa J. ;
Peterson, Andre D. H. ;
Kuhlmann, Levin ;
Lai, Alan ;
Goodarzy, Farhad ;
Cook, Mark J. .
CURRENT NEUROLOGY AND NEUROSCIENCE REPORTS, 2015, 15 (11)
[8]   Is seizure frequency variance a predictable quantity? [J].
Goldenholz, Daniel M. ;
Goldenholz, Shira R. ;
Moss, Robert ;
French, Jacqueline ;
Lowenstein, Daniel ;
Kuzniecky, Ruben ;
Haut, Sheryl ;
Cristofaro, Sabrina ;
Detyniecki, Kamil ;
Hixson, John ;
Karoly, Philippa ;
Cook, Mark ;
Strashny, Alex ;
Theodore, William H. .
ANNALS OF CLINICAL AND TRANSLATIONAL NEUROLOGY, 2018, 5 (02) :201-207
[9]   A multi-dataset time-reversal approach to clinical trial placebo response and the relationship to natural variability in epilepsy [J].
Goldenholz, Daniel M. ;
Strashny, Alex ;
Cook, Mark ;
Moss, Robert ;
Theodore, William H. .
SEIZURE-EUROPEAN JOURNAL OF EPILEPSY, 2017, 53 :31-36
[10]   Confusing placebo effect with natural history in epilepsy: A big data approach [J].
Goldenholz, Daniel M. ;
Moss, Robert ;
Scott, Jonathan ;
Auh, Sungyoung ;
Theodore, William H. .
ANNALS OF NEUROLOGY, 2015, 78 (03) :329-336