Assessment of a Digital Symptom Checker Tool's Accuracy in Suggesting Reproductive Health Conditions: Clinical Vignettes Study

被引:3
作者
Peven, Kimberly [1 ]
Wickham, Aidan P. [1 ,5 ]
Wilks, Octavia [1 ]
Kaplan, Yusuf C. [1 ]
Marhol, Andrei [1 ]
Ahmed, Saddif [1 ]
Bamford, Ryan [1 ]
Cunningham, Adam C. [1 ]
Prentice, Carley [1 ]
Meczner, Andras [2 ]
Fenech, Matthew [3 ]
Gilbert, Stephen [4 ]
Klepchukova, Anna [1 ]
Ponzo, Sonia [1 ]
Zhaunova, Liudmila [1 ]
机构
[1] Flo Hlth UK Ltd, London, England
[2] Your MD Ltd, London, England
[3] Hlth GmbH, Hamburg, Germany
[4] TUD Dresden Univ Technol, Else Kroner Fresenius Ctr Digital Hlth, Dresden, Germany
[5] Flo Hlth UK Ltd, 27 Old Gloucester St, London WC1N 3AX, England
来源
JMIR MHEALTH AND UHEALTH | 2023年 / 11卷
关键词
women's health; symptom checkers; symptom checker; digital health; chatbot; accuracy; eHealth apps; mobile phone; mobile health; mHealth; mobile health app; polycystic ovary syndrome; gynecology; digital health tool; endometriosis; uterus; uterine; uterine fibroids; vignettes; clinical vignettes; POLYCYSTIC-OVARY-SYNDROME; QUALITY-OF-LIFE; UTERINE FIBROIDS; DIAGNOSTIC DELAY; ENDOMETRIOSIS; PREVALENCE; WOMEN; BURDEN; FEATURES; IMPACT;
D O I
10.2196/46718
中图分类号
R19 [保健组织与事业(卫生事业管理)];
学科分类号
摘要
Background: Reproductive health conditions such as endometriosis, uterine fibroids, and polycystic ovary syndrome (PCOS) affect a large proportion of women and people who menstruate worldwide. Prevalence estimates for these conditions range from 5% to 40% of women of reproductive age. Long diagnostic delays, up to 12 years, are common and contribute to health complications and increased health care costs. Symptom checker apps provide users with information and tools to better understand their symptoms and thus have the potential to reduce the time to diagnosis for reproductive health conditions. Objective: This study aimed to evaluate the agreement between clinicians and 3 symptom checkers (developed by Flo Health UK Limited) in assessing symptoms of endometriosis, uterine fibroids, and PCOS using vignettes. We also aimed to present a robust example of vignette case creation, review, and classification in the context of predeployment testing and validation of Methods: Independent general practitioners were recruited to create clinical case vignettes of simulated users for the purpose of testing each condition symptom checker; vignettes created for each condition contained a mixture of condition-positive and condition-negative outcomes. A second panel of general practitioners then reviewed, approved, and modified (if necessary) each vignette. A third group of general practitioners reviewed each vignette case and designated a final classification. Vignettes were then entered into the symptom checkers by a fourth, different group of general practitioners. The outcomes of each symptom checker were then compared with the final classification of each vignette to produce accuracy metrics including percent agreement, sensitivity, specificity, positive predictive value, and negative predictive value. Results: A total of 24 cases were created per condition. Overall, exact matches between the vignette general practitioner classification and the symptom checker outcome were 83% (n=20) for endometriosis, 83% (n=20) for uterine fibroids, and 88% (n=21) for PCOS. For each symptom checker, sensitivity was reported as 81.8% for endometriosis, 84.6% for uterine fibroids, and 100% for PCOS; specificity was reported as 84.6% for endometriosis, 81.8% for uterine fibroids, and 75% for PCOS; positive predictive value was reported as 81.8% for endometriosis, 84.6% for uterine fibroids, 80% for PCOS; and negative predictive value was reported as 84.6% for endometriosis, 81.8% for uterine fibroids, and 100% for PCOS. Conclusions: The single-condition symptom checkers have high levels of agreement with general practitioner classification for endometriosis, uterine fibroids, and PCOS. Given long delays in diagnosis for many reproductive health conditions, which lead to increased medical costs and potential health complications for individuals and health care providers, innovative health apps and symptom checkers hold the potential to improve care pathways.
引用
收藏
页数:13
相关论文
共 79 条
  • [1] Uterine Fibroids: Burden and Unmet Medical Need
    Al-Hendy, Ayman
    Myers, Evan Robert
    Stewart, Elizabeth
    [J]. SEMINARS IN REPRODUCTIVE MEDICINE, 2017, 35 (06) : 473 - 480
  • [2] Chronic Fatigue, Physical Impairments and Quality of Life in Women with Endometriosis: A Case-Control Study
    Alvarez-Salvago, Francisco
    Lara-Ramos, Ana
    Cantarero-Villanueva, Irene
    Mazheika, Maryna
    Mundo-Lopez, Antonio
    Galiano-Castillo, Noelia
    Fernandez-Lao, Carolina
    Arroyo-Morales, Manuel
    Ocon-Hernandez, Olga
    Artacho-Cordon, Francisco
    [J]. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH, 2020, 17 (10)
  • [3] [Anonymous], 2022, ERICSSON MOBILITY RE
  • [4] [Anonymous], The world's most popular female health app
  • [5] [Anonymous], 2021, Digital health trends 2021: innovation, evidence, regulation, and adoption
  • [6] The prevalence and features of the polycystic ovary syndrome in an unselected population
    Azziz, R
    Woods, KS
    Reyna, R
    Key, TJ
    Knochenhauer, ES
    Yildiz, BO
    [J]. JOURNAL OF CLINICAL ENDOCRINOLOGY & METABOLISM, 2004, 89 (06) : 2745 - 2749
  • [7] Azziz R, 2007, Androgen Excess Disorders in Women Polycystic Ovary Syndrome and Other Disorders, V2, P353, DOI DOI 10.1007/978-1-59745-179-6_32
  • [8] Polycystic ovary syndrome
    Azziz, Ricardo
    Carmina, Enrico
    Chen, ZiJiang
    Dunaif, Andrea
    Laven, Joop S. E.
    Legro, Richard S.
    Lizneva, Daria
    Natterson-Horowtiz, Barbara
    Teede, Helena J.
    Yildiz, Bulent O.
    [J]. NATURE REVIEWS DISEASE PRIMERS, 2016, 2
  • [9] Impact of endometriosis on women's health: comparative historical data show that the earlier the onset, the more severe the disease
    Ballweg, ML
    [J]. BEST PRACTICE & RESEARCH CLINICAL OBSTETRICS & GYNAECOLOGY, 2004, 18 (02) : 201 - 218
  • [10] Quality of life and psychological well being in polycystic ovary syndrome
    Barnard, L.
    Ferriday, D.
    Guenther, N.
    Strauss, B.
    Balen, A. H.
    Dye, L.
    [J]. HUMAN REPRODUCTION, 2007, 22 (08) : 2279 - 2286