Click here to load reader

The Effectiveness of Symptom Checkers for Self-Diagnosis ... · PDF file checkers on appropriate triage advice ranged from 33% (95% CI 19 to 48) to 78% (95% CI 64 to 91) of SP evaluations

  • View
    2

  • Download
    0

Embed Size (px)

Text of The Effectiveness of Symptom Checkers for Self-Diagnosis ... · PDF file checkers on...

  • Confidential: For Review O

    nly

    The Effectiveness of Symptom Checkers for Self-Diagnosis

    and Triage: Beyond “Googling” Symptoms

    Journal: BMJ

    Manuscript ID: BMJ.2015.025489.R1

    Article Type: Research

    BMJ Journal: BMJ

    Date Submitted by the Author: 14-May-2015

    Complete List of Authors: Semigran, Hannah; Harvard Medical School, Health Care Policy Linder, Jeffrey; Brigham and Women's Hospital, Medicine; Harvard Medical School, Gidengil, Courtney; RAND Corporation, ; Boston Children's Hospital, Infectious Diseases

    Mehrotra, Ateev; Harvard Medical School, Health Care Policy; Beth Israel Deaconess Medical Center, General Internal Medicine and Primary Care

    Keywords: Symptom checkers, Internet, Antibiotic prescribing, mHealth

    https://mc.manuscriptcentral.com/bmj

    BMJ

  • Confidential: For Review O

    nly

    1

    The Effectiveness of Symptom Checkers for Self-Diagnosis and Triage:

    An Audit Study

    Hannah L. Semigran, 1

    Jeffrey A. Linder, 2 Courtney Gidengil,

    3,4 Ateev Mehrotra,

    1,5

    1 Department of Health Care Policy, Harvard Medical School, Boston, MA

    2 Division of General Medicine and Primary Care, Brigham and Women’s Hospital, Boston, MA

    3 RAND Corporation, Boston, MA

    4 Division of Infectious Diseases, Boston Children’s Hospital, Boston, MA

    5 Division of General Internal Medicine and Primary Care, Beth Israel Deaconess Medical Center,

    Boston, MA

    Corresponding Author:

    Ateev Mehrotra, MD, MPH

    Harvard Medical School

    180 Longwood Avenue

    Boston, MA 02115

    617-432-3905

    [email protected]

    Word Count: 3,689

    Tables: 4

    This study was funded by the United States’ National Institute of Health, (National Institute of Allergy

    and Infectious Disease - Grant # R21 AI097759-01).

    Contributors: Author Contributions: Study concept and design: Mehrotra, Semigran, Gidengil, Linder.

    Acquisition of data: Semigran. Analysis and interpretation of data: Semigran, Mehrotra. Drafting of the

    manuscript: Semigran. Critical revision of the manuscript for important intellectual content: Gidengil,

    Linder, Mehrotra. Statistical analysis: Semigran, Mehrotra. Administrative, technical, and material

    support: Mehrotra. Study supervision: Mehrotra

    Page 1 of 45

    https://mc.manuscriptcentral.com/bmj

    BMJ

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

  • Confidential: For Review O

    nly

    2

    Competing interests: All authors are affiliated with Harvard Medical School. Harvard Medical School’s

    Family Health Guide is used as the basis for one of the symptom checkers evaluated. None of the

    authors have been or plan to be involved in the development, evaluation, promotion or any other facet

    of Harvard Medical School-related symptom checker. All authors have completed the ICMJE uniform

    disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for

    the submitted work; no financial relationships with any organisations that might have an interest in the

    submitted work in the previous three years; no other relationships or activities that could appear to

    have influenced the submitted work.

    Ethical Approval: Not required

    Data Sharing: No additional data available

    Transparency: The senior author (the manuscript's guarantor) affirms that the manuscript is an honest,

    accurate, and transparent account of the study being reported; that no important aspects of the study

    have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered)

    have been explained.

    Page 2 of 45

    https://mc.manuscriptcentral.com/bmj

    BMJ

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

  • Confidential: For Review O

    nly

    3

    SUMMARY:

    Objectives: Use of the internet for self-diagnosis is common. Symptom checkers are online tools which

    use computer algorithms to help patients with self-diagnosis and/or self-triage. Despite the growth in

    use of such tools, the clinical accuracy of symptom checkers has not been assessed. Our objective was

    to determine the diagnostic and triage accuracy of online symptom checkers.

    Design: Audit study of web-based symptom checkers using standardized patient (SP) vignettes to assess

    their clinical performance

    Participants: We identified 23 symptom checkers that were in English, free, publicly-available, and

    provided advice across a range of conditions. We compiled 45 standardized patient (SP) vignettes

    equally divided into three categories of triage urgency: emergent care required (e.g., pulmonary

    embolism), non-emergent care reasonable (e.g., otitis media), and self-care reasonable (e.g., viral upper

    respiratory illness)

    Main outcome measures: For symptom checkers that provided a diagnosis, our main outcomes were

    whether the symptom checker listed the correct diagnosis first or within the first 20 potential diagnoses

    (n = 770 SP evaluations). For symptom checkers that provided a triage recommendation, our main

    outcomes were whether the symptom checker recommended emergent care, non-emergent care, or

    self-care (n = 532 SP evaluations).

    Results: The 23 symptom checkers provided the correct diagnosis first in 34% (95% confidence interval

    [CI], 31 to 37) of SP evaluations, listed the correct diagnosis within the top 20 diagnoses given in 58%

    (95% CI 55 to 62) of SP evaluations, and provided the appropriate triage advice in 57% (95% CI 52 to 61)

    of SP evaluations. Triage performance varied by condition urgency, with appropriate triage advice

    provided in 80% (95% CI 75 to 86) of emergent cases, 55% (95% CI 47 to 63) of non-emergent cases, and

    Page 3 of 45

    https://mc.manuscriptcentral.com/bmj

    BMJ

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

  • Confidential: For Review O

    nly

    4

    33% (95% CI 26 to 40) of self-care cases (p

  • Confidential: For Review O

    nly

    5

    The public is increasingly using the internet to research their health concerns. For example, the United

    Kingdom’s online patient portal for national health information, NHS Choices, reports over 15 million

    visits per month. 1 More than a third of adults in the United States regularly use the internet to self-

    diagnose what ails them, using it for both non-urgent and urgent symptoms such as chest pain. 2 3

    While

    there is a wealth of online resources to learn about specific conditions, self-diagnosis usually starts with

    search engines like Google, Bing, or Yahoo. 2 However, internet search engines can lead users to

    confusing and sometimes unsubstantiated information and people with urgent symptoms are not

    directed to seek emergent care. 3-6

    Recently, there has been a proliferation of more sophisticated

    programs called symptom checkers that attempt to more effectively diagnose patients and direct them

    to the appropriate care setting. 3 6-13

    Using computerized algorithms, symptom checkers ask users a series of questions about their symptoms

    or require users to input their symptoms themselves. The algorithms vary and may use branching logic,

    Bayesian inference, or other methods. Private companies and other organizations, including the

    National Health Service (NHS), the American Academy of Pediatrics, and the Mayo Clinic, have launched

    their own symptom checkers. One symptom checker, iTriage, reports 50 million uses per year. 14

    Typically symptom checkers are accessed via websites, but some are also available as apps for smart

    phones or tablets.

    Symptom checkers serve two main functions: facilitating self-diagnosis and assisting with triage. The

    self-diagnosis function provides a list of diagnoses usually rank ordered by likelihood. The diagnosis

    function is typically framed as helping educate patients on the range of diagnoses that might fit their

    symptoms. The triage function informs patients whether they should seek care at all and, if so, where

    (i.e. accident & emergency department, general practitioner’s (GP) clinic) and with what urgency (i.e.

    emergently or within a few days). Symptom checkers may supplement or replace telephone triage lines,

    Page 5 of 45

    https://mc.manuscriptcentral.com/bmj

    BMJ

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

  • Confidential: For Review O

    nly

    6

    which are common in primary care. 15-18

    To ensure the safety of medical mobile apps, the United States

    Congress is considering regulation of apps

Search related