Invited Commentary

Known Unknowns and Unknown Unknownsat the Point of CareGurpreet Dhaliwal, MD

Lifelong learning (LLL) in medicine is at once a value, a set ofskills, and a social pact. It is part of our professional identity,a core competency of the modern clinician, and an inviolableobligation to our patients. Despite its alignment with profes-sional ideals, LLL remains a voluntary and self-motivated pur-suit with no direct extrinsic rewards, which raises questionsabout what can be done to enhance its value and facilitate itswidespread enactment.

Clinicians consistently report that patient problems fromtheir own practices are their greatest motivators for learning,yet most of their questions go unanswered.1 Health systemshave an opportunity to capitalize on this motivation and un-met need by optimizing point-of-care learning (POCL), wherequestions triggered by patient care activities are answered inreal time. All physicians know, however, that this is easier saidthan done.

In this issue of JAMA Internal Medicine, Cook andcolleagues2 take a granular look at what stands in the way ofon-the-job POCL. Based on interviews with 50 physicians, theauthors model the multiple decisions clinicians face when con-templating information searching while caring for a patient,including whether to search, when to search, where to search,and when to stop. It will come as no surprise that insufficient

time was the greatest barrierto POCL (along with casecomplexity and informationoverload) and that efficiency

was the most important determinant in selecting an informa-tion source. These findings match those of a recent study ofPOC resource selection by internal medicine residents, who pri-oritized efficiency and credibility of sources (eg, UpToDate andGoogle) and cited time and information overload as their prin-cipal barriers. Both experienced physicians and residents sharea common goal of learning by finding “the most trusted andrelevant information in the shortest time.”3(p790)

The first node in the model described by Cook et al2

(whether to initiate POCL) may be more complicated thanmeets the eye. While most physicians would claim to be ca-pable of determining when they need assistance, studies con-sistently reveal that physicians’ ability to self-assess knowl-edge and skills is inaccurate and that we maintain blind spotsto our deficiencies.4 This misperception may arise in part fromthe universal human tendency to mistake confidence for com-petence. If confidence and skill were tightly calibrated, phy-sicians could rely on their confidence as a guide for when theyneed additional resources. Multiple studies have demon-strated that this is not the case.

In another study in this issue of JAMA Internal Medicine,Meyer and colleagues5 examine the diagnostic accuracy, con-fidence, and resource utilization of physicians who analyzed

web-based patient vignettes. Although physicians’ diagnos-tic accuracy declined between easy cases (55% of physiciansmade the correct diagnosis) and difficult cases (6%), physi-cians’ confidence in their diagnostic accuracy (measured be-fore the correct diagnosis was revealed) remained largely un-changed across both case types. Greater case difficulty wascorrelated with increased requests for reference materials butnot increased input from colleagues in the form of second opin-ions, curbside consultations, or referrals. These findings sug-gest that physicians may not request the full complement ofPOC resources—references and colleagues—to facilitate diag-nosis when we need it the most, and that confidence is not areliable indicator of when we need help at the POC.

Taken together, the studies by Cook et al2 and Meyer et al5

suggest that physicians who want to provide optimal care re-quire systems that facilitate POCL, both when they know-ingly want information and when they are unaware that theyneed education. Not every barrier to POCL can be overcome.There is little hope of decreasing case complexity or increas-ing physician time (although care delivery systems could beredesigned so that physicians see only the most complex caseswith sufficient time). However, technical and organizationalsolutions to issues of resource selection, search failure, and in-formation overload are achievable.

The development of computer systems that can analyze,understand, and generate human languages (natural lan-guage processing) will be instrumental in making POCL a re-ality in the busy clinical environment. Such systems wouldhave the capacity to accept statements and questions such as“show me 10 images of typical lichen planus” or “what per-centage of patients receiving vancomycin develop eosino-philia?” and reply with a direct answer or succinct search re-sults on the first query.

In addition to on-demand learning, intelligent electronichealth record (EHR) systems could identify the issues physiciansare contending with based on real-time text entry or the patternof tests and treatments being ordered and offer a menu of brieflearning opportunities at precisely the times when physicians’confidence leads them to believe it is unnecessary. As a physi-cian is typing, the computer may offer the following: “It lookslike you are treating chest pain. Click here for an update on thesensitivity of the new troponin assay. Click here if you would liketo receive a secure e-mail if this patient has an elevated troponinlevel or undergoes a stress test or cardiac catheterization in thenext 30 days.” IBM is training its supercomputer Watson to “lis-ten in” as physicians are typing and offer this form of unsolic-ited learning and clinical decision support in real time.6 Pilotstudies of EHR-embedded real-time learning programs suggestthat they are easy to use and are valued by clinicians becauseof their integration with the patient care process.7

Related articles

Known and Unknown Unknowns at the Point of Care Invited Commentary

jamainternalmedicine.com JAMA Internal Medicine Published online August 26, 2013 E1

Downloaded From: http://archinte.jamanetwork.com/ by a University Of Connecticut Health Center User on 08/27/2013

Until that high level of automation is realized, we will con-tinue to rely on colleagues to buttress our knowledge and skills.Medical librarians are far more adept at navigating the entirecanon of medical knowledge than are physicians, but their skillshave not been leveraged for POCL. While a librarian on staffin every office or on every medical team may be fanciful, a re-mote librarian service is not. In one study, a just-in-time in-formation consultation service was shown to deliver useful an-swers in less than 15 minutes to clinicians’ real-world questionssubmitted via smartphone.8 Health care systems should con-sider investing in informaticians who search the medical lit-erature, patient records, and the system’s own “big data” toprovide answers to clinicians’ real-time inquiries.

Colleagues are frequently the most efficient and usefulsources of information and answers, but ambiguous normsand preferences around POC inquiries—phone vs text,expected response time, curbside vs formal consultation—limit utilization and heighten concerns about inconvenienc-ing consultants. Systems and groups should clarify andinternally post these preferences in advance so that

obstacles to efficient communication at the time of a patientencounter are minimized.

Reducing the barriers to POCL and making appeals to pro-fessional values will not be sufficient to transform the behav-ior of physicians. Integration with external motivators such asspecialty maintenance of certification (MOC) will be neces-sary to catalyze POCL. The American Board of Internal Medi-cine MOC program already offers a POCL module, which phy-sicians preferred to multiple-choice examinations, citinggreater relevance to their practice and credit for self-directedlearning.9 A more streamlined version could be developed forsmartphones, tablets, and computers, perhaps even the veryEHR note the physician is working on.

Studies demonstrate that top-performing physicians ap-proach their work with a conscious intent to learn from theirpatients rather than welcoming learning as a chancebyproduct.10 All physicians want to be lifelong learners. We canadvance technology, practices, certification, and ultimatelymindsets to give physicians the nudge they need to move to-ward learning when so many other forces pull them away.

ARTICLE INFORMATION

Author Affiliations: Department of Medicine,University of California, San Francisco; MedicalService, San Francisco Veterans Affairs MedicalCenter, San Francisco, California.

Corresponding Author: Gurpreet Dhaliwal, MD,San Francisco Veterans Affairs Medical Center, 4150Clement St (111), San Francisco, CA 94121 (gurpreet.dhaliwal@ucsf.edu).

Published Online: August 26, 2013.doi:10.1001/jamainternmed.2013.7494.

Conflict of Interest Disclosures: None reported.

Additional Contributions: The author thanks EricHolmboe, MD, and Vishnu Mohan, MD, MBI, fortheir comments on this article.

REFERENCES

1. Campbell C, Parboosingh J, Gondocz T,Babitskaya G, Pham B. Study of the factorsinfluencing the stimulus to learning recorded byphysicians keeping a learning portfolio. J ContinEduc Health Prof. 1999;19(1):16-24.

2. Cook DA, Sorensen KJ, Wilkinson JM, Berger JM.Barriers and decisions when answering clinicalquestions at the point of care: a grounded theorystudy [published online August 26, 2013]. JAMAIntern Med. doi:10.1001/jamainternmed.2013.10103.

3. Duran-Nelson A, Gladding S, Beattie J, Nixon LJ.Should we Google it? resource use by internalmedicine residents for point-of-care clinicaldecision making. Acad Med. 2013;88(6):788-794.

4. Eva KW, Regehr G. Self-assessment in the healthprofessions: a reformulation and research agenda.Acad Med. 2005;80(10)(suppl):S46-S54.

5. Meyer AND, Payne VL, Meeks M, Rao R, Singh H.Physicians’ diagnostic accuracy, confidence, andresource requests: a vignette study [publishedonline August 26, 2013]. JAMA Intern Med.doi:10.1001/jamainternmed.2013.10081.

6. Giles T, Wilcox R. IBM Watson and medicalrecords text analytics: presented at the 2011Healthcare Information and Management Systems

Society meeting. http://www-01.ibm.com/software/ebusiness/jstart/downloads/MRTAWatson-HIMSS.pdf. Accessed June 21, 2013.

7. Borbolla D, Gorman P, Del Fiol G, et al.Physicians’ perceptions of an educational supportsystem integrated into an electronic health record.Stud Health Technol Inform. 2013;186:125-129.

8. McGowan J, Hogg W, Campbell C, Rowan M.Just-in-time information improved decision-makingin primary care: a randomized controlled trial. PLoSOne. 2008;3(11):e3785.

9. Green ML, Reddy SG, Holmboe E. Teaching andevaluating point of care learning with aninternet-based clinical-question portfolio. J ContinEduc Health Prof. 2009;29(4):209-219.

10. Sargeant J, Mann K, Sinclair D, et al. Learning inpractice: experiences and perceptions ofhigh-scoring physicians. Acad Med.2006;81(7):655-660.

Invited Commentary Known and Unknown Unknowns at the Point of Care

E2 JAMA Internal Medicine Published online August 26, 2013 jamainternalmedicine.com

Downloaded From: http://archinte.jamanetwork.com/ by a University Of Connecticut Health Center User on 08/27/2013