iHealth 2017 Clinical Informatics Conference Q&A bookletand using risk scores. a) Interventions may be too costly or niche in nature to justify care management resources even if high

iHealth 2017 Clinical Informatics Conference

Q&A booklet

for Self Assessment

and

Maintenance of Certification (MOC) Part II Credit

2 | P a g e

iHealth 2017 Self Assessment and Maintenance of Certification (MOC) Part II Q&A booklet Purpose AMIA invited the iHealth 2017 workshop and session presenters to submit multiple choice questions (MCQ) associated with their presentations. This was to encourage interactivity as well as to comply with Maintenance of Certification guidelines from the American Board of Preventive Medicine and the American Board of Pathology. All attendees of iHealth 2017 may use these MCQs to test their knowledge of what they learned in these sessions. All board-certified clinical informaticians must engage in answering the MCQs associated with the sessions for which they will claim MOC-II credit. American Board of Pathology-boarded clinical informaticians: acceptable performance in this self assessment activity is defined as answering correctly 75% of the MCQs for the sessions you attended. Copy the text on the next page to send an attestation of your self assessment to [email protected]. The ACGME competencies addressed by this activity are: Patient Care, Medical Knowledge, Practice-based Learning and Improvement, Interpersonal & Communications Skills, Professionalism, and Systems-based Practice. For a full explanation of the ACGME competencies in the context of clinical informatics, see Safran C et al. Program requirements for fellowship education in the subspecialty of clinical informatics. J Am Med Inform Assoc. 2009 Mar-Apr;16(2):158-66. MOC Committee Reviewers and Disclosures: The following reviewers disclose that they and/or their life partners do not have relevant financial relationships with commercial interests: Thomas Agresta; Anupam Goel; Eunice Huang; Robert Marshall; Julie Massey; Sandeep Soman; Keith Woeltje

The following reviewers disclose that they and/or their life partners do have relevant financial relationships with commercial interests:

Name of individual

Name of commercial interest Nature of the relationship

Amy Y. Wang Intelligent Medical Objects

Stockholder

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2649323/


3 | P a g e

Instructions The first half of this booklet of about 130 MCQs provides the question and answer options organized by day and by session number. The second half provides the correct answers per question. Each answer is followed with an explanations and at least one referenc. Clinical Informatics Diplomates: How to claim MOC-II credit • Log into your account at www.amia.org to access AMIA Central • On the right side, click “My Events” • Click “Apply for Credits” under “iHealth 2017 Clinical Informatics Conference” • For “Select Credit Type” click “Physician” • For “Select Physician Credit Type” click “MOC-II” and select your MOC-II sessions attended

--You must claim your MOC-II credit first for AMIA to keep an accurate record of your MOC-II credits • Click “Save and Add Additional Credit Type” at bottom of screen • Now for “Select Physician Credit Type” click “Physician” on drop-down menu, and claim your regular CME sessions • Click Submit • You will be prompted to download your certificate

Claim credit by June 5, 2017. AMIA reports MOC-II credits to the ABPM and to the American Board of Pathology. We pull data to send reports only within the same year as the activity.

Philip Smith Intelligent Medical Objects, Inc.;

Consultant

Remote Patient Monitoring Inc.;

Other Financial or Material Support

Arc Devices Stock Shareholder (directly purchased)

4 | P a g e

Email attestation text for American Board of Pathology-boarded clinical informaticians:

To: [email protected]

From: [Use the email you have on file with the American Board of Pathology]

Subject: MOC-II credits iHealth 2017

I attest that I attended the live activity iHealth 2017 and used the self assessment booklet as my Self Assessment Module (SAM) for those sessions designated as offering MOC-II credit in the activity.

I attest that I achieved at least a 75% score on the questions I answered that were associated with the MOC-II designated sessions that I attended.

Name:

____________________________________________

5 | P a g e

Multiple Choice Questions for iHealth 2017 Self-Assessment: TUESDAY, MAY 2: WS01: Workshop – Clinical Knowledge Management Dominik Aronsky, Dirk Wenke, Asli Ozdas-Weitkamp WS01-1: What is a logical first step in starting a clinical knowledge management program within the iterative knowledge management framework?

a) Create a catalog of existing knowledge assets including relationships with metadata

b) Build standards-based interoperable knowledge assets

c) Centralize the process of creating knowledge assets

d) Establish a knowledge asset lifecycle

WS01-2: What activities are least likely within the scope of a clinical knowledge management framework?

a) Integrate knowledge assets into clinical workflow

b) Build a sustainable knowledge asset framework with reusable components

c) Maintain the representation of knowledge assets

d) Provide knowledge lifecycle support

WS01-3: Which topic is least likely associated with clinical knowledge management?

a) Institutional data governance

b) Lifecycle management of knowledge assets

c) Management of knowledge meta-data

d) Single-login management for clinical workstations

6 | P a g e

WS01-4: Which process is not directly supported by a clinical knowledge management system?

a) Centralized request management for changes in the knowledge base

b) Centralized prioritization of knowledge management implementation tasks

c) Monitoring clinicians’ usage of clinical decision support systems

d) Versioning of knowledge assets

e) Auditing of knowledge asset changes

WS01-5: Which characteristic(s) is/are considered best practice(s) for creating knowledge assets?

a) Modular

b) Reusable

c) Interoperable

d) All of the above

WS01-6: Which of the following is a/are critical method(s) applied in clinical knowledge management?

a) Formal representation of knowledge assets

b) Versioning of knowledge assets

c) Terminology models as building blocks

d) All of the above

7 | P a g e

WS02 Workshop - Understanding and communicating risk: Critical competencies for the healthcare workforce

Greg Nelson, Monica Horvath

WS02-1:

A medical provider is reviewing the literature to understand the risk of colon cancer given different socio-economic backgrounds. However, the risk expressions are not identical among these resources. Of the following conclusions, which one has the greatest stated risk of colon cancer?

a) One out of 21 men will develop colon cancer in their lifetime.

b) 4.4% of women will develop colon cancer in their lifetime

c) 41 out of 1,000 men and women will develop new colon cancer cases each year

d) One out of 42 women will develop colon cancer in their lifetime

WS02-2:

A randomized controlled trial monitored 500 elderly (>65 years of age) individuals with periodic instances of vertigo for one year. By the end of one year, 125 had reported an injury requiring medical care occurring while walking, moving, or otherwise attempting to adjust their physical position. What are the odds of reporting an injury during this study?

a) 18%

b) 25%

c) 33%

d) 75%

WS02-3:

For the case in question #2, what is the incidence of patients with vertigo during the study period?

a) 18%

8 | P a g e

b) 25%

c) 33%

d) 75%

WS02-4:

The prevalence of a specific disease is 0.5% of the population. A new test accurately detects the disease 90% of the time, but falsely identifies 5% of healthy people as having the disease. What is the probability of seeing a true positive test result in a clinic?

a) ~ 8%

b) ~ 9%

c) ~ 85%

d) ~ 92%

WS02-5:

Select the statement below that represents a common misperception regarding interpreting and using risk scores.

a) Interventions may be too costly or niche in nature to justify care management resources even if high risk

b) Patient risk score is a prediction of the likelihood of a patient to be in a certain situation in the future

c) Patients with the highest readmission risk scores are those most impactable and thus deserving of care management interventions

d) Seeing ROI from an intervention guided by a risk score is highly dependent on being able to discern those patients most likely to benefit

WS02-6:

Successful implementation of risk prediction models requires a multidisciplinary team of medical, nursing, technical, financial, and statistical staff. Which of the following activities are clinical informaticists best suited to own and resolve given their unique perspective?

a) Charting a path to manage the lack of uniform standards for payer-provided claims data

b) Developing rules describing patient attribution to an appropriate provider under a risk-contract or alternative payment model

9 | P a g e

c) Ensuring that risk scores have meaning and the rationale behind score creation is understood by end user clinicians.

d) Model adjustments for member churn and out of network utilization of care services

e) Negotiation with vendors providing risk scores based on predictive models

WS03: Workshop Building an innovation ecosystem: strategies and lessons learned in spreading IT innovations using the Diffusion of Excellence model Elizabeth Floto, Vanessa Coronel, Shereef Elnahal, Andrea Ippolito, Thalia Sirjue WS03-1: An innovation ecosystem refers to the dynamics of the complex relationships that are formed between _______ and _______ whose functional goal is to enable technology development, and innovation.

a) IT professionals, clinicians

b) the Business & Accounting department, the Patient Care Services department

c) actors, entities

d) financers, innovators

WS03-2: How does Everett Rogers categorize innovation adopters?

a) Early Adopters, Early Majority, Late Majority, Laggards, Traditionalists

b) Innovators, Early Adopters, Early Majority, Late Majority, Laggards

c) Innovators, Early Adopters, Majority, Laggards, Antagonists

d) Early Adopters, Majority, Laggards, Traditionalists, Antagonists

WS03-3:

10 | P a g e

A patient at Happy Healthy Medical Center (HHMC) shares with Nurse Andrea that his Primary Care Physician did not wash his hands before beginning the visit. She apologizes, assures the patient that this is not regular practice, and promises to bring the issue up at a future staff meeting. However, psychological safety is low at HHMC so Nurse Andrea does not feel comfortable raising the issue, or presenting the idea she has to help address it. She confides in a colleague, who shares an anonymized version of the story via an employee engagement survey. The HHMC Employee Engagement team shares this comment with leadership. What of the following is the most appropriate HHMC leadership response to improve the organization’s Innovation Ecosystem?

a) Nothing. The comment represents the views of one staff member. His or her reluctance to share does not indicate a system-wide problem.

b) Work with the Employee Engagement and Internal Communications teams to develop a multi-modal plan to improve the Innovation Ecosystem. The plan should include appropriate activities and effective communication to ensure all levels of staff know they are encouraged, and empowered to ask questions, identify problems, and pilot innovative solutions to addressing them. HHMC leadership should also ensure all managers and executives are aware of and empowered to provide resources and support where needed.

c) Send out an all-employee email reminding employees of the facility’s “safe space” policy encouraging staff to talk to their managers about challenges and ideas for developing creative solutions.

WS03-4: A clinical informatics team noticed rates of specialty-referral completions were especially low in the primary care department of one facility in the Roy G. Biv Healthcare System (RGBHS). With the permission of facility leadership, the team worked with the clinical staff to determine the root cause of the problem. Referrals were in fact being completed. However, they were not accurately reported in RGBHS’s EHR software. To combat the reporting issue, the team developed and piloted a new referral template and protocols in the EHR software. Within 90 days of implementing the new process, the referral completion rates jumped from 29% to 74%. If they are following the principles of innovation dissemination, which of the following is the most appropriate next step?

a) Report the successful innovation to leadership, conduct research to determine whether or not the problem and root cause are the same at other facilities. Upon confirming presence of the problem, request permission to update the referral templates and protocols for the other six RGBHS facilities.

11 | P a g e

b) Immediately roll out the new template and protocol across all RGBHS facilities

c) Develop a report that includes the research findings, pilot results, updated template fields, and protocol codes. Save it on the shared drive so other facilities can access it, and hope they see it.

WS03-5: Which of the following best describes why it is so difficult for organizations to change?

a) Fear of the unknown or unfamiliar

b) Union representatives of bargaining agreements

c) Organization leadership

d) People are likely to keep their belief system and continue current behavior even if data contradicts it

WS04: Workshop - Leveraging Electronic Health Records Using Recurrent Neural Networks

David Ledbetter, Long Ho, Abel Brown

WS04-1:

Which of the following algorithms explicitly takes advantage of the temporal component of data?

a) Logistic Regressions

b) Support Vector Machine

c) Random Forest

d) Recurrent Neural Network

WS04-2:

HDF stands for...

a) Hierarchical Data Format

12 | P a g e

b) High-Definition File

c) HemoDynamic Fluctuation

d) Hoggs Data Filter

WS04-3:

The primary rationale to normalize data with an RNN is to...

a) Make the numbers smaller for machine precision

b) Make magnitude of variations of variables similar

c) Fit the data into a 16-bit floating point variable

d) Make it easier to plot multiple variables on a single canvas

WS04-4:

The best way to impute data is to...

a) Forward fill

b) Linear interpolation

c) Cubic splines

d) There is no agreed upon standard practice for data imputation

WS04-5:

Deep learning algorithms are typically trained using Graphics Processing Units (GPUs) because...

a) A desktop PC can house up to 4 GPUs (as opposed to just 1 or 2 CPU)

b) Many deep learning algorithms rely extensively on large-scale matrix multiplications which can be easily paralleled to many smaller, simpler processors

c) It gives you a good excuse to write off your gaming PC as a tax deduction

d) Most deep learning packages are only able to perform computations on a GPU

13 | P a g e

WS04-6:

A particular advantage of an RNN over static methods (such as PIM2 or PRISM3) is that...

a) It is able to continuously update as new information becomes available

b) It is able to able to utilize hundreds of variables simultaneously

c) Expert knowledge can be incorporated

d) The contribution of each input can be calculated

14 | P a g e

WS05: Workshop – Predictive Analytics Using Open Source Machine Learning WEKA Software

Robert Hoyt, Dallas Snider

WS05-1:

Your academic center has been funded as part of a Precision Medicine grant. You are in the process of integrating genomic data with phenotypical data you have archived derived from your electronic health record. The primary goal of your data analytics team is to find “undiscovered meaningful patterns” in the data. If you utilize machine learning for your inquiry what type of approach will you most likely use?

a) Classification

b) Regression

c) Association

d) Clustering

WS05-2:

Your hospital has EHR data from the last year, archived in your enterprise data warehouse. The administration is alarmed about possible lost revenue due to an increased readmission rate. Your data analytics team is going to tackle heart failure readmissions first. Your goal is to create a predictive analytics model to identify those factors that lead to the binary outcome (readmitted or not readmitted.

Which machine learning approach would you most likely use?

a) Classification

b) Regression

c) Association

d) Clustering

15 | P a g e

WS05-3:

Your data analytics team is planning on creating some new clinical decision support rules to alert clinicians about patients who are at high risk for medication non-adherence. They would like to create IF-THEN rules with 95% confidence they can implement.

Which machine learning approach will you most likely use to accomplish this?

a) Classification

b) Regression

c) Association

d) Clustering

WS05-4:

As the head of the data analytics team you have been tasked by the CFO to create a model to help predict wait times in the ER using common parameters such as age, APACHE II disease severity score, gender (male = 0, female = 1) and insurance status (uninsured = 0, insured = 1). The most likely machine learning approach would be:

a) Classification

b) Regression

c) Association

d) Clustering

WS05-5:

Your data analytics team has created a predictive model using a machine learning classification approach (Naive Bayes) for readmissions for COPD. Your team believes your model is unique and worthy of publication. What is the most common data representation choice for a binary outcome in a publication?

a) Precision

b) Specificity

c) C-statistic or ROC area under the curve

d) Effect size

16 | P a g e

WS05-6:

Your data analytics team is beginning to analyze a complicated data set from your EHR. You are in the initial data preprocessing phase of your analysis and using the machine learning software WEKA. What feature in WEKA would you most likely use to address missing data, normalization and discretization of data?

a) Filter

b) Select attributes

c) Edit

d) Clustering

WS06: Workshop – Agile Clinical Decision Support DuWayne Willett, Vaishnavi Kannan, Mujeeb Basit WS06-1: On rapid-cycle agile development projects, requirements are often kept lightweight, for example with a User Story and associated Acceptance Criteria. The most typical template for a User Story is: "As a XXXXX, I want YYYYY so that ZZZZZ ". When using this template to write a User story for a clinical decision support (CDS) tool which will be displayed to clinicians in an EHR, the best entry for the placeholder is:

a) The executive sponsor of the project

b) The clinician who will see and use the CDS tool

c) The EHR analyst building the CDS tool

d) The patient who will ultimately benefit from best practices promoted by use of CDS

17 | P a g e

WS06-2: The acronym INVEST is a widely-accepted set of criteria for assessing the quality of a User Story, and its readiness to undergo development. Which of the following is one of the INVEST criteria?

a) Independent

b) Visible

c) Straightforward

d) Trainable

WS06-3: Epidural catheters for pain control are frequently placed by anesthesiologists for patients primarily managed by other services. Administering anticoagulants to patients with an epidural catheter may be associated with increased risk of epidural hematoma and neurologic damage. You've been asked to create an alert for anyone ordering an anticoagulant or antiplatelet medication for a patient with an epidural catheter, to alert them of this risk. As you listen to the experts discuss when the alert should fire, you learn that exception conditions apply, e.g. for low-dose VTE prophylaxis. You want to capture the logic to define unequivocally for which sets of conditions the alert should and should not appear. The optimal model for depicting this logic is a:

a) Swimlane Workflow Diagram

b) User Interface (UI) Storyboard

c) Use Case Diagram

d) Object Diagram

e) Decision Tree

18 | P a g e

WS06-4: You're embarking on an initiative to redesign venous thromboembolism (VTE) prophylaxis risk profiling and prophylaxis for patients admitted to your hospital. This project is envisioned to involve capturing of VTE risk data by patients (via your EMR's patient portal), by clinic staff and pre-surgery clinic visits, and/or by hospital nurses and physicians/surgeons following admission, in any of several combinations or sequences. In addition, clinical decision support is desired to prompt ordering of VTE prophylaxis from the options most appropriate for a given patient's VTE risk level and other clinical factors suchas renal function. The best diagram for depicting on a single page the full scope of desired activities (user goals) along with which user roles would participate in each activity would be a:

a) Use Case Diagram

b) Swimlane Workflow Diagram

c) Data Flow Diagram

d) Decision Tree

WS06-5: An automated acceptance test is written before development of a clinical decision support alert, specifying the decision logic it should employ when evaluating whether to display to clinicians. The automated test is red (fails) initially, and then becomes green (passes) once the EHR analyst has successfully finished their development. Which best describes this use of automated testing?

a) Regression testing

b) Test-driven development

c) Exploratory testing

d) Unit testing

WS06-6:

19 | P a g e

Your liver specialists want to increase the rate at which patients with cirrhosis are screened for hepatocellular carcinoma, in line with published guidelines. Accordingly, a clinical decision support alert has been reminding providers to order a liver imaging test if one has not been performed within the recommended screening interval. The alert checks for presence of cirrhosis on the Problem List, and for a completed order in your EMR for any of a list of imaging procedures. Your CDS monitoring program recently detected a sharp rise in the frequency at which this alert is displaying to providers. What is the most likely cause for the abrupt increase in frequency of alert display?

a) The national society guidelines were recently updated to eliminate the screening recommendations, and the physicians' practice reflects the new guidelines.

b) Your local physicians have collectively decided that the screening practice is low-yield and not cost-effective, and have abandoned it.

c) A recent operating system software security patch on your workstations has caused the CDS alert to display inappropriately

d) Radiology has begun using a new orderable in your EMR which is not contained in the "Liver imaging" procedure list used by the CDS alert, causing it to display inappropriately.

WS07: Workshop - Fast Healthcare Interoperability Resources (FHIR): A Workshop for Implementers

Charles Jaffe, Grahame Grieve, Alistair Erskine, Viet Nguyen, Russ Leftwich, Stan Huff

WS07-1:

What are the requirements for use of FHIR in open-source or commercial products?

a) FHIR is provided through a creative commons license.

b) HL7 membership

c) FHIR Foundation membership

d) A license fee based upon utilization.

WS07-2:

What individuals may contribute to FHIR development?

20 | P a g e

a) HL7 members

b) Argonaut Project members

c) FHIR Foundation members

d) FHIR development is open to anyone who registers

WS07-3:

FHIR is rapidly integrating with other HL7 standards, including V2 messaging and CDA. On the current trajectory,

a) V2 and CDA will be sunset by the end of the decade.

b) FHIR will replace V3

c) Service oriented architecture (SOA)

d) FHIR will coexist with enhancements to V2 and CDA, which are effective for purpose

WS07-4:

What is the principle objective of the FHIR Foundation?

a) Accelerate development of FHIR resources and profiles.

b) Provide a collaborative environment for FHIR implementation.

c) Enable a funding source for subject matter experts.

d) Provide support for the US Realm.

WS07-5:

An important advantage of using FHIR based APIs versus standard messaging interfaces is:

a) Patient data can be accessed directly from the original source without making a duplicate copy.

b) FHIR does not require the use of standard code systems like LOINC, SNOMED, or RxNorm.

c) The FHIR resources are more mature than the messaging standards.

d) FHIR provides superior security.

21 | P a g e

WS07-6:

One important benefit of the Argonaut work is:

a) It establishes broad semantic interoperability of healthcare data based on FHIR profiles.

b) It provides a pathway for requesting the addition of new concepts to standard coding systems.

c) It establishes a baseline for security and authentication.

d) It has created FHIR profiles for a core set of common data elements for use in the US.

WS08: Application of Model-based Computer Simulation in Medical Informatics

Duane Steward

WS08-1:

Which of the following best outlines the discipline of model-based simulation? A. 1) Enumerate independent and dependent variables 2) Choose appropriate models to emulate the system features of interest 3) Analyze variance to determine number of runs required for replication experiments 4) Render simulation output to enable debugging, performance measurement and inter-rater agreement 5) Iterate until output converges within specified confidence intervals for chosen outputs B. 1) Articulate purpose, scope, granularity and performance measures 2) Choose entities, resources, and context of investigation 3) Construct a network of models, input and output variables using an iterative process 4) Frame assumptions that best fit the question investigated 5) Analyze results of replicated results only after establishing confidence in the decision makers C. 1) Choose entities, system resources, and context of investigation 2) Enumerate independent and dependent variables 3) Frame assumptions and implement a directed graph of models 4) Analyze variance to determine number of runs required for replication experiments 5) Iterate until output converges within specified confidence intervals for chosen outputs

22 | P a g e

D. 1) Frame assumptions, enumerating independent and dependent variables 2) Analyze variance to determine number of runs to adjust for confounders in experiments 3) Choose appropriate models to emulate the system features of interest 4) Articulate purpose, scope, granularity and performance measures 5) Analyze results of replicated results only after output converges within accepted confidence intervals WS08-2: The use of model-based simulation was asserted to be the testing of assumptions--i.e., the assigning of assumed values to variables and model parameters, running the simulation and observing what values emerge in output variables. Insofar as that is true, the entire exercise pivots on the quality of the data embedded in or serving as the basis for the assumptions. If the solution sought is for a system that does not yet exist or has yet to be exercised, the following question becomes vitally important. What source of data is most ill-advised for model-based simulation where the objective is to support decision making in scenarios for which data does not yet exist?

a) Creative data

b) Biased data

c) Data from other systems not within the scope of the investigation

d) Simulator output

e) Data representing assumptions of the decision maker

WS08-3: Which of the following simulation study projects are likely to require custom programming?

a) Queuing of admitted patients between emergency department disposition and hospital admission

b) Impact of patient reported outcomes on a clinic’s throughput

c) Spread of disease over an urban population

d) Integration of discharge summary and outcomes feedback to reduce readmission

WS08-4:

23 | P a g e

Which of the following is not a form of model-based simulation?

a) Agent-based Simulation of social activity in a food court.

b) Process flow simulation in surgical prep

c) System dynamics simulation of drug perfusion in vivo.

d) Emergency Response Team simulation of assigned roles

e) Organ failure rate simulation over time comparing toxin levels

WS08-5: An ophthalmology clinic requests help in evaluating operations. They want to know whether distributing the allocation of exam rooms unevenly is warranted in light of multiple ophthalmologist with variation in practice style. Among four clinicians, one has expressed no interest in improving Press Gainey scores and insists on rapid efficient exam room visits to increase how many patients can be seen on non-surgical clinic days. Another of the four ophthalmologists prides himself in taking as long as the patient needs to answer questions at a more relaxed pace and has an average exam duration that everyone suspects is greater than others. The clinic administrative team would like to know if the clinic throughput would be higher if the clinic schedule was arranged so that the faster and slower clinicians were scheduled on the same day with more rooms allocated to the faster paced clinician balanced by a reduced number of rooms allocated to the slower paced clinician. In this scenario, which of the following are not likely useful as simulation output variables?

a) Patient Diagnosis

b) Duration of Exam

c) Length of Stay

d) Resource utilization

WS08-6: In the scenario of the previous question, the simulation results in no significant difference in throughput (patients per day) despite a significant difference between the average length of patient-provider exam room encounter. What is the most fallible explanation for this result?

24 | P a g e

a) Clinic workflow segments outside the provider exam contribute an impact on length of visit that eliminates the impact of physician pace during patient encounters (e.g., mydriasis).

b) System resource bottlenecks (e.g., rooming by overburdened staff, limited number of shared instruments) are rate limiting features with greater impact on throughput than pace of physician encounter

c) A bug in the software may account for the unanticipated finding.

d) Impact on throughput may exist but not in the range of resource levels tested, at least not that impact overall performance measured. Context specific limitations preclude sound conclusions about alternate contexts without changes to the simulation.

e) Simulation verified to be working as intended and validated as representative of observed experience indicates that no benefit to throughput should be anticipated by unequal allocation of exam rooms based on clinician pace for the specific context studied in this simulation.

25 | P a g e

WEDNESDAY, MAY 3

S01: Panel - Rethinking Documentation: Streamlining Communication and Workflow in the Inpatient Setting

Evan Orenstein, Subha Airan-Javia, Mark Mai, Eric Shelov

S01-1:

Your hospital identifies a safety event in which an immunocompromised patient with recent history of extended-spectrum beta-lactamase resistant E. coli bacteremia was not started on sufficiently broad antibiotics when she developed a fever overnight, and effective antibiotics were not instituted until the error was caught 18 hours later. During root cause analysis, your team identifies that the patient’s progress notes made clear that the patient should be started specifically on vancomycin, imipenem, and amikacin in the setting of a new fever, however the written handoff did not include this information. Which of the following systems would be most likely to prevent this error?

a) Instituting a policy in which all febrile, immunocompromised patients are immediately transferred to the intensive care unit.

b) An easily accessible documentation template for fever in immunocompromised patients.

c) Educating providers about the importance of situational awareness in the written handoff.

d) Populating the progress note & written handoff from a single source of truth

S01-2:

A healthcare organization aims to increase the number of problems added to the problem list in the inpatient setting. Which of the following inpatient documentation strategies is most likely to increase the number of documented problems per admitted patient?

a) Force clinicians to enter each problem on the problem list before being able to document the plan related to that problem.

b) Institute educational sessions for all clinicians emphasizing the importance of the problem list for quality improvement, research, and billing.

c) Audit inpatient progress notes weekly and contact clinicians about possible missed problems that should be added to the problem list.

d) Audit inpatient progress notes and problem lists and let clinicians know where they rank in terms of problem list completeness compared to their peers.

26 | P a g e

S01-3:

A research team is working to understand the causes of medical errors overnight and wants to understand the relationship between contingency plans made by the daytime clinical team and overnight clinical decisions. In which inpatient document should overnight contingency plans be found to minimize communication errors?

a) Progress note

b) Problem List

c) To-Do List

d) Handoff

S02: Presentations – Population Health Querying Electronic Health Data for Population Health Activities using PopMedNetTM

Jessica Malenfant

S02-1: The PopMedNetTM (PMN) platform was enhanced to enable users to create and send custom data queries to multiple data sources using different database management systems (e.g. SQL server and Oracle). In what way does this new functionality facilitate a sharable, scalable query infrastructure?

a) PMN was extended to allow a single query to target multiple data models at once.

b) The tool was purpose-built to separate the front and back-end components to enable projects that use different data models to more easily leverage existing work (e.g. Race Term) to target additional data models.

c) The system now allows for users to see which database management system is used at each site.

d) Menu-driven queries can be used to generate patient lists that can be shared with the investigator who submitted a query.

27 | P a g e

S02: Presentations – Population Health Defining Patient Sub-Populations with SNOMED Vaishnavi Kannan S02-2: Your cancer center is offering state-of-the-art treatments for care of patients with renal cell carcinoma. The expert working group of medical oncologists, urologists, and pathologists who jointly provide this care wish to have a list of their patients with renal cell carcinoma available within their common EHR. They also envision real-time clinical decision support tools within the EHR targeted specifically to this population of patients. They do not want other forms of kidney cancer included (e.g. not kidney sarcoma, nephroblastoma, or transitional cell carcinoma). In your health system, physicians use a single EHR for both clinical documentation and billing, in both ambulatory and inpatient settings. Your EHR employs a clinical terminology set of diagnoses supplied by a vendor, pre-mapped to ICD-9, ICD-10, and SNOMED codes. The most streamlined option to generate such an actionable list is:

a) Export clinical and billing data to your enterprise data warehouse (EDW); extract data from the EDW into a Renal Cell Carcinoma registry database by using ICD-10 codes, and seek to interface this database with your EHR.

b) Generate a diagnosis grouper in your EHR with ICD-9 codes, and use that to create an EHR-based renal cell carcinoma registry.

c) Generate a diagnosis grouper in your EHR with ICD-10 codes, and use that to create an EHR-based renal cell carcinoma registry.

d) Generate a diagnosis grouper in your EHR with SNOMED codes, and use that to create an EHR-based renal cell carcinoma registry.

S02: Presentations – Population Health

Enabling Prospective Analysis of Care Redesign with reCAP, The REal-Time Care Analysis Platform

Michael Ripperger

S02-3:

A team of physicians and software developers are tasked with building and implementing a new clinical support tool. The support tool is being designed to decrease the time physicians spend on a complicated task. Once a month, the team meets to assess state of the software. So far, the software has been robust and has many features, but has been consistently critiqued for not being effective. The physicians report that using the software is time-consuming. What

28 | P a g e

would be the most effective change this team could make to reduce the time it takes to complete the complicated task?

a) Bring additional software developers on the team to speed up the existing software.

b) Meet more frequently and have physicians walk through how they use the tool.

c) Deploy the tool to other physicians and analyze uptake.

d) Scrap the software and reassess the usefulness of such a tool.

S03: Presentations - Telemedicine

Telemedicine to manage HIV patients within Pennsylvania Prisons

Jennifer Aldrich

S03-1: Which of the following best describes HIV telemedicine in the Pennsylvania prison system?

a) While geographically and temporally efficient, HIV telemedicine has not been shown to be cost efficient.

b) 62% of HIV-infected prisoners in Pennsylvania are from Philadelphia.

c) Telemedicine promotes continuity of care after release from prison.

d) The issue of EMR system incompatibility was fortunately addressed quickly and with satisfactory resolution by the IT teams of the two respective platforms at the beginning of the Temple/PA DOC partnership.


Feasibility, Acceptability and Impact of a Pediatric Teledermatology Mobile Health Application

Lindsay Berrigan

S03-2:

In this single group, prospective study of a store-and-forward pediatric teledermatology mobile application, which of the following was associated with increased user satisfaction?

a) Time to take photos within the app

b) User-reported ease of taking photographs

c) Receipt of a prescription

d) Time to receive a diagnosis from the dermatologist

29 | P a g e


Analysis of Abnormal Heart Rates Recorded when Patients were Invited to Upload Personal Fitness Device Data to the Electronic Health Record

Joshua Pevnick

S03-3: Patient-generated data, including patient-entered text in Open Notes, patient-reported outcomes, and biosensor data is increasingly prevalent. Challenges associated with incorporating this patient-generated data into the electronic health record include:

a) Continuously monitoring large numbers of patients’ electronic charts to detect abnormal data

b) Instantly presenting providers with all of the data generated by each of their patients

c) Understanding that abnormal findings are more likely to represent pathology than normal variation, user misunderstanding or error, or device error

d) Making providers responsible for identifying and responding to abnormal patient-generated data

S04: Panel -

Implementing EHR Patient Management Tools in the Pediatric Medical Home: Moving Toward Value-Based Care

Anthony Luberti, Elizabeth Brooks, Jonathan Crossette, Matt Dye

S04-1:

EMR-based patient registries and utilization reports should be used in which of the following ways to enhance care coordination and improve clinical outcomes:

a) Look for opportunities for revenue enhancement

b) Find opportunities to assist clinical staff by automating common processes

c) Rapidly identify gaps in care for specific populations and encourage targeted patient outreach

d) Locate patients who are over-using services

S04-2:

Having access to payer claims data in a hospital setting is beneficial because:

30 | P a g e

a) A health system can determine how much comparable hospitals charged the payer for their services

b) Payer claims allow a health system to examine patient utilization outside of their hospital and obtain a more complete picture of patient care and care gaps.

c) Payer claims give hospitals a way to validate activity in the hospital EMR and monitor how well providers are documenting encounter-related information.

d) Payer claims allow hospitals to analyze patient satisfaction at other health systems in order to gauge how satisfied their patients are compared to market.

S04-3:

Population Health tools, deployed correctly in the EMR, will enhance revenue by:

a) Creating improved access to care, increasing visit volume and thus revenue

b) Delivering point of care reminders for clinical services due, thus increasing “bill above” charges

c) Facilitating better and more efficient management of patient populations thus improving performance on payer quality incentive plans

d) Identifying and recruiting new patients into the practice, improving per member per month (PMPM) cap payments

S05: Presentation – Data Sciences

Effects of digoxin and diltiazem on mortality among atrial fibrillation patients

Shrie Raam Sathyanarayanan

S05-1:

Among various combinations of race and sex (e.g., Asian males) in this study, which group deviated from the trend of digoxin-associated mortality rates exceeding those for diltiazem?

a) Hispanic males

b) Asian females

c) Caucasian males

d) Hispanic females

31 | P a g e

S05: Presentations – Data Sciences

Estimating Data Requirements to Detect Pediatric Critical Decompensation

Melissa Aczon

S05-2:

Which of the following best describes the recommended strategy for improving the RNN model's performance in predicting critical decompensation of floor patients?

a) Increase the number of parameters to make the RNN larger and deeper.

b) Collect and use more data for training, including higher resolution time series.

c) Optimize the initialization points of the RNN's weight matrices using a separate validation set.

d) Use the GRU (gated recurrent unit) architecture instead of LSTM (long-short term memory).


Mining Care Pathways of Multimorbid Patients from Electronic Health Records

Yiye Zhang

S05-3:

This study proposes a new approach to incorporate medical costs explicitly in the chronologically ordered, clinical pathways (CPs) of patient experiences (Zhang, 2016). We analyzed CPs of patients with multiple chronic conditions using a cost-centered perspective, as well as a clinically focused perspective, to show alignment in some subgroups and significant variations in others. Which of the following best explains our finding, by comparing patient cohorts and their CPs categorized by the two approaches?

a) We found variations in costs among clinically similar patients, and variations in clinical complexity among patients with similar costs.

b) We found that there is a substantial overlap among patients who are relative low-spenders of medical services and patients who are clinically highly complex.

c) We found decisive evidence of excessive use of medical services by patients who were classified as having relatively low clinical needs.

d) We found exact overlap among the CPs of patients with medium level of medical spending and patients with medium clinical needs.

32 | P a g e

S06: Presentations – Clinical Decision Support

Development of a Web-based Decision Support Tool for Operationalizing and Optimizing Management of Hyperbilirubinemia in Preterm Infants

Yassar Arain

S06-1:

Current guidelines for hyperbilirubinemia treatment in premature infants are based on expert-consensus recommendations. There is wide variability in management of hyperbilirubinemia in this patient population, given the dearth of evidence-based recommendations. Our CDS tool, Premie BiliRecs, has operationalized expert-consensus recommendations by providing discrete Total Bilirubin thresholds for initiating phototherapy, based on the patient's Post-Menstrual Age. (The PMA is similar term for “gestational age,” referencing the infant. For example, a baby born at 27 weeks and 2 days who is 7 days old, has a PMA of 28 weeks and 2 days.) With EMR-integration, the greatest value of our CDS tool is in its ability to do which of the following?

a) Decrease use of phototherapy

b) Generate new data

c) Improve workflow

d) Standardize care

S06: Presentations – Clinical Decision Support Design and Implementation of a Real Time Acuity Detection System in a Pediatric ICU Eric Shelov S06-2: You are creating Clinical Decision Support for your patient acuity system. Your team has gathered historical data on a cohort of patients and has identified patients who would have been triggered by the CDS system and those who went on to deteriorate. You then categorized the patients and the triggers into a 2x2 table to look at true positives, false positives, true negatives and false negatives. Which one of the following should primarily drive the intrusiveness and strength of the recommendation embedded in the CDS?

a) Sensitivity

b) Specificity

c) Positive Predictive Value

d) Negative Predictive Value

33 | P a g e

S06: Presentations: Extracting the amount of alcohol consumption from clinical narratives using Natural Language Processing for use in clinical decision support tools Rajeev Chaudhry S06-3: Alcohol consumption by patients is documented as free text in the clinical notes narrative at your institute. Which of the following is a true statement regarding abstraction of the units of alcohol consumption from the clinical notes for a defined group of patients at your institute?

a) Extraction of alcohol consumption from the notes is easy as it is documented in discrete format

b) Chief complaint part of the notes is best to target for determining alcohol consumption

c) Use of alcohol consumption is needed for determining individualized risk for osteoporosis and bleeding for anticoagulants

d) Alcohol use is protective for heart and bone health

S07: Panel - Social Media Language Analysis for Healthcare

Lyle Ungar, Anneke Buffone, J. Eichstaedt, Hansen Schwartz

S07-1:

The Yelp™ website is comprised of user-generated content. Users post reviews and rate their experiences with businesses they have patronized, including health care providers and hospitals. Give the best answer concerning Yelp™ reviews and formal HCAHPS reviews of hospitals:

a) HCAHPS covers many topics that Yelp™ misses

b) Yelp™ covers many topics missed by HCAHPS

c) Each covers many topics the other misses

d) Both have similar coverage

34 | P a g e

S07-2:

A recent study by Eichstaedt et al. looked at the language expressed on Twitter to characterize community-level psychological correlates of age-adjusted mortality from atherosclerotic heart disease (AHD). Based on that study, how well did Twitter predict county-level heart disease rates, when compared to traditional methods?

a) About as well as demographics

b) About as well as health risk factors

c) About as well as income and education

d) Significantly better than all of the above combined

S07-3:

Which of the following is true of caregiver empathy?

a) Empathy is a clearly defined construct with a widely agreed upon definition for researchers across disciplines. It means putting yourself in the shoes of your patient.

b) The more empathic a caregiver is, the better patient outcomes. Caregivers who are more empathetic also tend to be more satisfied with their jobs, which positively affects their health and well-being.

c) Empathy has a widely agreed upon definition across academic disciplines. Social media language can identify empathic language accurately and provide insights into the emotional, cognitive, and behavioral realities of those higher and lower in empathy. We find empathy to predict higher scores for provider health, job satisfaction, and well-being and for patient satisfaction.

d) Empathy can have both positive and negative consequences for health and well-being of caregivers. There are at least two separate forms of empathy, one health-demoting where the caregiver takes on the patient’s feelings and cognitively merges with the patient, and one health-promoting where the caregiver has warm, tender, and concerned feelings for the patient, but does not cognitively or emotionally merge with the patient. Both of these can be detected in human language.

35 | P a g e

S08: Panel - Improving Access to Care and Efficiency: Patient-Centered Methods and Strategies

Bradley Doebbeling, Tammy Toscos, Kislaya Kunjan, Huanmei Wu

S08-1: Positive deviance, a strategy from complexity science, is a useful framework in designing system redesigns because it:

a) Helps identify approaches that work in your environment

b) Identifies people who are not trustworthy

c) Finds stakeholders who do not have time to participate

d) Has not been shown to work for multiple social, public health and clinical problems

S08-2: What was the most important function of the data warehouse in this PCORI project?

a) Patient Visit Planning/Morning Huddles

b) Improve Clinical Quality Measures

c) Support Scheduling Dashboards

d) Claims Reporting

S08-3:

What is the most significant challenge for health centers in terms of reporting and analytics?

a) Poor EHR Interoperability/HIE

b) Resource Constraints (Personnel, Capital)

c) Inefficient Clinic Workflow

d) High EHR Implementation Cost

36 | P a g e

S09: Presentations - Interoperability and Informatics Infrastructure

If (Integrating Data, Design, And Technology): Print(“Value Institute Design Lab”)

Rebecca Kowalski

S09-1:

Sepsis has become of increasing concern to public health in the past 30 years. This is due to the high cost of hospitalizing patients with sepsis, which accounted for $20 billion of total hospital costs in 2011, and its high mortality rate, ranging from 25-50% depending on severity. Because of this, people are working hard to find a standard way of defining and responding to sepsis. Although this has not yet been achieved, CMS does have guidelines on what defines sepsis.

Which of the options represents the current CMS guidelines for sepsis?

a) Suspected Infection

b) Meeting two SIRS (Systematic Inflammatory Response Syndrome ) criteria + suspected infection

c) Organ failure

d) Meeting two SIRS (Systematic Inflammatory Response Syndrome ) criteria + suspected infection + organ failure

S09-2:

We've established that sepsis is a problem and a financial burden for hospitals. The expense mainly comes from hospitalization for cases of severe sepsis, but how prevalent is severe sepsis? What is the national estimate of severe sepsis according to a study done on data from 2001-2004?

a) 0.05%

b) 0.007%

c) 0.7%

d) 1.2%

37 | P a g e


Leveraging search patterns in electronic health records to make information retrieval more efficient

Titus Schleyer, Xia Ning, Martin Doug

S09-3:

Typically, the information retrieval workflows in electronic health records (EHR) are not oriented around clinical conditions, forcing clinicians to retrieve relevant data from a variety of places in the EHR. Thus, physicians must spend significant amounts of time retrieving and synthesizing data from EHRs. Which of the following EHR features can help address this problem?

a) clinical decision support algorithms

b) overview screens showing the most recent information about the patient

c) disease-specific dashboards

d) Google-like search

S09-4:

Both custom-designed, condition-specific dashboards and recommender systems can make information retrieval from EHRs more efficient. Recommender systems analyze past search and information retrieval patterns in EHRs to identify information that may be relevant to a clinician’s review of a patient case. Compared to dashboards, recommender systems:

a) may cause clinicians to miss important data.

b) are laborious to implement for multiple conditions and diseases.

c) always produce predictable results.

d) require medical expert opinion for design.

38 | P a g e

S10: Panel – Immediate Adaptability

Yalini Senathirajah. Andre Kushniruk, Jon Patrick, Ross Koppel, Elizabeth Borycki

S10-1:

The most important value of Immediate Adaptability is:

a) Eliminate workarounds in workflow

b) Prevent workflow from being altered by staff

c) Ensure that a system is optimal for the community of practice

d) Abandon traditional EMR technologies

S10-2:

The best method for determining user needs in immediate adaptability (for making changes to systems) is:

a) Rapid-low cost usability testing

b) Clinical simulations

c) Workflow analysis

d) Observation

S10-3:

You are in an area of healthcare in which several things are happening simultaneously as a result of fiscal and technological change. These include the need to incorporate many new sources of information, such as app data from patients, feeds from other institutions, algorithmic approaches to decision support based on population statistics, and the need to present information rapidly in a way doctors can grasp quickly. At the same time concerns about technology induced errors are increasing.

In this type of situation one of the main most important advantages of immediate adaptability are:

a) The architecture allows rapid incorporation of new information sources and visualizations

b) User control of an adaptable system can allow new insights

c) It requires increased thinking and design in creation of the system, but lower costs of change and redevelopment/new creation

d) There are no particular advantages

39 | P a g e

S11: Presentations – Health Policy and Payment Reform

Decision Modeling of Clinical Content for Collaborative Documentation Purposes in the Management of Chronic Conditions

Hari Nandigam

S11-1:

Clinical Informaticians at a hospital wanted to develop clinical content for Emergency Department. They wanted to store their information in HL7 standard resource model. What is the best approach in their modeling process for their content development?

a) Conceptual, Physical, Logical

b) Physical, Logical, Conceptual

c) Conceptual, Logical, Physical

d) Logical, Conceptual, Physical


An Informatics Framework for Value-based Care at NewYork-Presbyterian

Gilad Kuperman

S11-2:

A health care provider organization is planning to enter into a value-based care contract with a commercial payer. The payer will reward the provider for decreasing the total cost of care for the payer’s beneficiaries that receive the bulk of their care from the provider.

Which data resource should the provider organization use to identify the population of patients for which it will be responsible under the contract and could benefit the most from expensive care management interventions?

a) The provider organization’s clinical database

b) An epidemiologic database provided by the local public health department

c) A database of claims transactions provided by the payer for all patients that had care at the provider

d) The Death Master File provided by the Social Security Administration

e) The Medicare claims database

40 | P a g e

S11: Presentation – Health Policy and Payment Reform

Re-Identification Risk in HIPAA Safe Harbor De-Identified Datasets: Using non-HIPAA identifiers / the case of the MVA Attack

Peter Elkin

S11-3:

You are an ER physician when a patient is brought in with life-threatening injuries from motor vehicle accident involving a drunk driver. After the patient has been stabilized, your third year medical student, who diligently obtained information about the accident from the EMTs, remarks that the accident is 'all over the news.' You review the medical student's note and are concerned that there is a risk that it could be used to re-identify the patient if released in the future even if HIPAA identifiers are removed. What information is it most important for you to redact from the note to protect patient privacy while preserving the value of the anonymized note for research?

a) The time of day (hour and minute) the accident took place

b) The name of the hospital to which the patient was taken

c) The model of car the patient was driving

d) The ICD-10 diagnosis codes for the patient's injury

S12: Presentations – Analytics and the Learning Health System

Predictive Analytics for Pediatric Diabetic Keto-Acidosis (DKA) using Features Extracted from Asynchronous Multivariate Data Streams

Rema Padman

Extracting predictive features from asynchronous multivariate data streams for early prediction of diabetic ketoacidosis (DKA) in pediatric type 1 diabetes patients is a challenging problem. This study proposes a smoothing technique for segmenting measurements collected at irregularly spaced time points into trend and value abstractions that can be added as new features to standard prediction models.

Suggested by this study, what benefit can be obtained from building prediction models that include trend and value abstractions from asynchronous multivariate EHR data streams?

a) Confidence intervals for DKA prediction b) Improved prediction accuracy c) Identification of the best interventions for an individual patient d) Identification of patient clusters that may benefit from an intervention

41 | P a g e


Identifying Patterns of Co-occurring Medical Conditions through Topic Models of Electronic Health Records

Moumita Bhattacharya

As discussed in the abstract, we represent each patient file as a probability distribution over SNOMED codes, where each such distribution is referred to as a topic. Each distribution carries certain information contents indicating how informative the associations among codes are, where a uniform distribution, which assigns an equal probability to all codes, carries the lowest information contents.

Each of the three graphs shown below plots a probability distribution over 180 SNOMED codes, thus each representing a topic, denoted Topic A, Topic B and Topic C. The x-axis of each plot corresponds to the 180 codes, while the y-axis shows the conditional probability per code to occur in the respective topic.

Which of the three topics shown below provides the most information about likely associations among conditions?

42 | P a g e

Page 43 of 174

a) Topic A

b) Topic B

c) Topic C

d) None of the above. All topics show the same information contents.


Developing and Implementing a Reliable and Validated Solution for Big Data Text Analytics (NLP, Text Mining) at a Tertiary Pediatric Hospital. Luis Ahumada, William Nieczpiel, John Martin In our proposed approach, we have combined techniques from several areas, including NLP, knowledge engineering, and machine learning to implement an automated framework for annotating physician notes. Additionally, as an immediate benefit, the produced framework facilitates the retrieval of notes on demand, and allows the collection of empirical domain knowledge that can be formalized with the help of an ontology to also permit reasoning. Given that all the following tasks are part of the general Text Analytics framework, which group of tasks is more appropriate to produce more informative features from physician notes using an NLP engine?

a) Text categorization, document clustering and information retrieval

b) Document summarization, information extraction, question answering

c) Ambiguity, data sparsity, redundancy

d) Bootstrapping, labeled and unlabeled data

Page 44 of 174

S13: Panel - Should Drug-Drug Interaction Seriousness Ratings Be Taken Seriously? Interactive Panel

Scott Nelson, John Horn, Joan Kapusnik-Uner, Bimal Desai, Richard Boyce

S13-1:

Which of the following organizations has drug-drug interaction clinical significance code value set for drug utilization reviews (DUR) messaging?

a) HL7 - Health Level-7

b) NCPDP - National Council for Prescription Drug Programs

c) PQA – Pharmacy Quality Alliance

d) AHRQ - Agency for Healthcare Research and Quality

e) NIST - National Institute of Standards and Technology

S13-2:

Which one of the following statements does not apply to the implementation of patient specific DDI alerts?

a) The total number of alerts triggered per patient will not be affected.

b) Alert filters can be adjusted to reflect the requirements of the user.

c) The ordering of an interacting drug pair will trigger an alert each time the pair is ordered.

d) Common filters include drug doses, routes of administration, and laboratory values

Page 45 of 174

S13-3: Beth is reviewing the physician’s discharge order for Maria. Maria is a 72-year old woman who was admitted to the hospital with acute decompensated heart failure. While reviewing Maria’s medications, Beth sees an alert that informs her that Maria is being discharged with spironolactone, a potassium-sparing diuretic that could potentially interact with the potassium chloride that Maria had been taking at home to treat low potassium levels. The alert also notes that Maria is being discharged with an ACE inhibitor lisinopril for heart failure. Which of the following is most appropriate additional information that the alert should provide to help Beth ensure that Maria's medication therapy is safe and effective?

a) That the potential interaction is considered ""serious"".

b) That the concomitant exposure to potassium chloride and spironolactone increases Maria’s risk of hyperkalemia

c) That ACE inhibitors can also increase potassium levels.

d) The specific lab results, concomitant medications, medication dosing, and health factors that increase or mitigate the likelihood that Maria will experience hyperkalemia, and how to manage the interaction such as by discontinuing or changing the dose of one of Maria’s medications

S14: Panel – Data and Network Security – What You Need to Know!

Paul DeMuro, John Rasmussen, Cathy Beech

S14-1:

When conducting a breach risk assessment, what threshold must be assessed to consider a disclosure a breach?

a) High risk of harm

b) Patient safety risk

c) Low probability of compromise

d) Excessive risk of disclosure

S14-2:

Which of these controls represent a “physical safeguard” under HIPAA?

a) Audit logging

b) Identity badge policy

c) Anti-virus

d) Security cameras

Page 46 of 174

S14-3:

Which of the following represents the BEST strategy for recovering from a ransomware infection?

a) Tape backup

b) User education

c) Anti-virus

d) Data loss prevention

S14-4:

The HIPAA Privacy Rule

a) Provides state protections for individually identifiable health information held by covered entities and their business associates.

b) Requires that covered entities and their business associates perform a risk assessment.

c) Provides federal protections for individually identifiable health information held by covered entities and their business associates.

d) Requires implementation of security measures that can help prevent the introduction of malware, including ransomware.

S14-5:


a) Protects all health information wherever it is found.

b) Does not protect all health information wherever it is found.

c) Applies to health information about an individual that has been de-identified.

d) Requires covered entities and business associates to implement policies and procedures that can assist an entity in responding to and recovering from a ransomware attack.

Page 47 of 174

S14-6:

The HIPAA Security Rule does not

a) Require that covered entities and their business associates perform a security risk assessment.

b) Require covered entities and business associates to implement policies and procedures that can assist an entity in responding to and recovering from a ransomware attack.

c) Require training users on malicious software protection.

d) Extend to all wearable fitness trackers.

S15: Presentations – Care Coordination

Integrating mHealth medication reconciliation and symptom reporting for patient-centered ambulatory care

Lisa Grossman, Rui Sim, Ruth Masterson Creber

S15-1:

How does electronic symptom assessment and tracking benefit patients?

a) Enables participating patients to more consistently track and monitor their symptoms

b) Engages patients with low health literacy in symptom tracking

c) Engages a greater percentage of patients in tracking their symptoms

d) Engages patients with low technology literacy in symptom tracking

Theresa Cullen [email protected]


Developing an electronic care plan to improve longitudinal care coordination in chronic kidney disease

Jenna Norton

S15-2:

A healthcare system wants to improve information sharing during transitions of care in chronic diseases. They decide to implement an electronic care plan to facilitate sharing of key patient information across providers within the system. They elect to focus their initial efforts on

mailto:[email protected]

Page 48 of 174

chronic kidney disease (CKD). Which of the following factors best support their selection of CKD for initial efforts to implement an electronic care plan?

a) CKD is defined primarily by objective laboratory data, requires interdisciplinary management including a variety of provider types, and is subject to frequent transitions of care.

b) CKD patients are cared for only by nephrologists and primary care providers, so the care plan would only need to be implemented within a few settings.

c) CKD has few comorbid conditions, limiting concerns regarding reconciliation of data between CKD and other chronic conditions.

d) All data elements relevant to CKD care have existing standards to enable electronic sharing of data in a consistent fashion.


Reducing Readmission and Post-Hospital Mortality through Data beyond Your Borders

Philip Smith

S15-3:

With a national drop in hospital 30-day readmissions,

a) Overall readmission penalties under CMS have decreased.

b) Less hospitals are receiving fines for Readmission penalties.

c) The percent hospitals fined, the total fines, and the percentage of fines are increasing.

d) Predictive analytics are the de facto standard for determining an individual’s readmission and mortality outcomes.

Page 49 of 174

THURSDAY, MAY 4

S16: Panel - EHR 2020 After 2 Years: What Have We Done For You(r EHR) Lately?

Sarah Corley, Theresa Cullen, Gil Kuperman, Thomas Payne, Charlene Weir

S16-1:

The Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs was designed to

a) Identify near term strategies to address EHR challenges

b) Propose changes to software design and style sheets

c) Identify CMS specific changes in reimbursement that reflected the impact of EHRs

d) Support the development of a comprehensive learning health care system

S16-2:

Recommendations on the person centered care delivery model in the AMIA 2020-EHR Task Force include

a) Incorporating social determinants of health in the care record

b) Limitations on patient entered data into the care record

c) Sharing of school health records within the HIT system

S16-3:

Transparency to aid selection of an EHR, as discussed in the EHR 2020 report, includes:

a) Posting recordings of the process of demonstrating that a product meets the MU functional objectives.

b) Sharing source code of vendor EHRs.

c) Interviewing physicians and patients to learn their satisfaction with EHRs and publishing results.

d) Permitting the public to view all screens seen by a practitioner using the EHR during a physician visit.

Page 50 of 174

S16-4:

Regulation of EHR certification, use, and functionality, according to the EHR 2020 report…

a) Should be changed but not abandoned.

b) Plays no further constructive role in national efforts to improve use of EHRs.

c) Should be extended to cover personal health tracking devices.

d) Is key to advancing innovation in EHRs and supporting technologies.

S18: Panel - Clinician Engagement in Informatics: The Experience of the Pediatric Residency NERD Squad at the Children’s Hospital of Philadelphia

Mark Mai, Evan Orenstein, Ivor Asztalos, Anthony Luberti, George Dalembert

S18-1:

Why is it important to combine didactic education along with a self-directed project when crafting an informatics elective for a trainee?

a) The project can tackle a lingering task on your queue

b) The project reinforces principles learned in the didactics

c) Both are needed to meet a minimum amount of time commitment

d) A tangible project is something the student can put on their resume

S18-2:

As a medical educator, you would like to teach interns about pitfalls to avoid when using an EHR, such as medication errors and failing to identify concerning clinical trends. Which of the following educational activities can best teach effective use of the EHR in the clinical setting to identify common EHR error types?

a) Informatics simulation cases

b) Passive lectures

c) Flipped classroom

d) Case reports

Page 51 of 174

S18-3:

While you are attending on the general medicine service, one of your residents mentions the challenges and the time needed to place orders for a patient admitted for asthma. She suggests that the hospital should design an order set containing appropriate orders that would streamline the front line clinician workflow. The resident has demonstrated a penchant for identifying workflow challenges and appropriate informatics interventions. Which of the following is the most appropriate next step of action?

a) Construct the order set for the resident

b) Tell the resident to submit an information services ticket

c) Inform the resident that there are too many requests and not enough analysts to complete the project

d) Discuss with the resident her ideas and offer to provide further resources and mentoring to complete the project

S19: Panel - Optimal Use of Health IT Requires Competency of Clinicians and Patients William Hersh, Michelle Troseth, Susan Woods S19-1:

Which of the following competencies in clinical informatics is most appropriate for a healthcare provider?

a) Analyze a genomic data set to look for clinical correlations

b) Establish a regional health information exchange organization

c) Perform SQL queries in a patient data warehouse

d) Retrieve all relevant data from a patient record in an EHR

Page 52 of 174

S19-2:

Which of the following have been shown to positively affect patient trust and loyalty with primary care providers?

a) Lower co-pay amounts for visits

b) Provider offering secure email

c) Provider offering secure email and sharing clinic notes online

d) Provider sharing clinic notes online

Page 53 of 174

S20: Presentations: Safety and Quality

Reduced Rates of Hospital-Associated Clostridium difficile Infection Associated with a Clinical Surveillance System

Stephen Ross, Sue Ie, Sharad Manaktala, Justin Clark

S20-1:

On the Hospital Compare site, your hospital's standardized infection ratio (SIR) for hospital-acquired Clostridium difficile infection is significantly higher than other hospitals in your region. In a presentation to the CEO and CIO of your hospital, you are justifying investment in an automated surveillance system to assist your antimicrobial stewardship and infection control teams. Which of the following statements most accurately reflects the current evidence for such systems?

a) Although not required for all hospitals, implementation of an automated surveillance system for antimicrobial stewardship and infection control is strongly endorsed in the proposed new "conditions of participation" from the Center for Medicare and Medicaid services.

b) There is currently no evidence that automated surveillance systems reduce the use or expense of antibiotics that may contribute to hospital-acquired Clostridium difficile infection.

c) Without automated surveillance systems in place, antimicrobial stewardship programs have not been demonstrated to reduce rates of hospital-acquired Clostridium difficile infection.

d) Clinical trials establish that automated surveillance systems reduce rates of hospital-acquired Clostridium difficile infection.

S20: Presentations – Safety and Quality

Making Acute Care More Patient-Centered: Implementing a Learning Lab

Theresa Fuller

S20-2:

This Patient Safety Learning Laboratory combines multiple units and clinical services in the implementation of a suite of patient- and provider-facing tools all geared toward improving patient safety. There are benefits and drawbacks for each element in being part of such a large implementation. This shows through in the “stepped wedge” design of the project (where units/services are randomized into “steps,” and each step goes live incrementally). This design

Page 54 of 174

is intended to reduce the temporal, contaminating effects of a single go-live date. We learned that it also comes with a series of challenges.

Which of these was the most significant learning of our experience in mitigating those challenges?

a) Holding to your original go-live dates

b) Defining your “step” design with input from unit leadership

c) “Rolling out” all tools together

d) Maintaining a constant physical presence from the research team on the units


First Contact Provider (FCP): Improving Inpatient Critical Results Reporting

Anisha Chandiramani

S20-3:

Reporting critical results of tests and diagnostic procedures in a timely manner is a Hospital National Patient Safety Goal developed by The Joint Commission. The objective of the patient safety goal is to “…provide the responsible licensed caregiver these results within an established time frame so that the patient can be promptly treated.”* The University of Chicago Medicine (UCM) was not meeting its goal of timely reporting of critical lab values to Licensed Independent Practitioners (LIPs).

What did UCM implement to meet this Joint Commission goal?

a) An EPIC-based way to enter, store, and display each inpatient’s primary provider, called the First Contact Provider (FCP)

b) A tool in Epic for a Licensed Independent Practitioner (LIP) to dictate urgent treatments and place orders

c) A written procedure for managing the critical results of tests and diagnostic procedures

S21: Panel - Optimizing Strategies for Clinical Decision Support

Marianne Hamilton Lopez, James Tcheng, Jonathan Teich, Blackford Middleton, Scott Weingarten, Kensaku Kawamoto

You have just been elected to be the Chair of Informatics of your professional medical association. Prior work has confirmed that only about half of the patients in the United States receive care according to the evidence-based best practices recommended by your medical

Page 55 of 174

association. You have been asked by the leadership of the medical association to attend the AMIA iHealth meeting and to identify the challenges and potential solutions for leveraging clinical decision support (CDS) to ensure that all patients receive the best possible care based on your medical association’s evidence-based practice guidelines.

S21-1:

What is one of the largest barriers to widespread dissemination of evidence-based practice guidelines using CDS?

a) There is generally no room for improvement – clinicians by and large already follow evidence-based care guidelines almost all of the time.

b) Fee-for-service payment models provide little incentives for higher quality, evidence-based care.

c) Electronic health record systems generally do not have mechanisms for providing CDS to its end-users.

d) Electronic medical records already provide the best evidence.

S21-2:

What is a best practice for ensuring that CDS has the intended impact?

a) Do not measure impact; CDS interventions always have the desired impact.

b) Provide the right information, to the right person, in the right CDS intervention format, through the right channel, and at the right time in workflow (CDS 5 Rights).

c) Do not worry about the volume or accuracy of alerts – “alert fatigue” is not something the vast majority of clinicians’ experience.

d) Have every clinician see the same level of alerts so that nothing is missed.

S21-3:

What is one reason we do not yet have widespread standards-based interoperability for CDS?

a) There are simply no interoperability standards available in the CDS arena.

b) Major electronic health record (EHR) vendors provide no support for standards-based interoperability in general.

c) We are still in early stages of EHR vendor adoption of CDS interoperability standards.

d) The industry has already widely adopted standards-based interoperability for CDS?

Page 56 of 174

S21-4:

You are tasked with the responsibility of implementing a clinical decision support rule in your EHR system. The EHR “Meaningful Use” regulations stipulate the use of which of the following expression languages for clinical decision support?

a) Arden Syntax

b) CQL (Clinical Quality Language)

c) GELLO

d) No specification in the regulations

S22: Presentations – Bridging Analytics, Bedside Care, and Education Granular ED Throughput Data: Helping Providers Find Agency in Process Improvement Tom Spiegel S22-1: One of the many benefits of the Electronic Health/Medical Records is the collection and storage of healthcare data that allows for data mining that in part fuels quality improvement. Our QI project was born out of recognition that "patient length of stay" and "relative value units" were too ambiguous and therefore not sufficiently motivating to our residents. Unlike our "purple time," they were unable to provide the insight we needed to improve performance. What underlying principle of data granularity is illustrated by this effective use of "purple time”?

a) Granulated data are more easily retrieved than aggregate data

b) Data are most helpful when they are appropriately granular and defined to a level of detail specific to your needs

c) Granular data have been shown to improve accuracy of medical billing and coding

d) Increasingly granulated data hinder EHR interoperability

S22-2: Which of the following is considered a “Visual Analysis Best Practice” as discussed in the presentation:

a) Select creatively non-meaningful colors to your data

b) Avoid cultural significant colors when making your palate choices

c) When color choices are not obvious, ensure you use label or a clear legend

d) Avoid adding color to more than 10 distinct values

Page 57 of 174

S22: Presentation: Bridging Analytics, Bedside care, and Education:

Manual, automated, or derived measures: The value of variability in the meaningful use of vital sign data

Keith Feldman

S22-3:

Which of the following is an important consideration for researchers when examining vital sign data procured from the electronic medical record?

a) Vital sign data procured from the standard electronic medical record, unlike vital sign data procured from paper medical records, more consistently identifies patient deterioration.

b) B. Vital sign data procured from the electronic medical record, unlike vital sign data procured from paper medical records, are less likely to include erroneous values.

c) The use of vital sign data procured from the electronic medical record improves study validity, as data recordings are made electronically and at set intervals, thereby removing provider bias.

d) There may be greater variability in vital sign data procured from the electronic medical record compared to data procured from paper medical records, because a broader range of methods may be used to collect and record the data.

S22-4:

Which of the following methods for collecting heart rate values from a patient is associated with greatest variability of values across standard deviation and range?

a) Manual collection of discrete values (i.e.: using a stethoscope) by a provider, at standard intervals, with manual documentation into the medical record.

b) Manual selection of values by a provider, at standard intervals, from values generated by a bedside ECG monitor, with manual documentation into the medical record.

c) Manual validation of electronically documented values, imported into the medical record from a bedside ECG monitor, at standard intervals.

d) Electronic documentation of waveform data, imported into the medical record from a bedside ECG monitor.

Page 58 of 174

S23: Presentations – Workflow

Prototyping The Future of CPOE: Starting With Indications

Aaron Nathan

S23-1:

Which of the following is a significant benefit of indications-based prescribing?

a) The indication will always be put on the medication label

b) Drug costs will decrease when indications are included

c) Patients will know the reason why they are taking each of their medications

d) Brand name drugs will get selected more often

S23: Presentations – Workflow Design and Application of a Bariatric Ambulatory Workflow for the Pre-operative Evaluation of Bariatric Surgical Candidates Tatyan Clarke S23-2: The Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSAQIP) encourages standardization in the evaluation and management of Bariatric patients. To this end, our Bariatric Ambulatory Workflow provides a system to compile the elements of this complex work-up for each patient. Which of the following describes additional benefits of this form of data capture?

a) It dictates the care of each patient, ensuring every patient has an identical evaluation.

b) It guides each provider to review the detailed results of each element of the evaluation at every encounter.

c) The workflow can be queried to assess outcomes and guide quality-improvement strategies.

d) The content of the flowsheet directly bills for each element of the evaluation.

e) The flowsheets auto-populate all parts of the progress note, making documentation effortless.

Page 59 of 174

S23: Presentations - Workflow

Leveraging Advanced Physician Documentation for Improving Quality, Accuracy, and Compliance

Kang Hsu

S23-3:

Coders can only code from:

a) Nursing Notes

b) Radiology Reports

c) Physician Documentation

d) Laboratory Reports

S24: Panel - A Proposed National Research and Development Agenda for Population Health Informatics: Case Studies and Future Directions

Hadi Kharrazi, Steve Fihn, William Yasnoff, Bharat Sutariya, Jonathan Weiner, Aneel Advani

S24-1:

Which specification is more accurate about the “Population Health Informatics” (PopHI) domain? (based on https://www.ncbi.nlm.nih.gov/pubmed/27018264)

a) The focus of PopHI solutions is on the treatment of individual patients at the point of care

b) The key stakeholders of PopHI include providers and exclude payers and local communities

c) The main action arm of PopHI activities is the federal government (e.g., CDC) and excludes provider or payer organizations

d) The common intervention target of Pop HI is large population denominators attributed to an entity or geographic boundary

S24-2:

Page 60 of 174

Which of the following items are identified as one of the major challenges for “Population Health Informatics” (PopHI) research and development? (based on https://www.ncbi.nlm.nih.gov/pubmed/27018264)

a) Extensive use of patient identification methods to link medical, public health, and social service data systems

b) Alignment of individual and population-based measures for ACOs and PCMHs

c) The lack of a clear vision and lack of a shared model for a PopHI infrastructure

d) Lack of variability of data sources and data patterns in population health

S24-3:

Which of the following items are identified as an emergent “Population Health Informatics” (PopHI) research and development agenda? (based on https://www.ncbi.nlm.nih.gov/pubmed/27018264)

a) Developing information solutions to integrating PopHI at the point of care

b) Developing a scientific evidence and knowledge base across different sectors affecting population health

c) Developing new ideas to merge genomic data with clinical data

d) Developing a market strategy to translate PopHI findings into business ideas

S25: Panel - CMIO + CNIO + CIO Leadership: Improving Outcomes at the System Level

Bimal Desai, Kisha Hawthorne, Kimberly Burress, Diane Humbrecht, Colleen Saul, Jon Sternlieb

S25-1:

You are a provider and your medical center is facing increasing financial pressures. Payers are negotiating lower contracted rates, and your organizations costs are going up. Leadership is advocating cuts to staff that would lead to more work by providers. Which is the best option?

a) Accelerate quality reporting initiatives, despite increased costs, to improve contract negotiation strength.

b) Agree to staffing cuts with a plan to wait until the current financial challenges pass

c) Contract only with large insurance companies to ensure higher coverage for the most commonly billed physician visits

d) Cut back on expansion plans that would have increased the population of patients who receive care at the medical center

https://www.ncbi.nlm.nih.gov/pubmed/27018264

Page 61 of 174

S25-2:

You are an informatics leader and your medical center has acquired several new practice sites and a new hospital has joined the network. Everyone is talking about improving clinical integration; however, it is likely that all sites will eventually migrate to the same EHR platform. Since the change to a single platform is expected to take three years, which option appears best as an interim strategy for clinical content access?

a) Apply secure WhatsApp for communication across the enterprise for easy communication.

b) Deploy targeted FHIR applications to share key clinical data.

c) Perform a standard ETL (extract, transform, load) process so all information becomes easily available in a shared repository.

d) Use auto-fax to distribute all visit notes across all sites to improve communication.

S25-3:

Your organization, one of several in a city, is contemplating a telemedicine initiative with a goal of significantly reducing in-person visits at your medical center given the existence of newly supportive contacts in this regard. Which approach appears both most cost-effective and likely to achieve this goal?

a) Develop and place kiosks for real-time health management at the local airport, and at central bus and subway stations.

b) Identify high volume low, complexity conditions and develop algorithms to manage conditions asynchronously through the existing patient portal.

c) Increase the call center personnel with staff who have at least a community college degree to cut down on wait times for returned calls.

d) Outsource after-hours visits to management by a telemedicine company that has received a high rating from the Wall Street Journal.

S26: Data and Network Security

A New Approach to Patient Privacy Monitoring using Machine Learning

Daniel Fabbri

S26-1: Which of the following best describes access control policies in modern electronic medical record systems?

Page 62 of 174

a) Provide too narrow access rights

b) Provide too broad access rights

c) Correctly limit access

d) Provide no access

S26-2: Employees have used their EMR access rights to inappropriate access patient data in the past. Which of the following best describes a process to detect one-off inappropriate access?

a) Manual review of all accesses

b) Credit card fraud system

c) Statistical anomaly detection systems

d) Context-based access monitoring systems

S26-3: The explanation-based auditing system discovers what type of pattern from the EMR database?

a) Medical appointment patterns

b) Access frequency patterns

c) A connection between the patient and the employee accessing the patient’s record using EMR data.

d) Accesses from employees to neighbor records.

Page 63 of 174

Answers: TUESDAY, MAY 2 WS01: Workshop – Clinical Knowledge Management Dominik Aronsky, Dirk Wenke, Asli Ozdas-Weitkamp WS01-1: What is a logical first step in starting a clinical knowledge management program within the iterative knowledge management framework?

a) Create a catalog of existing knowledge assets including relationships with metadata

b) Build standards-based interoperable knowledge assets

c) Centralize the process of creating knowledge assets

d) Establish a knowledge asset lifecycle

Answer: a) Create a catalog of existing knowledge assets including relationships with metadata Explanation: An initial assessment of all institutionally available knowledge assets is desirable to understand the level of existing knowledge, identify redundancies, identify gaps, and responsibilities. Reference Greenes RA, ed. Clinical Decision Support: The Road to Broad Adoption. 2nd ed. Waltham, MA: Elsevier; 2014. WS01-2: What activities are least likely within the scope of a clinical knowledge management framework?

a) Integrate knowledge assets into clinical workflow

b) Build a sustainable knowledge asset framework with reusable components

c) Maintain the representation of knowledge assets

d) Provide knowledge lifecycle support

Answer: a) Integrate knowledge assets into clinical workflow

Page 64 of 174

Explanation: Knowledge management deals with the creation and maintenance of institution knowledge assets. This includes sustaining the knowledge assets, maintaining the representation and providing strict lifecycle support. The aspects of workflow integration are commonly not represented in a clinical knowledge management environment. Reference Greenes RA, ed. Clinical Decision Support: The Road to Broad Adoption. 2nd ed. Waltham, MA: Elsevier; 2014. WS01-3: Which topic is least likely associated with clinical knowledge management?

a) Institutional data governance

b) Lifecycle management of knowledge assets

c) Management of knowledge meta-data

d) Single-login management for clinical workstations

Answer: d) Single-login management for clinical workstations Explanation: Single-login management is least likely to be modeled in a clinical knowledge management environment. The other topics are typical KM activities. Reference: Greenes RA, ed. Clinical Decision Support: The Road to Broad Adoption. 2nd ed. Waltham, MA: Elsevier; 2014. WS01-4: Which process is not directly supported by a clinical knowledge management system?

a) Centralized request management for changes in the knowledge base

b) Centralized prioritization of knowledge management implementation tasks

c) Monitoring clinicians’ usage of clinical decision support systems

d) Versioning of knowledge assets

e) Auditing of knowledge asset changes

Page 65 of 174

Answer: c) Monitoring clinicians’ usage of clinical decision support systems Explanation: Once a knowledge asset, such as a rule, an order set, or pathway is implemented for operational use in the target system, its monitoring is accomplished in other environments, such as the target system itself, a business intelligence (BI) or enterprise data warehouse (EDW), or similar. Reference Greenes RA, ed. Clinical Decision Support: The Road to Broad Adoption. 2nd ed. Waltham, MA: Elsevier; 2014. WS01-5: Which characteristic(s) is/are considered best practice(s) for creating knowledge assets?

a) Modular

b) Reusable

c) Interoperable

d) All of the above

Answer: d) All of the above. Explanation: All three characteristics are desirable and considered best practices Reference Greenes RA, ed. Clinical Decision Support: The Road to Broad Adoption. 2nd ed. Waltham, MA: Elsevier; 2014. WS01-6: Which of the following is a/are critical method(s) applied in clinical knowledge management?

a) Formal representation of knowledge assets

b) Versioning of knowledge assets

c) Terminology models as building blocks

d) All of the above

Page 66 of 174

Answer: d) All of the above. Explanation: All three techniques are - among others - considered important for a successful and sustainable knowledge management strategy.

Reference Greenes RA, ed. Clinical Decision Support: The Road to Broad Adoption. 2nd ed. Waltham, MA: Elsevier; 2014.

WS02 Workshop - Understanding and communicating risk: Critical competencies for the healthcare workforce

Greg Nelson, Monica Horvath

WS02-1:

A medical provider is reviewing the literature to understand the risk of colon cancer given different socio-economic backgrounds. However, the risk expressions are not identical among these resources. Of the following conclusions, which one has the greatest stated risk of colon cancer?

a) One out of 21 men will develop colon cancer in their lifetime.

b) 4.4% of women will develop colon cancer in their lifetime

c) 41 out of 1,000 men and women will develop new colon cancer cases each year

d) One out of 42 women will develop colon cancer in their lifetime

Answer: a) One out of 21 men will develop colon cancer in their lifetime.

Explanation:

1/21 is equivalent to 4.7% and 41 out of 1,000 is equivalent to 4.1%. (a) is the highest.

Reference:

Woloshin, S. Know Your Changes: Understanding Health Statistics. 1st ed. University of California Press; 2008.

Page 67 of 174

WS02-2:

A randomized controlled trial monitored 500 elderly (>65 years of age) individuals with periodic instances of vertigo for one year. By the end of one year, 125 had reported an injury requiring medical care occurring while walking, moving, or otherwise attempting to adjust their physical position. What are the odds of reporting an injury during this study?

a) 18%

b) 25%

c) 33%

d) 75%

Answer: c) 33%

Explanation:

Odds are calculated as the number of people in whom the outcome occurred divided by the number of people in whom the outcome did not occur

Reference:

Hancock M, Kent P. Interpretation of dichotomous outcomes: risk, odds, risk ratios, odds ratios and number needed to treat. J. Physiother. 2016;62(3):284-7

WS02-3:

For the case in question #2, what is the incidence of patients with vertigo during the study period?

a) 18%

b) 25%

c) 33%

d) 75%

Answer: b) 25%

Explanation:

The incidence is the proportion of monitored individuals in whom the outcome of interest occurs

Page 68 of 174

Reference:

Hancock M, Kent P. Interpretation of dichotomous outcomes: risk, odds, risk ratios, odds ratios and number needed to treat. J. Physiother. 2016;62(3):284-7

WS02-4:

The prevalence of a specific disease is 0.5% of the population. A new test accurately detects the disease 90% of the time, but falsely identifies 5% of healthy people as having the disease. What is the probability of seeing a true positive test result in a clinic?

a) ~ 8%

b) ~ 9%

c) ~ 85%

d) ~ 92%

Answer: a) ~ 8%

Explanation:

P(Disease | Screen Positive) = P(Screen Positive | Disease) * P(Disease)/ P(Screen Positive)

The small number of false positives in the large population is greater than the majority of true positives in the smaller subpopulation that actually have the disease.

Reference:

Operskalski JT, Barbey AK. Risk literacy in medical decision-making. Science. 2016;352(6284):413-4

WS02-5: Select the statement below that represents a common misperception regarding interpreting and using risk scores.

a) Interventions may be too costly or niche in nature to justify care management resources even if high risk

b) Patient risk score is a prediction of the likelihood of a patient to be in a certain situation in the future

c) Patients with the highest readmission risk scores are those most impactable and thus deserving of care management interventions

d) Seeing ROI from an intervention guided by a risk score is highly dependent on being able to discern those patients most likely to benefit

Page 69 of 174

Answer: c). Patients with the highest readmission risk scores are those most impactable and thus deserving of care management interventions

Explanation:

Patients at high risk of readmission are not necessarily those most impactable by a given intervention because the root cause of the risk may not be able to be modified by changes in clinical care.

Reference:

Lewis GH. “Impactibility Models”: Identifying the Subgroup of High-Risk Patients Most Amenable to Hospital-Avoidance Programs. The Milbank Quarterly. 2010;88(2):240-255.

WS02-6:

Successful implementation of risk prediction models requires a multidisciplinary team of medical, nursing, technical, financial, and statistical staff. Which of the following activities are clinical informaticists best suited to own and resolve given their unique perspective?

a) Charting a path to manage the lack of uniform standards for payer-provided claims data

b) Developing rules describing patient attribution to an appropriate provider under a risk-contract or alternative payment model

c) Ensuring that risk scores have meaning and the rationale behind score creation is understood by end user clinicians.

d) Model adjustments for member churn and out of network utilization of care services

e) Negotiation with vendors providing risk scores based on predictive models

Answer: c) Ensuring that risk scores have meaning and the rationale behind score creation is understood by end user clinicians.

Explanation:

Risk score transparency is essential to successful model deployment. Front-line clinicians will be mostly likely to adopt a model that they understand and can provide valuable feedback during any model validation or refinement iterations.

Reference:

Kivlahan C, Gaus C, Webster AM, Ferrans R, Larimer CF, Rosenberg MA, Patnode NJ. High risk patient identification: Strategies for success [Internet]. Washington: Association of American

Page 70 of 174

Medical Colleges; Sept 2016. [cited 2017 Mar 1]. Available from: ttps://www.aamc.org/download/470456/data/riskid.pdf

WS03: Workshop Building an innovation ecosystem: strategies and lessons learned in spreading IT innovations using the Diffusion of Excellence model Elizabeth Floto, Vanessa Coronel, Shereef Elnahal, Andrea Ippolito, Thalia Sirjue WS03-1: An innovation ecosystem refers to the dynamics of the complex relationships that are formed between _______ and _______ whose functional goal is to enable technology development, and innovation.

a) IT professionals, clinicians

b) the Business & Accounting department, the Patient Care Services department

c) actors, entities

d) financers, innovators

Answer: c) actors, entities Explanation: “An innovation ecosystem models the economic dynamics of the complex relationships that are formed between actors or entities whose functional goal is to enable technology development and innovation. In this context, the actors would include the material resources (funds, equipment, facilities, etc.) and the human capital (students, faculty, staff, industry researchers, industry representatives, etc.) that make up the institutional entities participating in the ecosystem (e.g. the universities, colleges of engineering, business schools, business firms, venture capitalists, industry-university research institutes, federal or industrial supported Centers of Excellence, and state and/or local economic development and business assistance organizations, funding agencies, policy makers, etc.).” Quote from Jackson’s white paper. Reference: Jackson, Deborah J. What is an Innovation Ecosystem [white paper] http://erc-assoc.org/content/what-innovation-ecosystem

http://erc-assoc.org/content/what-innovation-ecosystem

http://erc-assoc.org/content/what-innovation-ecosystem

Page 71 of 174

WS03-2: How does Everett Rogers categorize innovation adopters?

a) Early Adopters, Early Majority, Late Majority, Laggards, Traditionalists

b) Innovators, Early Adopters, Early Majority, Late Majority, Laggards

c) Innovators, Early Adopters, Majority, Laggards, Antagonists

d) Early Adopters, Majority, Laggards, Traditionalists, Antagonists

Answer: b) Innovators, Early Adopters, Early Majority, Late Majority, Laggards Explanation: This answer provides Rogers’s cateogorization of innovation adopters. Individuals in a social system do not all adopt an innovation at the same time. Rather, they adopt in an over-time sequence, so that individuals can be classified into adopter categories on the basis of when they first begin using a new idea. Reference: Rogers, Everett. Diffusion of Innovation, 5th ed. Free Press; 2003. WS03-3: A patient at Happy Healthy Medical Center (HHMC) shares with Nurse Andrea that his Primary Care Physician did not wash his hands before beginning the visit. She apologizes, assures the patient that this is not regular practice, and promises to bring the issue up at a future staff meeting. However, psychological safety is low at HHMC so Nurse Andrea does not feel comfortable raising the issue, or presenting the idea she has to help address it. She confides in a colleague, who shares an anonymized version of the story via an employee engagement survey. The HHMC Employee Engagement team shares this comment with leadership. What of the following is the most appropriate HHMC leadership response to improve the organization’s Innovation Ecosystem?

a) Nothing. The comment represents the views of one staff member. His or her reluctance to share does not indicate a system-wide problem.

b) Work with the Employee Engagement and Internal Communications teams to develop a multi-modal plan to improve the Innovation Ecosystem. The plan should include appropriate activities and effective communication to ensure all levels of staff know they are encouraged, and empowered to ask questions, identify problems, and pilot innovative solutions to addressing them. HHMC leadership should also ensure all managers and executives are aware of and empowered to provide resources and support where needed.

c) Send out an all-employee email reminding employees of the facility’s “safe space” policy encouraging staff to talk to their managers about challenges and ideas for developing creative solutions.

Page 72 of 174

Answer: b) Work with the Employee Engagement and Internal Communications teams to develop a multi-modal plan to improve the Innovation Ecosystem. The plan should include appropriate activities and effective communication to ensure all levels of staff know they are encouraged, and empowered to ask questions, identify problems, and pilot innovative solutions to addressing them. HHMC leadership should also ensure all managers and executives are aware of and empowered to provide resources and support where needed. Explanation: Creating and maintaining a healthy Innovation Ecosystem requires multiple layers of communication, collaboration and cooperation from all organizational levels, and leadership buy-in. Response B best addresses this. Reference: Morgan, Jacob. The Innovation Ecosystem for the Future of Work. Forbes; 2015 Retrieved from: https://www.forbes.com/sites/jacobmorgan/2015/08/12/innovation-ecosystem-future-of-work/#5ca63bea2eec WS03-4: A clinical informatics team noticed rates of specialty-referral completions were especially low in the primary care department of one facility in the Roy G. Biv Healthcare System (RGBHS). With the permission of facility leadership, the team worked with the clinical staff to determine the root cause of the problem. Referrals were in fact being completed. However, they were not accurately reported in RGBHS’s EHR software. To combat the reporting issue, the team developed and piloted a new referral template and protocols in the EHR software. Within 90 days of implementing the new process, the referral completion rates jumped from 29% to 74%. If they are following the principles of innovation dissemination, which of the following is the most appropriate next step?

a) Report the successful innovation to leadership, conduct research to determine whether or not the problem and root cause are the same at other facilities. Upon confirming presence of the problem, request permission to update the referral templates and protocols for the other six RGBHS facilities.

b) Immediately roll out the new template and protocol across all RGBHS facilities

c) Develop a report that includes the research findings, pilot results, updated template fields, and protocol codes. Save it on the shared drive so other facilities can access it, and hope they see it.

Answer: a) Report the successful innovation to leadership, conduct research to determine whether or not the problem and root cause are the same at other facilities. Upon confirming

https://www.forbes.com/sites/jacobmorgan/2015/08/12/innovation-ecosystem-future-of-work/#5ca63bea2eec


Page 73 of 174

presence of the problem, request permission to update the referral templates and protocols for the other six RGBHS facilities. Explanation: The principles of innovation dissemination include creating ideas, testing/piloting ideas, learning and iterating, then scaling ideas. Response A is the only option that completes this sequence. Reference: Morgan, Jacob. The Innovation Ecosystem for the Future of Work. Forbes; 2015 Retrieved from: https://www.forbes.com/sites/jacobmorgan/2015/08/12/innovation-ecosystem-future-of-work/#5ca63bea2eec WS03-5: Which of the following best describes why it is so difficult for organizations to change?

a) Fear of the unknown or unfamiliar

b) Union representatives of bargaining agreements

c) Organization leadership

d) People are likely to keep their belief system and continue current behavior even if data contradicts it

Answer: d) People are likely to keep their belief system and continue current behavior even if data contradicts it Explanation: Although all responses impact organizational readiness to change, option D is the most difficult to overcome since it requires shifting the needle on individual readiness to change. Reference: Weiner, Bryan J. A theory of organizational readiness for change. Implementation Science 4(67); October 19, 2009. Retrieved from: http://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-4-67



http://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-4-67

Page 74 of 174

WS04: Workshop - Leveraging Electronic Health Records Using Recurrent Neural Networks

David Ledbetter, Long Ho, Abel Brown

WS04-1:

Which of the following algorithms explicitly takes advantage of the temporal component of data?

a) Logistic Regressions

b) Support Vector Machine

c) Random Forest

d) Recurrent Neural Network

Answer: D) Recurrent Neural Network

Explanation:

Recurrent Neural Networks have a “cell state” which maintains a history of the previous timesteps which is used to inform the current state.

Reference:

Andrej Karpathy blog. http://karpathy.github.io/2015/05/21/rnn-effectiveness/

WS04-2:

HDF stands for...

a) Hierarchical Data Format

b) High-Definition File

c) HemoDynamic Fluctuation

d) Hoggs Data Filter

Answer: A) Hierarchical Data Format

Explanation:

HDF is a hierarchical data structure which was designed to store and access large amounts of scientific data. It is faster, more efficient, and scales better to large scale applications than CSVs.

Page 75 of 174

Reference:

www.hdfgroup.org

WS04-3:

The primary rationale to normalize data with an RNN is to...

a) Make the numbers smaller for machine precision

b) Make magnitude of variations of variables similar

c) Fit the data into a 16-bit floating point variable

d) Make it easier to plot multiple variables on a single canvas

Answer: b) Make magnitude of variations of variables similar

Explanation:

Since the gradients provide a measure of contribution of each input variable the normalization helps ensure that the gradient magnitudes are commensurate. Additionally, optimization converges more quickly with normalized inputs.

Reference:

Sola J, Sevilla J. Importance of input data normalization for the application of neural networks to complex industrial problems. IEEE Transactions on Nuclear Science. 1997 Jun;44(3):1464-8.

WS04-4:

The best way to impute data is to...

a) Forward fill

b) Linear interpolation

c) Cubic splines

d) There is no agreed upon standard practice for data imputation

Answer: d) There is no agreed upon standard practice for data imputation

Explanation:

Page 76 of 174

Imputation is dependent on both the nature of the underlying data and the particulars of the algorithm implementation. The best method to determine the best imputation method is to create an objective metric with appropriate training, testing, and validation sets to assess the performance of particular approaches.

Reference:

Friedman J, Hastie T, Tibshirani R. The elements of statistical learning. Springer, Berlin: Springer series in statistics; 2001.

WS04-5:

Deep learning algorithms are typically trained using Graphics Processing Units (GPUs) because...

a) A desktop PC can house up to 4 GPUs (as opposed to just 1 or 2 CPU)

b) Many deep learning algorithms rely extensively on large-scale matrix multiplications which can be easily paralleled to many smaller, simpler processors

c) It gives you a good excuse to write off your gaming PC as a tax deduction

d) Most deep learning packages are only able to perform computations on a GPU

Answer: b) Many deep learning algorithms rely extensively on large-scale matrix multiplications which can be easily paralleled to many smaller, simpler processors

Explanation:

The thousands of CUDA cores presented in a GPU are able to perform simple computations (such as matrix multiplications) extremely efficiently. Although not as sophisticated as a CPU core, due to their efficiency they are able to achieve high computations/watt.

Reference:

Fatahalian K, Sugerman J, Hanrahan P. Understanding the efficiency of GPU algorithms for matrix-matrix multiplication. InProceedings of the ACM SIGGRAPH/EUROGRAPHICS conference on Graphics hardware 2004 Aug 29 (pp. 133-137). ACM.

Page 77 of 174

WS04-6:

A particular advantage of an RNN over static methods (such as PIM2 or PRISM3) is that...

a) It is able to continuously update as new information becomes available

b) It is able to able to utilize hundreds of variables simultaneously

c) Expert knowledge can be incorporated

d) The contribution of each input can be calculated

Answer: a) It is able to continuously update as new information becomes available

Explanation:

While B, C, and D are true of an RNN, they are also true of most static methods. The primary advantage of the RNN is the ability to maintain a temporal history about the patient rather than relying on a single static snapshot in time.

References:

Aczon M, Ledbetter D, Ho L, Gunny A, Flynn A, Williams J, Wetzel R. Dynamic Mortality Risk Predictions in Pediatric Critical Care Using Recurrent Neural Networks. arXiv preprint arXiv:1701.06675. 2017 Jan 23.

Page 78 of 174

WS05: Workshop – Predictive Analytics Using Open Source Machine Learning WEKA Software

Robert Hoyt, Dallas Snider

WS05-1:

Your academic center has been funded as part of a Precision Medicine grant. You are in the process of integrating genomic data with phenotypical data you have archived derived from your electronic health record. The primary goal of your data analytics team is to find “undiscovered meaningful patterns” in the data. If you utilize machine learning for your inquiry what type of approach will you most likely use?

a) Classification

b) Regression

c) Association

d) Clustering

Answer: d. Clustering

Explanation:

Clustering is commonly used to identify previously unrecognized patterns in genomic data

Reference:

Witten IH, Frank E, Hall MA. Data Mining: Practical Machine Learning Tools and Techniques. 3rd edition. Morgan Kaufmann. 2011

WS05-2:

Your hospital has EHR data from the last year, archived in your enterprise data warehouse. The administration is alarmed about possible lost revenue due to an increased readmission rate. Your data analytics team is going to tackle heart failure readmissions first. Your goal is to create a predictive analytics model to identify those factors that lead to the binary outcome (readmitted or not readmitted.

Which machine learning approach would you most likely use?

a) Classification

b) Regression

c) Association

d) Clustering

Page 79 of 174

Answer: a) Classification

Explanation:

Classification is used when the desired binary outcome is categorical (nominal) data.

Reference:


WS05-3:

Your data analytics team is planning on creating some new clinical decision support rules to alert clinicians about patients who are at high risk for medication non-adherence. They would like to create IF-THEN rules with 95% confidence they can implement.

Which machine learning approach will you most likely use to accomplish this?

a) Classification

b) Regression

c) Association

d) Clustering

Answer: c) Association

Explanation:

Association algorithms create IF-THEN like statements that could be used to create clinical decision support .

Reference:


WS05-4:

As the head of the data analytics team you have been tasked by the CFO to create a model to help predict wait times in the ER using common parameters such as age, APACHE II disease severity score, gender (male = 0, female = 1) and insurance status (uninsured = 0, insured = 1). The most likely machine learning approach would be:

Page 80 of 174

a) Classification

b) Regression

c) Association

d) Clustering

Answer: b) Regression

Explanation:

The desired outcome here involves numerical data so regression and not classification is used

Reference:


WS05-5:

Your data analytics team has created a predictive model using a machine learning classification approach (Naive Bayes) for readmissions for COPD. Your team believes your model is unique and worthy of publication. What is the most common data representation choice for a binary outcome in a publication?

a) Precision

b) Specificity

c) C-statistic or ROC area under the curve

d) Effect size

Answer: c) statistic or ROC area under the curve

Explanation:

The validation outcome most commonly associated with classification is the receiver operator characteristic curve (area under the curve), also known as the c-statistic

Reference:


Page 81 of 174

WS05-6:

Your data analytics team is beginning to analyze a complicated data set from your EHR. You are in the initial data preprocessing phase of your analysis and using the machine learning software WEKA. What feature in WEKA would you most likely use to address missing data, normalization and discretization of data?

a) Filter

b) Select attributes

c) Edit

d) Clustering

Answer: a) Filter

Explanation:

The filter option in WEKA enables the user to deal with missing data and data sets that need to be normalized or converted to discrete data

Reference:


Page 82 of 174

WS06: Workshop – Agile Clinical Decision Support DuWayne Willett, Vaishnavi Kannan, Mujeeb Basit WS06-1: On rapid-cycle agile development projects, requirements are often kept lightweight, for example with a User Story and associated Acceptance Criteria. The most typical template for a User Story is: "As a XXXXX, I want YYYYY so that ZZZZZ ". When using this template to write a User story for a clinical decision support (CDS) tool which will be displayed to clinicians in an EHR, the best entry for the placeholder is:

a) The executive sponsor of the project

b) The clinician who will see and use the CDS tool

c) The EHR analyst building the CDS tool

d) The patient who will ultimately benefit from best practices promoted by use of CDS

Answer: b) The clinician who will see and use the CDS tool explanation Explanation: User stories are written in the voice of the person who will actually be using the product. The executive sponsor for a CDS tool may have an institutional goal in mind, but is unlikely to be a representative CDS user: any requirements or constraints from the sponsor can go in the Acceptance Criteria. Using the voice of the developer/analyst is not the point of a User Story, and is incorrect. The benefit to the patient would better be in the placeholder. A helpful adage for more successful CDS tools is "do CDS with your users, not to them". Writing CDS user stories in the voice of the clinician who will use them reinforces exploring what benefit valued by the clinician is being sought (such as improved efficiency, and/or an improvement in their patient's outcome). References:

• Larman C. Agile and Iterative Development: a Manager's Guide. Addison Wesley; 2004.

• Cohn M. User stories applied: for agile software development. Boston: Addison-Wesley; 2004.

• Osheroff J, Teich J, Levick D, Saldana L, et al. Improving Outcomes with Clinical Decision

Support: An Implementer's Guide . Taylor & Francis; 2012.

Page 83 of 174

• Wright A, Hickman T-TT, Mcevoy D, Aaron S, Ai A, Andersen JM, et al. Analysis of clinical decision support system malfunctions: a case series and survey. Journal of the American Medical Informatics Association J Am Med Inform Assoc. 2016

WS06-2: The acronym INVEST is a widely-accepted set of criteria for assessing the quality of a User Story, and its readiness to undergo development. Which of the following is one of the INVEST criteria?

a) Independent

b) Visible

c) Straightforward

d) Trainable

Answer: a) Independent Explanation: The INVEST acronym states that a good user story should be: "I" ndependent (of all others) "N" egotiable (not a specific contract for features) "V" aluable (or vertical) "E" stimable (to a good approximation) "S" mall (so as to fit within an iteration) "T" estable (in principle, even if there isn't a test for it yet) Following this checklist helps ensure a user story can be scheduled into and developed within a single iteration, and provide some tested, valuable functionality. Keeping user stories independent enables re-prioritizing of user story development order based on business value. Keeping them small (by "splitting" or "slicing" as needed) helps ensure each user story can be completed in a single iteration. Requiring testability ensures that the acceptance criteria have been thought through enough to validate successful completion of the story, and helps reduce defects. References:



• Osheroff J, Teich J, Levick D, Saldana L, et al. Improving Outcomes with Clinical Decision Support: An Implementer's Guide . Taylor & Francis; 2012.

Page 84 of 174


WS06-3: Epidural catheters for pain control are frequently placed by anesthesiologists for patients primarily managed by other services. Administering anticoagulants to patients with an epidural catheter may be associated with increased risk of epidural hematoma and neurologic damage. You've been asked to create an alert for anyone ordering an anticoagulant or antiplatelet medication for a patient with an epidural catheter, to alert them of this risk. As you listen to the experts discuss when the alert should fire, you learn that exception conditions apply, e.g. for low-dose VTE prophylaxis. You want to capture the logic to define unequivocally for which sets of conditions the alert should and should not appear. The optimal model for depicting this logic is a:

a) Swimlane Workflow Diagram

b) User Interface (UI) Storyboard

c) Use Case Diagram

d) Object Diagram

e) Decision Tree

Answer: e) Decision Tree Explanation: Decision trees (or alternatively, decision tables) depict clearly the logic by which any rule should determine a specified output given any set of inputs. This is the most appropriate diagram to depict the sets of conditions under which a CDS alert should or should not display to a single clinician . The remaining model types all have different uses. Swimlane workflow diagrams display the flow of work among multiple roles, and graphically depict the sequence of work and handoffs involved. UI Storyboards are a mockup of what a user will see on screen, and are useful accompaniments to use case step-by-step descriptions. Use Case Diagrams depict the variety of Actors that interact with a system, and the specific Use Cases each can be involved with, without any sequencing implied. An Object Diagram is a structural model, not a process model, showing the objects that make up a system, and how they are related. References:

Page 85 of 174





WS06-4: You're embarking on an initiative to redesign venous thromboembolism (VTE) prophylaxis risk profiling and prophylaxis for patients admitted to your hospital. This project is envisioned to involve capturing of VTE risk data by patients (via your EMR's patient portal), by clinic staff and pre-surgery clinic visits, and/or by hospital nurses and physicians/surgeons following admission, in any of several combinations or sequences. In addition, clinical decision support is desired to prompt ordering of VTE prophylaxis from the options most appropriate for a given patient's VTE risk level and other clinical factors suchas renal function. The best diagram for depicting on a single page the full scope of desired activities (user goals) along with which user roles would participate in each activity would be a:

a) Use Case Diagram

b) Swimlane Workflow Diagram

c) Data Flow Diagram

d) Decision Tree

Answer: a) Use Case Diagram Explanation: A use case diagram provides a visual "table of contents" for all the activities or user goals envisioned for a system, with each "actor" or user role shown connected to the activities they participate in. This helps provide a common language for referring to the dynamic activities within the system. Each "use case" or oval on the diagram can later become a user story for development --or frequently broken down into smaller user stories for scheduling into an iteration. A swimlane workflow diagram depicts the end-to-end flow of a single repeatable business process as the work flows from the initiating event to the final delivered product or service showing which user roles are responsible for each step. Because of the anticipated variability here in timing of activities, and in who performs them, it would be difficult to depict the full range of desired capabilities in a single workflow diagram. (It would still be helpful here to show a workflow diagram for each of several common, illustrative possible sequences). A data flow diagram shows what information flows between systems

Page 86 of 174

and/or persons which are part of an overall system, but does not directly depict activities or use cases of the system. A decision tree depicts business logic unambiguously, but would not meet the stated purpose here. References:





WS06-5: An automated acceptance test is written before development of a clinical decision support alert, specifying the decision logic it should employ when evaluating whether to display to clinicians. The automated test is red (fails) initially, and then becomes green (passes) once the EHR analyst has successfully finished their development. Which best describes this use of automated testing?

a) Regression testing

b) Test-driven development

c) Exploratory testing

d) Unit testing

Answer: b) Test-driven development Explanation: In Test-driven development (TDD), an automated test is written before development begins, to unequivocally specify a requirement to be fulfilled (in this case the alert's decision logic). The test is designed to fail prior to development, and only pass once successful development has been accomplished. Regression tests on the other hand are meant to always be green (pass): failure of a regression test implies that software previously working successfully is no longer doing so. Exploratory (or ad hoc) testing refers to unscripted testing of a system under development, commonly by end users. Unit testing refers to testing of a single,

Page 87 of 174

smallest testable part of an application's code; unit testing can be employed in testing either after development has occurred, or up front in test-driven development. References:





WS06-6: Your liver specialists want to increase the rate at which patients with cirrhosis are screened for hepatocellular carcinoma, in line with published guidelines. Accordingly, a clinical decision support alert has been reminding providers to order a liver imaging test if one has not been performed within the recommended screening interval. The alert checks for presence of cirrhosis on the Problem List, and for a completed order in your EMR for any of a list of imaging procedures. Your CDS monitoring program recently detected a sharp rise in the frequency at which this alert is displaying to providers. What is the most likely cause for the abrupt increase in frequency of alert display?

a) The national society guidelines were recently updated to eliminate the screening recommendations, and the physicians' practice reflects the new guidelines.

b) Your local physicians have collectively decided that the screening practice is low-yield and not cost-effective, and have abandoned it.

c) A recent operating system software security patch on your workstations has caused the CDS alert to display inappropriately

d) Radiology has begun using a new orderable in your EMR which is not contained in the "Liver imaging" procedure list used by the CDS alert, causing it to display inappropriately.

Page 88 of 174

Answer: d) Radiology has begun using a new orderable in your EMR which is not contained in the "Liver imaging" procedure list used by the CDS alert, causing it to display inappropriately. Explanation: Clinical decision support monitoring systems can detect anomalies prior to being reported by clinicians. An abrupt increase or decrease in the rate a given CDS alert displays to clinical users most likely results from a change affecting the logic of the alert. Such possibilities include a change to: the EHR software, the configuration of the CDS tool, or in the discrete data being evaluated by the logic of the CDS tool. In this case, the alert checks for an imaging procedure order from a specified list (or "grouper"). Use of a new imaging procedure orderable without also updating the CDS alert's list would immediately result in the alert displaying inappropriately, and is the most likely cause among the listed reasons. A change in the Windows or other operating system software would be much less likely to affect the logic of this particular CDS alert in isolation. Changes in clinical practice that follow or precede changes in clinical guidelines typically manifest gradually, not abruptly. References:





WS07: Workshop - Fast Healthcare Interoperability Resources (FHIR): A Workshop for Implementers

Charles Jaffe, Grahame Grieve, Alistair Erskine, Viet Nguyen, Russ Leftwich, Stan Huff

WS07-1:

What are the requirements for use of FHIR in open-source or commercial products?

a) FHIR is provided through a creative commons license.

b) HL7 membership

c) FHIR Foundation membership

d) A license fee based upon utilization.

Answer: a) FHIR is provided through a creative commons license.

Page 89 of 174

Explanation:

Like all HL7 intellectual property, FHIR is licensed free of charge.

Reference:

FHIR Release 3 (STU). License and legal terms. April 19, 2017. Available at https://www.hl7.org/fhir/license.html

WS07-2:

What individuals may contribute to FHIR development?

a) HL7 members

b) Argonaut Project members

c) FHIR Foundation members

d) FHIR development is open to anyone who registers

Answer: d) FHIR development is open to anyone who registers

Explanation:

Anyone can contribute to FHIR development. Only HL7 members may vote on FHIR ballots.

Reference:

FHIR Release 3 (STU). Getting Started with FHIR. April 19, 2017. Available at https://www.hl7.org/fhir/modules.html

WS07-3:

FHIR is rapidly integrating with other HL7 standards, including V2 messaging and CDA. On the current trajectory,

a) V2 and CDA will be sunset by the end of the decade.

b) FHIR will replace V3

c) Service oriented architecture (SOA)

d) FHIR will coexist with enhancements to V2 and CDA, which are effective for purpose

https://www.hl7.org/fhir/license.html

https://www.hl7.org/fhir/modules.html

Page 90 of 174

Answer: d) FHIR will coexist with enhancements to V2 and CDA, which are effective for purpose

Explanation:

Many HL7 standards will remain suitable for specific needs in the foreseeable future. FHIR integration will be an incremental process.

Reference:

HL7 International. Standards – Product Brief. Section 3: Clinical and Administrative Domains, Section 5: Implementation Guides: HL7 Fast Healthcare Interoperability Resources Specification (FHIR®), DSTU Release 1. © 2007-2017. Available at http://www.hl7.org/implement/standards/product_brief.cfm?product_id=343

WS07-4:

What is the principle objective of the FHIR Foundation?

a) Accelerate development of FHIR resources and profiles.

b) Provide a collaborative environment for FHIR implementation.

c) Enable a funding source for subject matter experts.

d) Provide support for the US Realm.

Answer: b) Provide a collaborative environment for FHIR implementation.

Explanation:

FHIR.org, the home of the FHIR Foundation, provides opportunities for FHIR implementers to collaborate, review best practices, share technology and solutions, and test conformance.

Reference:

FHIR. About fhir.org and the FHIR Foundation. Available at http://www.fhir.org/about

http://www.hl7.org/implement/standards/product_brief.cfm?product_id=343

http://www.fhir.org/about

Page 91 of 174

WS07-5:

An important advantage of using FHIR based APIs versus standard messaging interfaces is:

a) Patient data can be accessed directly from the original source without making a duplicate copy.

b) FHIR does not require the use of standard code systems like LOINC, SNOMED, or RxNorm.

c) The FHIR resources are more mature than the messaging standards.

d) FHIR provides superior security.

Answer: a) Patient data can be accessed directly from the original source without making a duplicate copy.

Explanation:

FHIR allows access to data where it resides. FHIR does require the use of standard codes. The definition of FHIR resources are still evolving and are not yet as stable as the older messaging standards.

Reference:

United State Office of the National Coordinator for Health IT (ONC). 2015. Connecting Health and Care for the Nation: A Shared Nationwide Interoperability Roadmap. FINAL Version 1.0. https://www.healthit.gov/sites/default/files/hie-interoperability/nationwide-interoperability-roadmap-final-version-1.0.pdf

WS07-6:

One important benefit of the Argonaut work is:

a) It establishes broad semantic interoperability of healthcare data based on FHIR profiles.

b) It provides a pathway for requesting the addition of new concepts to standard coding systems.

c) It establishes a baseline for security and authentication.

d) It has created FHIR profiles for a core set of common data elements for use in the US.

Answer: d) It has created FHIR profiles for a core set of common data elements for use in the US.

https://www.healthit.gov/sites/default/files/hie-interoperability/nationwide-interoperability-roadmap-final-version-1.0.pdf

https://www.healthit.gov/sites/default/files/hie-interoperability/nationwide-interoperability-roadmap-final-version-1.0.pdf

Page 92 of 174

Explanation:

The Argonauts have created useful FHIR profiles for resources that correspond to common data elements required for meaningful use in the US. Additional FHIR profiles are needed to establish semantic interoperability. The Argonauts are not adding new codes to the standard coding systems.

Reference:

HL7 International. Welcome to the Argonaut Project. April 26, 2017. Available at http://argonautwiki.hl7.org/index.php?title=Main_Page

WS08: Application of Model-based Computer Simulation in Medical Informatics

Duane Steward

WS08-1:

Which of the following best outlines the discipline of model-based simulation? A. 1) Enumerate independent and dependent variables 2) Choose appropriate models to emulate the system features of interest 3) Analyze variance to determine number of runs required for replication experiments 4) Render simulation output to enable debugging, performance measurement and inter-rater agreement 5) Iterate until output converges within specified confidence intervals for chosen outputs B. 1) Articulate purpose, scope, granularity and performance measures 2) Choose entities, resources, and context of investigation 3) Construct a network of models, input and output variables using an iterative process 4) Frame assumptions that best fit the question investigated 5) Analyze results of replicated results only after establishing confidence in the decision makers C. 1) Choose entities, system resources, and context of investigation 2) Enumerate independent and dependent variables 3) Frame assumptions and implement a directed graph of models 4) Analyze variance to determine number of runs required for replication experiments

http://argonautwiki.hl7.org/index.php?title=Main_Page

Page 93 of 174

5) Iterate until output converges within specified confidence intervals for chosen outputs D. 1) Frame assumptions, enumerating independent and dependent variables 2) Analyze variance to determine number of runs to adjust for confounders in experiments 3) Choose appropriate models to emulate the system features of interest 4) Articulate purpose, scope, granularity and performance measures 5) Analyze results of replicated results only after output converges within accepted confidence intervals Answer: B. 1) Articulate purpose, scope, granularity and performance measures 2) Choose entities, resources, and context of investigation 3) Construct a network of models, input and output variables using an iterative process 4) Frame assumptions that best fit the question investigated 5) Analyze results of replicated results only after establishing confidence in the decision makers Explanation: A. Selection of variables without knowing purpose, scope and context leads rapidly to inefficiency. This outline fits classical statistic analysis better than model-based simulation. B. This is the outline presented explicitly in the workshop. C. Selection of entities, system resources and context before purpose, scope and performance measures can lead to irrelevant simulation ill-positioned to support decisions. Analysis without prior verification and validation of the simulator will likely be unconvincing and/or misleading. D. Similar to A & C with additional elements out of order--e.g., articulate purpose, scope, granularity and performance measures were asserted as the place to start. Reference: Pritsker, A.A.B., and O’Reilly, J.J. Simulation with Visual SLAM and AweSim, 2nd ed. Wiley, New York, p. 828, 1999. WS08-2: The use of model-based simulation was asserted to be the testing of assumptions--i.e., the assigning of assumed values to variables and model parameters, running the simulation and observing what values emerge in output variables. Insofar as that is true, the entire exercise pivots on the quality of the data embedded in or serving as the basis for the assumptions. If the

Page 94 of 174

solution sought is for a system that does not yet exist or has yet to be exercised, the following question becomes vitally important. What source of data is most ill-advised for model-based simulation where the objective is to support decision making in scenarios for which data does not yet exist?

a) Creative data

b) Biased data

c) Data from other systems not within the scope of the investigation

d) Simulator output

e) Data representing assumptions of the decision maker

Answer: d) Simulator output is most ill-advised. Explanation: a) Creative data -- The workshop addressed how in the absence of data, features of subnetworks that comprise the system in question can be used to construct empirical means to generate data on a sound basis. This can result in assumptions and simulated sub-system behavior that is more reliable and emulates with greater fidelity that expert opinion or other substitutes in the absence of historical data. b) Biased data -- Usually associated with negative consequence, bias can nonetheless be adjusted for if well understood. This we know from traditional statistics. Adjusted for bias, derived behavior can easily be more sound than guesswork or subjective opinion of experts. Furthermore, bias when adjusted for rationally can be very convincing and help motivate adoption of results for decision making. c) Data from other systems not within the scope of the investigation -- Data from other systems that share features (assumed or historically known) with the system in question represents a good source of data for assumptions with evidence-based quality. So long as care is taken to match system features and apply the match carefully to appropriate subcomponents of the unknown system, this source of proxy data is a great resource with perhaps more visible rationale. d) Simulator output -- Using the output of simulation as a source of data to make assumptions for simulation is backwards at face value. Although this does represent a means to create data where it does not exist, it fails to provide a basis for data quality and assumptions used to create the simulation. e) Data representing assumptions of the decision maker -- In the face of simulation with no historical basis, expert opinion has often been used. While the quality of the assumptions

Page 95 of 174

derived may be lower than other sources, if based on sound interpretation of experience, the results are likely better than pure guesswork or throwing darts. This was illustrated by one case study exposed in the workshop yet to be published. Using the assumptions of the decision maker can promote confidence in the simulator output, especially if couched appropriately. Reference: • Steward, Duane; Hofler, Richard; Thaldorf, Carey; Milov, David E. A Method For

Understanding Some Consequences Of Bringing Patient Generated Data Into Health Care Delivery, Journal of Medical Decision Making, July/August, 2010.

• Steward, Duane and Wan, Thomas T.H. Role of Simulation and Modeling In Disaster

Management, Journal of Medical Systems, Journal of Medical Systems, March 21, 2007. WS08-3: Which of the following simulation study projects are likely to require custom programming?

a) Queuing of admitted patients between emergency department disposition and hospital admission

b) Impact of patient reported outcomes on a clinic’s throughput

c) Spread of disease over an urban population

d) Integration of discharge summary and outcomes feedback to reduce readmission

Answer: d) Integration of discharge summary and outcomes feedback to reduce readmission is most likely to require custom programming. Explanation:

a) Queuing is a common part of process modeling and supported by nearly all simulation software tool libraries

b) Clinic workflow is readily modeled as a network of resource service activities in parallel, series or both. These are common constructs of process simulation.

c) Assumptions regarding changes of state for individuals in a population with interactions are easily represented as parameters of building blocks commonly found in simulation software.

d) Discharge summaries represent a sophisticated abstraction of information for which it would be most reasonable to anticipate some deconstruction and interpretation.

Page 96 of 174

In this description, no fundamental modeling strategy is revealed to suggest which off-the-shelf construct or model node is an obvious choice for simulating system behavior. The integration of sample discharge summary elements likely requires natural language processing of free text or utilization of a machine learned algorithm inferred from structured data schemas. These notions involve levels of complexity and design alternatives that are not unimaginable in model-based simulation but do indicate a need for non-trivial implementation to emulate system behavior and support decision making. Reference: Pritsker, A.A.B., and O’Reilly, J.J. Simulation with Visual SLAM and AweSim, 2nd ed. Wiley, New York, p. 828, 1999. WS08-4: Which of the following is not a form of model-based simulation?

a) Agent-based Simulation of social activity in a food court.

b) Process flow simulation in surgical prep

c) System dynamics simulation of drug perfusion in vivo.

d) Emergency Response Team simulation of assigned roles

e) Organ failure rate simulation over time comparing toxin levels

Answer: d) Emergency Response Team simulation of assigned roles is not a form of model-based simulation. Explanation: A. Agent-based simulation is one mode of model-based simulation B. Process simulation is a synonym for discrete-event-model based simulation C. System Dynamics is a form of model-based simulation dominated by flows and reservoirs D. Role-based simulation, where humans walk through roles, is a form of simulation not overtly model-based, depends upon actors and is much more constrained to real time execution. E. Simulation of continuous processes over time is a form of model-based simulation. Reference: Karnon J, Stahl J, Brennan A, Caro JJ, Mar J, Moller J (2012) Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-4. Med Decis Making 32 (5):701-711. doi:10.1177/0272989X12455462

Page 97 of 174

WS08-5: An ophthalmology clinic requests help in evaluating operations. They want to know whether distributing the allocation of exam rooms unevenly is warranted in light of multiple ophthalmologist with variation in practice style. Among four clinicians, one has expressed no interest in improving Press Gainey scores and insists on rapid efficient exam room visits to increase how many patients can be seen on non-surgical clinic days. Another of the four ophthalmologists prides himself in taking as long as the patient needs to answer questions at a more relaxed pace and has an average exam duration that everyone suspects is greater than others. The clinic administrative team would like to know if the clinic throughput would be higher if the clinic schedule was arranged so that the faster and slower clinicians were scheduled on the same day with more rooms allocated to the faster paced clinician balanced by a reduced number of rooms allocated to the slower paced clinician. In this scenario, which of the following are not likely useful as simulation output variables?

a) Patient Diagnosis

b) Duration of Exam

c) Length of Stay

d) Resource utilization

Answer: a) Patient Diagnosis is not likely to be an output variable. Explanation: A. The diagnosis is not a result of the clinic process. The simulation will require an assumption about the diagnosis of each patient to appropriately control the flow through the clinic if diagnosis is suspected to play any role in the throughput. Assumptions are input variables, not observed outcomes. B. The duration of work segments in the system is usually an assumption required for classic service system simulations. However, when the examination is a work segment composed of other smaller work segments, there may be greater fidelity to system behavior achieved by modeling the total duration of examination as an output of the assumptions for constituent subelements. In such cases it could be argued the duration of exam is an intermediate output variable that generates a value to be used as an input variable for other parts of the system simulated. C. Length of Stay is a common performance measure used in clinic operation analysis. Performance measures are output variables. D. Resource utilization is a measure of how busy resources are observed to be over periods of time. In contrast to assumed values for input variables, observations are output variable data points. Reference: Pritsker, A.A.B., and O’Reilly, J.J. Simulation with Visual SLAM and AweSim, 2nd ed. Wiley, New York, p. 828, 1999.

Page 98 of 174

WS08-6: In the scenario of the previous question, the simulation results in no significant difference in throughput (patients per day) despite a significant difference between the average length of patient-provider exam room encounter. What is the most fallible explanation for this result?

a) Clinic workflow segments outside the provider exam contribute an impact on length of visit that eliminates the impact of physician pace during patient encounters (e.g., mydriasis).

b) System resource bottlenecks (e.g., rooming by overburdened staff, limited number of shared instruments) are rate limiting features with greater impact on throughput than pace of physician encounter

c) A bug in the software may account for the unanticipated finding.

d) Impact on throughput may exist but not in the range of resource levels tested, at least not that impact overall performance measured. Context specific limitations preclude sound conclusions about alternate contexts without changes to the simulation.

e) Simulation verified to be working as intended and validated as representative of observed experience indicates that no benefit to throughput should be anticipated by unequal allocation of exam rooms based on clinician pace for the specific context studied in this simulation.

Answer: e) Simulation indicates that no benefit ... for the specific context studied in this simulation would be the most fallible explanation for the stated simulation output. Explanation:

a) This is a plausible explanation. It represents the potential for confounding factors in complex system behavior that may obscure truth about untested assumptions.

b) This is a plausible explanation, again representing potential for confounding factors of another nature in complex system behavior. The plurality of such factors may account for summative or synergistic impact on performance measures.

c) This is an important plausible explanation, which must be ruled out by validation effort to ensure the software emulates real system behavior faithfully and that implementation is verified to execute the code as intended without the artificial introduction of misleading output.

d) Consider the following rewording of the candidate answer. “In the range of specific values used for simulation (available rooms, exam duration, diagnostic context, profile of additional resources—e.g., context of additional clinicians’ pace with shared resources), impact on throughput may exist but not at levels that impact overall performance. Such context specific limitations preclude sound conclusions about alternate contexts without significant changes to the simulation.”

Page 99 of 174

The first sentence is arguably true, citing the limitations of context variables for strict application of findings. The second sentence might be argued either fallible or correct. On one hand, a simulator with input and output variables need only be altered by the change of values assumed for those variables and thereby reflect sound predictions for alternate contexts. While that is strictly speaking a change in the simulator, it is far from starting over with coding a new simulation. On the other hand, insights gained in the course of revealing such negative findings may nonetheless provide profound insight to the nature of system behavior in ways that may support significant qualitative insights that are sound in the absence of further simulation. So this answer is debatably fallible or not.

e) It is important to understand the nature of model-based simulation as a practice of testing assumptions. Before conclusions are drawn, this must be acknowledged. Any conclusion is contingent upon the assumptions tested. In this case, a negative conclusion about benefit without acknowledgment of assumptions would be erroneous. Alternate assumptions (e.g., longer durations for atomic work flow segments, altered resource levels, different traffic patterns, more rooms, alternate clinic work-day hours and staffing) or values outside the range of the simulation could alter results and conclusions.

Reference: Karnon J, Stahl J, Brennan A, Caro JJ, Mar J, Moller J (2012) Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-4. Med Decis Making 32 (5):701-711. doi:10.1177/0272989X12455462

Page 100 of 174

WEDNESDAY, MAY 3

S01: Panel - Rethinking Documentation: Streamlining Communication and Workflow in the Inpatient Setting

Evan Orenstein, Subha Airan-Javia, Mark Mai, Eric Shelov

S01-1:

Your hospital identifies a safety event in which an immunocompromised patient with recent history of extended-spectrum beta-lactamase resistant E. coli bacteremia was not started on sufficiently broad antibiotics when she developed a fever overnight, and effective antibiotics were not instituted until the error was caught 18 hours later. During root cause analysis, your team identifies that the patient’s progress notes made clear that the patient should be started specifically on vancomycin, imipenem, and amikacin in the setting of a new fever, however the written handoff did not include this information. Which of the following systems would be most likely to prevent this error?

a) Instituting a policy in which all febrile, immunocompromised patients are immediately transferred to the intensive care unit.

b) An easily accessible documentation template for fever in immunocompromised patients.

c) Educating providers about the importance of situational awareness in the written handoff.

d) Populating the progress note & written handoff from a single source of truth

Answer: d) Populating the progress note & written handoff from a single source of truth

Explanation:

In addition to writing and updating admission notes, progress notes, and discharge summaries, front line clinicians (FLCs) must also update plans of care and problem lists in their handoffs. Often, these documents do not communicate with each other, forcing FLCs to maintain each document independently. As length of stay for an individual patient and team census/workload increases, progress notes and handoffs are known to drift away from each other in content and decrease in overall quality (1). In the absence of links from information in the electronic health record (EHR) to elements of the written handoff, inconsistencies are known to occur for the list of problems, code status, medications, and allergies (2-4). In this scenario, the key piece of information that was lost between documents (a contingency plan in case this patient developed a fever) is unlikely to be captured discretely in the EHR. Thus, a unified documentation system that populates both the progress note and the written handoff from a single source of truth would most likely have avoided this error since the FLC would only have had to make an update in one location instead of two.

Page 101 of 174

Instituting a policy requiring transfer of febrile, immunocompromised patients to the intensive care unit would have introduced another handoff and required substantial time to accomplish. The intensive care unit clinicians would still have needed to have read the most recent progress note in addition to any transfer notes or the handoff in order to have caught this mistake, which may not be part of their workflow. Thus, this approach is not likely to prevent this error in addition to leading to many potentially unnecessary intensive care unit admissions.

While a documentation template for fever in immunocompromised patients can provide guidance to FLCs to include explicit fever plans, the problem in this case is not that the patient had no fever plan. Rather this patient’s individualized, unusual fever plan was not copied over from the progress note to the handoff. Thus, a documentation template alone would probably not have prevented this error.

While it is certainly important to educate providers about the importance of situational awareness in the written handoff, high census, individual patient complexity, and increased length of stay make it increasingly difficult for FLCs to keep all inpatient documents updated and high quality. Thus, education alone is unlikely to prevent the error seen in this patient without technological solutions that streamline workflow and decrease the documentation burden (2).

References:

1. Miller DM, Schapira MM, Visotcky AM, et al. Changes in written sign-out composition across hospitalization: Handoffs and Written Sign-out Quality. J Hosp Med 2015;10:534–6. doi:10.1002/jhm.2390

2. Airan-Javia SL, Kogan JR, Smith M, et al. Effects of Education on Interns’ Verbal and Electronic Handoff Documentation Skills. J Grad Med Educ 2012;4:209–14. doi:10.4300/JGME-D-11-00017.1

3. Aylward MJ, Rogers T, Duane PG. Inaccuracy in Patient Handoffs: Discrepancies between Resident-Generated Reports and the Medical Record. Minn Med 2011;94:38–41.

4. Derienzo C, Lenfestey R, Horvath M, Goldberg R, Ferranti J. Neonatal intensive care unit handoffs: a pilot study on core elements and epidemiology of errors. J Perinatol. 2014;34(2):149–152.

S01-2:

A healthcare organization aims to increase the number of problems added to the problem list in the inpatient setting. Which of the following inpatient documentation strategies is most likely to increase the number of documented problems per admitted patient?

a) Force clinicians to enter each problem on the problem list before being able to document the plan related to that problem.

Page 102 of 174

b) Institute educational sessions for all clinicians emphasizing the importance of the problem list for quality improvement, research, and billing.

c) Audit inpatient progress notes weekly and contact clinicians about possible missed problems that should be added to the problem list.

d) Audit inpatient progress notes and problem lists and let clinicians know where they rank in terms of problem list completeness compared to their peers.

Answer: a) Force clinicians to enter each problem on the problem list before being able to document the plan related to that problem.

Explanation:

In addition to simplifying administrative efforts, quality improvement, and billing, accurate problem list documentation improves patient outcomes (1). However, because problem list entry is often not a part of the standard workflow of front-line clinicians, problem list completeness and accuracy are often poor. Forcing clinicians to enter a problem on the problem list before being able to document a plan for that problem, mandatory problem-oriented charting, is the most effective method for encouraging problem list utilization (2). While this approach is highly effective for this purpose, clinical informaticists should be aware that such rigid structures and forcing functions may aggravate clinicians due to limited expressivity (3,4).

Auditing inpatient progress notes to contact clinicians about possible missed problems, gap reporting, improves problem list usage but has less impact than mandatory problem-oriented charting. This approach is also resource intensive as it often requires substantial clinical knowledge to assess inconsistencies between the problem list and the rest of the clinical documents. Auditing inpatient progress notes in order to notify clinicians where they rank against their peers would be similarly resource intensive without evidence of improving problem list completeness.

Educational sessions may help produce an organizational culture that expects problem list completeness, but is less effective than mandatory problem-oriented charting.

References:

1. Hartung DM, Hunt J, Siemienczuk J, Miller H, Touchette DR. Clinical implications of an accurate problem list on heart failure treatment. J Gen Intern Med. 2005 Feb;20(2):143–7.

2. Wright A, McCoy AB, Hickman T-TT, Hilaire DS, Borbolla D, Bowes WA, et al. Problem list completeness in electronic health records: A multi-site study and assessment of success factors. Int J Med Inf. 2015 Oct;84(10):784–90.

Page 103 of 174

3. Rosenbloom ST, Denny JC, Xu H, Lorenzi N, Stead WW, Johnson KB. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc. 2011 Mar 1;18(2):181–6.

4. Wright A, Maloney FL, Feblowitz JC. Clinician attitudes toward and use of electronic problem lists: a thematic analysis. BMC Med Inform Decis Mak. 2011;11(1):1.

S01-3:

A research team is working to understand the causes of medical errors overnight and wants to understand the relationship between contingency plans made by the daytime clinical team and overnight clinical decisions. In which inpatient document should overnight contingency plans be found to minimize communication errors?

a) Progress note

b) Problem List

c) To-Do List

d) Handoff

Answer: d) Handoff

Explanation:

Front-line clinicians in the inpatient setting are responsible for multiple clinical documents for each admitted patient. These documents contain information with different goals and audiences. Of the choices listed, the primary goal of the handoff for each patient is to communicate between daytime and overnight/weekend providers so that the oncoming provider can accomplish the necessary tasks and make appropriate clinical decisions as problems arise. The IPASS© method for handoffs has shown decreased medical errors associated with communication between providers and includes an assessment of illness severity, patient summary, action items, situational awareness/contingency planning, and synthesis by the receiver (1). Thus, the handoff would be the most appropriate place to find information on contingency plans.

The intended audience for the progress note is wider and includes not only other clinicians caring for the patient but also billers and regulatory agencies, leading to additional included information. Thus, while the progress note often contains much of the same information as a handoff, it is often more difficult to find key information on task management and situational awareness that are of extra importance to a cross-covering provider.

Page 104 of 174

The key function of the problem list is to allow any provider to immediately make medical decisions in context for a particular patient (2). In addition, it provides structured data that can drive clinical decision support, quality improvement initiatives, research, and billing. While it is often incorporated into a handoff and plans about each problem may include contingency plans, the problem list itself generally does not contain a dedicated section for contingency planning where cross-covering providers can look.

The To-Do List is also often incorporated into a handoff and contains discrete action items that should be completed in a timely manner. It is not intended to help cross-covering providers make medical decisions in case of a change in clinical status.

References:

1. Starmer AJ, Spector ND, Srivastava R, West DC, Rosenbluth G, Allen AD, et al. Changes in Medical Errors after Implementation of a Handoff Program. N Engl J Med. 2014 Nov 6;371(19):1803–12.

2. Weed LL. Medical Records that Guide and Teach. N Engl J Med. 1968;278(11):593–600.

3. Weed L. Medical Records that Guide and Teach [Internet]. Grand Rounds presented at; 1971; Emory University. Available from: http://www.visualdx.com/company/larry-weed-1971-grand-rounds-at-emory-video

S02: Presentations – Population Health Querying Electronic Health Data for Population Health Activities using PopMedNetTM

Jessica Malenfant

S02-1: The PopMedNetTM (PMN) platform was enhanced to enable users to create and send custom data queries to multiple data sources using different database management systems (e.g. SQL server and Oracle). In what way does this new functionality facilitate a sharable, scalable query infrastructure?

a) PMN was extended to allow a single query to target multiple data models at once.

b) The tool was purpose-built to separate the front and back-end components to enable projects that use different data models to more easily leverage existing work (e.g. Race Term) to target additional data models.

c) The system now allows for users to see which database management system is used at each site.

d) Menu-driven queries can be used to generate patient lists that can be shared with the investigator who submitted a query.

Page 105 of 174

Answer: b) The tool was purpose-built to separate the front and back-end components to enable projects that use different data models to more easily leverage existing work (e.g. Race Term) to target additional data models. Explanation: While a single PMN menu-driven query could technically target different data models that share terms (e.g. using the Race Term in a query of different data models that both capture race data), the question or query may not make sense for an investigator to send based on the type of data stored in each data mode. The system is not aware of which RDBMS is used for each data source; the technology utilizes service providers that translate the PMN query to the appropriate SQL syntax for the database upon query execution. MDQs do not return row level data; only counts are returned, which can be stratified by different categories. This project focuses on developing a new approach to Menu-Driven-Queries (MDQ) in PMN that are scalable and extensible and enable efficient querying within a diverse health data network. The legacy query composer in PMN built for MDPHnet was not suitable for use in the diverse ecosystem because was not scalable and much of it was hardcoded for a specific RDBMS and could not easily be repurposed. With advancements in technology, the PMN platform was substantially enhanced to introduce the Microsoft Entity Framework and custom workflow engines to produce the new MDQ tools that enable querying across RDBMS. References:

1. Curtis LC, Brown JS, Platt R. Four Health Data Networks Illustrate The Potential For A Shared National Multipurpose Big-Data Network. Health Affairs. 2014; 33(7):1178-1186.

2. ESP Overview [Internet]. Esphealth.org. 2016. Available from: http://www.esphealth.org/index.php/overview/

3. Data Quality Review and Characterization Programs [Internet]. Sentinelsystem.org. 2016. Available from:

https://www.sentinelsystem.org/sentinel/data/distributed-database-common-data-model/112\

4. Vogel J, Klompas M, Brown JS, Land T, and Platt R. MDPHnet: Secure, Distributed Sharing of Electronic Health Record Data for Public Health Surveillance, Evaluation, and Planning. American Journal of Public Health. 2014 Dec;0: e1-e6.

5. Fleurence RL, Curtis LC, Califf RM, Platt R, Selby JV, Brown JS. Launching PCORnet, a national patient-centered clinical research network. J Am Med Inform Assoc. 2014 Jul-Aug;21(4):578-82.

http://www.esphealth.org/index.php/overview/

https://www.sentinelsystem.org/sentinel/data/distributed-database-common-data-model/112/

https://www.sentinelsystem.org/sentinel/data/distributed-database-common-data-model/112/

Page 106 of 174

S02: Presentations – Population Health Defining Patient Sub-Populations with SNOMED Vaishnavi Kannan S02-2: Your cancer center is offering state-of-the-art treatments for care of patients with renal cell carcinoma. The expert working group of medical oncologists, urologists, and pathologists who jointly provide this care wish to have a list of their patients with renal cell carcinoma available within their common EHR. They also envision real-time clinical decision support tools within the EHR targeted specifically to this population of patients. They do not want other forms of kidney cancer included (e.g. not kidney sarcoma, nephroblastoma, or transitional cell carcinoma). In your health system, physicians use a single EHR for both clinical documentation and billing, in both ambulatory and inpatient settings. Your EHR employs a clinical terminology set of diagnoses supplied by a vendor, pre-mapped to ICD-9, ICD-10, and SNOMED codes. The most streamlined option to generate such an actionable list is:

a) Export clinical and billing data to your enterprise data warehouse (EDW); extract data from the EDW into a Renal Cell Carcinoma registry database by using ICD-10 codes, and seek to interface this database with your EHR.

b) Generate a diagnosis grouper in your EHR with ICD-9 codes, and use that to create an EHR-based renal cell carcinoma registry.

c) Generate a diagnosis grouper in your EHR with ICD-10 codes, and use that to create an EHR-based renal cell carcinoma registry.

d) Generate a diagnosis grouper in your EHR with SNOMED codes, and use that to create an EHR-based renal cell carcinoma registry.

Answer: d) Generate a diagnosis grouper in your EHR with SNOMED codes, and use that to create an EHR-based renal cell carcinoma registry. Explanation: SNOMED includes Renal Cell Carcinoma as a specific concept in its ontology, both as a Disorder and as a Morphologic Abnormality. The Disorder "Renal Cell Carcinoma" is a subtype of "Primary malignant neoplasm of the kidney" and can be cleanly separated from other types of kidney cancers. Using a diagnosis grouper directly in the EHR streamlines EHR-based registry creation and enables real-time clinical decision support within the EHR. The other options are less optimal. Exporting data to the EDW and constructing a standalone Renal Cell Carcinoma registry database may prove desirable for research purposes; however by defining renal cell carcinoma outside the EHR, real-time clinical decision support cannot be provided, absent the additional complexity of constructing a reverse interface. Defining renal cell carcinoma directly in the EHR is simpler.

Page 107 of 174

Neither ICD-9 nor ICD-10 differentiate renal cell carcinoma from other types of kidney cancers, and so lack the clinical granularity needed for this registry. Reference: Benson T, Grieve G. Principles of health interoperability: SNOMED CT, HL7 and FHIR. 3rd ed. London: Springer-Verlag; 2016.

S02: Presentations – Population Health

Enabling Prospective Analysis of Care Redesign with reCAP, The REal-Time Care Analysis Platform

Michael Ripperger

S02-3:

A team of physicians and software developers are tasked with building and implementing a new clinical support tool. The support tool is being designed to decrease the time physicians spend on a complicated task. Once a month, the team meets to assess state of the software. So far, the software has been robust and has many features, but has been consistently critiqued for not being effective. The physicians report that using the software is time-consuming. What would be the most effective change this team could make to reduce the time it takes to complete the complicated task?

a) Bring additional software developers on the team to speed up the existing software.

b) Meet more frequently and have physicians walk through how they use the tool.

c) Deploy the tool to other physicians and analyze uptake.

d) Scrap the software and reassess the usefulness of such a tool.

Answer: b) Meet more frequently and have physicians walk through how they use the tool.

Explanation:

The problem here is most likely that the developed clinical support tool has not been designed to fit into the physicians' workflows. The software is able to complete the task, but does not do so in a way that isstreamlined for what is done in practice. The team would most likely benefit from designing the tool with continual user feedback on usability. This can be accomplished by meeting more frequently and walking through how exactly the tool should work. Since the software has been robust and extensively developed, hiring additional developers would most

Page 108 of 174

likely not increase the capabilities of the software nor lead to a more specific application design. If the tool does not solve the problem more efficiently, it is unlikely to be used by other physicians and so deploying the application would not address the problem. Additionally, scrapping the tool would most likely be a waste of resources since effective software development utilizes changing user requirements anyway.

Reference:

Coiera E. Interaction design theory. International journal of medical informatics. 2003 Mar 31;69(2):205-22.


Telemedicine to manage HIV patients within Pennsylvania Prisons

Jennifer Aldrich

S03-1: Which of the following best describes HIV telemedicine in the Pennsylvania prison system?

a) While geographically and temporally efficient, HIV telemedicine has not been shown to be cost efficient.

b) 62% of HIV-infected prisoners in Pennsylvania are from Philadelphia.

c) Telemedicine promotes continuity of care after release from prison.

d) The issue of EMR system incompatibility was fortunately addressed quickly and with satisfactory resolution by the IT teams of the two respective platforms at the beginning of the Temple/PA DOC partnership.

Answer: d) Telemedicine promotes continuity of care after release from prison. Explanation: Telemedicine promotes continuity of care after release from prison by allowing inmates to establish an ongoing relationship with their outpatient provider while still incarcerated. Many HIV-infected prisoners do not have consistent HIV care before they are in prison, and others are diagnosed during their stay. 40% of inmates in the State of Pennsylvania are from the Philadelphia area, where Temple's providers are located. Reference Jeremy D. Young, MD, MPH, and Mahesh Patel, MD, HIV Subspecialty Care in Correctional Facilities Using Telemedicine. Journal of Correctional Health Care, 2015, Vol. 21(2) 177-185

Page 109 of 174


Feasibility, Acceptability and Impact of a Pediatric Teledermatology Mobile Health Application

Lindsay Berrigan

S03-2:

In this single group, prospective study of a store-and-forward pediatric teledermatology mobile application, which of the following was associated with increased user satisfaction?

a) Time to take photos within the app

b) User-reported ease of taking photographs

c) Receipt of a prescription

d) Time to receive a diagnosis from the dermatologist

Answer: c) Receipt of a prescription

Explanation:

Adjusting for covariates, parents who reported receiving a prescription for their child through this mobile teledermatology app were more likely to be satisfied with the app (Odds Ratio (OR) 3.2 (95% CI: 1.4, 7.7)), more likely to recommend it to friends and family (OR 2.7 (1.0, 7.3)), and more likely to use the app again (OR 3.3 (1.2, 9.1)). Standardized for covariates, the proportion of users reporting receiving a prescription who were satisfied with the app was 81.7% compared to 59.2% among those not reporting receiving a prescription.

Minimizing medication overuse is important. Techniques such as suggesting actions parents can take to reduce symptoms without medication and explicitly ruling out the need for a prescription may reduce the risk of overprescribing while satisfying families. Determining how such communication techniques may best be integrated into teledermatology, and telemedicine more broadly, is an area warranting additional investigation.

Reference:

Resneck JS, Abrouk M, Steuer M, Tam A, Yen A, Lee I, Kovarik CL, Edison KE. Choice, transparency, coordination, and quality among direct-to-consumer telemedicine websites and apps treating skin disease. JAMA dermatology. 2016 Jul 1;152(7):768-75.

Page 110 of 174


Analysis of Abnormal Heart Rates Recorded when Patients were Invited to Upload Personal Fitness Device Data to the Electronic Health Record

Joshua Pevnick

S03-3: Patient-generated data, including patient-entered text in Open Notes, patient-reported outcomes, and biosensor data is increasingly prevalent. Challenges associated with incorporating this patient-generated data into the electronic health record include:

a) Continuously monitoring large numbers of patients’ electronic charts to detect abnormal data

b) Instantly presenting providers with all of the data generated by each of their patients

c) Understanding that abnormal findings are more likely to represent pathology than normal variation, user misunderstanding or error, or device error

d) Making providers responsible for identifying and responding to abnormal patient-generated data

Answer: a) Continuously monitoring large numbers of patients’ electronic charts to detect abnormal data Explanation: A is correct because systems must be put in place to monitor patient-generated data, which differs from most prior EHR data in that physicians haven't initiated data collection (e.g. physician-ordered blood tests). B is wrong because instead of instantly presenting providers with all of the data generated by each of their patients, informaticists will need to organize, simplify, and clearly present patient-generated data to the relevant clinicians. C is wrong because abnormal findings are less likely to represent pathology than they are to represent normal variation, user misunderstanding or error, or device error. D is wrong because informaticists must work to develop organizational rules regarding whether patients, providers, or information technology departments are responsible for identifying and responding to abnormal patient-generated data. Reference: Accenture USA, Department of Health and Human Services (HHS) U.S. Conceptualizing a Data Infrastructure for the Capture, Use, and Sharing of Patient-Generated Health Data in Care

Page 111 of 174

Delivery and Research through 2024, Draft White Paper for a PGHD Policy Framework. (2016) Available at: https://www.healthit.gov/sites/default/files/Draft_White_Paper_PGHD_Policy_Framework.pdf S04: Panel -

Implementing EHR Patient Management Tools in the Pediatric Medical Home: Moving Toward Value-Based Care

Anthony Luberti, Elizabeth Brooks, Jonathan Crossette, Matt Dye

S04-1:

EMR-based patient registries and utilization reports should be used in which of the following ways to enhance care coordination and improve clinical outcomes:

a) Look for opportunities for revenue enhancement

b) Find opportunities to assist clinical staff by automating common processes

c) Rapidly identify gaps in care for specific populations and encourage targeted patient outreach

d) Locate patients who are over-using services

Answer: c) Rapidly identify gaps in care for specific populations and encourage targeted patient outreach

Explanation:

Patient registries and utilization reports in the EMR can be used effectively to determine which patients are not up-to-date on recommended screenings, labs, office visits, prescriptions, etc. With that information, then care coordination staff can target outreach to patients to encourage them to attend appointments and follow up on labs and screenings.

The intention of EMR-based population management tools is not to increase revenue, automate processes, or identify patients who are using many services. Instead, these tools can be used to effectively redistribute work among both clinical and non-clinical staff and promote the most effective outreach. Although patient registries can help you to track and analyze utilization of services, there are many strategies designed to address over-utilization that do not involve automatically shifting a patient’s care over to another provider.

https://www.healthit.gov/sites/default/files/Draft_White_Paper_PGHD_Policy_Framework.pdf

Page 112 of 174

Reference:

JW McAllister, E Presler, WC Cooley, "Practice-based care coordination: a medical home essential". Pediatrics (vol 120:3), Sept. 2007

S04-2:

Having access to payer claims data in a hospital setting is beneficial because:

a) A health system can determine how much comparable hospitals charged the payer for their services

b) Payer claims allow a health system to examine patient utilization outside of their hospital and obtain a more complete picture of patient care and care gaps.

c) Payer claims give hospitals a way to validate activity in the hospital EMR and monitor how well providers are documenting encounter-related information.

d) Payer claims allow hospitals to analyze patient satisfaction at other health systems in order to gauge how satisfied their patients are compared to market.

Answer: b) Payer claims allow a health system to examine patient utilization outside of their hospital and obtain a more complete picture of patient care and care gaps.

Explanation:

Through an arrangement with a payer like an Accountable Care Organization, a health system can review claims data on all care that their patients receive, even outside of their hospital. This allows a care provider to see a much fuller picture of where their patients are going outside of their walls, and potentially determine ways to encourage more appropriate utilization of care (such as discouraging over-use of the emergency room).

With the sharing of claims data from a payer, you would not be able to see what other hospitals were charging for services. Claims data is also not an accurate way to validate any clinical documentation in the EMR, and will not give you information about patient satisfaction.

Reference:

AM Epstein, AK Jha, EJ Orav, DL Liebman, et al. "Analysis of early accountable care organizations defines patient, structural, cost, and quality-of-care characteristics". Health Affairs (vol 33:1), Jan. 2014

Page 113 of 174

S04-3: Population Health tools, deployed correctly in the EMR, will enhance revenue by:

a) Creating improved access to care, increasing visit volume and thus revenue

b) Delivering point of care reminders for clinical services due, thus increasing “bill above” charges

c) Facilitating better and more efficient management of patient populations thus improving performance on payer quality incentive plans

d) Identifying and recruiting new patients into the practice, improving per member per month (PMPM) cap payments

Answer: c) Facilitating better and more efficient management of patient populations thus improving performance on payer quality incentive plans

Explanation:

There are increasing revenue opportunities in payer performance incentive plans. Targeted improvement measures include population health management objectives such as completion of early childhood immunization series, completion of HPV vaccine (by age 13), and reduction of ED utilization. Measures such as these require providers be able to analyze and manage the entire population of patients, identifying gaps in care and initiating outreach interventions.

Reference:

DM Berwick, TW Nolan, J Whittington, "The triple aim: care, health, and cost". Health affairs (vol 27:3), May 2008

S05: Presentation – Data Sciences

Effects of digoxin and diltiazem on mortality among atrial fibrillation patients

Shrie Raam Sathyanarayanan

S05-1:

Among various combinations of race and sex (e.g., Asian males) in this study, which group deviated from the trend of digoxin-associated mortality rates exceeding those for diltiazem?

a) Hispanic males

b) Asian females

c) Caucasian males

d) Hispanic females

Answer: d) Hispanic females

Page 114 of 174

Explanation:

Hispanic was the only racial minority group studied to have higher mortality rates associated with diltiazem (2.9%) than with digoxin (2.6%), although the difference was not statistically significant. Examining the Hispanic group by sex showed rates of mortality were higher for Hispanic males taking digoxin, but among females rates were higher for those taking diltiazem. Although differences between drug-associated mortality rates among Hispanics overall and within males and females were not statistically significant, these patterns deviated from the overall trend of digoxin being associated with higher mortality rates. This suggests an interaction with the effect of drug on mortality risk depending on the values of race and gender.

Reference:

https://sites.uni.edu/butlera/courses/org/modmed/moderator_mediator.htm


Estimating Data Requirements to Detect Pediatric Critical Decompensation

Melissa Aczon

S05-2:

Which of the following best describes the recommended strategy for improving the RNN model's performance in predicting critical decompensation of floor patients?

a) Increase the number of parameters to make the RNN larger and deeper.

b) Collect and use more data for training, including higher resolution time series.

c) Optimize the initialization points of the RNN's weight matrices using a separate validation set.

d) Use the GRU (gated recurrent unit) architecture instead of LSTM (long-short term memory).

Answer: b) Collect and use more data for training, including higher resolution time series.

Explanation:

While A, C and D are possible options to consider, the main point of this presentation is the RNN's data requirements. The work showed that increasing the number of patient encounters used in training the RNN led to improved performance, where performance was measured in terms of AUROC.

Page 115 of 174

Reference:

Goodfellow I, Bengio Y, Courville A. Deep learning. MIT Press; 2016 Nov 10.


Mining Care Pathways of Multimorbid Patients from Electronic Health Records

Yiye Zhang

S05-3:

This study proposes a new approach to incorporate medical costs explicitly in the chronologically ordered, clinical pathways (CPs) of patient experiences (Zhang, 2016). We analyzed CPs of patients with multiple chronic conditions using a cost-centered perspective, as well as a clinically focused perspective, to show alignment in some subgroups and significant variations in others. Which of the following best explains our finding, by comparing patient cohorts and their CPs categorized by the two approaches?

a) We found variations in costs among clinically similar patients, and variations in clinical complexity among patients with similar costs.

b) We found that there is a substantial overlap among patients who are relative low-spenders of medical services and patients who are clinically highly complex.

c) We found decisive evidence of excessive use of medical services by patients who were classified as having relatively low clinical needs.

d) We found exact overlap among the CPs of patients with medium level of medical spending and patients with medium clinical needs.

Answer: a) We found variations in costs among clinically similar patients, and variations in clinical complexity among patients with similar costs.

Explanation:

Our methods identified three CP-based, and four cost-based, patient subgroups. Two sets of subgroups from each approach indicated some clinical similarity in terms of average statistics, such as number of diagnoses and medication needs. However, the CP-based subgroups displayed significant variation in costs; conversely, large differences in clinical needs were observed among cost-based subgroups.

Reference:

Page 116 of 174

Zhang Y, Padman R. Data-driven clinical and cost pathways for chronic care delivery. The American journal of managed care. 2016;22(12):816-820.

S06: Presentations – Clinical Decision Support

Development of a Web-based Decision Support Tool for Operationalizing and Optimizing Management of Hyperbilirubinemia in Preterm Infants

Yassar Arain

S06-1:

Current guidelines for hyperbilirubinemia treatment in premature infants are based on expert-consensus recommendations. There is wide variability in management of hyperbilirubinemia in this patient population, given the dearth of evidence-based recommendations. Our CDS tool, Premie BiliRecs, has operationalized expert-consensus recommendations by providing discrete Total Bilirubin thresholds for initiating phototherapy, based on the patient's Post-Menstrual Age. (The PMA is similar term for “gestational age,” referencing the infant. For example, a baby born at 27 weeks and 2 days who is 7 days old, has a PMA of 28 weeks and 2 days.) With EMR-integration, the greatest value of our CDS tool is in its ability to do which of the following?

a) Decrease use of phototherapy

b) Generate new data

c) Improve workflow

d) Standardize care

Answer: b) Generate new data

Explanation:

With EMR-integration, our CDS tool facilitates improved workflow and standardization fo care. However, the greatest potential to improve care is in its ability to gather patient-specific data for premature infants treated for hyperbilirubinemia. Analyzing the outcomes of these infants will add to the evidence base available for optimization of the current consensus-based guidelines. As guidelines are updated, their incorporation into clinical practice can be supported through iterative updates to the recommendations provided by our CDS tool.

Reference:

Jackson, Terri. Building the ‘continuous learning’ healthcare system. HIM J. 2014;43(1):4-5.

Page 117 of 174

S06: Presentations – Clinical Decision Support Design and Implementation of a Real Time Acuity Detection System in a Pediatric ICU Eric Shelov S06-2: You are creating Clinical Decision Support for your patient acuity system. Your team has gathered historical data on a cohort of patients and has identified patients who would have been triggered by the CDS system and those who went on to deteriorate. You then categorized the patients and the triggers into a 2x2 table to look at true positives, false positives, true negatives and false negatives. Which one of the following should primarily drive the intrusiveness and strength of the recommendation embedded in the CDS?

a) Sensitivity

b) Specificity

c) Positive Predictive Value

d) Negative Predictive Value

Answer: c) Positive Predictive Value Explanation: The Positive Predictive Value, or true positive rate, provides the best indicator of how imperative the message in the CDS should be. Strong recommendations or interruptive CDS paired with a low PPV risks inappropriate action and alert fatigue. Reference: Osheroff JA. Improving outcomes with clinical decision support: an implementer's guide. HIMSS. S06: Presentations: Extracting the amount of alcohol consumption from clinical narratives using Natural Language Processing for use in clinical decision support tools Rajeev Chaudhry S06-3: Alcohol consumption by patients is documented as free text in the clinical notes narrative at your institute. Which of the following is a true statement regarding abstraction of the units of alcohol consumption from the clinical notes for a defined group of patients at your institute?

Page 118 of 174

a) Extraction of alcohol consumption from the notes is easy as it is documented in discrete format

b) Chief complaint part of the notes is best to target for determining alcohol consumption

c) Use of alcohol consumption is needed for determining individualized risk for osteoporosis and bleeding for anticoagulants

d) Alcohol use is protective for heart and bone health

Answer: c) Use of alcohol consumption is needed for determining individualized risk for osteoporosis and bleeding for anticoagulants Explanation:

A. is wrong as alcohol consumption is documented as text and NLP is needed

B. is wrong as social history part of clinical note is used by clinicians to document alcohol use

C. Is correct as amount of alcohol consumed daily or weekly changes risk for osteoporosis and bleeding from anticoagulants.

D. is wrong as alcohol use in moderation may be protective for heart health, its use (especially abuse) can lead to osteoporosis.

References: • Pisters, R., et al. (2010) "A novel user-friendly score (HAS-BLED) to assess 1-year risk of

major bleeding in patients with atrial fibrillation: the Euro Heart Survey." Chest Journal 138, No. 5, 1093-1100.

• Kanis, J.A., Johansson, H., Johnell, O. et al. (2005) “Alcohol intake as a risk factor for

fracture.” Osteoporosis International 16: 737.

Page 119 of 174

S07: Panel - Social Media Language Analysis for Healthcare

Lyle Ungar, Anneke Buffone, J. Eichstaedt, Hansen Schwartz

S07-1:

The Yelp™ website is comprised of user-generated content. Users post reviews and rate their experiences with businesses they have patronized, including health care providers and hospitals. Give the best answer concerning Yelp™ reviews and formal HCAHPS reviews of hospitals:

a) HCAHPS covers many topics that Yelp™ misses

b) Yelp™ covers many topics missed by HCAHPS

c) Each covers many topics the other misses

d) Both have similar coverage

Answer: b) Yelp™ covers many topics missed by HCAHPS

Explanation:

The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey is the US standard for evaluating patients’ experiences after hospitalization. We compared the content of Yelp narrative reviews of hospitals to the topics in the HCAHPS survey, called domains in HCAHPS terminology. While the domains included in Yelp™ reviews covered the majority of HCAHPS domains, Yelp™ reviews covered an additional twelve domains not found in HCAHPS. The majority of Yelp™ topics that most strongly correlate with positive or negative reviews are not measured or reported by HCAHPS. The large collection of patient- and caregiver-centered experiences found on Yelp can be analyzed with natural language processing methods, identifying for policy makers the measures of hospital quality that matter most to patients and caregivers. The Yelp™ measures and analysis can also provide actionable feedback for hospitals.

Reference:

Ranard BL, Werner RM, Antanavicius T, Schwartz HA, Smith RJ, Meisel ZF, et al.. Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care. Health Affairs. 2016;35(4): 697-705.

S07-2:

A recent study by Eichstaedt et al. looked at the language expressed on Twitter to characterize community-level psychological correlates of age-adjusted mortality from atherosclerotic heart

Page 120 of 174

disease (AHD). Based on that study, how well did Twitter predict county-level heart disease rates, when compared to traditional methods?

a) About as well as demographics

b) About as well as health risk factors

c) About as well as income and education

d) Significantly better than all of the above combined

Answer: d) Significantly better than all of the above combined

Explanation:

We used language expressed on Twitter to characterize community-level psychological correlates of age-adjusted mortality from atherosclerotic heart disease (AHD). A cross-sectional regression model based only on Twitter language predicted AHD mortality significantly better than did a model that combined 10 common demographic, socioeconomic, and health risk factors, including smoking, diabetes, hypertension, and obesity. Capturing community psychological characteristics through social media is feasible, and these characteristics are strong markers of cardiovascular mortality at the community level.

Reference:

Eichstaedt JC, Schwartz, HA, Kern ML, Park G, Labarthe DR, Merchant RM, et al. Psychological language on Twitter predicts county-level heart disease mortality. Psychological Science. 2015;26(2): 159-169.

S07-3:

Which of the following is true of caregiver empathy?

a) Empathy is a clearly defined construct with a widely agreed upon definition for researchers across disciplines. It means putting yourself in the shoes of your patient.

b) The more empathic a caregiver is, the better patient outcomes. Caregivers who are more empathetic also tend to be more satisfied with their jobs, which positively affects their health and well-being.

c) Empathy has a widely agreed upon definition across academic disciplines. Social media language can identify empathic language accurately and provide insights into the emotional, cognitive, and behavioral realities of those higher and lower in empathy. We find empathy to predict higher scores for provider health, job satisfaction, and well-being and for patient satisfaction.

Page 121 of 174

d) Empathy can have both positive and negative consequences for health and well-being of caregivers. There are at least two separate forms of empathy, one health-demoting where the caregiver takes on the patient’s feelings and cognitively merges with the patient, and one health-promoting where the caregiver has warm, tender, and concerned feelings for the patient, but does not cognitively or emotionally merge with the patient. Both of these can be detected in human language.

Answer: d) Empathy can have both positive and negative consequences for health and well-being of caregivers. There are at least two separate forms of empathy, one health-demoting where the caregiver takes on the patient’s feelings and cognitively merges with the patient, and one health-promoting where the caregiver has warm, tender, and concerned feelings for the patient, but does not cognitively or emotionally merge with the patient. Both of these can be detected in human language.

Explanation:

This work is based on previous research by Anneke Buffone that showed evidence for a form of self-other merging empathy (pathogenic empathy) that is associated with a physiological stress response (HPA + SAM activation) and a form of empathy (salutogenic empathy) that is associated with a challenge response (SAM activation). We created a survey to discern these two distinct empathic types, salutogenic empathy which is associated with an appropriate level of self-other distance, warmth and compassion, more deliberate reasoning, and greater other focus than self focus and pathogenic empathy, an empathic response that is associated with making another person’s suffering your own, equal self and other focus, automatic reasoning and negative affect. We identified both empathic types in Facebook language and built predictive models to assess both of them in large samples. Pathogenic empathy was found to predict stress, lower self-rated health, higher rates of problem drinking, and several medical diagnoses. Salutogenic empathy was associated with lower stress and better health outcomes.

Reference:

Buffone AEK, Abdul-Mageed M, Eichstaedt JC, Giorgi S, Peng H, Smith LK, et al. Health-promoting and health-demoting forms of empathy: Data-driven insights into characteristics of two types of prosocial responders – and associations with perceived stress and well-being. Forthcoming 2017.

Page 122 of 174

S08: Panel - Improving Access to Care and Efficiency: Patient-Centered Methods and Strategies

Bradley Doebbeling, Tammy Toscos, Kislaya Kunjan, Huanmei Wu

S08-1: Positive deviance, a strategy from complexity science, is a useful framework in designing system redesigns because it:

a) Helps identify approaches that work in your environment

b) Identifies people who are not trustworthy

c) Finds stakeholders who do not have time to participate

d) Has not been shown to work for multiple social, public health and clinical problems

Answer: a) Helps identify approaches that work in your environment Explanation: Positive Deviance is a social change strategy. Researchers call individuals who employ effective behaviors "positive deviants." Positive deviants are the minority of people who employ beneficial health behaviors while the majority do not. These beneficial behaviors are likely to be “affordable, acceptable, and sustainable because they are already practiced by at risk people, they do not conflict with local culture, and they work.” (Marsh et al, 2004). The positive deviance approach is a way to engage stakeholders from the bottom up, since the beneficial strategies are already in use by some in the community. This approach can be used to identify effective change approaches, incorporate innovative measurement approaches, and to internally implement and spread adoption of positive behaviors. References: Marsh DR, Schroeder DG, Dearden KA, Sternin, J, Sternin M. The power of positive deviance BMJ. 2004 Nov 13;329(7475):1177-9. Rose AJ, McCullough MB. A Practical Guide to Using the Positive Deviance Method in Health Services Research. Health Serv Res. 2016 Jun 28. 1475. S08-2: What was the most important function of the data warehouse in this PCORI project?

a) Patient Visit Planning/Morning Huddles

Page 123 of 174

b) Improve Clinical Quality Measures

c) Support Scheduling Dashboards

d) Claims Reporting

Answer: c) Support Scheduling Dashboards Explanation: While the data warehouse contained data elements pertaining to clinical, operational, quality, and billing, the focus of this project was on improving access through schedule optimization. Dashboards were used as a means to monitor and optimize the open access intervention. Reference: Kunjan K, Toscos T, Turkcan A, Doebbeling BN. A Multidimensional Data Warehouse for Community Health Centers. In AMIA Annual Symposium Proceedings 2015 (Vol. 2015, p. 1976). American Medical Informatics Association. S08-3:

What is the most significant challenge for health centers in terms of reporting and analytics?

a) Poor EHR Interoperability/HIE

b) Resource Constraints (Personnel, Capital)

c) Inefficient Clinic Workflow

d) High HER Implementation Cost

Answer: b) Resource Constraints (Personnel, Capital)

Explanation:

We found that although most community health centers had implemented an EHR, they typically did not have the resources in terms of personnel or capital, to be able to create the reports needed for analytics.

Reference:

Kunjan K, Toscos T, Turkcan A, Doebbeling BN. A Multidimensional Data Warehouse for Community Health Centers. In AMIA Annual Symposium Proceedings 2015 (Vol. 2015, p. 1976). American Medical Informatics Association.

Page 124 of 174


If (Integrating Data, Design, And Technology): Print(“Value Institute Design Lab”)

Rebecca Kowalski

S09-1:

Sepsis has become of increasing concern to public health in the past 30 years. This is due to the high cost of hospitalizing patients with sepsis, which accounted for $20 billion of total hospital costs in 2011, and its high mortality rate, ranging from 25-50% depending on severity. Because of this, people are working hard to find a standard way of defining and responding to sepsis. Although this has not yet been achieved, CMS does have guidelines on what defines sepsis.

Which of the options represents the current CMS guidelines for sepsis?

a) Suspected Infection

b) Meeting two SIRS (Systematic Inflammatory Response Syndrome ) criteria + suspected infection

c) Organ failure

d) Meeting two SIRS (Systematic Inflammatory Response Syndrome ) criteria + suspected infection + organ failure

Answer: b) Meeting two SIRS (Systematic Inflammatory Response Syndrome ) criteria + suspected infection

Explanation:

CMS guidelines currently define sepsis as meeting two SIRS (Systematic Inflammatory Response Syndrome ) criteria + suspected infection. Severe sepsis is defined as sepsis + evidence of organ disfunction including Lactate > 2, platelets < 100,000, creatinine > 2.0, and others.

Reference:

Dellinger et al (2013). Surviving sepsis campaign: International Guidelines for Management of Severe Sepsis and Septic Shock: 2012. Critical Care Medicine 41(2): 580-637.

Page 125 of 174

S09-2:

We've established that sepsis is a problem and a financial burden for hospitals. The expense mainly comes from hospitalization for cases of severe sepsis, but how prevalent is severe sepsis? What is the national estimate of severe sepsis according to a study done on data from 2001-2004?

a) 0.05%

b) 0.007%

c) 0.7%

d) 1.2%

Answer: c) 0.7%

Explanation:

The national estimate of sepsis according to our study was 0.7%, equal to about 2.3 million people out of 331.5 million adult ED visits.

Reference:

Wang HE, Shapiro NI, Angus DC, Yealy DM (2007). National estimates of severe sepsis in United States emergency departments. Crit Care Med 35(8):1928-1936, 2007.


Leveraging search patterns in electronic health records to make information retrieval more efficient

Titus Schleyer, Xia Ning, Martin Doug

S09-3:

Typically, the information retrieval workflows in electronic health records (EHR) are not oriented around clinical conditions, forcing clinicians to retrieve relevant data from a variety of places in the EHR. Thus, physicians must spend significant amounts of time retrieving and synthesizing data from EHRs. Which of the following EHR features can help address this problem?

a) clinical decision support algorithms

b) overview screens showing the most recent information about the patient

c) disease-specific dashboards

d) Google-like search

Page 126 of 174

Answer: c) disease-specific dashboards

Explanation: The key to this question is to recognize that data relevant to a patient condition are typically spread out in various parts of the EHR. While clinical decision support algorithms can highlight important information at particular points in time, they typically do not offer a collection of relevant data to help clinicians make a diagnosis or select a therapy. Overview screens showing the most recent information about a patient can be useful to get a sense of “what is urgent and relevant” to address right now. However, such overview screens are typically not focused on single conditions/problems. Google-like search functionality can help make information retrieval more efficient, but clinicians have to search for each desire data element separately. The correct answer is C, “disease-specific dashboards,” which are specifically designed to display the most relevant data about a patient condition.

Reference: Koopman RJ, Kochendorfer KM, Moore JL, Mehr DR, Wakefield DS, Yadamsuren B, et al. A diabetes dashboard and physician efficiency and accuracy in accessing data needed for high-quality diabetes care. Ann Fam Med. 2011;9(5):398–405.

S09-4:

Both custom-designed, condition-specific dashboards and recommender systems can make information retrieval from EHRs more efficient. Recommender systems analyze past search and information retrieval patterns in EHRs to identify information that may be relevant to a clinician’s review of a patient case. Compared to dashboards, recommender systems:

a) may cause clinicians to miss important data.

b) are laborious to implement for multiple conditions and diseases.

c) always produce predictable results.

d) require medical expert opinion for design.

Answer: a) may cause clinicians to miss important data.

Explanation: Recommender systems analyze past information search and retrieval patterns of clinicians to predict what information might be relevant for a particular patient case. Developing recommendation system entails designing algorithms that generate the most appropriate recommendations based on information search and retrieval patterns. Once the algorithms are written, they can be easily implemented for multiple conditions and diseases, since they are content-neutral. Recommender systems don’t always produce predictable results because they are driven by search patterns of individual or groups of clinicians. Thus, a data item that a clinician expects might not be displayed if other clinicians have not retrieved it frequently. Recommender systems do not require medical expert opinion for design. Condition-

Page 127 of 174

specific dashboards, on the other hand, do. Because recommender systems make certain data much more easily accessible to clinicians, they may cause clinicians to avoid the effort of searching for additional data. Thus, important data might be missed.

Reference:

King AJ, Cooper GF, Hochheiser H, Clermont G, Visweswaran S. Development and preliminary evaluation of a prototype of a learning electronic medical record system. AMIA Annu Symp Proc. 2015;1967–75.

S10: Panel – Immediate Adaptability

Yalini Senathirajah. Andre Kushniruk, Jon Patrick, Ross Koppel, Elizabeth Borycki

S10-1:

The most important value of Immediate Adaptability is:

a) Eliminate workarounds in workflow

b) Prevent workflow from being altered by staff

c) Ensure that a system is optimal for the community of practice

d) Abandon traditional EMR technologies

Answer: c) Ensure that a system is optimal for the community of practice

Explanation:

A system can only be optimal for a community of practice if it is customizable to their needs and support their Continuous Process Improvement strategies. If an EMR is immutable, CPI cannot be achieved.

Reference:

Patrick J. Part One: Immediate Adaptability (IA). March 9, 2016. Available at: http://www.jon-patrick.com/2016/03/immediate-adaptability/

S10-2:

The best method for determining user needs in immediate adaptability (for making changes to systems) is:

http://www.jon-patrick.com/2016/03/immediate-adaptability/

http://www.jon-patrick.com/2016/03/immediate-adaptability/

Page 128 of 174

a) Rapid-low cost usability testing

b) Clinical simulations

c) Workflow analysis

d) Observation

Answer: a) Rapid-low cost usability testing

Explanation:

Rapid low-cost usability testing methods can be used to determine how to change, modify or customize a user interface or system in order to attain adaptability in an efficient and effective way.

References:

Baylis, T. B., Kushniruk, A. W., & Borycki, E. M. (2011). Low-cost rapid usability testing for health information systems: is it worth the effort? Studies in Health Technology and Informatics, 180, 363-367. Kushniruk, A. W., & Borycki, E. M. (2006). Low-cost rapid usability engineering: designing and customizing usable healthcare information systems. Healthcare Quarterly, 5(2), 98-102.

S10-3:

You are in an area of healthcare in which several things are happening simultaneously as a result of fiscal and technological change. These include the need to incorporate many new sources of information, such as app data from patients, feeds from other institutions, algorithmic approaches to decision support based on population statistics, and the need to present information rapidly in a way doctors can grasp quickly. At the same time concerns about technology induced errors are increasing.

In this type of situation one of the main most important advantages of immediate adaptability are:

a) The architecture allows rapid incorporation of new information sources and visualizations

b) User control of an adaptable system can allow new insights

c) It requires increased thinking and design in creation of the system, but lower costs of change and redevelopment/new creation

d) There are no particular advantages

Page 129 of 174

Answer: c) It requires increased thinking and design in creation of the system, but lower costs of change and redevelopment/new creation

Explanation:

The lower costs of change and development of new functions and features (which can include visualizations and information sources) mean problems can be solved more quickly and a great variety of options tried.

References:

Senathirajah Y, Bakken S, Kaufman D. The clinician in the Driver's Seat: part 1 - a drag/drop user-composable electronic health record platform. J Biomed Inform. 2014 Dec;52:165-76. doi: 10.1016/j.jbi.2014.09.002. Epub 2014 Sep 18. PMID: 25240253

Senathirajah Y, Kaufman D, Bakken S.The clinician in the driver's seat: part 2 - intelligent uses of space in a drag/drop user-composable electronic health record. J Biomed Inform. 2014 Dec;52:177-88. doi: 10.1016/j.jbi.2014.09.008. Epub 2014 Oct 24. PMID:25445921


Decision Modeling of Clinical Content for Collaborative Documentation Purposes in the Management of Chronic Conditions

Hari Nandigam

S11-1:

Clinical Informaticians at a hospital wanted to develop clinical content for Emergency Department. They wanted to store their information in HL7 standard resource model. What is the best approach in their modeling process for their content development?

a) Conceptual, Physical, Logical

b) Physical, Logical, Conceptual

c) Conceptual, Logical, Physical

d) Logical, Conceptual, Physical

Answer: c) Conceptual, Logical, Physical Explanation: The three levels of data modeling are conceptual data model, logical data model, and physical data model.

Page 130 of 174

References: http://www.1keydata.com/datawarehousing/data-modeling-levels.html https://en.wikipedia.org/wiki/Data_modeling


An Informatics Framework for Value-based Care at NewYork-Presbyterian

Gilad Kuperman

S11-2:

A health care provider organization is planning to enter into a value-based care contract with a commercial payer. The payer will reward the provider for decreasing the total cost of care for the payer’s beneficiaries that receive the bulk of their care from the provider.

Which data resource should the provider organization use to identify the population of patients for which it will be responsible under the contract and could benefit the most from expensive care management interventions?

a) The provider organization’s clinical database

b) An epidemiologic database provided by the local public health department

c) A database of claims transactions provided by the payer for all patients that had care at the provider

d) The Death Master File provided by the Social Security Administration

e) The Medicare claims database

Answer: c) A database of claims transactions provided by the payer for all patients that had care at the provider

Explanation:

The payer’s claims data will identify all of the payer’s beneficiaries and all of the care that the patient received, regardless of location. The provider’s clinical database will only include care provided at the provider organization so will not include care provided elsewhere; the contract will require knowledge of care provided at other locations. The public health database will not identify patients by provider or payer. The Death Master File is irrelevant to the needed analysis. The Medicare claims are not relevant because this is a commercial payer.

http://www.1keydata.com/datawarehousing/data-modeling-levels.html

https://en.wikipedia.org/wiki/Data_modeling

Page 131 of 174

Reference:

Bailit M, Hughes C. Key design elements of shared-savings payment arrangements. Issue Brief (Commonw Fund). 2011 Aug;20:1-16.

S11: Presentation – Health Policy and Payment Reform

Re-Identification Risk in HIPAA Safe Harbor De-Identified Datasets: Using non-HIPAA identifiers / the case of the MVA Attack

Peter Elkin

S11-3:

You are an ER physician when a patient is brought in with life-threatening injuries from motor vehicle accident involving a drunk driver. After the patient has been stabilized, your third year medical student, who diligently obtained information about the accident from the EMTs, remarks that the accident is 'all over the news.' You review the medical student's note and are concerned that there is a risk that it could be used to re-identify the patient if released in the future even if HIPAA identifiers are removed. What information is it most important for you to redact from the note to protect patient privacy while preserving the value of the anonymized note for research?

a) The time of day (hour and minute) the accident took place

b) The name of the hospital to which the patient was taken

c) The model of car the patient was driving

d) The ICD-10 diagnosis codes for the patient's injury

Answer: c) The model of car the patient was driving

Explanation:

It is the most identifying and is often tracked in news articles.

References:

1. L Sweeney. k-anonymity: a model for protecting privacy. International Journal on Uncertainty, Fuzziness and Knowledge-based Systems, 10(5):557–70, 2002.

2. D McGraw. Building public trust in uses of health insurance portability and accountability act

https://urldefense.proofpoint.com/v2/url?u=https-3A__eresources.library.mssm.edu-3A2056_pubmed_-3Fterm-3DBailit-2520M-255BAuthor-255D-26cauthor-3Dtrue-26cauthor-5Fuid-3D21847849&d=DwMGaQ&c=apLCJo22jkVRpFivmRGGHnRn85FoLi_g9mEBSlVKwRY&r=G3I-a1_BJpFlwOy3wsU1sg&m=FGNgxKoYoBZIuyCKjvN87_ICY9_csPncjCo_U2sPWro&s=rq78BN72iJNk6y8Y933Smplvs1joOxHJAVOwehJN7Dk&e=

https://urldefense.proofpoint.com/v2/url?u=https-3A__eresources.library.mssm.edu-3A2056_pubmed_-3Fterm-3DHughes-2520C-255BAuthor-255D-26cauthor-3Dtrue-26cauthor-5Fuid-3D21847849&d=DwMGaQ&c=apLCJo22jkVRpFivmRGGHnRn85FoLi_g9mEBSlVKwRY&r=G3I-a1_BJpFlwOy3wsU1sg&m=FGNgxKoYoBZIuyCKjvN87_ICY9_csPncjCo_U2sPWro&s=1sjJZ4R7i65ljN3fEi5py_vkNU5HUOWFa1rKJ2zY6sY&e=

https://urldefense.proofpoint.com/v2/url?u=http-3A__www.commonwealthfund.org_-7E_media_files_publications_issue-2Dbrief_2011_aug_1539-5Fbailit-5Fkey-5Fdesign-5Felements-5Fsharedsavings-5Fib-5Fv2.pdf&d=DwMGaQ&c=apLCJo22jkVRpFivmRGGHnRn85FoLi_g9mEBSlVKwRY&r=G3I-a1_BJpFlwOy3wsU1sg&m=FGNgxKoYoBZIuyCKjvN87_ICY9_csPncjCo_U2sPWro&s=EWnK2ZJBq8245PotW4VqzL6qpt9mTEVQ0-y5rcnAS1Q&e=

https://urldefense.proofpoint.com/v2/url?u=https-3A__eresources.library.mssm.edu-3A2056_pubmed_-3Fterm-3D-2522Key-2BDesign-2BElements-2Bof-2BShared-2DSavings-2BPayment-2BArrangements-2522&d=DwMGaQ&c=apLCJo22jkVRpFivmRGGHnRn85FoLi_g9mEBSlVKwRY&r=G3I-a1_BJpFlwOy3wsU1sg&m=FGNgxKoYoBZIuyCKjvN87_ICY9_csPncjCo_U2sPWro&s=fvjLseQusWmnIocAbThE3zxFaR93qCwW2DCXtNOPgoo&e=

https://urldefense.proofpoint.com/v2/url?u=https-3A__eresources.library.mssm.edu-3A2056_pubmed_-3Fterm-3D-2522Key-2BDesign-2BElements-2Bof-2BShared-2DSavings-2BPayment-2BArrangements-2522&d=DwMGaQ&c=apLCJo22jkVRpFivmRGGHnRn85FoLi_g9mEBSlVKwRY&r=G3I-a1_BJpFlwOy3wsU1sg&m=FGNgxKoYoBZIuyCKjvN87_ICY9_csPncjCo_U2sPWro&s=fvjLseQusWmnIocAbThE3zxFaR93qCwW2DCXtNOPgoo&e=

Page 132 of 174

de-identified data. J Am Med Inform Assoc, 20(1):29–34, 2013.

3. P Kwok and D Lafky. Harder than you think: A case study of re-identification risk of hipaa-compliant records.

4. K El Emam, E Jonker, L Arbuckle, and B Malin. A systematic review of re-identification attacks on health data. PLoS One, 6(12):e28071, 2011.

5. L Zhao, J Lucado, and C Stocks. Emergency department visits associated with motor vehicle accidents, 2006


Predictive Analytics for Pediatric Diabetic Keto-Acidosis (DKA) using Features Extracted from Asynchronous Multivariate Data Streams

Rema Padman

Extracting predictive features from asynchronous multivariate data streams for early prediction of diabetic ketoacidosis (DKA) in pediatric type 1 diabetes patients is a challenging problem. This study proposes a smoothing technique for segmenting measurements collected at irregularly spaced time points into trend and value abstractions that can be added as new features to standard prediction models.

Suggested by this study, what benefit can be obtained from building prediction models that include trend and value abstractions from asynchronous multivariate EHR data streams?

a) Confidence intervals for DKA prediction

b) Improved prediction accuracy

c) Identification of the best interventions for an individual patient

d) Identification of patient clusters that may benefit from an intervention

Answer: b) Improved prediction accuracy

Explanation:

The study shows that using the abstracted temporal features in building prediction models improves predictive accuracy over models that solely consider summary statistics such as averages and counts, but does not provide confidence intervals for the predictions. The method also suggests possible interventions for each patient by highlighting the most important

Page 133 of 174

predictors, but does not identify the best intervention. Identifying clusters of patients who may benefit from a specific intervention cannot be obtained from the prediction models.

Reference:

Diwakar IB, Chouldechova A, Clements MA, Padman R. On Extracting Features from Asynchronous Multivariate Data Streams. Fourth international conference on Business Analytics and Intelligence (ICBAI), December 19-21 (2016), IISc., Bangalore, India.


Identifying Patterns of Co-occurring Medical Conditions through Topic Models of Electronic Health Records

Moumita Bhattacharya

As discussed in the abstract, we represent each patient file as a probability distribution over SNOMED codes, where each such distribution is referred to as a topic. Each distribution carries certain information contents indicating how informative the associations among codes are, where a uniform distribution, which assigns an equal probability to all codes, carries the lowest information contents.

Each of the three graphs shown below plots a probability distribution over 180 SNOMED codes, thus each representing a topic, denoted Topic A, Topic B and Topic C. The x-axis of each plot corresponds to the 180 codes, while the y-axis shows the conditional probability per code to occur in the respective topic.

Which of the three topics shown provides the most information about likely associations among conditions?

Page 134 of 174

a) Topic A

b) Topic B

c) Topic C

d) None of the above. All topics show the same information contents.

Answer: b) Topic B

Explanation: As can be seen from the graphs, Topic B is characterized by a small number of SNOMED codes that have a relatively high probability (between 0.05 and 0.25) of being associated with the topic. In contrast, Topic A and Topic C are both characterized by a high number of SNOMED codes (20 and 50, respectively) that all have a similar relatively low probability ( ≤ 0.05) to be associated with the respective topic. Since topics A and C assign similar low probabilities to a large number of codes, they carry much less information contents about any specific code-associations compared to Topic B.

Quantitatively, the entropy of Topic A, B and C is 4.32, 2.71 and 5.48 respectively. Topic B has a significantly lower entropy compared to that of topics A and C, which indicates that Topic B indeed bears higher information contents. Hence, Topic B provides the most information about likely associations among conditions.

Reference:

Lin, J. Divergence measures based on the Shannon entropy. IEEE Transactions on Information theory. 1991; 37(1): 145-151.


Developing and Implementing a Reliable and Validated Solution for Big Data Text Analytics (NLP, Text Mining) at a Tertiary Pediatric Hospital. Luis Ahumada, William Nieczpiel, John Martin In our proposed approach, we have combined techniques from several areas, including NLP, knowledge engineering, and machine learning to implement an automated framework for annotating physician notes. Additionally, as an immediate benefit, the produced framework

Page 135 of 174

facilitates the retrieval of notes on demand, and allows the collection of empirical domain knowledge that can be formalized with the help of an ontology to also permit reasoning. Given that all the following tasks are part of the general Text Analytics framework, which group of tasks is more appropriate to produce more informative features from physician notes using an NLP engine?

a) Text categorization, document clustering and information retrieval

b) Document summarization, information extraction, question answering

c) Ambiguity, data sparsity, redundancy

d) Bootstrapping, labeled and unlabeled data

Answer: b) Document summarization, information extraction, question answering Explanation: The ultimate goal of the proposed approach is to discover interesting and reliable facts about patients that clinicians can use to support quality improvement activities or help improve practice. Applying natural language processing to (B) document summarization and information extraction at the sentence- or paragraph- level provides more informative and useful clinical features. The result of this process is a rich and useful encoded set of clinical concepts that otherwise will not be generated by the bag-of-words approach using statistical and machine learning methods applied to (A) text categorization, document clustering and information retrieval at the document level. Answer option (C) represents known text mining problems and (D) represents known machine learning techniques. References: 1. Tonkin, E., & Tourte, G. J. L. (2016). Working with text: Tools, techniques and approaches for text mining. Cambridge, MA;Kidlington, UK;: Chandos Publishing. 2. Kao, A., & Poteet, S. R. (2007). Natural language processing and text mining. London: Springer.

Page 136 of 174

S13: Panel - Should Drug-Drug Interaction Seriousness Ratings Be Taken Seriously? Interactive Panel

Scott Nelson, John Horn, Joan Kapusnik-Uner, Bimal Desai, Richard Boyce

S13-1:

Which of the following organizations has drug-drug interaction clinical significance code value set for drug utilization reviews (DUR) messaging?

a) HL7 - Health Level-7

b) NCPDP - National Council for Prescription Drug Programs

c) PQA – Pharmacy Quality Alliance

d) AHRQ - Agency for Healthcare Research and Quality

e) NIST - National Institute of Standards and Technology

Answer: b) NCPDP - National Council for Prescription Drug Programs

Explanation: A code for the clinical significance is required by NCPDP

Reference:

http://www.quickscrip.net/new/uncategorized/dur-codes/

S13-2:

Which one of the following statements does not apply to the implementation of patient specific DDI alerts?

a) The total number of alerts triggered per patient will not be affected.

b) Alert filters can be adjusted to reflect the requirements of the user.

c) The ordering of an interacting drug pair will trigger an alert each time the pair is ordered.

d) Common filters include drug doses, routes of administration, and laboratory values

Answer: c) The ordering of an interacting drug pair will trigger an alert each time the pair is ordered.

Explanation:

Page 137 of 174

Patient specific DDI alerting is based on drug and patient features that have been shown to modify the risk of an adverse outcome if the potentially interacting drugs are coadministered. The common features that can be employed as filters include drug dose, duration, route of administration, order of administration, concurrent diseases, renal function, laboratory values, pharmacogenetics and age. Patient specific alerts are sensitive and specific, user customizable, and reduce the number of alerts triggered.

Reference:

Payne TH, Hines LE, Chan RC et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc. 2015;22:1243-50.

S13-3:

Beth is reviewing the physician’s discharge order for Maria. Maria is a 72-year old woman who was admitted to the hospital with acute decompensated heart failure. While reviewing Maria’s medications, Beth sees an alert that informs her that Maria is being discharged with spironolactone, a potassium-sparing diuretic that could potentially interact with the potassium chloride that Maria had been taking at home to treat low potassium levels. The alert also notes that Maria is being discharged with an ACE inhibitor lisinopril for heart failure. Which of the following is most appropriate additional information that the alert should provide to help Beth ensure that Maria's medication therapy is safe and effective?

a) That the potential interaction is considered ""serious"".

b) That the concomitant exposure to potassium chloride and spironolactone increases Maria’s risk of hyperkalemia

c) That ACE inhibitors can also increase potassium levels.

d) The specific lab results, concomitant medications, medication dosing, and health factors that increase or mitigate the likelihood that Maria will experience hyperkalemia, and how to manage the interaction such as by discontinuing or changing the dose of one of Maria’s medications

Answer: d) The specific lab results, concomitant medications, medication dosing, and health factors that increase or mitigate the likelihood that Maria will experience hyperkalemia, and how to manage the interaction such as by discontinuing or changing the dose of one of Maria’s medications

Explanation: Drug-drug interaction databases integrated into electronic health-record systems are intended to be useful tools to support general practitioners and improve drug-prescribing performance. However, one of the reported weakness of these systems is the risk of inappropriate alerts with low specificity, which cause ‘over-alerting’ (Böttiger, Ylva, et al. 2009)

Page 138 of 174

and ultimately reduce the usability of the decision support system. A vast amount of information regarding potential drug-drug interactions is published every day but there is no defined standard to ensure the effective use of the generated knowledge in computerized decision support systems. This issue was one of the topics addressed at a recent conference series funded by the United States Agency for Healthcare Research and Quality. Attendees at the conference series included key stakeholders from organizations that provide drug information for us in clinical settings (Tilson, McEvoy, et al. 2016; Payne, Hines, et al. 2016).

Among the key recommendations was the following suggested set of core information that should be included for every PDDI mentioned in a drug information resource:

• Clinical consequences • Contextual information/modifying factors • Evidence • Mechanism of the interaction • Recommended actions • Drugs Involved • Frequency of exposure • Frequency of harm • Seriousness rating

References:

• Böttiger, Ylva, et al. ""SFINX—a drug-drug interaction database designed for clinical decision support systems."" European journal of clinical pharmacology 65.6 (2009): 627-633.

• Tilson H, Hines LE, McEvoy G, Weinstein DM, Hansten PD, Matuszewski K, le Comte M, Higby-Baker S, Hanlon JT, Pezzullo L, Vieson K, Helwig AL, Huang SM, Perre A, Bates DW, Poikonen J, Wittie MA, Grizzle AJ, Brown M, Malone DC. Recommendations for selecting drug-drug interactions for clinical decision support. Am J Health Syst Pharm. 2016 Apr 15;73(8):576-85. doi: 10.2146/ajhp150565. PubMed PMID: 27045070; PubMed Central PMCID: PMC5064943

• Payne TH, Hines LE, Chan RC, Hartman S, Kapusnik-Uner J, Russ AL, Chaffee BW, Hartman C, Tamis V, Galbreth B, Glassman PA, Phansalkar S, van der Sijs H, Gephart SM, Mann G, Strasberg HR, Grizzle AJ, Brown M, Kuperman GJ, Steiner C, Sullins A, Ryan H, Wittie MA, Malone DC. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc.2015 Nov;22(6):1243-50. doi: 10.1093/jamia/ocv011. Review. PubMed PMID: 25829460."

Page 139 of 174

S14: Panel – Data and Network Security – What You Need to Know!

Paul DeMuro, John Rasmussen, Cathy Beech

S14-1: When conducting a breach risk assessment, what threshold must be assessed to consider a disclosure a breach?

a) High risk of harm

b) Patient safety risk

c) Low probability of compromise

d) Excessive risk of disclosure

Answer: c) Low probability of compromise

Explanation:

The HIPAA Security Rule provides that whether or not the presence of ransomware would be a breach is a fact specific determination and one of the considerations is whether there is a low probability that the PHI has been compromised.

References:

• Health Insurance Portability and Accountability Act, Pub. L. 104-191, 110 Stat. 1936 (1996); http://library.clerk.house.gov/reference-files/PPL_HIPAA_HealthInsurancePortabilityAccountabilityAct_1996.pdf

• Examining Oversight of the Privacy & Security of Health Data Collected by Entities Not Regulated by HIPAA. U.S. Department of Health and Human Services; https://www.healthit.gov/sites/default/files/non-covered_entities_report_june_17_2016.pdf

• FACT SHEET: Ransomware and HIPAA. http://www.hhs.gov/sites/default/files/RansomwareFactSheet.pdf

S14-2:

Which of these controls represent a “physical safeguard” under HIPAA?

a) Audit logging

Page 140 of 174

b) Identity badge policy

c) Anti-virus

d) Security cameras

Answer: d) Security cameras

Explanation: Of the choices, only security cameras are physical in nature.

References:




S14-3:

Which of the following represents the BEST strategy for recovering from a ransomware infection?

a) Tape backup

b) User education

c) Anti-virus

d) Data loss prevention

Answer: a) Tape backup

Explanation:

Of the choices, only a tape backup could have the information available prior to the infection.

References:

Page 141 of 174




S14-4:


a) Provides state protections for individually identifiable health information held by covered entities and their business associates.

b) Requires that covered entities and their business associates perform a risk assessment.

c) Provides federal protections for individually identifiable health information held by covered entities and their business associates.

d) Requires implementation of security measures that can help prevent the introduction of malware, including ransomware.

Answer: c) Provides federal protections for individually identifiable health information held by covered entities and their business associates.

Explanation:

HIPAA is a federal law and thus, does not provide state protections. Thus, answer a. is wrong. Answers b. and d. are from the HIPAA Security Rule.

References:



Page 142 of 174


S14-5:


a) Protects all health information wherever it is found.

b) Does not protect all health information wherever it is found.

c) Applies to health information about an individual that has been de-identified.

d) Requires covered entities and business associates to implement policies and procedures that can assist an entity in responding to and recovering from a ransomware attack.

Answer: b) Does not protect all health information wherever it is found.

Explanation:

HIPAA protects most individually identifiable health information held or transmitted by a covered entity or its business associate in any form or medium, whether electronic, on paper, or oral (PHI). It does not protect all health information and not wherever it is found. Thus, answer a. is incorrect. It does not apply to de-identified information and thus, answer c. is incorrect. Answer d. is from the HIPAA Security Rule.

Reference:




Page 143 of 174

S14-6:

The HIPAA Security Rule does not

a) Require that covered entities and their business associates perform a security risk assessment.

b) Require covered entities and business associates to implement policies and procedures that can assist an entity in responding to and recovering from a ransomware attack.

c) Require training users on malicious software protection.

d) Extend to all wearable fitness trackers.

Answer: d) Extend to all wearable fitness trackers.

Explanation:

The HIPAA Security Rule applies to answers a., b., and c. It does not apply to all wearable fitness trackers. Such devices were designed, created and used after the drafting of HIPAA in 1996 and thus, were not anticipated at the time.

References:




Page 144 of 174


Integrating mHealth medication reconciliation and symptom reporting for patient-centered ambulatory care

Lisa Grossman, Rui Sim, Ruth Masterson Creber

S15-1: How does electronic symptom assessment and tracking benefit patients?

a) Enables participating patients to more consistently track and monitor their symptoms

b) Engages patients with low health literacy in symptom tracking

c) Engages a greater percentage of patients in tracking their symptoms

d) Engages patients with low technology literacy in symptom tracking

Answer: a) Enables participating patients to more consistently track and monitor their symptoms

Explanation:

Studies of symptom assessment and tracking systems find that such systems help patients more consistently track their symptoms, as opposed to paper-based or no system (Answer A). By enabling patients to more consistently track their symptoms, electronic symptom assessment and tracking systems help patients better understand their symptoms’ association with their disease, and better communicate their symptoms to their physician. Despite the success of electronic symptom assessment and tracking symptoms thus far, only a small subset of technology literate and health literate patients tends to use them (Answers B, C, D). Future research should explore methods to make symptom assessment and tracking systems more accessible to low technology literacy and low health literacy patients.

Reference:

Patel RA, Klasnja P, Hartzler A, Unruh KT, Pratt W. Probing the benefits of real-time tracking during cancer care. AMIA Annual Symposium Proceedings. 2012;2012:1340-1349. Theresa Cullen [email protected]


Developing an electronic care plan to improve longitudinal care coordination in chronic kidney disease

Jenna Norton

mailto:[email protected]

Page 145 of 174

S15-2: A healthcare system wants to improve information sharing during transitions of care in chronic diseases. They decide to implement an electronic care plan to facilitate sharing of key patient information across providers within the system. They elect to focus their initial efforts on chronic kidney disease (CKD). Which of the following factors best support their selection of CKD for initial efforts to implement an electronic care plan?

a) CKD is defined primarily by objective laboratory data, requires interdisciplinary management including a variety of provider types, and is subject to frequent transitions of care.

b) CKD patients are cared for only by nephrologists and primary care providers, so the care plan would only need to be implemented within a few settings.

c) CKD has few comorbid conditions, limiting concerns regarding reconciliation of data between CKD and other chronic conditions.

d) All data elements relevant to CKD care have existing standards to enable electronic sharing of data in a consistent fashion.

Answer: a) CKD is defined primarily by objective laboratory data, requires interdisciplinary management including a variety of provider types, and is subject to frequent transitions of care.

Explanation: Because CKD patients experience frequent care transitions across numerous settings, including primary care and nephrology clinics, hospitals, and dialysis centers, enhancements in transitions of care for CKD patients may improve outcomes and reduce costs. Additionally, CKD patients benefit from interdisciplinary care by a variety of clinicians—including primary care providers, nephrologists, dietitians, and pharmacists, among others, requiring information to be shared across these providers. Although CKD is defined primarily by objective laboratory data with existing standards, the NKDEP CKD Care Plan Working Group is currently working with standard setting organizations to create necessary standards for inclusion in the CKD care plan. Given the frequency of comorbid conditions in CKD, reconciliation of e-care plan data across disease types for individuals with multiple chronic conditions remains a challenge for implementation of CKD (and other chronic disease) e-care plans.

Reference: Drawz PE, Archdeacon P, McDonald CJ, Powe NR, Smith KA, Norton J, et al. CKD as a Model for Improving Chronic Disease Care through Electronic Health Records. Clin J Am Soc Nephrol. 2015;10(8):1488-99.

Page 146 of 174


Reducing Readmission and Post-Hospital Mortality through Data beyond Your Borders

Philip Smith

S15-3: With a national drop in hospital 30-day readmissions,

a) Overall readmission penalties under CMS have decreased.

b) Less hospitals are receiving fines for Readmission penalties.

c) The percent hospitals fined, the total fines, and the percentage of fines are increasing.

d) Predictive analytics are the de facto standard for determining an individual’s readmission and mortality outcomes.

Answer: c) The percent hospitals fined, the total fines, and the percentage of fines are increasing.

Explanation:

A is false & C is correct because: Fines have increased in percentage at risk, dollar amounts and number of hospitals penalized, not decreased.

B is false, because: The number of hospitals fined has been increasing.

D is false, because: Predictive Analytics address probabilities of outcomes impacting populations of patients and not specific outcomes of the individual patient.

References:

1. New Data Shows Affordable Care Act Reforms Are Leading to Lower Hospital Readmission Rates for Medicare Beneficiaries. https://blog.cms.gov/2013/12/06/new-data-shows-affordable-care-act-reforms-are-leading-to-lower-hospital-readmission-rates-for-medicare-beneficiaries/

2. Medicare’s Readmission Penalties Hit New High. http://khn.org/news/more-than-half-of-hospitals-to-be-penalized-for-excess-readmissions/. August 2, 2016.

3. Medicare penalties lead to decline in hospital readmission rates. http://yaledailynews.com/blog/2017/01/17/medicare-penalties-lead-to-decline-in-hospital-readmission-rates/. January 17, 2017."

Page 147 of 174

THURSDAY, MAY 4

S16: Panel - EHR 2020 After 2 Years: What Have We Done For You(r EHR) Lately?

Sarah Corley, Theresa Cullen, Gil Kuperman, Thomas Payne, Charlene Weir

S16-1:

The Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs was designed to

a) Identify near term strategies to address EHR challenges

b) Propose changes to software design and style sheets

c) Identify CMS specific changes in reimbursement that reflected the impact of EHRs

d) Support the development of a comprehensive learning health care system

Answer: a) Identify near term strategies to address EHR challenges

Explanation:

This report focused on near -term strategies that were identified as potential high impact guidance for EHRs in a five year period from 2015-2020. The workgroup understood that technology continues to change, and issues of concern to providers may have novel solutions in the next ten years. Our focus was on identifying and understanding the issues, and helping guide near term EHR development and implementation to help resolve some of these concerns.

Reference:

Payne TH et al. Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs. J Am Med Inform Assoc. 2015 Sep;22(5):1102-10.

S16-2:

Recommendations on the person centered care delivery model in the AMIA 2020-EHR Task Force include

a) Incorporating social determinants of health in the care record

b) Limitations on patient entered data into the care record

c) Sharing of school health records within the HIT system

Answer: a) Incorporating social determinants of health in the care record

Page 148 of 174

Explanation:

Person-centered care requires additional data from disparate date sources. The report focused on a few of these, including social determinants of health, functional information, goals and population relevant information. Inclusion of patient entered data in the health record would support patient centered care. There was not a specific reference to school health records, though one could imagine that the inclusion of them in the future would be helpful to care delivery and improving health status.

Reference:


S16-3:

Transparency to aid selection of an EHR, as discussed in the EHR 2020 report, includes:

a) Posting recordings of the process of demonstrating that a product meets the MU functional objectives.

b) Sharing source code of vendor EHRs.

c) Interviewing physicians and patients to learn their satisfaction with EHRs and publishing results.

d) Permitting the public to view all screens seen by a practitioner using the EHR during a physician visit.

Answer: a) Posting recordings of the process of demonstrating that a product meets the MU functional objectives.

Explanation:

The report recommends that additional data about the certification process is critical to improving usability, patient safety and innovation. Increasing insight into how products meet MU criteria can help improve decisions into vendor selection as well as the potential impact on clinical workflow. Although the other answer options are other ways to increase transparency to aid selection of an EHR, these answer options are too weak or problematic to be included as task force recommendations.

The task force recognized that (B) vendors will protect their intellectual property and not share their proprietary source code with providers. Although (C) conducting satisfaction interviews with physicians and patients could be helpful in the EHR selection process, the task force

Page 149 of 174

emphasized the issue of safety rather than satisfaction. The task force reported that it is important for patients to have access to their health data, but did not recommend (D) permitting the public to view all screens seen by a practitioner to encourage transparency.

Reference:


S16-4:

Regulation of EHR certification, use, and functionality, according to the EHR 2020 report…

a) Should be changed but not abandoned.

b) Plays no further constructive role in national efforts to improve use of EHRs.

c) Should be extended to cover personal health tracking devices.

d) Is key to advancing innovation in EHRs and supporting technologies.

Answer: a) Should be changed but not abandoned.

Explanation: The certification process and regulations have helped accelerate the use of HIT. Changes are needed to the process but regulatory guidance has the potential to help improve data exchange and interoperability, prioritize patient outcomes, and support vendors and providers working together to enhance usability, safety and care delivery.

Reference:


S18: Panel - Clinician Engagement in Informatics: The Experience of the Pediatric Residency NERD Squad at the Children’s Hospital of Philadelphia

Mark Mai, Evan Orenstein, Ivor Asztalos, Anthony Luberti, George Dalembert

S18-1:

Why is it important to combine didactic education along with a self-directed project when crafting an informatics elective for a trainee?

a) The project can tackle a lingering task on your queue

Page 150 of 174

b) The project reinforces principles learned in the didactics

c) Both are needed to meet a minimum amount of time commitment

d) A tangible project is something the student can put on their resume

Answer: b) The project reinforces principles learned in the didactics

Explanation:

Students may have multitude of reasons for completing an informatics elective. Some may have a burgeoning idea that they want to complete and others may be looking for a more general introduction to the field. Whatever their rationale, projects are an important component in trainee informatics education, as they allow students to engage in active, collaborative learning on a project in which they may have some personal stake.

Reference:

Singer JS, Cheng EM, Baldwin K, Pfeffer MA, UCLA Health Physician Informaticist Committee. The UCLA Health Resident Informaticist Program - A Novel Clinical Informatics Training Program. J Am Med Inform Assoc [Internet]. 2017 Jan 23; Available from: http://dx.doi.org/10.1093/jamia/ocw174

S18-2:

As a medical educator, you would like to teach interns about pitfalls to avoid when using an EHR, such as medication errors and failing to identify concerning clinical trends. Which of the following educational activities can best teach effective use of the EHR in the clinical setting to identify common EHR error types?

a) Informatics simulation cases

b) Passive lectures

c) Flipped classroom

d) Case reports

Answer: a) Informatics simulation cases

Explanation:

While each of the answer options provides an opportunity to learn about potential EHR errors, informatics simulation cases offer hands-on opportunity to identify these errors in a realistic

Page 151 of 174

clinical scenario that allows learners to compare their current use of the EHR to an optimized workflow in real-time. This allows trainees to practice and implement practical changes to their management of clinical scenarios.

Reference:

March CA, Steiger D, Scholl G, Mohan V, Hersh WR, Gold JA. Use of simulation to assess electronic health record safety in the intensive care unit: a pilot study. BMJ Open [Internet]. 2013 Apr 10;3(4). Available from: http://dx.doi.org/10.1136/bmjopen-2013-002549

S18-3:

While you are attending on the general medicine service, one of your residents mentions the challenges and the time needed to place orders for a patient admitted for asthma. She suggests that the hospital should design an order set containing appropriate orders that would streamline the front line clinician workflow. The resident has demonstrated a penchant for identifying workflow challenges and appropriate informatics interventions. Which of the following is the most appropriate next step of action?

a) Construct the order set for the resident

b) Tell the resident to submit an information services ticket

c) Inform the resident that there are too many requests and not enough analysts to complete the project

d) Discuss with the resident her ideas and offer to provide further resources and mentoring to complete the project

Answer: d) Discuss with the resident her ideas and offer to provide further resources and mentoring to complete the project

Explanation:

Each of these answer choices may be a reasonable next step. However, when presented with a learner who shows promise in developing a future career that integrates clinical informatics, only option D provides both an opportunity to develop a clinically useful feature, as well as steps that nurture an interest in the field.

Reference:

Page 152 of 174

Ostrovsky A, Barnett M. Accelerating change: Fostering innovation in healthcare delivery at academic medical centers. Healthc (Amst). 2014 Mar;2(1):9-13.. S19: Panel - Optimal Use of Health IT Requires Competency of Clinicians and Patients William Hersh, Michelle Troseth, Susan Woods S19-1:

Which of the following competencies in clinical informatics is most appropriate for a healthcare provider?

a) Analyze a genomic data set to look for clinical correlations

b) Establish a regional health information exchange organization

c) Perform SQL queries in a patient data warehouse

d) Retrieve all relevant data from a patient record in an EHR

Answer: d) Retrieve all relevant data from a patient record in an EHR Explanation: Healthcare providers need the most competence in using EHRs and other clinical information systems. Reference: Hersh, WR, Gorman, PN, et al. (2014). Beyond information retrieval and EHR use: competencies in clinical informatics for medical education. Advances in Medical Education and Practice. 5: 205-212. http://www.dovepress.com/beyond-information-retrieval-and-electronic-health-record-use-competen-peer-reviewed-article-AMEP

S19-2:

Which of the following have been shown to positively affect patient trust and loyalty with primary care providers?

a) Lower co-pay amounts for visits

b) Provider offering secure email

c) Provider offering secure email and sharing clinic notes online

d) Provider sharing clinic notes online

Answer: c) Provider offering secure email and sharing clinic notes online

http://www.dovepress.com/beyond-information-retrieval-and-electronic-health-record-use-competen-peer-reviewed-article-AMEP

http://www.dovepress.com/beyond-information-retrieval-and-electronic-health-record-use-competen-peer-reviewed-article-AMEP

Page 153 of 174

Explanation: The most positive effect on patient trust and loyalty with primary care providers comes from when the provider offers secure email and sharing clinic notes online.

References:

• Marianne Turley Terhilda Garrido Alex Lowenthal Yi Yvonne Zhou Association between personal health record enrollment and patient loyalty. The American journal of managed care. , 2012, Vol.18(7), p.e248-e253 http://www.ajmc.com/journals/issue/2012/2012-7-vol18-n7/association-between-personal-health-record-enrollment-and-patient-loyalty/p-2

• Garrido T, Meng D, Wang JJ, Palen TE, Kanter MH. Secure e-mailing between physicians and patients: transformational change in ambulatory care. J Ambul Care Manage. 2014 Jul-Sep;37(3):211-8. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4215918/pdf/jamcm-37-211.pdf

• When doctors share visit notes with patients: a study of patient and doctor perceptions of documentation errors, safety opportunities and the patient-doctor relationship. Bell SK, Mejilla R, Anselmo M, Darer JD, Elmore JG, Leveille S, Ngo L, Ralston JD, Delbanco T, Walker J. BMJ Qual Saf. 2017 Apr;26(4):262-270. http://qualitysafety.bmj.com.liboff.ohsu.edu/content/qhc/26/4/262.full.pdf

http://www.ajmc.com/journals/issue/2012/2012-7-vol18-n7/association-between-personal-health-record-enrollment-and-patient-loyalty/p-2

http://www.ajmc.com/journals/issue/2012/2012-7-vol18-n7/association-between-personal-health-record-enrollment-and-patient-loyalty/p-2

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4215918/pdf/jamcm-37-211.pdf

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4215918/pdf/jamcm-37-211.pdf

http://qualitysafety.bmj.com.liboff.ohsu.edu/content/qhc/26/4/262.full.pdf

Page 154 of 174

S20: Presentations: Safety and Quality

Reduced Rates of Hospital-Associated Clostridium difficile Infection Associated with a Clinical Surveillance System

Stephen Ross, Sue Ie, Sharad Manaktala, Justin Clark

On the Hospital Compare site, your hospital's standardized infection ratio (SIR) for hospital-acquired Clostridium difficile infection is significantly higher than other hospitals in your region. In a presentation to the CEO and CIO of your hospital, you are justifying investment in an automated surveillance system to assist your antimicrobial stewardship and infection control teams. Which of the following statements most accurately reflects the current evidence for such systems?

a) Although not required for all hospitals, implementation of an automated surveillance system for antimicrobial stewardship and infection control is strongly endorsed in the proposed new "conditions of participation" from the Center for Medicare and Medicaid services.

b) There is currently no evidence that automated surveillance systems reduce the use or expense of antibiotics that may contribute to hospital-acquired Clostridium difficile infection.

c) Without automated surveillance systems in place, antimicrobial stewardship programs have not been demonstrated to reduce rates of hospital-acquired Clostridium difficile infection.

d) Clinical trials establish that automated surveillance systems reduce rates of hospital-acquired Clostridium difficile infection.

Answer: a) Although not required for all hospitals, implementation of an automated surveillance system for antimicrobial stewardship and infection control is strongly endorsed in the proposed new "conditions of participation" from the Center for Medicare and Medicaid services.

Explanation:

The proposed conditions of participation require surveillance of infections and implementation of antimicrobial stewardship programs. The interpretive guidance clearly endorses the use of "'automated surveillance' by way of analyzing useful information from infection control data through the systematic application of medical informatics and computer science technologies." While strongly endorsed, such systems are not required for all hospitals, perhaps because investment in these systems may be less justified in smaller critical access hospitals.

B is not correct because there multiple studies have demonstrated the benefits of surveillance systems in process measures of antimicrobial use.

Page 155 of 174

C is not correct because antimicrobial stewardship programs have by themselves demonstrated reductions in the important clinical outcome of hospital-acquired Clostridum difficile infections - a reduction of roughly half in a meta-analysis.

D is not correct because while the observational results reported in this presentation suggest an association between implementing a surveillance system and a reduction in hospital-acquired Clostridium difficile infection, there have been no clinical trials that have firmly established that automated clinical surveillance systems reduce rates of hospital-acquired Clostridium difficile infection.

Reference:

Wright, M. Automated Surveillance and Infection Control: Toward a better tomorrow. Am J Infect Control 2008; 36:S1-S5.


Making Acute Care More Patient-Centered: Implementing a Learning Lab

Theresa Fuller

This Patient Safety Learning Laboratory combines multiple units and clinical services in the implementation of a suite of patient- and provider-facing tools all geared toward improving patient safety. There are benefits and drawbacks for each element in being part of such a large implementation. This shows through in the “stepped wedge” design of the project (where units/services are randomized into “steps,” and each step goes live incrementally). This design is intended to reduce the temporal, contaminating effects of a single go-live date. We learned that it also comes with a series of challenges.

Which of these was the most significant learning of our experience in mitigating those challenges?

a) Holding to your original go-live dates

b) Defining your “step” design with input from unit leadership

c) “Rolling out” all tools together

d) Maintaining a constant physical presence from the research team on the units

Answer: b) Defining your “step” design with input from unit leadership

Explanation:

Learnings of this project include: 1) a flexible perspective on the ‘correct’ way to implement the tools, 2) early and open lines of communication with all stakeholders from floor staff to hospital

Page 156 of 174

administrators 3) faith that a culture change can be supported by unit staff without a constant presence of research team members. All of that is dependent on a strong original study design. Without a clear understanding of the units’ workflow before determining the logistics of the project, it is impossible to balance the many competing difficulties of a stepped wedge design.

Reference:

Prost A, Binik A, Abubakar I, Roy A, De Allegri M, Mouchoux C, et al. Logistic, ethical, and political dimensions of stepped wedge trials: critical review and case studies.Trials. 2015 Aug 17;16:351


First Contact Provider (FCP): Improving Inpatient Critical Results Reporting

Anisha Chandiramani

Reporting critical results of tests and diagnostic procedures in a timely manner is a Hospital National

Patient Safety Goal developed by The Joint Commission. The objective of the patient safety goal is to “…provide the responsible licensed caregiver these results within an established time frame so that the patient can be promptly treated.”* The University of Chicago Medicine (UCM) was not meeting its goal of timely reporting of critical lab values to Licensed Independent Practitioners (LIPs).

What did UCM implement to meet this Joint Commission goal?

a) An EPIC-based way to enter, store, and display each inpatient’s primary provider, called the First Contact Provider (FCP)

b) A tool in Epic for a Licensed Independent Practitioner (LIP) to dictate urgent treatments and place orders

c) A written procedure for managing the critical results of tests and diagnostic procedures

Answer: a) An EPIC-based way to enter, store, and display each inpatient’s primary provider, called the First Contact Provider (FCP)

Explanation:

UCM implemented a new, EPIC-based way to enter, store, and display each inpatient’s primary provider, called the First Contact Provider (FCP), that:

Page 157 of 174

1. Uses existing EPIC infrastructure: the Treatment Team

2. Mandates compliance with entering this data

3. Displays the information clearly for all users involved in patient care

Reference:

* Hospital: 2016 National Patient Safety Goals [Internet]. [cited 2016 Oct 5]. Available from: https://www.jointcommission.org/hap_2016_npsgs/

S21: Panel - Optimizing Strategies for Clinical Decision Support

Marianne Hamilton Lopez, James Tcheng, Jonathan Teich, Blackford Middleton, Scott Weingarten, Kensaku Kawamoto

You have just been elected to be the Chair of Informatics of your professional medical association. Prior work has confirmed that only about half of the patients in the United States receive care according to the evidence-based best practices recommended by your medical association. You have been asked by the leadership of the medical association to attend the AMIA iHealth meeting and to identify the challenges and potential solutions for leveraging clinical decision support (CDS) to ensure that all patients receive the best possible care based on your medical association’s evidence-based practice guidelines.

S21-1:

What is one of the largest barriers to widespread dissemination of evidence-based practice guidelines using CDS?

a) There is generally no room for improvement – clinicians by and large already follow evidence-based care guidelines almost all of the time.

b) Fee-for-service payment models provide little incentives for higher quality, evidence-based care.

c) Electronic health record systems generally do not have mechanisms for providing CDS to its end-users.

d) Electronic medical records already provide the best evidence.

Answer: b) Fee-for-service payment models provide little incentives for higher quality, evidence-based care.

https://www.jointcommission.org/hap_2016_npsgs/

Page 158 of 174

Explanation:

Fee-for-service payment models generally do not provide incentives for delivering higher quality care. As a result, there is little incentive to invest in mechanisms for improving quality such as CDS.

Reference:

Burwell SM. Setting value-based payment goals--HHS efforts to improve U.S. health care. N Engl J Med. 2015 Mar 5;372(10):897-9.

S21-2:

What is a best practice for ensuring that CDS has the intended impact?

a) Do not measure impact; CDS interventions always have the desired impact.

b) Provide the right information, to the right person, in the right CDS intervention format, through the right channel, and at the right time in workflow (CDS 5 Rights).

c) Do not worry about the volume or accuracy of alerts – “alert fatigue” is not something the vast majority of clinicians’ experience.

d) Have every clinician see the same level of alerts so that nothing is missed.

Answer: b) Provide the right information, to the right person, in the right CDS intervention format, through the right channel, and at the right time in workflow (CDS 5 Rights).

Explanation:

The CDS 5 Rights is a well-accepted framework for helping to ensure that CDS has the desired impact.

Reference:

Campbell R. The five "rights" of clinical decision support. J AHIMA. 2013 Oct;84(10):42-7; quiz 48.

S21-3:

What is one reason we do not yet have widespread standards-based interoperability for CDS?

a) There are simply no interoperability standards available in the CDS arena.

Page 159 of 174

b) Major electronic health record (EHR) vendors provide no support for standards-based interoperability in general.

c) We are still in early stages of EHR vendor adoption of CDS interoperability standards.

d) The industry has already widely adopted standards-based interoperability for CDS?

Answer: c) We are still in early stages of EHR vendor adoption of CDS interoperability standards.

Explanation: We are still in early stages of EHR vendor adoption of CDS interoperability standards.

Answer “a” is incorrect because there are, in fact, standards available.

Answer “b” is incorrect because major EHR vendors do in fact support standards-based interoperability to varying degrees.

Answer “c” is correct because we are in fact still in the early stages of EHR vendor adoption of CDS interoperability standards. While there are other reasons for limited standards-based CDS interoperability, they are not included as potential answer options.

Answer “d” is incorrect, since we are still in the early stages of adoption.

Reference:

Office of the National Coordinator for Health IT. 2016 Interoperability Standards Advisory. https://www.healthit.gov/sites/default/files/2016-interoperability-standards-advisory-final-508.pdf

S21-4:

You are tasked with the responsibility of implementing a clinical decision support rule in your EHR system. The EHR “Meaningful Use” regulations stipulate the use of which of the following expression languages for clinical decision support?

a) Arden Syntax

b) CQL (Clinical Quality Language)

c) GELLO

d) No specification in the regulations

Answer: d) No specification in the regulations

Page 160 of 174

Explanation:

There is no certification or other specification for an expression language for CDS in EHR systems.

References:

EHR Incentive Programs for Eligible Professionals: What You Need to Know for 2016 Tipsheet https://www.federalregister.gov/documents/2015/10/16/2015-25595/medicare-and-medicaid-programs-electronic-health-record-incentive-program-stage-3-and-modifications

Kawamoto K et al. Standards for Scalable Clinical Decision Support: Need, Current and Emerging Standards, Gaps, and Proposal for Progress. Open Med Inform J. 2010; 4: 235–244

S22: Presentations – Bridging Analytics, Bedside Care, and Education Granular ED Throughput Data: Helping Providers Find Agency in Process Improvement Tom Spiegel S22-1: One of the many benefits of the Electronic Health/Medical Records is the collection and storage of healthcare data that allows for data mining that in part fuels quality improvement. Our QI project was born out of recognition that "patient length of stay" and "relative value units" were too ambiguous and therefore not sufficiently motivating to our residents. Unlike our "purple time," they were unable to provide the insight we needed to improve performance. What underlying principle of data granularity is illustrated by this effective use of "purple time”?

a) Granulated data are more easily retrieved than aggregate data

b) Data are most helpful when they are appropriately granular and defined to a level of detail specific to your needs

c) Granular data have been shown to improve accuracy of medical billing and coding

d) Increasingly granulated data hinder EHR interoperability

Answer: b) Data are most helpful when they are appropriately granular and defined to a level of detail specific to your needs Explanation: For data to be actionable, an appropriate level of granularity is required. This presentation was not a mandate for granular data, but for our purposes, specific provider and

https://www.federalregister.gov/documents/2015/10/16/2015-25595/medicare-and-medicaid-programs-electronic-health-record-incentive-program-stage-3-and-modifications

https://www.federalregister.gov/documents/2015/10/16/2015-25595/medicare-and-medicaid-programs-electronic-health-record-incentive-program-stage-3-and-modifications

Page 161 of 174

patient visit data allowed us to root out the variability in patient length of stay. By providing resident specific feedback and partitioning patient flow to a greater degree, we turned a vague measurement into an actionable one. While answers A, C, and D can be argued to have stand-alone validity, the principle of data granularity used in this project relates to defining specific detail to clinical actions, making answer B the correct answer to the question. References:

• Tange HJ, Schouten HC, Kester ADM, Hasman A. The Granularity of Medical Narratives and Its Effect on the Speed and Completeness of Information Retrieval. Journal of the American Medical Informatics Association : JAMIA. 1998;5(6):571-582.

• Weiskopf NG, Hripcsak G, Swaminathan S, Weng C. Defining and measuring completeness of electronic health records for secondary use. Journal of biomedical informatics. 2013;46(5):10.1016/j.jbi.2013.06.010. doi:10.1016/j.jbi.2013.06.010

S22-2: Which of the following is considered a “Visual Analysis Best Practice” as discussed in the presentation:

a) Select creatively non-meaningful colors to your data

b) Avoid cultural significant colors when making your palate choices

c) When color choices are not obvious, ensure you use label or a clear legend

d) Avoid adding color to more than 10 distinct values

Answer: c) When color choices are not obvious, ensure you use label or a clear legend Explanation: Choice C will assist the viewer in understanding and interpreting your visual data display. Choice A is incorrect because one should select semantically meaningful colors if they apply to the data. Choice B is not correct because assessing the cultural impact of color choices is important and the immediate recognition by users can be helpful and should not necessarily be avoided. For example, in much of the world, the colors of a traffic light can be used to immediately confer green=positive, red=negative, and yellow=cautionary. Choice D is incorrect because one should avoid adding color to more than 12 distinct values.

Page 162 of 174

Reference: Visual Analysis Best Practices: Simple Techniques for Making Every Data Visualization Useful and Beautiful. Tableau Software, Inc. https://www.tableau.com/sites/default/files/media/whitepaper_visual-analysis-guidebook_0.pdf

S22: Presentation: Bridging Analytics, Bedside care, and Education:

Manual, automated, or derived measures: The value of variability in the meaningful use of vital sign data

Keith Feldman

S22-3:

Which of the following is an important consideration for researchers when examining vital sign data procured from the electronic medical record?

a) Vital sign data procured from the standard electronic medical record, unlike vital sign data procured from paper medical records, more consistently identifies patient deterioration.

b) B. Vital sign data procured from the electronic medical record, unlike vital sign data procured from paper medical records, are less likely to include erroneous values.

c) The use of vital sign data procured from the electronic medical record improves study validity, as data recordings are made electronically and at set intervals, thereby removing provider bias.

d) There may be greater variability in vital sign data procured from the electronic medical record compared to data procured from paper medical records, because a broader range of methods may be used to collect and record the data.

Answer: d. There may be greater variability in vital sign data procured from the electronic medical record compared to data procured from paper medical records, because a broader range of methods may be used to collect and record the data.

Explanation:

Choice A is incorrect because there is currently a lack of evidence demonstrating the use of medical record data to consistently identify clinically relevant patient deterioration, as well as a lack of evidence demonstrating the use of standard electronic medical record data as superior/inferior for this purpose. Choice B is incorrect because electronically recorded data is more likely to contain artefactual (erroneous) data. Choice C is incorrect because not all vital

Page 163 of 174

sign data procured from the electronic medical record are necessarily made electronically – there is opportunity for providers to adjust values and insert manually collected values.

Reference:

Schulman, Christine S., and LuAnn Staul. "Standards for frequency of measurement and documentation of vital signs and physical assessments." Critical care nurse 30.3 (2010): 74-76.

S22-4:

Which of the following methods for collecting heart rate values from a patient is associated with greatest variability of values across standard deviation and range?

a) Manual collection of discrete values (i.e.: using a stethoscope) by a provider, at standard intervals, with manual documentation into the medical record.

b) Manual selection of values by a provider, at standard intervals, from values generated by a bedside ECG monitor, with manual documentation into the medical record.

c) Manual validation of electronically documented values, imported into the medical record from a bedside ECG monitor, at standard intervals.

d) Electronic documentation of waveform data, imported into the medical record from a bedside ECG monitor.

Answer: d) Electronic documentation of waveform data, imported into the medical record from a bedside ECG monitor.

Explanation:

Choice D is correct because it is the only choice that does not describe a method for documenting interval/discrete values. Waveform collection, compared to any type of manual, discrete value collection, is associated with greatest variability of values across standard deviation and range. Comprehensive measures generated from automatically collected waveform data may capture additional information not discernible to providers upon examination of a series of heart rate values.

Reference:

Mok WQ, Wang W, Liaw SY. Vital signs monitoring to detect patient deterioration: An integrative literature review. Int J Nurs Pract. 2015;21 Suppl 2:91-8."

Page 164 of 174

S23: Presentations – Workflow

Prototyping The Future of CPOE: Starting With Indications

Aaron Nathan

S23-1:

Which of the following is a significant benefit of indications-based prescribing?

a) The indication will always be put on the medication label

b) Drug costs will decrease when indications are included

c) Patients will know the reason why they are taking each of their medications

d) Brand name drugs will get selected more often

Answer: d) Patients will know the reason why they are taking each of their medications

Explanation:

The answer is not A because even if the indication does not get put on the medication label, it will be included on the patient's medication list.

The answer is not B because the authors found no examples of insurers incentivizing the addition of indications to medications with a decrease in medication cost (although the authors do note that some medications require adding a diagnosis for coverage, it is not a ubiquitous enough policy to validate option B as the most correct answer).

The answer is not D because the authors have found no evidence to support the claim that adding an indication causes or even correlates with brand-name drug selection.

The correct answer is C because whether it is on the medication bottle label or in the patient's medication list, adding indications improves patient knowledge about the reasons for taking their medications, which is paramount to promoting informed patient decisions in the modern patient-centered care model.

Reference:

Indications-Based Prescribing Research Project (AHRQ-BWH) [Internet]. Google Sites. AHRQ-BWH Indications-based Prescribing Team; [cited 2017Feb20]. Available from: https://sites.google.com/site/indicationsrx/

Page 165 of 174

S23: Presentations – Workflow Design and Application of a Bariatric Ambulatory Workflow for the Pre-operative Evaluation of Bariatric Surgical Candidates Tatyan Clarke S23-2: The Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSAQIP) encourages standardization in the evaluation and management of Bariatric patients. To this end, our Bariatric Ambulatory Workflow provides a system to compile the elements of this complex work-up for each patient. Which of the following describes additional benefits of this form of data capture?

a) It dictates the care of each patient, ensuring every patient has an identical evaluation.

b) It guides each provider to review the detailed results of each element of the evaluation at every encounter.

c) The workflow can be queried to assess outcomes and guide quality-improvement strategies.

d) The content of the flowsheet directly bills for each element of the evaluation.

e) The flowsheets auto-populate all parts of the progress note, making documentation effortless.

Answer: c) The workflow can be queried to assess outcomes and guide quality-improvement strategies. Explanation: During the pre-operative evaluation process, patients are assessed for a variety of obesity-associated medical problems. The medical clearance process often differs between patients, based largely on their overall clinical status, addressing both known and discovered comorbid conditions. Our Bariatric Ambulatory Workflow, helps to ensure adherence to MBSAQIP and national standards, while allowing customization of each patient’s multidisciplinary assessment. Each step of their evaluation is streamlined by the protocol-based flowsheets, and their content informs all providers who audit the chart, preventing redundant data collection. As components of the Electronic Medical Record, the resulting flowsheets of the Workflow can be queried in a program-wide audit to assess outcomes and guide quality-improvement strategies. To support encounter documentation required for billing, the data entered into the flowsheets is designed to auto-populate into the relevant sections of the progress note for that visit. Details of the physical exam and assessment and plan will always need to be completed separately in the context of the individual visit. Reference:

Page 166 of 174

American College of Surgeons, American Society for Metabolic and Bariatric Surgery. Standards Manual: Resources for Optimal Care of the Metabolic and Bariatric Surgery Patient. Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program. 2016;V2.0: 1-62.

S23: Presentations - Workflow

Leveraging Advanced Physician Documentation for Improving Quality, Accuracy, and Compliance

Kang Hsu

S23-3:

Coders can only code from:

a) Nursing Notes

b) Radiology Reports

c) Physician Documentation

d) Laboratory Reports

Answer: c) Physician Documentation

Explanation:

Coders can only code from Physician Documentation. They cannot code from Nursing Reports, Laboratory, Radiology, EKG, Pathology, or symbols such as ? or ↓. Thus, if a Radiology Report has an impression from the radiologist it cannot be coded until the Attending Physician documents the finding in his documentation. Or, if a sodium is 120 in a lab report the patient cannot be coded to have hyponatremia until the physician interprets the lab report and documents it in his documentation.

Reference:

Russo, R. Documentation and Data Improvement Fundamentals. American Health Information Management Association, 2004.

Page 167 of 174

S24: Panel - A Proposed National Research and Development Agenda for Population Health Informatics: Case Studies and Future Directions

Hadi Kharrazi, Steve Fihn, William Yasnoff, Bharat Sutariya, Jonathan Weiner, Aneel Advani

S24-1:

Which specification is more accurate about the “Population Health Informatics” (PopHI) domain? (based on https://www.ncbi.nlm.nih.gov/pubmed/27018264)

a) The focus of PopHI solutions is on the treatment of individual patients at the point of care

b) The key stakeholders of PopHI include providers and exclude payers and local communities

c) The main action arm of PopHI activities is the federal government (e.g., CDC) and excludes provider or payer organizations

d) The common intervention target of Pop HI is large population denominators attributed to an entity or geographic boundary

Answer: d) The common intervention target of Pop HI is large population denominators attributed to an entity or geographic boundary

Explanation:

PopHI usually targets a specific denominator of a population (e.g., patient population of a healthcare provider/system or the population residing in a specific geographical boundary)

Reference:

Kharrazi H et al. A proposed national research and development agenda for population health informatics: summary recommendations from a national expert workshop. J Am Med Inform Assoc. 2017 Jan;24(1):2-12. https://www.ncbi.nlm.nih.gov/pubmed/27018264 -- See figure 1A (page 4)

S24-2:

Which of the following items are identified as one of the major challenges for “Population Health Informatics” (PopHI) research and development? (based on https://www.ncbi.nlm.nih.gov/pubmed/27018264)

https://www.ncbi.nlm.nih.gov/pubmed/27018264

Page 168 of 174

a) Extensive use of patient identification methods to link medical, public health, and social service data systems

b) Alignment of individual and population-based measures for ACOs and PCMHs

c) The lack of a clear vision and lack of a shared model for a PopHI infrastructure

d) Lack of variability of data sources and data patterns in population health

Answer: c) The lack of a clear vision and lack of a shared model for a PopHI infrastructure

Explanation:

PopHI suffers from an unclear vision for a shared model of PopHI infrastructure. Both the use of patient identifiers to link various data sources (clinical and non-clinical) and the existence of individual and population-level metric alignment enable PopHI solutions to propel population health. The high variability of data sources for population limits the development of PopHI solutions.

Reference:

Kharrazi H et al. A proposed national research and development agenda for population health informatics: summary recommendations from a national expert workshop. J Am Med Inform Assoc. 2017 Jan;24(1):2-12. https://www.ncbi.nlm.nih.gov/pubmed/27018264 --See table 1 (page 5)

S24-3:

Which of the following items are identified as an emergent “Population Health Informatics” (PopHI) research and development agenda? (based on https://www.ncbi.nlm.nih.gov/pubmed/27018264)

a) Developing information solutions to integrating PopHI at the point of care

b) Developing a scientific evidence and knowledge base across different sectors affecting population health

c) Developing new ideas to merge genomic data with clinical data

d) Developing a market strategy to translate PopHI findings into business ideas

Answer: b) Developing a scientific evidence and knowledge base across different sectors affecting population health

Page 169 of 174

Explanation:

Further knowledge is needed to advance the integration and mining of data across clinical, social, community and non-health factors. PopHI is of need of a scientific evidence and knowledge base across different stakeholders. Integrating PopHI solutions at the point of care, incorporating genomic data, and developing business strategies, although helpful, are not ‘emergent’ to the development of PopHI at this time.

Reference:

Kharrazi H et al. A proposed national research and development agenda for population health informatics: summary recommendations from a national expert workshop. J Am Med Inform Assoc. 2017 Jan;24(1):2-12. https://www.ncbi.nlm.nih.gov/pubmed/27018264 -- See figure 2 (page 10)

S25: Panel - CMIO + CNIO + CIO Leadership: Improving Outcomes at the System Level

Bimal Desai, Kisha Hawthorne, Kimberly Burress, Diane Humbrecht, Colleen Saul, Jon Sternlieb

S25-1:

You are a provider and your medical center is facing increasing financial pressures. Payers are negotiating lower contracted rates, and your organizations costs are going up. Leadership is advocating cuts to staff that would lead to more work by providers. Which is the best option?

a) Accelerate quality reporting initiatives, despite increased costs, to improve contract negotiation strength.

b) Agree to staffing cuts with a plan to wait until the current financial challenges pass

c) Contract only with large insurance companies to ensure higher coverage for the most commonly billed physician visits

d) Cut back on expansion plans that would have increased the population of patients who receive care at the medical center

Answer: a) Accelerate quality reporting initiatives, despite increased costs, to improve contract negotiation strength.

Explanation:

A is correct because it establishes a compelling platform for future negotiations and subsequent stabilization of the operating margin.

Page 170 of 174

B is incorrect because, as a singular reactive strategy, it has no flexibility to respond to continued future pricing pressures and may lead to a downward spiral.

C is incorrect because reimbursement rates from insurance companies with larger market share are negotiating lower rates for providers.

D is incorrect because it conflicts with structural changes underway in the health care system focusing on population based coverage for which larger populations are necessary to achieve financial viability.

References:

Roberts ET, Chernew ME, McWilliams JM. Market Share Matters: Evidence of Insurer and Provider Bargaining Over Prices. Health Aff (Millwood). 2017 Jan 1;36(1):141-148.

Gesme DH, Wiseman M. How to negotiate with health care plans. J Oncol Pract. 2010 Jul;6(4):220-2.

S25-2:

You are an informatics leader and your medical center has acquired several new practice sites and a new hospital has joined the network. Everyone is talking about improving clinical integration; however, it is likely that all sites will eventually migrate to the same EHR platform. Since the change to a single platform is expected to take three years, which option appears best as an interim strategy for clinical content access?

a) Apply secure WhatsApp for communication across the enterprise for easy communication.

b) Deploy targeted FHIR applications to share key clinical data.

c) Perform a standard ETL (extract, transform, load) process so all information becomes easily available in a shared repository.

d) Use auto-fax to distribute all visit notes across all sites to improve communication.

Answer: b) Deploy targeted FHIR applications to share key clinical data.

Explanation:

A is incorrect because while some communication may be improved, there is no platform for direct access to clinical content.


Page 171 of 174

B is correct because FHIR offers the prospect of building an expanding channel for increasing availability of clinical content across sites.

C is incorrect because the typical time required for these efforts exceeds that available for an interim strategy; in particular, master data management quickly becomes overwhelming.

D is incorrect because pushing content that may not be of interest to recipients will quickly overwhelm each practice site’s resources.

Reference:

Mandel JC, et al. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc (2016) 23 (5): 899-908.

S25-3:

Your organization, one of several in a city, is contemplating a telemedicine initiative with a goal of significantly reducing in-person visits at your medical center given the existence of newly supportive contacts in this regard. Which approach appears both most cost-effective and likely to achieve this goal?

a) Develop and place kiosks for real-time health management at the local airport, and at central bus and subway stations.

b) Identify high volume low, complexity conditions and develop algorithms to manage conditions asynchronously through the existing patient portal.

c) Increase the call center personnel with staff who have at least a community college degree to cut down on wait times for returned calls.

d) Outsource after-hours visits to management by a telemedicine company that has received a high rating from the Wall Street Journal.

Answer: b) Identify high volume low, complexity conditions and develop algorithms to manage conditions asynchronously through the existing patient portal.

Explanation:

A is incorrect because it is not targeted toward the established patients of the medical center, and is therefore unlikely to reduce volume (but may assist in introducing new patients).

B is correct because telemedicine is well positioned for management of straightforward and self-limited conditions such as URI, UTI, joint sprains, and diarrhea that contribute to significant numbers of urgent care visits.

https://academic.oup.com/jamia/article/23/5/899/2379865/SMART-on-FHIR-a-standards-based-interoperable-apps

https://academic.oup.com/jamia/article/23/5/899/2379865/SMART-on-FHIR-a-standards-based-interoperable-apps

Page 172 of 174

C is incorrect because as phrased it is not clear that simply responding more rapidly to phone calls with clerical staff will contribute to decreased in-person visits, and may increase the volume of scheduled patients.

D is incorrect because it is not the best answer since there will likely be contractual costs for the outsourced vendor and, as stated, without targeting “one and done” clinical scenarios, the program may simply serve to re-direct patients to their physicians the next day without impacting in-person volume.

Reference:

Bashshur RL et al. The Empirical Foundations of Telemedicine Interventions in Primary Care. Telemed J E Health. 2016 May;22(5):342-75.

S26: Data and Network Security

A New Approach to Patient Privacy Monitoring using Machine Learning

Daniel Fabbri

S26-1: Which of the following best describes access control policies in modern electronic medical record systems?

a) Provide too narrow access rights

b) Provide too broad access rights

c) Correctly limit access

d) Provide no access

Answer: b) Provide too broad access rights Explanation: Most modern EMR systems have an open-access environment, which means that after a user enters their credentials and is authenticated, the user can access any patient’s record. As a result, most employees have the ability to access more patient records than those they are supporting. Reference: Ferreira A, Cruz-Correia R, Antunes L, et al. Access control: how can it improve patients healthcare? Stud Health Technol Inform 2007;127:65–76

Page 173 of 174

S26-2: Employees have used their EMR access rights to inappropriate access patient data in the past. Which of the following best describes a process to detect one-off inappropriate access?

a) Manual review of all accesses

b) Credit card fraud system

c) Statistical anomaly detection systems

d) Context-based access monitoring systems

Answer: d) Context-based access monitoring systems Explanation: Modern EMRs receive millions of accesses per day, making manual analysis infeasible. Moreover, given the dynamic nature of patient care, statistical anomaly detection systems are often unable to detect inappropriate behavior (beyond large scale scraping when an employee assesses many more records than normal). Credit card fraud systems, which rely on value, location and timing of purchase, are also not effective for health care given patient care dynamics (i.e., employees work different shifts, cover varying patients and may even write notes after getting home from a shift). To effectively monitor for inappropriate accesses, a system should leverage clinical and operational context (e.g., appointments, medications, etc.) to filter out appropriate accesses, so privacy officers can focus on suspicious behavior. Reference: Fabbri D, LeFevre K. Explaining accesses to electronic medical records using diagnosis information. Journal of the American Medical Informatics Association: JAMIA. 2013;20(1):52-60. doi:10.1136/amiajnl-2012-001018. S26-3: The explanation-based auditing system discovers what type of pattern from the EMR database?

a) Medical appointment patterns

b) Access frequency patterns

c) A connection between the patient and the employee accessing the patient’s record using EMR data.

d) Accesses from employees to neighbor records.

Page 174 of 174

Answer: c) A connection between the patient and the employee accessing the patient’s record using EMR data. Explanation: The explanation-based auditing system learns connections between the patient and the employee accessing the patient’s record using EMR data. These connections often can identify the reason why the access occurred. Auditors can use these explanations to quickly filter out appropriate access, so they can focus on suspicious behavior. Reference: Daniel Fabbri and Kristen LeFevre. Explanation-based auditing. Proc. VLDB Endow. 5, 1. 2011.

Documents

iHealth 2017 Clinical Informatics Conference Q&A bookletand using risk scores. a) Interventions may be too costly or niche in nature to justify care management resources even if high