Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
1
THE EDWARD J. STEMMLER, MD
MEDICAL EDUCATION RESEARCH FUND
2014‐2015 INVITED PROPOSAL
Title: Assessment of Resident Decision Making and Patient Safety: A
Randomized Trial of Inpatient Medical Attending Supervision of Trainees
Primary Investigator: Kathleen M Finn M.D. M.Phil
Co‐Investigators: Christiana Iyasere M.D., M.B.A.
Joshua Metlay M.D., PhD., Hasan Bazari M.D,
Jay Vyas M.D., PhD., Yuchiao Chang, PhD.,
and Elyse Park PhD., MPH.
School: Harvard Medical School
Massachusetts General Hospital
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
2
A. Letter of Intent Application. (Cover Sheet)
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
3
Letter of Intent
Study Objectives and Research Question/Hypothesis
Graduate physician training is based the apprenticeship model where residents provide care to
patients under the supervision of a senior physician. Residents gain increased patient care
responsibility, learn critical thinking and make medical decisions with a gradual reduction in
supervision as the trainee develops mastery. In this model, performance is increased via
explicit instruction from a learned mentor or teacher to allow individualized diagnosis of errors,
informative feedback and remedial part training. (1)
Despite the historical legacy of the apprenticeship model and the importance of clinical
mentorship, what constitutes “supervision” is not clearly defined. Resident oversight by
attending physicians on the general inpatient medical wards varies widely. For example, some
attendings independently review all laboratory values and imaging and impose minor changes
in treatments, others do not. The right balance between autonomy and supervision to promote
both learning and ensure excellent patient care is unknown and difficult to achieve. We
propose a study that begins to investigate these questions – what is a reasonable level of
resident oversight? What are the consequences, both intended and unintended, of increased
resident supervision on the medical wards? What models can we employ to assess these
outcomes?
Following the death of Libby Zion, the Bell Commission cited both fatigue and inadequate
supervision of residents as contributing factors to her untimely death. (2) As a result, both New
York State and the Accreditation Council for Graduate Medical Education (ACGME) focused on
resident work hour limitations in an attempt to improve patient safety by reducing medical
errors. However, duty‐hour reform has not demonstrated the improved patient safety
outcomes anticipated, and work hour restriction has raised questions about a decline in
educational opportunities. (3, 4) \
Given the fundamental importance of improving patient safety, the emphasis is now shifting
from hours worked to the level of supervision of residents. If improved duty hours have not
reduced medical errors and improved patient safety, perhaps increased attending supervision
will? In response to these concerns, some residency programs have intensified senior clinician
involvement on the medical wards. In these new models, attendings now fully participate in
both new patient presentations and old patient work‐rounds, are present on the wards for
most of the day and are engaged in all details of patient care. Residents’ decision making is
monitored, their plans are closely reviewed, and attendings are directly involved with patients
and their families.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
4
Deeper attending involvement in patient care and medical decision making seems to be a
logical next step to improve patient safety. The underlying assumption is – more supervision,
better patient outcomes and resident training. Yet two recent studies comparing increased
nighttime Intensive Care Unit supervision to prior more limited staffing models found no
difference in mortality or patient safety outcomes. (5, 6) Similarly, heightened attending
involvement in medical decision making may be at odds with the way residents learn best.
Adult learning theory emphasizes adults are self‐directed learners and best discover knowledge
for themselves without being told. (7) Many physicians report that independent decision
making was crucial for their development as physicians and maturation of clinical thinking. (2)
Some would argue closer attending supervision may be detrimental to residents’ education, a
theme echoed in a recent perspective piece in the New England Journal of Medicine which
raised concerns about the effect of increasing levels of supervision on resident education and
critical thinking. (2)
It is within this context that we propose to investigate the hypothesis that increased attending
supervision in resident decision making improves patient safety
Rationale for Proposed Research
Massachusetts General Hospital is a 1100 bed tertiary/quaternary medical center. The Internal
Medicine Residency Program has 185 residents who rotate through both outpatient clinics and
inpatient wards during their tenure. On the inpatient wards the residents interface with 30 core
teaching faculty, those who have distinguished themselves as medical teachers and are fluent
in inpatient care.
The general medical inpatient service has a mixed model of supervision. For new admissions
the attending is deeply involved in initial critical thinking and decision making. Attendings and
residents round together for two hours every morning examining and discussing the new
admissions. However, resident decisions regarding ongoing management and care of existing
patients on the service occur independently of the supervising clinician. After the new patients
are presented and discussed with the attending, the residents go on bedside work rounds to
evaluate the existing patients without the attending. During these bedside work rounds
residents examine all the existing patients, discuss updates, make decisions about plans for the
day, call consults and enter orders. The attending for the team briefly provides advice later in
the afternoon to help adjust and refine treatment plans on these existing patients.
To investigate whether increased attending supervision in resident decision making improves
patient safety, we plan to have these 30 core faculty join bedside work rounds with the
residents. Since faculty members will attend on service multiple times during the study period,
we will randomize all attendings to both the current system (usual care) and the increased
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
5
supervision model – participating in work rounds (intervention). Attendings will serve as their
own control. Will this increased supervision improve medical decision making by residents and
reduce medical errors?
To better understand the effects of attending input and participation on resident decision
making during work rounds we will assess the following:
Does attending presence influence: Metric of Measurement
Patient treatment plans Change in frequency of written orders
Time spent per patient at the bedside and in
discussion
Time motion study of work round length and
participant location
Quantity of resident input into decision
making
Vocalization of residents during work rounds
Quality of resident input into decision making Blinded review of work round transcripts by
established experts in the field
Resident perception of learning environment,
quality of care and self evaluation of
competency
Survey and focus groups
Attending perceptions of learning
environment, resident competency and
patient care
Survey and focus groups
Patient outcomes Medical errors, preventable and avoidable
adverse events during time of study, cost per
patient, # ICU transfers, LOS and # of pre‐noon
discharges
Traditionally, resident knowledge has served as a surrogate for overall competency, but
knowledge alone provides limited insight into how residents make decisions and the quality of
the decisions that are made. It is fundamentally physician decision making that addresses
competence. This research study will directly asses resident medical decision making, the
consequences of attending participation in this process and the effect on medical education
and patient outcomes.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
6
Given there are no prior studies assessing the impact of attending supervision on resident
critical decision making and patient outcomes in medicine, we believe this study would be
unique. We will utilize novel methods to collect data including time motion studies, direct
observation and review of transcripts from work rounds for analysis. This study is of particular
import as we strive to better understand what environments drive optimal decision making and
resident training. If increased supervision is mandated by the ACGME without research to
confirm its value to both patient safety and resident education, the outcome could be increased
costs to the health care system and possibly less competent physicians. Alternatively, through
better understanding of the inputs to optimal performance and the balance between
supervision and autonomy we can focus our efforts and resources on proven interventions.
Description of Methods
To understand the effects of attending participation on work rounds (the intervention) we will
randomize supervising attendings to two weeks on service with a medical team where they
attend work rounds and two weeks on service on the traditional model (attending participation
on new attending rounds only with post hoc advice for previous patients). All teams will consist
of two attendings, five interns and one junior resident. The study will be done from November
to late June 2015 in order to avoid the “July effect” where it is suspected there are more near
misses due to resident inexperience. (8) Given that approximately 80 novel patients are
admitted to a team per month, over the ensuing eight months over 1200 resident/attending
discussions of patient cases will be evaluated.
The primary outcome will be medical errors, preventable and avoidable adverse events and
near misses per 100‐admissions. These will be collected by two clinical research nurses through
chart review, review of all orders, daily solicited error reports from residents and attendings
and a review of all formal incident reports. Similar to a recent paper assessing rates of medical
errors and preventable adverse events during handoffs, all incidents will be classified as adverse
events, non‐intercepted potential adverse events, intercepted potential adverse events and
error with little potential for harm. (8) All recorded events will be blinded and adjudicated by
three research physicians as to whether they are real errors, adverse events and near misses.
Disagreements will be resolved by discussion.
Secondary outcomes will include changes in medical orders between 12pm and the 7pm (when
the team signs out). This should reflect whether the teams had to reverse the orders they wrote
during morning bedside work rounds. We will also assess cost per patient. This should reflect
whether closer supervision during decision making on work rounds reduces unnecessary orders
and helps reduce expensive test ordering. Other patient safety results will be obtained from
hospital data including length of stay, pre‐noon discharges, transfers to the ICU and
readmissions. Additional secondary outcomes include patient and family satisfaction with MD
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
7
communication (part of the hospital collected post discharge survey results). We will survey
residents about perception of autonomy and satisfaction and perceptions about education. We
will also survey faculty about their perception of resident autonomy and education. We will
asses both the residents and faculty’s perception of how frequently they believe faculty input
changed plans on new and old patient work rounds.
Finally we plan to do a time motion study of work rounds. We will record actual time of work
rounds to learn whether the presence of an attending increases the length of rounds. Also
recorded will be counts of who is speaking during bedside work rounds and how often the
intern is interrupted. We will also evaluate transcripts of the work rounds for resident decision
making thought processes.
All of these outcomes will assess and probe resident decision making and its impact on patient
safety testing the hypothesis that increased attending supervision in resident decision making
improves patient safety.
Key References
1. Ericsson, KA, Krampe RT and Tesch‐Romero, C. The Role of Deliberate Practice in the Acquisition of Expert Performance. Psychological Review, 1993, Vol 100, No 3 363‐406.
2. Halpern S and Detsky A. Graded Autonomy in Medical Education – Managing Things That Go Bump in the Night. NEJM. 2014;370;12:1086‐1089.
3. Volpp, K, Rosen A, Rosenbaum P et al. Mortality Among Hospitalized Medicare Beneficiaries in the First Two Years Following ACGME Resident Duty Hour Reform. JAMA. 2007;298(9):975‐983
4. Desai S, Feldman L, Brown L et al. Effect of the 2011 vs. 2003 Duty Hour Regulation – Compliant Models on Sleep Duration, Trainee Education, and Continuity of Patient Care Among Intern Medicine House Staff. JAMA Intern Med. 2013;173(8):649‐655.
5. Kerlin M, Small D, Cooney E et al. A Randomized Trial of Nighttime Physician Staffing in an Intensive Care Unit. NEJMB 2013;368:2201‐9.
6. Garland A, Roberts D, GraffL. Twenty‐four Hour Intensivists Presence: A Pilot Study of Effects on Intensive Care Unit Patients, Families, Doctors and Nurses. Am J Respir Crit Care Med 2012;185:738‐43.
7. Kaufman, D. Applying Educational Theory in Practice. BMJ. 2003. 326(7382):213‐216.
8. Starmer A, Sectish T, Simon D et al. Rates of Medical Errors and Preventable Adverse Events Among Hospitalized Children Following Implementation of a Resident Handoff Bundle. JAMA. 2013;310(21):2262‐2270.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
8
B. A Description of Changes to the Proposed Research
We have made the following changes:
1. We now plan to use one research nurse instead of two.
2. The study dates will now be Sept 30, 2015 to June 7, 2016.
3. In the proposal we have clarified many of the outcomes.
4. We will be submitting our IRB application in early February.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
9
C. Table of Contents
A1. Application Cover Sheet Page 2
A2. Letter of Intent Page 3
B. Description of Changes Page 8
C. Table of Contents Page 9
D. Proposal Narrative Page 10
E. Budget Form Page 25
F. Budget Narrative Page 26
G. Project Timeline Page 27
H. Primary Qualifications Page 28
I. Biographical Data Forms Page 30
J. Appendices
References Page 43
IRB Page 44
Non Profit Status IRS 501(c)(3) Page 45
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
10
D. Proposal Narrative
An Assessment of Patient Safety in Resident Education: A Randomized Trial of Inpatient Medical Attending Supervision of Trainees.
1. Background Information
Graduate physician training has been based on the apprenticeship model where residents
provide care to patients under the supervision of an attending physician. As trainees gain patient
care experience they are given increasing responsibility with a gradual reduction in supervision.
This model of progressive independence allows clinical oversight as trainees master clinical
reasoning and decision making skills. The goal is to develop competent and independent
practitioners. [1] This progressive supervision is provided to residents as a mixture of scheduled
time with an attending physician and independent work time. In a paper defining degrees of
supervision, Kennedy et al call this traditional model “routine oversight.” [2] In most training
programs routine oversight consists of scheduled morning attending rounds where cases are
presented to the supervising physician for discussion and correction of residents’ plans.
Historically, the rest of the time trainees worked independently with the ability to call the
supervising physician if needed. [3] Much of the independent time occurred at night and on the
weekends when attending physicians were not in the hospital. [4]
In the last decade this independent work time has been scrutinized largely driven by the
patient safety movement. This traditional model of supervision was first called into question after
the Bell Commission investigation into the death of Libby Zion in a New York teaching hospital.
The investigation found both resident fatigue and inadequate clinical supervision as contributing
causes in the death. [5]
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
11
With pressures from the public, government and patient safety advocates the
Accreditation Council for Graduate Medical Education (ACGME) and the Institute of Medicine
(IOM) focused new policies on resident duty hours. The goal was to improve patient safety by
reducing medical errors through restriction of work hours; less fatigued residents would make
fewer mistakes. [3, 6] Starting in 2003, and again in 2011, resident work hours were limited.
However, duty-hour reform has not demonstrated the improved patient safety outcomes
anticipated. [7-9] A study of Medicare data found no mortality difference in medical or surgical
patients after the 2003 work hour change. [7] Following the 2011 changes, a study of 2,323
medical interns noted they worked fewer hours, but did not gain additional sleep and self-
reported more medical errors. [9] A second study showed an increase in patient hand-offs and a
decrease in both continuity of care and educational opportunities.[8]
Without the patient safety outcomes hoped for, there is now more focus on the second
recommendation of the Bell Committee: increased supervision. But will increased clinical
supervision improve patient safety? A small body of literature demonstrates benefits from
increased supervision or complications from its absence. [2, 10-12] One study found increased
resident compliance with emergency room guidelines when residents were supervised.[10] Here
supervision was defined by whether the attending wrote a separate note. A study in anesthesia
showed a reduction in complications during intubation when an attending was present. A surgical
trauma review noted missed radiological diagnoses without attending supervision. [11, 12]
Similarly, a study in primary care demonstrated that attendings judged patients to be more
seriously ill and reported a change in management for 27% of the cases after having seen the
patients.[13]
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
12
Survey research looking at the impact of resident supervision from the trainee perspective
suggests benefits from a higher degree of supervision. A national survey of residents in multiple
specialties asked residents how often they cared for patients without “adequate supervision,”
with 21% of 3,604 residents reporting inadequate supervision at least once a week.[14] Better
supervision correlated with positive ratings of learning, increased time with attendings and better
residency experience. In four studies on increased faculty presence on the wards, either in the
afternoon or overnight, residents reported increased satisfaction with both faculty and education.
[15-18] These studies conclude that increased supervision would enhance education and assume
it would improve patient safety.
The patient safety movement, lead by the IOM with support from the government,
medical boards, clinical educators and the public has called for increased supervision. Some
policy changes around supervision have already been implemented. The 2011 ACGME work-
hour changes included a requirement that “the program must demonstrate that the appropriate
level of supervision is in place for all residents who care for patients.”[19] One-third of U.S.
hospitals increased nighttime intensive care unit (ICU) supervision. [3] And yet early studies on
this new policy have not shown patient safety benefits. [20, 21] A one year ICU study found no
change in length of stay or mortality especially for those patients admitted overnight.[20] An
accompanying perspective piece in the New England Journal proposes that we should have a
better understanding of the clinical and educational tradeoffs between supervision and resident
autonomy before blindly implementing policies. [3]
What is known about medicine’s model of clinical education and supervision? In a
review of the medical, social science and educational literature, Kennedy notes there is little
evidence to demonstrate the efficacy of the current graduate medical model of education.[1] This
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
13
progressive supervision model appears to have developed intuitively. In a review of effective
supervision in clinical practice settings, Kilminster and Jolly conclude “(clinical supervision) is
probably the least investigated, discussed and developed aspect of clinical teaching.” [22] While
it is commonly believed supervision is critical in the acquisition of clinical skills and
professional development of trainees, it is difficult to know what actual components of
supervision matter. The Kilminster review concludes by asking, “In what circumstances is
supervision necessary? What sort of supervision should this be? “What is the optimal length and
frequency for supervision?” [22] There are indeed the questions that we seek to answer in the
proposed study.
An evaluation of models of supervision needs to consider not only the impact on patient
outcomes but also the impact on physician skill development. Is there harm in too much
supervision? Many physicians recall their independent work time as formative in their
development as clinicians. The expertise literature supports this idea. Learners must be engaged
in active decision making and take responsibility for the results of their actions in order to
integrate new information into their understanding of situations. [1] There needs to be some
degree of independence in order to progress to expertise. Educational theory notes that learning
takes place when learners are challenged to work beyond the level at which they comfortable and
self directed learning occurs when there is appropriate space between teacher and learner. [4]
And sociology literature has illustrated the limitations when learning and evaluation occur at the
same time; a common practice in clinical supervision. Studies in medical students reveal they
disguise their lack of knowledge and do not ask questions in order to portray competence. [23]
In Kennedy’s review she notes excessive supervision without progressive independence may
hamper progression to competent practitioner.[1] A balance between supervision and autonomy
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
14
is required to facilitate resident’s development. In the short term limiting autonomy might
improve patient safety, but in the long term could have unintended consequences of creating
physicians who are not ready for independent practice. The question is what is the right amount
autonomy without compromising patient safety?
The growth of hospitalists over the last decade has provided more faculty presence on the
wards expanding supervision beyond “routine oversight”. Hospitalists are now more involved in
patient care details and double check residents’ work. This is defined as “responsive oversight”
in the framework of supervision created by Kennedy et al. [2] Hospitalists also provide direct
patient care without involvement of the resident and practice more “backstage oversight” defined
as reviewing all the patient’s care details without the trainee’s awareness. A study on the
introduction of hospitalists in a pediatric hospital, where attendings gave more “responsive
oversight” and did direct patient care, found the interns reported learning more and still felt they
could make decisions independently, but upper level residents reported a decrease in their
knowledge and supervisory skills and a loss in their ability to make independent decisions. [18]
This study raises questions about the optimal type of supervision and when it should be applied.
In this context it is clear that there is limited evidence as to the appropriate balance of
clinical supervision and autonomy for both patient safety and educational purposes. Multiple
medical educators have expressed the need for studies evaluating types of supervision and its
related outcomes. [3, 4, 22] With this is mind, we propose to study the effect of additional
attending “responsive oversight” on resident medical teams in terms of both patient safety and
resident learning. Given that the mission of the National Board of Medical Examiners is to
“protect the health of the public through the state of art assessment of health professions”,
clinical supervision clearly falls under this domain. Traditionally, resident knowledge has
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
15
served as a surrogate for overall competency, but knowledge alone provides limited insight into
how residents make decisions and the quality of the decisions that are made. It is fundamentally
physician decision making that addresses competence. This research study will directly asses
resident medical decision making, the consequences of attending participation in this process and
the effect on medical education and patient outcomes.
2. Hypothesis or Research Question
Background: Currently our general medical inpatient service employs the model of “routine
oversight”. During the scheduled daily two-hour morning attending rounds new admissions are
presented at the bedside. These rounds focus only on new admissions to the medical team. After
attending rounds, a resident leads team rounds on all of the previously admitted patients without
the presence of the attending, identifying new medical issues, discussing ongoing problems and
reviewing the management and care of these existing patients. This is frequently referred to as
“work rounds”. The attending for the team independently sees and evaluates the previously
admitted patients and briefly provide advice to the supervising resident later in the afternoon to
help adjust and refine treatment plans. With this background, our research aims are:
Specific Aims:
1. To investigate whether a rounding model of increased resident supervision by
including attending physicians on work rounds (responsive oversight) in addition to new
patient rounds (routine oversight) compared to attending physicians on new patient
rounds only (routine oversight) result in improved patient safety? This will be measured
by a reduction in medical errors (primary outcome) and reductions in length of stay,
intensive care unit transfers, inpatient morality and costs (secondary outcomes). Our
hypothesis is that the increased supervision model will improve patient outcomes.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
16
2. To investigate whether a rounding model of increased resident supervision by
including attending physicians on work rounds (responsive oversight) in addition to new
patient rounds (routine oversight) compared to attending physicians on new patient
rounds only (routine oversight) affect resident autonomy, decision making and learning?
This will be assessed by the percentage of time of resident communication on work
rounds and length of work rounds (primary outcome). Secondary outcomes will include
resident, attending and nurse perception of education and autonomy and a qualitative
analysis of content of discussion on work rounds and reason for attending interruptions.
Our hypothesis is that increased supervision will not affect educational outcomes for
residents.
Outcomes:
To better understand the effects of attending input and participation on patient safety and
resident education and autonomy during work rounds we will assess multiple aspects of the
supervisor-resident-patient triad. (Table 1) Each aim, (1) patient safety and (2) education and
autonomy, will have its own primary and secondary outcomes. Given the complexity of the
relationship between supervision, resident education and patient safety, we believe it is necessary
to evaluate both aims and their outcomes during this study. The results of one aim cannot be
interpreted without the results of the other.
Patient Outcomes:
Our primary outcome of patient safety which will be assessed by recording medical
errors using the standard definition that medical errors are preventable failures in the process of
care. Medical errors will include preventable adverse events, near misses and errors with little or
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
17
no potential for harm. Using an established surveillance process developed in several studies
evaluating medical errors in residency programs, we will measure the rate of medical errors per
100 admissions. [24-26] We will also collect mortality, length of stay, transfers to the intensive
care unit (ICU) and total costs, including the number of radiology studies as secondary
outcomes. Our hypothesis is “responsive oversight” will reduce medical errors, mortality, length
of stay, transfers to the ICU, costs and number of radiology studies as residents are supervised in
decision making.
Educational Outcomes:
To assess the effect on resident autonomy, decision making and learning, we plan to use a
mix methods approach during work rounds for both the intervention period (responsive
oversight) and the control period (routine oversight). Our primary outcome will be assessed by
the length of work rounds and the amount of time the resident is communicating during these
rounds. Using a time motion study of work rounds, both on the intervention and usual care
teams; we will measure length of work rounds and quantity of conversation by the resident,
attending and interns. Who is actually talking during the work rounds? Our secondary outcomes
will include both quantitative and qualitative components. At the conclusion of each 2 week
block, we will survey of residents, attendings and nurses about their perception of the learning
environment, autonomy and decision making and patient care. Given nursing also participate in
resident work rounds, their unique perspective as the patient advocate will be valuable. The
qualitative data collection will include content analysis of the conversations occurring during
work rounds. We will assess the conversational interactions around the following clinical areas:
(1) interpretation of the data (labs, vital signs, and physical exam), (2) identifying problems, (3)
generating differentials, (4) decision making or (5) teaching points. Which area is the attending
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
18
participating in? These analyses will be conducted comparing conversations with and without an
attending present, to explore which areas the supervising resident participates. We will also
assess the reasons for attending interruptions on work rounds.
Table 1 Outcomes Does attending presence influence: Metric of Measurement Patient outcomes Primary
Medical errors: preventable adverse events, near misses and errors with little harm.
Patient outcomes Secondary
Mortality, # ICU transfers, LOS, cost of hospitalization and number of radiology studies.
Educational Outcomes Primary
Length of work rounds and time/percentage of resident communication during rounds.
Educational Outcomes Secondary - quantitative
Surveys of residents, attendings and nurses perceptions about learning environment, autonomy, decision making and care
Educational Outcomes Secondary - qualitative
Content analysis of discussion on work rounds and reasons for attending interruptions
3. Study Design and Methodology
Study Design: Our intervention is to expand supervision from “routine oversight” (attending
rounds on new patient only) to “responsive oversight” (attending rounds on new patients and
established patient work rounds). Since faculty members attend on service multiple times during
the study period, we will randomize all attendings to both the current system of “routine
oversight” (usual care) and the increased supervision model or “responsive oversight”
(intervention). In the intervention phase, the supervising physician will more actively participate
in work rounds, overseeing detailed resident discussions and decision making about existing
patients on the medical team.
We will conduct a randomized cross-over study, with each attending serving as his or her
own control. Faculty rotations are two weeks and they work with the same resident team for that
entire time. Faculty will be randomized to start with either the intervention or usual care for the
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
19
duration of the two weeks for their first rotation during the study period and then “cross-over” to
the other study arm for their next two week rotation. (Figure 1).
Figure 1
Study Settings: Massachusetts General Hospital (MGH) is a 1100 bed tertiary/quaternary
medical center. The study will take place on the MGH general medical service which consists of
5 identical teams on similar nursing floors. Patients are randomly assigned to teams by the
admitting department based on bed availability. The study will occur September 30, 2015 to June
7, 2016. We plan to start in late September to avoid the “July Effect” of new trainees, a time
when we would expect more medical errors.[27]
Participants: The Internal Medicine Residency Program has 185 residents who rotate through
both outpatient clinics and inpatient wards during their training. Inpatient resident teams consist
of one resident, 5 interns and two attendings and care for 20-24 patients. Resident teams rotate
Responsive Oversight (+attending rounds, + work rounds)
Routine Oversight (+attending rounds, ‐ work rounds)
2 weeks 2 weeks
Attending A
AttendingAttending B
Attending
Intervention
Arm
Control A
rm
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
20
every 2-4 weeks, with upper level residents working on this service 6-8 weeks a year and interns
working about 4 months a year. We anticipate the study will involve 30 upper level residents
and nearly all 85 interns. On the inpatient wards the residents interface with 20 core teaching
faculty, who have distinguished themselves as medical teachers and are fluent in inpatient care.
These 20 experienced clinician educators have a background in inpatient medicine and
frequently attend on service.
Attendings participating in this study will receive a one-hour training session on
“responsive oversight” with a discussion of expectations for joining work hours. Residents and
interns will receive an orientation to this process and will receive reassurances that the
measurement of medical errors will be blinded and there will be no consequences to their
reporting such errors. Nursing on the floors will receive the same orientation and reassurance.
Data Collection and Analysis:
Patient Outcomes: For the primary outcome of medical errors we will apply standard
definitions. [24, 25] Medical errors, as noted previously, are preventable failures in the process
of care and include: (1) preventable adverse events, (2) near misses (where the error was caught
before anything could happen) and (3) errors with little or no potential for harm. We will also
collect non-preventable adverse events, and since these events cannot be prevented, this event
rate should be similar in both arms. Using an established surveillance process, one trained
research nurse will review all medical records and orders on the medical teams, 5 days a week,
with Mondays review to include a review of the weekends. [24, 25] There will be a locked-box
on each floor where nurses and residents can anonymously submit possible medical errors and
near misses. The nurse will also ask the team each day about possible errors, as well as
pharmacy, and will check the hospital reporting system for events. All possible medical errors
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
21
will be reviewed by two blinded physician investigators. They will independently classify each
event as a preventable adverse event, near miss, error with little potential for harm, non-
preventable adverse event or exclusion (not a medical error). The preventability of the adverse
event will be rated on a 4-point Likert scale: (definitely preventable, probably preventable,
probably not preventable, or definitely not preventable) which will be dichotomized into
preventable versus non-preventable before analysis. We will be following the methodology of
the IPASS study looking at medical errors in hand offs. [24] We will also collect severity of
harm using the National Coordination Council for Medication Error Reporting and Prevention
Index for Categorizing Errors. (MCC MERP). [28] Any disagreements in error assessment will
be summarized using a kappa statistic and resolved by discussion between the physician
reviewers. Secondary patient outcomes (morality, number of ICU transfers, length of stay, cost
of hospitalization and number of radiology studies) will be obtained from the hospital data base.
In the primary analysis, we will use an intention-to-treat approach for each patient. That
is, patients’ group assignment will be determined by the status of the first responding clinician
they encounter during the study period. We will compare the patient characteristics between
patients under routine oversight and patients under responsive oversight. The potential
confounding factors will be included in the regression model if any imbalance exists. We will
compare between the two groups using Poisson regression models for medical error rate, number
of ICU transfers, and number of radiology studies and quantile regression models for length of
stay and cost of hospitalization. We will use the mixed effects model approach to take into
account the clustering of patients within each responding attending and model the care team
(residents, interns and nurses) as random effects.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
22
In a secondary analysis, we will determine patient’s group assignment on a daily basis
according to the status of the assigned attending since patients’ hospitalization might span across
faculty rotations. Poisson regression models will be used to compare medical error rate between
the two groups with group considered as a time-varying variable.
We will conduct two pre-specified exploratory analyses. The first is looking at error rates
comparing the first 4 months to the last 4 months of the study. Studies on residents are
complicated by the fact they gain knowledge and improve through time. Our faculty will be
randomized throughout the year to try and mitigate this, but we plan to evaluate error rates based
on time of year. We will also compare error rates looking at the order in which attendings are
randomized. If an attending does the “responsive oversight” arm first, will this affect their
ability to return to “routine oversight”?
Educational Outcomes: For the primary educational outcome we will have a research assistant
join resident work rounds both on the intervention team and usual care team. For this time-
motion study, we anticipate they will join rounds approximately 5 times during a two-week
block. They will record rounds with permission granted by the resident team, nurse, attending
and patients. Using an iPad device, they will record the number of times and length each
individual speaks. The total length of these work rounds will be recorded. We plan to collect the
length of work rounds on all teams on weekdays. On the days the research assistant cannot join
rounds, they will ask the team for the start and stop time of work rounds. We will first examine
the distribution of the length of work rounds and time/percentage of resident communication
during rounds and perform variable transformations if necessary. These outcomes will be then
compared using a linear regression model with the Generalized Estimating Equations method to
take into account of the clustering within each attending.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
23
For perceptions, we will email surveys to the residents, attendings and nurses at the end
of every two week rotation to evaluate their perception of autonomy, the learning environment,
decision making and quality of care of patients. Survey responses will be compared between the
two groups using regression models taking into account of the individual responder effect.
Qualitative Data Analysis: Using purposive sampling, a random selection of recorded work
rounds with and without attending participation will be transcribed. We will transcribe both the
intervention work rounds and usual care work rounds. Each transcript will be coded by the
physician investigators and will explore the content of the discussion, including: (1)
interpretation of data, (2) identification of active medical problems, (3) the generation of
differential diagnoses for the active medical problems, (4) decision making regarding plans for
each problem and (5) teaching points around active or theoretical problems. We will initially
review a subset of transcripts and develop preliminary codes. We will then apply these final
codes to our random sampling using NVivo 10 qualitative data analysis software. Coding will
continue until level of agreement (kappa > 0.80) is reached. We will resolve discrepancies by
reviewing the transcript data for context. We will evaluate which areas the attendings are
involved in, how often they speak and who makes the final decision for each problem. All
analyses will be conducted comparing findings on rounds with attendings and without
attendings.
In a prior study looking at attending interruptions on new patient presentations, the
researchers used the following 5 categories to classify interruptions by the attendings: (1)
Probing for further data, (2) prompting for expected sequence, (3) teaching around the case, (4)
thinking out loud and (5) providing direction. [29] We will use these categories for analyzing
attending interruptions.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
24
Finally, we will explore whether there is a relationship between percentage of time
communicating on rounds, the content areas discussed, number of interruptions by the attending
and if any of these are related to medical error rates.
Sample Size: Given that each attending sees approximately 25 new admissions during each two-
week block, and all 20 attendings will be attending both in the usual arm and intervention arm,
we anticipate to enroll 500 patients in each arm. Since patients are clustered within each
attending, the inflation factor is 2.7 in an intra-class correlation coefficient of 0.07 and the
effective sample is 187 per group. Based on prior studies using the same surveillance methods
we anticipate 55 errors per 100 admissions from the routine oversight group and 33 errors per
100 admissions from the responsive oversight group, which only requires 178 patients per group
for 80% power and a 0.05 two-sided significance level. ([24, 26]
Conclusion: Given the limited number of studies assessing the impact of attending supervision
on resident decision making, autonomy and patient outcomes, we believe this study would be
unique. We will utilize a well established surveillance process to assess medical errors, a time
motion study, direct observation and a qualitative analysis of transcripts from work rounds. This
study is of particular import as we strive to better understand different aspects of supervision and
their affect on the development of independent and competence physicians. If increased
supervision is mandated by the ACGME without research to confirm its value to both patient
safety and resident education, the outcome could be increased costs to the health care system and
possibly less competent physicians. In order to ensure to the general public both short term and
long term patient safety, we need to find the right balance of supervision and autonomy through
proven interventions.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
25
E. Project Budget Form
PERSONNEL‐ DIRECT COSTS Year 1 Year 2 TOTAL
Compensation (Not including fringe benefits.)
A. Kathleen Finn, MD, Principal Investigator $0 $0 $0
B. Research RN $73,873 $0 $73,873
B. Research Asst $23,400 $0 $23,400
C. Fringe Benefits $35,991 $0 $35,991
OTHER‐ DIRECT COSTS
A. iPad $600 $0 $600
B. Travel $0 $0 $0
C. Materials and Supplies $0 $0 $0
D. Consultants/Contractual (Include both honorarium and travel costs for consultants. Provide a breakdown in the Budget Narrative)
$0 $0 $0
E. NVivo Software ‐ 10 $2,500 $0 $2,500
PROJECT ADMINISTRATIVE CHARGES
(Limited to 10% of the amount of Total Direct Costs) $13,636 $0 $13,636
TOTAL PROJECT BUDGET $150,000 $0 $150,000
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
26
F. Budget Narrative
Research Nurse Investigator: The clinical research nurse must be experienced in inpatient medicine and clinical medicine. He or she will be reviewing charts and medical orders searching for medical errors. He or she will also need to speak regularly with team members, nursing and pharmacy and review medical error reports. Given the identification of medical errors is a primary outcome for this study we anticipate hiring a more advanced clinical nurse investigator. At our institution the top rate is $70/hour. We anticipate he or she will need to work 8 hours per day. This person will be hired for 8‐9 months.
Research Assistant: This person will need to record work rounds for the qualitative component of the study, as well as measure the speaking time of each team member and total length of rounds. They will be transcribing the recordings in the afternoon. We will need someone with a college degree who can transcribe. At our institutions the lowest rate for a research assistant is $15/hour. This person will be hired for 8‐9 months.
IPad: This device will be used by the research assistant for both recording rounds and for counting speaking time. We will be searching or programming an App to help us with both the recording and counting components.
NVivo software: This commercial software is needed to qualitative analysis of content of work rounds.
Physician Investigators: The investigators’ time will be supported by Massachusetts General Hospital
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
27
G. Project Timeline
January 14, 2015 Submission of the Invited Proposal
February 3, 2015 IRB Submission to Harvard Medical School and Partners Healthcare
Summer, 2015 Hiring of Research Nurse and Research Assistant
Purchase of iPad and programming data collection application
Randomization of Faculty
September 2015 Training of Research Nurse and Research Assistant
Training of Clinical Faculty who will be attending on service
Orientation sessions for resident teams and floor nursing
September 30, 2015 Start of first 2 week block and beginning of study
October 2015 – September 2016: Physician investigators’ will start blinded reviews of potential medical errors and coding of content of transcribed work rounds.
June 7, 2016 End of data collection. Completion of last randomized 2 week block
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
28
H. Primary Qualifications of Research Team
Principal Investigator:
Dr. Kathleen Finn is an Assistant Professor of Medicine at Harvard Medical School and the
Inpatient Associate Program Director for Internal Medicine at Massachusetts General Hospital.
She is actively involved in the training and evaluation of medical residents. She was the PI for a
$100,000 Partners Reengineering Grant and led a team evaluating discharge facilitators
embedded in resident teams. The study was published in the Journal of Hospital Medicine. She
has also led several quality improvement projects and is actively involved in residency
education redesign.
Research Team:
Dr. Christiana Iyasere is an Instructor at Harvard Medical School and a member of the Inpatient
Clinician Educator Service at Massachusetts General Hospital. She is actively involved in the
training and evaluation of medical residents in addition to development of novel curricula in
medical leadership. Dr. Iyasere attended Columbia Medical School and Harvard Business
School. She has been actively involved in research projects looking at novel ways to promote
ongoing clinical mentorship of junior hospitalists, and the role of discharge facilitators on
resident teams. She will be co‐lead investigator in this project.
Dr. Joshua Metlay is Professor of Medicine at Harvard Medical School and Chief of the Division
of General Internal Medicine at Massachusetts General Hospital. He has led numerous multi‐
institutional clinical studies, including cluster randomized trials, and has specific expertise in
developing methods for primary and secondary data collection, outcome measurement and
analysis. He has also led several training programs, including serving as the PI of two federally
funded institutional Career Development studies. He is serving as lead advisor.
Dr. Hasan Bazari is Associate Professor of Medicine at Harvard Medical School and the Emeritus
Residency Program Director for Internal Medicine at Massachusetts General Hospital. He has
participated in numerous research projects evaluating resident education, work structure and
sleep deprivation and well being. He is serving as one of the physician Investigator to the
project.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
29
Dr. Jatin M. Vyas is Associate Professor of Medicine at Harvard Medical School and the
Residency Program Director for Internal Medicine at Massachusetts General Hospital. He is an
NIH‐funded investigator in the area of fungal immunology. He served as the Chief Resident in
Medicine after training and actively participates in resident education and direct observation of
housestaff for over 15 years. He is serving as one of the physician investigators.
Dr. Yuchiao Chang is Assistant Professor of Medicine at Harvard Medical School and Statistician
at Massachusetts General Hospital. She is currently supporting all research activities for
Emergency Department and the Division of General Internal Medicine at Massachusetts
General Hospital. Dr. Chang has been the principal statistician for more than 50
federally/industrially funded grants, including cluster randomized trials. She has extensive
experience with various types of clinical data and advanced statistical methodology as reflected
by her list of more than 250 publications. She will be the primary statistician.
Dr. Elyse Park is Associate Professor of Psychiatry at Harvard Medical School and Director of
Behavioral Health Research at the Benson‐Henry Institute for Mind Body Medicine at
Massachusetts General Hospital. Dr. Park is an expert if mix‐methods approach and qualitative
research. She has been on numerous grants and studies involving qualitative research around
smoking cessation, palliative care and relaxation response. She will be serving as advisor and
statistician for the qualitative component of this study.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
30
I. Biographical Data Form
BIOGRAPHICAL DATA FORM
1. Name/Position in Project: Kathleen Finn Principal Investigator 2. Education/Training:
Institution and Location Degree Year(s) Field of Study
University of Pennsylvania BA 1987 Anthropology
Oxford University M.Phil 1989 Social Anthropology
Bryn Mawr College 1990 Post‐Baccalaureate
Harvard Medical School M.D. 1995 Medicine
3. Research and Professional Experience: 1998‐2009 Instructor in Medicine Harvard Medical School 2004‐2006 Medical Director of the General Medical Service Brigham & Women’s Hosp 2008‐present Inpatient Associate Program Director, MGH Internal Medicine Residency Program Department of Medicine 2008‐present Director of MGH Annual Teaching Retreat MGH 2008‐2009 Principal Investigator Partners Physician Education Care Delivery Reengineering Innovation Grant. 2009 Harvard MACY program Educators in Health Professions 2009 Physician Leadership Development Certificate Program – MGPO MGH 2009 – present Assistant and Associate Editor Journal of Hospital Medicine 2010 Assistant Professor of Medicine Harvard Medical School 2012 Kranes Award for Excellence in Clinical Teaching MGH 2014 Charles Burnett Special Recognition Award MGH 2014 ACP Top Hospitalist’s for 2014
4. Publications: 1. Finn K, Heffner R, Chang Y, Bazari H, Hunt D, Pickell K, Berube R, Raju S, Farrell E, Iyasere
C, Thompson R, O’Malley T, O’Donnell W and Karson A. Improving the Discharge Process by
Embedding a Discharge Facilitator in a Resident Team. J Hosp Med 2011; 6(9):494‐500.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
31
2. Finn K, Chiappa V, Puig A, and Hunt DP. How to become a better clinical teacher: a collaborative peer observation process. Med Teach 2011; 33(2):151‐155.
3. Finn K and Greenwald J. Hospitalists and Alcohol Withdrawal: Yes Give Benzodiazepines, but is that the Whole Story? J Hosp Med. 6(8):435‐7, 2011 Oct 6. 4. Finn, K. Inpatient Management of Alcohol Withdrawal. Hospital Medicine Clinics. Volume 1, Issue 1, pages A1‐A10, e132‐147 (January 2012). 5. Finn, K and Hunt, D. Editors. Hospital Medicine Clinics. Volume 1, Issue 4, Pages A1‐A10, e427‐e558 (October 2012) 6. Soverow J, McGarrah R, Editors. Finn, K, Wright D and Puig A Co‐Editors. The Evidence: Classic and influential studies every medicine resident should know. Selected Nights, LLC. 2013. 7. Dankers, C and Finn, K. Non‐Invasive Mechanical Ventilation. In: Decision Support in Medicine, Hospital Medicine. Decision Support in Medicine, LLC. 2013 8. Journal Watch. Hospital Medicine/Pediatrics and Adolescent Medicine. August 19, 2013. The Check‐Out Checklist. Finn, KM and Dressler D, reviewing Soong C et al. J Hosp Med 2013 Aug. 9. Journal Watch. Hospital Medicine/Cardiology. Sept 9, 2013. Drug Safety: All Statins are Not Created Equal. Finn, KM and Dressler D, reviewing Naci H et al. Circ Cardiovasc Qual Outcomes 2013 Jul. 10. Ramani S, Finn K, Katz JT, Yialamas M. Beyond Show and Tell: Promoting physical examination skills as essential habits of reflective practice. Academic Internal Medicine Insight. 2014;12(1):7‐8,13.
11. Shoeb, M, Khanna R, Fang M, Sharpe B, Finn K, Ranji, S and Monash B. Internal Medicine Rounding Practices and the ACGME Core Competencies. J Hosp Med 2014; 9(4):239‐243.
12. Finn KM, Ginns CL, Robbins GK, Wu CC, Branda JA. Case records of the Massachusetts General Hospital. Case 20‐2014. A 65‐year‐old‐man with dyspnea and progressively worsening lung disease. NEJM. 370(26):2521‐30.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
32
BIOGRAPHICAL DATA FORM
Name/Position in Project: Christiana Iyasere, Co‐Lead Investigator Education/Training: Institution and Location Degree Year(s) Field of Study
Columbia University, College of Physicians and Surgeons MD 2002 Medicine
Harvard Business School MBA 2008 Business
Research and Professional Experience: Instructor in Medicine, Harvard Medical School 2006‐current
Clinician Educator Service, Massachusetts General Hospital 2008‐current
Associate Director, MGH Innovation Support Center 2008‐current
Publications: Christiana A. Iyasere, M.D., Leigh H. Simmons, M.D., Florian J. Fintelmann, M.D., and Anand S.
Dighe, M.D. Case 38‐2014 – An 87 Year‐Old Man with Sore Throat, Hoarseness, Fatigue and
Dyspnea. N Engl J Med 2014; 371:2321‐2327December 11, 2014.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
33
BIOGRAPHICAL DATA FORM
Name/Position in Project: Joshua Metlay Senior Advisor – Physician Investigator
Education/Training: Institution and Location Degree Year(s) Field of Study
Yale University, New Haven, CT B.A. 05/84 Biology Rockefeller University, New York, NY Ph.D. 06/90 Immunology Cornell University, New York, NY M.D. 05/91 Medicine
Harvard School of Public Health, Boston, MA M.Sc. 06/97 Health Policy/Management
Research and Professional Experience: 1995‐97 Clinical and Research Fellow in Medicine, Massachusetts General Hospital, Boston
1997‐2006 Assistant Professor of Medicine and Epidemiology, University of Pennsylvania
1997‐2013 Senior Scholar, Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania
1997‐2009 Staff Physician, Veterans Affairs Medical Center, Philadelphia, PA 1997‐2013 Senior Fellow, Leonard Davis Institute of Health Economics,University of Penn
2006‐2010 Associate Professor of Medicine and Epidemiology (Tenure), University of Penn
2006‐2010 Program Leader, Doris Duke Clinical Research Fellowship, University of Penn
2006‐2013 Co‐Director, Robert Wood Johnson Foundation Clinical Scholars Program
University of Pennsylvania School of Medicine
2009‐2013 Chief, Section of Hospital Medicine, University of Pennsylvania School of Med
2009‐2013 Director, Center for Healthcare Improvement and Patient Safety,
University of Pennsylvania School of Medicine
2010‐2013 Professor of Medicine, Emergency Medicine and Epidemiology, University of
Pennsylvania
2013‐ Chief, Division of General Medicine, Massachusetts General Hospital, Boston
2013‐ Professor of Medicine, Harvard Medical School
Honors
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
34
1982 Phi Beta Kappa, Yale University
1989 Alpha Omega Alpha, Cornell University Medical College
1995 National Associates Award for Outstanding Research, Society of General Internal
Medicine
1999 Robert Wood Johnson Foundation Generalist Physician Faculty Scholar
2003 Robert Austrian Faculty Research Award. Department of Medicine. University of
Pennsylvania
2005 Penn Pearls Teaching Award, University of Pennsylvania School of Medicine
2008 Christian and Mary Lindback Foundation Award for Distinguished Teaching
2009 Samuel Martin Health Evaluation Sciences Research Award, University of Penn
2010 Mid‐Career Research and Mentorship Award, Society of General Internal Med
2011 Arthur Asbury Outstanding Faculty Mentor Award, University of Pennsylvania
2012 American Epidemiological Society
2014 Award for Excellence in Research, Society of Hospital Medicine
Publications:
1. Metlay JP, Lautenbach E, Li Y, Shults J, Edelstein PH: The changing role of exposure to children as a risk factor for bacteremic pneumococcal disease in the post conjugate vaccine era. Archives of Internal Medicine. 2010;170:725‐731. NIHMS 15969
2. Soneji S, Metlay J. Mortality reductions for older adults differ by race/ethnicity and gender since the introduction of adult and pediatric pneumococcal vaccines. Public Health Reports. 2011;126:259‐269. PMCID: PMC3056039
3. Feemster KA, Li Y, Localio AR, Shults J, Edelstein P, Lautenbach E, Smith T, Metlay JP: Risk of invasive pneumococcal disease varies by neighborhood characteristics: Implications for prevention policies. Epidemiology and InfectionEpub ahead of print Oct, 2012. PMCID: PMC Journal‐In Process.
4. Gonzales R, Anderer T, McCulloch CE, Maselli JH, Bloom FJ, Graf TR, Stahl M, Yefko M, Molecavage J, Metlay JP: A cluster‐randomized trial of decision support strategies for reducing antibiotic use for acute bronchitis. JAMA Internal Medicine. 173:267‐273, 2013.PMCID:PMC3582762
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
35
BIOGRAPHICAL DATA FORM
Name/Position in Project: Hasan Bazari Physician Investor
Education/Training: Institution and Location Degree Year(s) Field of Study
1976 B.A Biology Columbia College, New York
1978 M.A. Biology Columbia University New York
1979 M.Ph Biology( Dr.Cyrus Levinthal) Columbia University, New York
1983 M.D. Medicine Albert Einstein College of Medicine, New York.
Research and Professional Experience: Program Director emeritus
Director, Swartz Initiative
Associate Professor of Medicine
2010 Winner of the Alfred Krane Award
2011 Honor Roll for the Partners Program Director Award
2011 Principal Clinical Experience Teaching Award
2011 Gold Humanism Award
2012 Excellence in Clinical Teaching
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
36
2013 Outstanding Program Director Award
Publications: a.Ripp J, Babyatsky M, Fallar R, Bazari H, Bellini L, Kapadia C, Katz J, Pecker M, Korenstein D. The
incidence and predictors of job burnout in first‐year residents: A five institution study.
Academic Medicine 2011;86:1304‐1310.
b. Finn K, Heffner R, Chang Y, Bazari H, Hunt D, Pickell K, Berube R, Raju S, Farrell E, Iyasere C,
Thompson R, O’Malley T, O’Donnell W and Karson A. Improving the Discharge Process by
Embedding a Discharge Facilitator in a Resident Team. Jour of Hospital Medicine 2011:6:494‐
500
c.Caverzagie KJ, Iobst WF, Aagard EM, Hood S, Chick DA, Kane GC, Brigham TP, Swing SR, Meade
LB, Bazari H, Bush RW, Kirk LM, Green ML, Hinchey KT and Smith,CD. The Internal Medicine
Reporting Milestones and the Next Accreditation System. Annals of Internal Medicine
2013;158:557‐9.
d.. Leaf DE, Pereira RC, Bazari H, Juppner H. Oncogenic Osteomalacia due to FGF‐23 Expressing
Colon Adenocarcinoma. The Journal of Clinical Endocrinology and Metabolism 2013
Mar;98(3):887‐91.
e..Intravenous moderate‐dose bumetanide continuous infusion and severe musculoskeletal
pain Vaduganathan M, Allegretti AS, Manchette AM, Patel SS, Olson KR, Bazari, H. Int J Cardiol
2013 Sept 20:168(1):e29‐31.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
37
BIOGRAPHICAL DATA FORM
Name/Position in Project: Jatin M. Vyas, MD, PhD ‐ Residency Program Director Physician Investigator Education/Training: Institution and Location Degree Year(s) Field of Study
University of Texas at Austin B.A. (plan II) 1985‐1989 Liberal Arts Honors Program
Baylor College of Medicine Ph.D. 1989‐1994 Microbiology and Immunology
Baylor College of Medicine M.D. 1989‐1996 Medicine
Research and Professional Experience: 2007‐present Faculty, Division of Infectious Disease, Department of Medicine, Massachusetts
General Hospital, Harvard Medical School
2011‐present PI on 2 NIH R01 to fund basic investigations in the Innate Immune Responses to
Fungal Pathogens
2014‐present Residency Program Director for the Department of Medicine, Massachusetts
General Hospital
Publications:
1. Tam JM, Mansour MK, Khan NS, Yoder NC, Vyas JM. Use of fungal derived polysaccharide‐
conjugated particles to probe Dectin‐1 responses in innate immunity. 2012. Integrative Biology
(Camb). Feb 1;4(2) 220‐7.
2. Huett A, Heath RJ, Begun J, Sassi SO, Baxt LA, Vyas JM, Goldberg MB, Xavier RJ. The LRR‐ and RING‐
domain protein LRSAM1 is an E3 ubiquitin ligase crucial for ubiquitin‐dependent autophagy of
intracellular Salmonella typhimurium. 2012 Cell: Host and Microbe. Dec 13;12(6):778‐90
3. Vyas, JM, Marasco WA. Fatal Fulminant Hepatic Failure from Adenovirus in Allogeneic Bone Marrow Transplant Patients. Case Reports in Infectious Disease. 2012.2012:463569.
4. Bell T. Vyas, JM. Prosthetic Joint Infections. Hospital Medicine Clinics. 2012. 1 e498‐e507.
5. Mansour MK, Tam JM, Vyas JM. The Cell Biology of the Innate Immune Response to Aspergillus fumigatus. Annals of the New York Academy of Sciences. 2012. Dec;1273(1):78‐84.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
38
6. Vyas JM. The Dendritic Cell: The General of the Army. Virulence. 2012. Nov 15;3(7):601‐602.
7. Vyas JM. Insights into Dendritic Cell Function Using Advanced Imaging Modalities. Virulence. 2012. Nov 15;3(7): 690‐694.
8. Grimm, MJ, Vethanayagam RR, Almyroudis NG, Dennis CG, Khan ANH, D’Auria1 A, Singel KL,
Davidson BA, Knight PR, Blackwell TS, Hohl TM, Mansour MK, Vyas JM, Röhm M, Urban CF, Kelkka T,
Holmdahl R, Segal BH. Monocyte and macrophage‐targeted NADPH oxidase mediates antifungal host
defense and regulation of acute inflammation in mice. 2013. The Journal of Immunology. Apr
15;190(8):4175‐84.
9. Mansour MK, Tam JM, Khan NS, Seward M, Davids PJ, Puranam S, Sokolovska A, Sykes DB, Dagher Z,
Becker C, Tanne A, Reedy JL, Stuart LM, and Vyas JM. Dectin‐1 activation controls maturation of β‐1,3‐
glucan‐containing phagosomes. 2013. Journal of Biological Chemistry. Apr 22. 288(22):16043‐54
10. Vyas JM, González RG, Pierce VM. Case records of the Massachusetts General Hospital. A 76‐Year‐Old Man with Fever, Worsening Renal Function, and Altered Mental Status. New England Journal of Medicine. 2013. May 16;368(20):1919‐27.
11. Kasper L, Seider K, Gerwien F, Allert S, Brunke S, Schwarzmüller T, Ames L, Barrera CZ, Mansour MK, Becken U, Barz D, Vyas JM, Reiling N, Haas A, Haynes K, Kuchler K and Hube B. Identification of Candida glabrata genes involved in pH modulation and modification of the phagosomal environment in macrophages. 2014. PloS One. May 1;9(5):e96015.
12. Tam JM, Mansour MK, Khan NS, Seward M, Puranam S, Tanne A, Sokolovska A, Becker CE, Acharya
M, Baird MA, Choi AMK, Davidson MW, Segal BH, Lacy‐Hulbert A, Stuart LM, Xavier RJ, and Vyas JM.
Dectin‐1‐Dependent LC3 Recruitment to Phagosomes Enhances Fungicidal Activity in Macrophages.
2014. The Journal of Infectious Disease. Dec 1;210(11):1844‐54.
13. Klassert TE, Hanisch A, Bräuer J, Klaile E, Heyl KA, Mansour MM, Tam JM, Vyas JM, Slevogt H.
Modulatory role of vitamin A on the Candida albicans‐induced immune response in human monocytes.
2014. Medical Microbiology and Immunology. Dec;203(6):415‐24
14. Mansour MK, Reedy JL, Tam JM and Vyas JM. Macrophage‐Cryptococcus Interactions: An Update.
2014. Current Fungal Infections Report. Mar 1;8(1):109‐115.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
39
BIOGRAPHICAL DATA FORM
Name/Position in Project: Yuchiao Chang, Statistician
Education/Training:
Institution and Location Degree Year(s) Field of Study
National Taiwan University, Taipei, Taiwan BS 06/86 Agronomy
Yale University, New Haven, Connecticut MA 05/89 Statistics
Carnegie Mellon University, Pittsburgh, PA PhD 12/93 Statistics
Research and Professional Experience:
Positions:
1986‐1987 Research Assistant, Biometry Laboratory, National Taiwan University 1991‐1992 Summer Lecturer, Department of Statistics, Carnegie Mellon University 1989‐1993 Teaching Assistant, Department of Statistics, Carnegie Mellon University 1998‐2001 Statistical Consultant, Gastroenterology 1993‐2005 Instructor in Medicine, Department of Medicine, Harvard Medical School 1993‐2006 Assistant Biostatistician, Massachusetts General Hospital, Boston, MA 1998‐2007 Statistical Consultant, Journal of Clinical Anesthesia 2005‐current Assistant Professor in Medicine, Department of Medicine, Harvard Medical
School 2006‐2012 Associate Biostatistician, Massachusetts General Hospital, Boston, MA 2012‐current Biostatistician, Massachusetts General Hospital, Boston, MA Honors: 1993 Biometric Society Student Award 1995 American Cancer Society Institutional Research Grant Award Publications:
Winickoff JP, Nabi‐Burza E, Chang Y, Finch S, Regan S, Wasserman R, Ossip D, Woo H, Klein J,
Dempsey J, Drehmer J, Hipple B, Weiley V, Murphy S, Rigotti NA. Implementation of a
Parental Tobacco Control Intervention in Pediatric Practice. Pediatrics. 2013 Jun 24. [Epub
ahead of print] PubMed PMID: 23796741; PubMed Central PMCID: PMC3691536.
Kruse GR, Chang Y, Kelley JH, Linder JA, Einbinder JS, Rigotti NA. Healthcare system effects of
pay‐for‐performance for smoking status documentation. Am J Manag Care. 2013
Jul;19(7):554‐61. PubMed PMID: 23919419.
Rodriguez F, Hong C, Chang Y, Oertel LB, Singer DE, Green AR, López L. Limited english
proficient patients and time spent in therapeutic range in a warfarin anticoagulation clinic. J
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
40
Am Heart Assoc. 2013 Jul 5;2(4):e000170. doi: 10.1161/JAHA.113.000170. PubMed PMID:
23832325.
Mejaddam AY, Elmer J, Sideris AC, Chang Y, Petrovick L, Alam HB, Fagenholz PJ. Prolonged
Emergency Department Length of Stay is Not Associated with Worse Outcomes in Traumatic
Brain Injury. J Emerg Med. 2013 Jun 12. doi:pii: S0736‐4679(13)00452‐6.
10.1016/j.jemermed.2013.04.015. [Epub ahead of print] PubMed PMID: 23769388.
Lobachova L, Brown DF, Sinclair J, Chang Y, Thielker KZ, Nagurney JT. Patient and Provider
Perceptions of Why Patients Seek Care in Emergency Departments. J Emerg Med. 2013 Sep
21. doi:pii: S0736‐4679(13)00830‐5. 10.1016/j.jemermed.2013.04.063. [Epub ahead of print]
PubMed PMID: 24063881.
Brouwers HB, Chang Y, Falcone GJ, Cai X, Ayres AM, Battey TW, Vashkevich A, McNamara KA,
Valant V, Schwab K, Orzell SC, Bresette LM, Feske SK, Rost NS, Romero JM, Viswanathan A,
Chou SH, Greenberg SM, Rosand J, Goldstein JN. Predicting Hematoma Expansion After
Primary Intracerebral Hemorrhage. JAMA Neurol. 2013 Dec 23. doi:
10.1001/jamaneurol.2013.5433. [Epub ahead of print] PubMed PMID: 24366060.
Sepucha K, Feibelmann S, Chang Y, Hewitt S, Ziogas A. Measuring the quality of surgical
decisions for Latina breast cancer patients. Health Expect. 2014 May 12. doi:
10.1111/hex.12207. [Epub ahead of print] PubMed PMID: 24813584.
Rigotti NA, Regan S, Levy DE, Japuntich S, Chang Y, Park ER, Viana JC, Kelley JH, Reyen M,
Singer DE. Sustained care intervention and postdischarge smoking cessation among
hospitalized adults: a randomized clinical trial. JAMA. 2014 Aug 20;312(7):719‐28. doi:
10.1001/jama.2014.9237. PubMed PMID: 25138333.
Kimo Takayesu J, Ramoska EA, Clark TR, Hansoti B, Dougherty J, Freeman W, Weaver KR,
Chang Y, Gross E. Factors Associated With Burnout During Emergency Medicine
Residency. Acad Emerg Med. 2014 Sep;21(9):1031‐1035. doi: 10.1111/acem.12464. PubMed
PMID: 25269584.
White BA, Chang Y, Grabowski BG, Brown DF. Using lean‐based systems engineering to
increase capacity in the emergency department. West J Emerg Med. 2014 Nov;15(7):770‐6.
doi: 10.5811/westjem.2014.8.21272. Epub 2014 Oct 10. PubMed PMID: 25493117; PubMed
Central PMCID: PMC4251218.
Baggett TP, Chang Y, Singer DE, Porneala BC, Gaeta JM, O'Connell JJ, Rigotti NA. Tobacco‐,
Alcohol‐, and Drug‐Attributable Deaths and Their Contribution to Mortality Disparities in a
Cohort of Homeless Adults in Boston. Am J Public Health. 2014 Dec 18:e1‐e9. [Epub ahead of
print] PubMed PMID: 25521869.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
41
BIOGRAPHICAL DATA FORM
Name/Position in Project: Elyse R. Park Qualitative Researcher
Education/Training:
Institution and Location Degree Year(s) Field of Study
Tufts University, Medford, MA B.A. 1988 Social Psychology Yeshiva University, Bronx, NY Ph.D. 1997 Clinical Health
Harvard School of Public Health, Boston, MA MPH 2007 Behavioral Medicine Public Health
Research and Professional Experience:
1990‐1991 Research Intern, Memorial Sloan Kettering Cancer Center, New York, NY.
1993‐1995 Graduate Research Assistant, Albert Einstein College of Medicine, Bronx, 1995‐1996 Psychology Intern, APA approved Clinical Psychology/Behavioral Medicine
Program. Union Memorial Hospital, Baltimore, MD. 1996‐1998 Behavioral Medicine Fellow. Brown Medical School, Providence, RI. 1998‐2000 Research Associate, Dana‐Farber Cancer Institute, Boston, MA. 2001‐2006 Instructor, Department of Psychiatry, Harvard Medical School, Boston, M.
2006‐2011 Assistant Professor, Department of Psychiatry, Harvard Medical School, B 2007‐2010 Chief of Behavioral Health Research, Benson‐Henry Institute for Mind 2009‐ Director of Behavioral Sciences, MGH Center for Psychiatric Oncology and
Behavioral Sciences at the Cancer Center 2009‐ Director of Behavioral Science Research, MGH Tobacco Research & Treatment Center
2010‐ Director of Behavioral Health Research, Benson‐Henry Institute for Mind Body Med 2011‐ Associate Professor, Department of Psychiatry, Harvard Medical School
Honors
1991‐1994 Academic Scholarship (Yeshiva University, Bronx, NY)
1993‐1994 Jewish Foundation for Education of Women Award
2005 Author of top 25 most read articles of 2005 (Health Affairs)
2006 Behavioral Medicine Excellence in Mentorship Award (MGH, Boston, MA)
2008 4th Biennial Survivorship Research Conference Meritorious Presentation
2010 Clinical Innovator Award, MGH Cancer Center’s Psychiatric Oncology and
Behavioral Sciences Center
2014 Mentor: Best poster, 11th annual American Psychosocial Oncology Society
5. Publications: Mueller E, Park ER, Davis M. What the Affordable Care Act Means for Survivors of Pediatric Cancer. Journal of Clinical Oncology. 2014; 32:615‐7.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
42
El‐Jawahri A, Traeger L, Park ER, Greer JA, Pirl WF, Lennes IT, Jackson VA, Gallagher ER, Temel JS.Associations among prognostic understanding, quality of life, and mood in patients with advanced cancer. Cancer. 2014; 15: 278‐285. Kirchhoff AC, Montenegro RE, Warner EL, Wright J, Fluchel M, Stroup AM, Park ER, Kinney AY. Support Care Cancer. Childhood cancer survivors' primary care and follow‐up experiences. 2014; 22: 1629‐35. Traeger L, Cannon S, Keating NL, Pirl WF, Lathan C, Martin MY, He Y, Park ER. Race by sex differences in depression symptoms and psychosocial service use among non‐Hispanic black and white patients with lung cancer. J Clin Oncol. 2014; 32(2):107‐13. Gareen I, Duan F, Greco EM, Snyder BS, Boiselle PM, Park ER, Fryback D, Gatsonis C. Impact of Lung Cancer Screening Results On Participant Health‐Related Quality Of Life and State anxiety in the National Lung Screening Trial. Cancer. 2014. 120: 3401‐9. Vranceanu A, Gonzalez A, Denninger J, Baim P, Park ER. Exploring the effectiveness of a modified comprehensive mind‐body intervention for medical and psychological symptom relief. Psychosomatics. 2014; 55:386‐91. Gonzalez A., Keating N, Japuntich S, He L, Wallace R, Park ER. Pain Experiences among a Population‐Based Cohort of Current, Former and Never Regular Smokers with Lung and Colorectal Cancer. Cancer 2014; 120:3554‐61. Back AL, Park ER, Greer J, Jackson V, Temel JS. Clinician roles in early integrated palliative care for patients with advanced cancer: a qualitative study. Journal of Palliative Medicine. 2014; 17:1244‐8. Rigotti NA, Regan S, Levy DE, Japuntich S, Chang Y, Park ER, Viana JC, Kelley JH, Reyen M, Singer DE. Sustained care intervention and post‐discharge smoking cessation among hospitalized adults: a randomized clinical trial. JAMA. 2014; 312:719‐28. Vranceanu AM Merker VL, Plotkin SR, Park ER. The Relaxation Response Resiliency Program (3RP) in patients with NF1, NF2, and schwannomatosis: Results from a pilot study. Journal of Neurooncology 2014; 120:103‐9. Huffman J., Moore S., DuBois C., Mastromauro C., Suarez L. & Park ER. An exploratory mixed methods analysis of adherence predictors following acute coronary syndrome. Psychology, Health, and Medicine. 2014; 15: 1‐10.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
43
J. Appendices
References – Literature Cited
1. Kennedy, T.J., et al., Progressive independence in clinical training: a tradition worth defending? Acad Med, 2005. 80(10 Suppl): p. S106‐11.
2. Kennedy, T.J., et al., Clinical oversight: conceptualizing the relationship between supervision and safety. J Gen Intern Med, 2007. 22(8): p. 1080‐5.
3. Halpern, S.D. and A.S. Detsky, Graded autonomy in medical education‐‐managing things that go bump in the night. N Engl J Med, 2014. 370(12): p. 1086‐9.
4. Kennedy, T.J., Towards a tighter link between supervision and trainee ability. Med Educ, 2009. 43(12): p. 1126‐8.
5. Bell, B.M., Resident duty hour reform and mortality in hospitalized patients. JAMA, 2007. 298(24): p. 2865‐6; author reply 2866‐7.
6. Farnan, J.M., et al., Strategies for effective on‐call supervision for internal medicine residents: the superb/safety model. J Grad Med Educ, 2010. 2(1): p. 46‐52.
7. Volpp, K.G., et al., Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. JAMA, 2007. 298(9): p. 975‐83.
8. Desai, S.V., et al., Effect of the 2011 vs 2003 duty hour regulation‐compliant models on sleep duration, trainee education, and continuity of patient care among internal medicine house staff: a randomized trial. JAMA Intern Med, 2013. 173(8): p. 649‐55.
9. Sen, S., et al., Effects of the 2011 duty hour reforms on interns and their patients: a prospective longitudinal cohort study. JAMA Intern Med, 2013. 173(8): p. 657‐62; discussion 663.
10. Sox, C.M., et al., The effect of supervision of residents on quality of care in five university‐affiliated emergency departments. Acad Med, 1998. 73(7): p. 776‐82.
11. Velmahos, G.C., et al., Around‐the‐clock attending radiology coverage is essential to avoid mistakes in the care of trauma patients. Am Surg, 2001. 67(12): p. 1175‐7.
12. Schmidt, U.H., et al., Effects of supervision by attending anesthesiologists on complications of emergency tracheal intubation. Anesthesiology, 2008. 109(6): p. 973‐7.
13. Gennis, V.M. and M.A. Gennis, Supervision in the outpatient clinic: effects on teaching and patient care. Journal of general internal medicine, 1993. 8(7): p. 378‐80.
14. Baldwin, D.C., Jr., S.R. Daugherty, and P.M. Ryan, How residents view their clinical supervision: a reanalysis of classic national survey data. J Grad Med Educ, 2010. 2(1): p. 37‐45.
15. Phy, M.P., et al., Increased faculty presence on inpatient teaching services. Mayo Clin Proc, 2004. 79(3): p. 332‐6.
16. Defilippis, A.P., et al., On‐site Night Float by Attending Physicians: A Model to Improve Resident Education and Patient Care. J Grad Med Educ, 2010. 2(1): p. 57‐61.
17. Trowbridge, R.L., et al., The effect of overnight in‐house attending coverage on perceptions of care and education on a general medical service. J Grad Med Educ, 2010. 2(1): p. 53‐6.
18. Landrigan, C.P., et al., Effect of a pediatric hospitalist system on housestaff education and experience. Arch Pediatr Adolesc Med, 2002. 156(9): p. 877‐83.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
44
19. Questions, A.D.H.R.F.A. Duty Hours Requirements: Frequently Asked Questions. 2011 2015]; Available from: https://www.acgme.org/acgmeweb/Portals/0/PDFs/dh‐faqs2011.pdf.
20. Kerlin, M.P., et al., A randomized trial of nighttime physician staffing in an intensive care unit. N Engl J Med, 2013. 368(23): p. 2201‐9.
21. Garland, A., D. Roberts, and L. Graff, Twenty‐four‐hour intensivist presence: a pilot study of effects on intensive care unit patients, families, doctors, and nurses. Am J Respir Crit Care Med, 2012. 185(7): p. 738‐43.
22. Kilminster, S.M. and B.C. Jolly, Effective supervision in clinical practice settings: a literature review. Medical education, 2000. 34(10): p. 827‐40.
23. Lingard, L., et al., A certain art of uncertainty: case presentation and the development of professional identity. Social science & medicine, 2003. 56(3): p. 603‐16.
24. Starmer, A.J., et al., Rates of medical errors and preventable adverse events among hospitalized children following implementation of a resident handoff bundle. JAMA, 2013. 310(21): p. 2262‐70.
25. Starmer, A.J., et al., Changes in medical errors after implementation of a handoff program. The New England journal of medicine, 2014. 371(19): p. 1803‐12.
26. Kaushal, R., et al., Medication errors and adverse drug events in pediatric inpatients. JAMA, 2001. 285(16): p. 2114‐20.
27. Young, J.Q., et al., "July effect": impact of the academic year‐end changeover on patient outcomes: a systematic review. Annals of internal medicine, 2011. 155(5): p. 309‐15.
28. website, N.C.C.f.M.E.R.a.P. [cited January 9, 2015; Available from: http://www.nccmerp.org/types‐medication‐errors.
29. Goldszmidt, M., N. Aziz, and L. Lingard, Taking a detour: positive and negative effects of supervisors' interruptions during admission case review discussions. Academic medicine : journal of the Association of American Medical Colleges, 2012. 87(10): p. 1382‐8.
Institutional Review Board (IRB) Certification Status
Our application will be submitted February 2015. We are completing our data collection
documents and surveys.
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
45
Applicant’s Non Profit Status
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020
46
Downloaded From: https://jamanetwork.com/ by a Non-Human Traffic (NHT) User on 07/15/2020