Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Martin Pusic MD PhD on behalf of…Institute for Innovations in Medical Education
NYU School of Medicine
Observe-Interpret-Act:
An NYULMC Learning Analytics Case Study
LEARNING
ANALYTICS“Learning analytics refers to the interpretation of a wide range of data produced by and gathered on
behalf of students in order to assess academic
progress, predict future performance, and spot
potential issues”
- DOE 2012
How is Big Data Different?Traditional Big Data
Data Collection Purposeful Incidental / Opportunistic
Intrusiveness High Low
Data Acquisition Cost High Low
Frequency e.g. Quarterly Continuously
Temporality Static Reports Dynamic Dashboards
Hypotheses Causal Associations
Sample Size Small biopsies Large swaths
NYU Education Data Warehouse
Education Data
Warehouse
LMS
Curricular
Content
AdmissionsExams
ePortfolioPatient
Log
Learning
ModulesEvaluations
SIS
Reporting and Analytics
Data Marts for
Education Research
Simulation
Clinical Data
Warehouse
NYU Practice
NetworkEpic EMR
iBeacons
New “Listeners”
• iPads
• Point-of-care digital forms
• Immediate Lecture evaluations
• Learning interactions with EHR
• Anything a SmartPhone can do
NYU Education Data Warehouse
Education Data
Warehouse
LMS
Curricular
Content
AdmissionsExams
ePortfolioPatient
Log
Learning
ModulesEvaluations
SIS
Reporting and Analytics
Data Marts for
Education Research
Simulation
Clinical Data
Warehouse
NYU Practice
NetworkEpic EMR
NYU Education Data Warehouse
Education Data
Warehouse
LMS
Curricular
Content
AdmissionsExams
ePortfolioPatient
Log
Learning
ModulesEvaluations
SIS
Reporting and Analytics
Data Marts for
Education Research
Simulation
Clinical Data
Warehouse
NYU Practice
NetworkEpic EMR
Gartner Model
© Gartner Group
Path Diagram for the Class of 2014
• Path coefficients (with standard errors) were calculated using multiple regressions.
• Each variable is set as the dependent variable and all variables to its left as possible predictors.
• We only include the paths that are statistically significant.
• Think of each number in the following way – adjusted for all the other incoming arrows, a one standard deviation change in the predictor results in an x standard deviation change in the thing being predicted.
• For example, the regression equation predicting USMLE1 includes MCAT, College GPA and Medical School GPA. College GPA does not signfiicantly predict USMLE1 so there is no number. Adjusted for MCAT score, a one standard deviation rise in Med_GPA results in a 0.62 standard deviation rise in USMLE1 score.
MCAT
College GPA
USMLE1
0.26(0.067)
0.37(0.085) 0.62(0.073) USMLE2SHELF
0.36(0.09)
0.42(0.09)
0.29(0.09)
0.57(0.08)Med_GPA
NYU Analytics Center
Clinical Curriculum
High level
view for
chairs
Curriculum Stage
LEARNING
ANALYTICS“Learning analytics refers to the interpretation of a wide range of data produced by and gathered on
behalf of students in order to assess academic
progress, predict future performance, and spot
potential issues”
- DOE 2012
LEARNING
ANALYTICS“Quality improvement refers to the interpretationof a wide range of data produced by and gathered
on behalf of clinicians in order to assess progress,
predict future performance, and spot potential
issues”
- DOE 2012
Quality
Improvement
The New CME
• Un-Announced Standardized Patients
• In-Situ Simulations
• Patient Reported Outcome Measures
• Process Metrics
– E.g. Door to Needle Time in Stroke Activations
– E.g. Surgical Video
How is Big Data Different?Traditional Big Data
Data Collection Purposeful Incidental / Opportunistic
Intrusiveness High Low
Data Acquisition Cost High Low
Frequency e.g. Quarterly Continuously
Temporality Static Reports Dynamic Dashboards
Hypotheses Causal Associations
Sample Size Small biopsies Large swaths
Feedback
Door to Needle Time
Discussion