On-Demand Clinical Intelligence Clinical Looking Glass Training

On-Demand Clinical Intelligence

Clinical Looking Glass Training

Don’t just sit there!

Login and Change Password

1. Open Internet Explorer

2. Enter “https://secure1.afms.mil/CLG” in address

3. Enter Username and Password

4. Under “Virtual Desktops” click on “SG-CLG” hyperlink for Citrix VDI Login

5. Enter “http://clgpoc.afms.mil/CLGNET” in address

6. Enter username: usually first initial+lastname

7. Enter generic password: clg123

8. Change password

9. Save (toolbar at the bottom)

10. Log out

https://secure1.afms.mil/

https://secure1.afms.mil/CLG

Today

1. Introductions

2. Ground Rules

3. Why Clinical Looking Glass?

4. Introductory Training

5. HIPAA

Welcome to the Future

The Future of Clinical Business Intelligence

Dr. Eran BellinVice President, IT Clinical Research and Development

Montefiore Medical Center

Ground Rules

• Use cell phones outside• Follow me when teaching

– There is time for hands on• Slow me down, ask questions

CLG: Included Data

Labs

Procs

Orders

Meds

DiagnosesDeath

Direct and Purchased Care

Direct Care Purchased Care

% of Encounters in CLG/HSDW

50% 50%

Has a Facility Value Yes No (All are “No Valid Match

Found”)

Associated Labs Yes No

Associated Meds Yes No

ICD9 Diagnoses Yes Yes

CPT and ICD Procedures Yes Yes

Disposition Yes No (all are “No Valid match

Found”)

CLG Value

Quick

Answers

Clinician

Empowerment

Culture

Change

• Ad hoc inquiry• Comparison studies• Patient work lists• Self service

• Performance awareness• Evaluation of interventions• Professional achievement• Creative potential

unleashed

• Measurable success• Clinical unit empowerment• Satisfied patients• Improved health

CLG Introductory Training

Learning Roadmap

Level: Introductory

1. CLG Core Concepts2. View Outcome Comparison Study

3. Modify a Cohort (Bactrim and Hyperkalemia)

4. Events, Attributes and Sets (Chronic Kidney Disease)

5. Build a Study (Congestive Heart Failure)

6. Smart Reports

CLG Core Concepts

1) Analytic rules are non-qualifying2) Method applied to one or more groups3) Methods include:

• Append study data• List / data grid• Crosstabs / pivots• Time to Outcome (survival)• Incidence Density• Time in Range• …more to come

1) Qualifying rules for inclusion2) Index Date (I)

• Patient specific start/enrollment date

3) Group Types• Cohorts – unique patients, 1 index

instance per person• Event Collections – events,

multiple index instances per person

3) Sources• Event Canvas – most flexible• Smart Reports – subject specific• Upload – from non-CLG source

Groups I

“Reusable research objects”

Analysis

Study = Groups X Analysis

Male diabetics I

2 year survival

Female diabetics I

% days A1C in control

5 yr visit hx per MD

2 yr history of HTN

Next Visit Alerts

When Data Are Not Patient-Centric

1/1/2005 1/1/2006 1/1/2007

0

0 = index date

(EG start therapy)

0000

00

000

1

2

3

4

5

6

7

8

9

10

Patient #

What % of new diabetic patients were controlled in the year 2005? 4 / 10 = 40%

Diabetes Control

= outcome(EG achieve lab value)

0 = patient experience

Patient-Based Analysis of Diabetes Control

Enrollment 1 Year 2 Years

0

0 = index date

= outcome

(EG start therapy)

(EG achieve lab value)

0 = patient experience

0

0

0

0

0

0

00

0

1

2

3

4

5

6

7

8

9

10

Patient #

What % of new diabetic patients were controlled within 1 year? 5 / 10 = 50%

Diabetes Control

(same data, re-sorted)

Cohort Paradigm: Patient-Centric

• Subject specific follow-up periods– Contra-indications taken into account– Stop looking for outcome when patient is no longer at risk

• Group summary is an aggregation of individual experiences

• Epidemiologic methods are ideal for retrospective, observational studies

Learning Roadmap

Level: Introductory

1. CLG Core Concepts

2. View Outcome Comparison Study3. Modify a Cohort (Bactrim and Hyperkalemia)



6. Smart Reports

Clinical Scenario:Bactrim & Hyperkalemia

Is this happening in MHS?

Study Group: • Patients greater than 65 years old • Outpatient prescription for Bactrim during 2008 to 2011• Outpatient prescription for an ACE ARB within 365 days before

the Bactrim prescription

Comparison Group: • Patients greater than 65 years old • Outpatient prescription for a macrolide (ERYTHROMYCIN

STEARATE, ERYTHROMYCIN BASE, CLARITHROMYCIN, AZITHROMYCIN) during 2008 to 2011

• Outpatient prescription for an ACE ARB within 365 days before the Bactrim prescription

Outcomes: • Potassium level of 5.5 or greater within 0 to 30 days of the

Bactrim prescription start date.

Bactrim may cause hyperkalemia when combined with ACE ARBs• Does Bactrim (trimethoprim/sulfamethoxazole) cause

hyperkalemia?• What is the mechanism?• Which patients are at risk for developing hyperkalemia

while on Bactrim?• Is it okay for patients taking other meds that increase

potassium (ACE inhibitors, ARBs, etc.) to take Bactrim?• When should you check potassium?• What are antibiotic alternatives to Bactrim?• How should TMP/SMX be dosed in renal insufficiency?• Is it okay for patients taking meds that increase

potassium to use salt substitutes?

Trimethoprim and Hyperkalemia

Trimethoprim is commonly used in combination with sulfamethoxazole (TMP/SMX cotrimoxazole, Bactrim, Septra, others) for the treatment of a variety of infections such as urinary tract infections. Although this medication has been available for many years, a recognized, but little-known adverse effect is hyperkalemia. This document discusses the clinical data, mechanism and risk factors for trimethoprim-induced hyperkalemia.

Login, Change PW, go to Study Designer

1. Open Internet Explorer

2. Enter “http://clgpoc.afms.mil/CLGNET” in address bar

3. Enter username: first initial+lastname

4. Enter generic password: clg123

5. Change password

6. Save (toolbar at the bottom)

7. Open Study Designer (Analysis Menu)

8. Open Bactrim study

Wait here

Study Designer

Demo

• Study Designer overview• Results• Criteria• Group definition using:

Skip to Exercise 1

TTO Method Criteria Entry

TTO Results - Demographics

TTO Results – Target Event

TTO Results – All Events

TTO Results – Patient List

Learning Roadmap

Level: Introductory


2. View Outcome Comparison Study

3. Modify a Cohort (Bactrim & Hyperkalemia)4. Events, Attributes and Sets (Chronic Kidney Disease)

5. Build a Cohort

6. Browse a Cohort: List Method


8. Smart Reports

Exercise 1Modify a Cohort

See handout In-Class Exercise: Bactrim 18 to 65

– Change Demographics to Age 18 to 65– Re-build your cohort– Rename and rerun the study

“antibiot acearb hyperk for cls”– Observe Results– What conclusion can be made about Bactrim in this

younger population?

Learning Roadmap

Level: Introductory






6. Smart Reports

Clinical Scenario 2Chronic Kidney Disease

Demo: Replicate CKD Study at MHS

• Create a cohort of patients>=18 years who had at least one admission with Chronic Kidney Disease during the year 2009 AND had a hemoglobin lab test done with a value>12g per deciliter around 90 days of the admission date AND received an inpatient med order of Epoetin Alfa within 30 days of the admission date.

• Next create a comparison group with the same criteria except the patients who did NOT received the inpatient med order of Epoetin Alfa.

• Then use time to outcome method to track primary end point events of mortality (6 months), readmission with MI, CHF and STROKE.

[Earliest of EV1-CKD-ADMIT (And) ] EV1-CKD-ADMIT: [ All of [CKD : InpatAdmit] WHEN IN [YR2005] WITH [age>18] ]

ANDEV2-HEM: [ All of [HEM : LabTestDate] within 0 to 90 Days Around Event: EV1-CKD-ADMIT ]

ANDEV3-MED: [NOT All of [MED : MedOrderStartDate] within 0 to 30 Days After Event: CKD-ADMIT ]

[Earliest of EV1-CKD-ADMIT (And) ] EV1-CKD-ADMIT: [ All of [CKD : InpatAdmit] WHEN IN [YR2009] WITH [ageGTE18] ] AND EV2-HEM GT12 90 ARND: [ All of [HEM : LabTestDate] within 0 to 90 Days Around Event: EV1-CKD-ADMIT ] AND EV3-MED EPO 30 AFT: [All of [MED : MedOrderStartDate] within 0 to 30 Days After Event: CKD-ADMIT

Event Canvas Looks Like…

Cohort 1: CKD-W-HEM>12-WITH-MED

Cohort 2: CKD-W-HEM>12-WITHOUT-MED

Learning Roadmap

Level: Introductory





5. Build a Study (Congestive Heart Failure)6. Smart Reports

Clinical Scenario 3Congestive Heart Failure

Exercise 2: Building a Study– Build a Cohort

• Discharges in March 2012 with CHF

– Add Two Methods• List • Add outcome: readmission

• See handout: In-Class Exercise Two

Learning Roadmap

Level: Introductory



3. Modify a Cohort (Bactrim & Hyperkalemia)



6. Smart Reports

Smart Reports

• Smart Reports in CLG are:– Focused reports usually oriented around a

single subject– Utilize CLG objects (groups, sets)– Often have an operational orientation

Diagnosis Summary Report

• A focused report that shows – all diagnoses and associated procedures for a

cohort of patients• for index event • and subsequent/prior events

• A way to explore patterns of care• The coding of this care

– indirect– purchased care

Diagnosis Summary Report Inputs

Demo

Diagnosis Summary Report for a CHF Cohort

Don’t Forget!

• CLG Help• HIPAA guidance• Citing CLG• Performance Improvement vs.

Research

CLG Help

• Online manuals– CLG User Manual– Ad hoc Reports– Events Definitions

• Streaming Video• Manuals and videos are also available for download from

the Web at: http://exploreclg.montefiore.org/clg-resources/becoming-a-clg-user/MHC-Resources.aspx

• See http://exploreclg.montefiore.org for more information

http://exploreclg.montefiore.org/clg-resources/becoming-a-clg-user/MHC-Resources.aspx




http://exploreclg.montefiore.org/

CLG and HIPAA

CLG gives you access to most all patient data.

Discuss: why is this risky?

HIPAA Protections:• Most analysis done with “limited data set”• Supervisor authorization required to access identifiers• You are challenged when requesting identifiers:

• QI Project• IRB approved research• Patient worklist

• Off-site use of CLG requires encryption tool• You are audited annually

Citing CLG

Dozens of posters and manuscripts enabled by CLG.

Give CLG a shout out!

Find methods verbiage in your training folder or request it from CLGMHSAdministrator.

PI vs. Research

What distinguishes Performance Improvement from Research?

In your Training Packet:• Registering PI Projects with QM Dept• The QI-Research Divide and the Need for External Oversight• Oversight of QI: Focusing on Benefits and Risks

(request from CLGMHSAdministrator if needed)

Institutional Review Board (IRB):• Special addendum needed if project accesses data via CLG

Q & A

Learning Roadmap ADVANCED

Level: Advanced

1. Review of Introductory Concepts

2. Temporality in Groups

3. Event Collections

4. Upload Groups

5. Time to Outcome• Simple Mode• Advanced Mode

6. Time in Range

7. List Method

8. HANDS ON: Individual Clinical Questions

Review: Map the CKD Study

• Study Group: patients>=18 years who had at least one admission with Chronic Kidney Disease during the year 2005 AND had a hemoglobin lab test done with a value>12g per deciliter around 90 days of the admission date AND received an inpatient med order of Epoetin Alfa within 30 days of the admission date.

• Comparison Group: same as study group but without Epoetin Alfa within 30days of the admission

• Outcomes: mortality within 6mo, readmission within 6mo all cause, with MI, with CHF.

Exercise: fill in the CLG Study Template handout

CKD Events Diagram

Study Designer

Review: Map the CKD Study

• Study Group: patients>=18 years who had at least one admission with Chronic Kidney Disease during the year 2005 AND had a hemoglobin lab test done with a value>12g per deciliter around 90 days of the admission date AND received an inpatient med order of Epoetin Alfa within 30 days of the admission date.

• Comparison Group: same as study group but without Epoetin Alfa within 30days of the admission

• Outcomes: mortality within 6mo, readmission within 6mo all cause, with MI, with CHF.

Exercise: fill in the CLG Study Template handout

CKD Events Diagram

Epoetin Alfa

Effects of Epoetin Alfa on Hemoglobin Levels in CKD

CKD Patients With Epoetin Alfa

Admissions

Lab Test

Med Order

>= 18 Years Old

2005 When In

Within

Within

Around 90 Days Admissions(90 Days Before & After)

30 Days After Admissions

CKD Events Diagram

Epoetin Alfa



Admissions

Lab Test

Med Order

>= 18 Years Old

2005 When In

Within

Within



NOT

Epoetin Alfa



Admissions

Lab Test

Med Order

>= 18 Years Old

2005 When In

Within

Within



NOT

Mortality within 6 Months

Readmission within 6 Months

X

X

CKD Events Diagram


Level: Advanced




4. Upload Groups


6. Time in Range

7. List Method


Temporality in Groups

• WHEN IN – a calendar period (e.g., 1/1/12 – 3/31/12)– a clinical duration (e.g., admit – discharge)

• WITHIN– time from another event (e.g., 30d before

admit)

Temporality: WHEN IN Calendar Duration

Gatifloxacin was prescribed between 1/1/04 – 3/6/06

Temporality:WHEN IN Duration

Durational Events• Events with inherent start and stop times (e.g. Inpatient Admissions, Medication Orders)

Create a cohort of patients who had an ejection fraction of < 35 DURING an admission for CHF in 2009

Temporality:WITHIN Event

HGB test > 12 WITHIN 90 days of the inpatient admission for CKD


Level: Advanced



3. Event Collections4. Upload Groups


6. Time in Range

7. List Method


Event Collections: Non-Unique Patients

MRN INDEX …

111 5/1/10

111 3/13/10

111 1/5/10

222 4/1/10

333 4/15/10

333 4/30/10

MRN INDEX …

111 5/1/10

222 4/1/10

333 4/30/10

Cohort: Latest of …Event Collection: All of …

• Good for process of care, productivity and throughput analysis• Only difference in Event Canvas is “ALL” at the root condition (INDEX)


Level: Advanced




4. Upload Groups5. Time to Outcome

• Simple Mode• Advanced Mode

6. List Method

7. Time in Range


Demo: Upload Cohort

• Get template• Paste in MRN and index• Optional to submit end dates• Upload• Validate unmatched MRN’s


Level: Advanced




4. Upload Groups


6. Time in Range

7. List Method


Time to OutcomeSimple Mode

Compare hospitalization rates of well-controlled vs. poorly controlled diabetics

Well Controlled Diabetics = 1 hBa1c >9.5 June 2002-June 2003, subsequent hBa1c <7 within 180-365 days of initial bad test

Poorly Controlled Diabetics = 1 hBa1c >9.5 June 2002-June 2003, no subsequent hBa1c < 7 within 180-365 days of initial bad test

Take a Look

Demo• New study• Cohort definitions: controlled vs. uncontrolled

diabetics• TTO simple mode: outcome is inpatient admission

– Method options

• Export patient list

Finished study:

Skip to: Advanced Mode

Create New Study1) Click the “+” next to the work studies in Management Panel

2) Study Designer Shows you 3 tabs:

Main – for entering study name and description – this is where you first land

Groups – this is where you can define up to eight study groups (cohorts or event collections)

Methods – you can apply three method types to your study: 1. Time to Outcome2. List3. Time in Range

Define outcome

Start = either from index date or with blackout period (where outcomes are ignored)

Outcome = an Event Definition (simple mode) or Analysis Definition (advanced) mode

End = Risk window (for Event Definition only)

Method Name and DescriptionEach method has some method-specific required info

Select Study Groups

Click … button to go to Event Canvas to define a cohort/event collection

Click + to add new groups

Enter group names

Define groups in Event Canvas

TTO: Define the Outcome

1. Click on Methods Tab2. Click on the ? Outcome shape

TTO: Define the Outcome

1. You will see a dropdown of Event Definitions2. Choose one or the […] button to go to Event

Definition Builder

TTO: Define New Event as Outcome

Right-click on event definitions to add a new event definition

TTO: Event Definition Builder

1 2

3

1. Enter a name for this Event Definition2. Choose an Event3. Right click on the Definition icon to add defining Event

Attributes4. Click Save or Save as, X to exit

Note, the Event Definition itself has no reference to temporality

TTO: Temporality

1) After creating or choosing your Event Definition, click the down arrow

2) Optional Blackout Period: If you want to ignore outcomes for a certain number of days from index

3) Enter Risk window

TTO: Define End Point

Note, the analysis definitions available to terminate the evaluation window are only earliest/latest types

TTO: Other Options

And then click Run Method to generate results

TTO: generating results

TTO Results: Demographics

• 4 tabs:– Demographics– Target Event– All Events– Patient List

TTO Results: Target Event

X axis = timeY = percent of group achieving the result

Below this are point estimates with % of each group achieving the result within specified periods

TTO Results: All Events

TTO Results: Patient List

1. Click on the tab of the group you want to see the patient list for2. Time to = time to first event3. Occurred = 1 occurred , 0 did not occur4. Total = total number of outcomes occurred5. Outcome Event Date = date of first outcome

TTO Results: Exporting Patient List

1. Choose Export type (excel 2003, 2007 or csv2. Click the Export button3. After results are exported, pull the dropdown next to

View the exported results and choose data file.4. An excel window will open


Level: Advanced




4. Upload Groups


6. Time in Range

7. List Method


TTO: Advanced Mode

Scenario

Study Group: controlled

Comparison Group: uncontrolled diabetics

Outcome: time to next “follow-up” defined as either– next clinic visit, or – next glucose test

Advanced because: two events in combo outcome

Simple Mode Advanced ModeEvent Definitions (Evs) Analysis Definitions (ADs)

Only One Event in EVs Multiple Events in ADs

No Temporality ADs have Temporality Built In (WhenIN, WITHIN)Note: Only ADs with earliest or latest at the root are accessible in TTO

TTO: Advanced vs Simple Mode

Take a look

Demo• Method 2:

Add a New Method to Your Study

1) Click on Methods Tab, then click on the “+”

2) You will see the New Method dialog. Select Time to outcome as Method Type

How to Get to TTO Advanced Mode

Click the To Advance label to get to TTO advanced mode

TTO Advanced Mode

Choose an AD from the dropdown or click the … button to enter the Analysis Definition canvas

AD Based on Earliest of 2 Events

Original Outcome: time to next “follow-up” defined as either

next clinic visit, ORnext glucose test

Is the CLG criterion line defined correctly?

Run Method vs. Run All

You can have multiple methods in a study.

Click Run Method to run only the current method, Run All to run all methods in the study

Be sure to press the Save icon to save results once our study is finished running

Punch line: Follow-up is the Same


Level: Advanced




4. Upload Groups


6. Time in Range

7. List Method8. HANDS ON: Individual Clinical Questions

TIR Summarizes for each Individual the amount of time spent in a particular range of values

TIR Summarizes for the ENTIRE cohort of patients the %time spent in a particular range of values

Used Mainly for Event Types with Continuous Values (Labs, Findings)

Time in Range (TIR)

Research Question:

What percent of the last year did this cohort spend in the following hematocrit value ranges:

Must interpolate between lab values.

Time in Range (TIR)

Range Label Lab value limits Meaning

Too High Hct >= 32 Bad

Normal Hct >= 29 and Hct < 32

Good

Too Low Hct < 29 Bad

Linear interpolation between lab values

Interpolation Interval: – define max distance between values to

allow interpolation

Carry Forward: – Define how long to assume constant lab

value if next result is beyond the interpolation interval

Interpolation Rules

Time

Hem

ato

crit

32 -

29 -

X

X

X

X

X

Consider 1 patient’s Hct’s

interpolationcarry forward

missing missing

TIR Interpolation Rules (cont.)

Time

Hem

ato

crit

32 -

29 -X

X

X

X

X

Add quality categories

interpolationlifespan

missing missingLow

Target

High

TIR Interpolation Rules (cont.)

Take a look

Demo

control of Hematocrit over 2011

Go to: Questions

TIR Rule Set

TIR Method

TIR Method

3.35% of cohort’s patient days spent in “good” category (aka “Target”)


Level: Advanced




4. Upload Groups


6. Time in Range

7. List Method8. HANDS ON: Individual Clinical Questions

List Method: Advanced Topics

Scenario: CHF cohort, look for latest troponin values, latest sysBP, latest LDL

• Method 1 for basics (as needed)

• Method 2: advanced mode with long view• Method 3: multiple ADs for covariates

Go to: Orders with text

Demo: Study Designer List, Advanced mode, all Troponin within 60 days

• Utilizes analysis definitions – all, first or last instance of an AD value can be used

• Analysis definitions can utilize temporality• You can use multiple events in an AD• Users can add multiple outcomes to the same list

method

Click on To Advanced

Study Designer: List – Advanced Mode

Click on … button to create a new Analysis Definition

List Method:Analysis Definition

Note analysis definition can utilize all, earliest or latest at the root

Note analysis definition can utilize within or when in

List Mode Advanced: Wide view

One row per patient with potentially multiple outcomes per row in adjacent columns

List Mode Advanced: Long View

Multiple rows per patient , one for each outcome

List Method, Advanced -- Covariates

• Utilizing earliest or latest at the root of analysis definitions you can create a patient list with covariate measures such as:– Last systolic blood pressure– Last LDL– First hba1c > 7– Date of last colonoscopy

List Method Advanced – Covariates Examples

Utilize earliest or latest at the root.

Note, we can use a WHEN IN with calendar dates

List Method Advanced, Covariates

• To show multiple covariates, successively create and choose analysis definitions:

List Method Advanced, Covariates

• Note multiple analysis definitions and their attributes on left panel.

• Choose attributes from each AD and drag over to the right panel. Run in wide mode

List Method, Covariates output

Note primary care measures last LDL and last systolic BP next to each other on the same line

LDL Value LDL Test Date Systolic Value Systolic Date

Transferring Files from VDI to Local

From the military virtual desktop use your Email:

1.Save the files (.xls, .pdf) from CLG to a preferred location on the military virtual desktop. – Recommended locations: Desktop or My Documents

2.Using Internet Explorer navigate to webmail – webmail.health.mil

3.Login to webmail

7. Compose a new mail message

8. Address mail message to yourself.

9. Attach relevant files to mail message by browsing for files saved to preferred location, refer to step 2.

10. Complete the “Subject” and “Body” of the mail message.

11. Send the message

12. Access mail message from your Email on your local workstation

13. Save files to a preferred location on your local workstation.

Transferring Files from VDI to Local

Q & A