Employment OntarioLiteracy and Basic Skills

Performance Management Reports Training

For Service Providers

2

Evaluations

Complete the first two pages and put them in the back of your participant guide. You will

work with them at the end of each unit and at the end of the day.

3

Introductions

NamePositionOrganization

4

Key Message

Today’s training supports the LBS third-party network to build capacity by way of report analysis and by improving the quality of the reports (data integrity).

5

Target Audience

Service Provider staff

Especially those who collect data, or input data into CaMS, or perform report analysis and continuous improvement.

6

MTCU Expectations for 2014-15 and Beyond

Achieve contracted commitmentsIntegrate continuous improvementCollect quality site level data

7

Purpose of the Training

Importance of data integrityBusiness IntelligenceHow data is used to evaluate site

performance and make decisionsImportance of performing service

delivery analysis

8

Unit 1

Performance Management System

9

Unit 1 Objectives

Who is at the centre of LBS System‘Above and below the waterline’Performance Management SystemWhat Business Intelligence Entails

10

Above and Below the Waterline

11

Components of a PMS

12

Continuous Improvement

13

Unit 1 Objectives Review

Customer is at the centre of LBS SystemSolid business foundation supports

service deliveryPMS components and components of

componentsBusiness Intelligence: technology; business

processes; people

14

Unit 2

LBS Program Services And Client

Path

15

Unit 2 Objectives

LBS Client PathService Quality StandardImportance of data integrityWhere report data is input into CaMS

Client Path Diagram (top portion)16

secondary school credit

Info

rma

tio

n a

nd

Re

ferr

al

Inta

ke

as

ses

sme

nt

Le

arn

er

Pla

n C

rea

tio

n

Se

rvic

e P

lan

(E

OIS

-Ca

MS

)

Le

arn

ing

Ac

tivi

tie

s

Mil

es

ton

es

Cu

lmin

ati

ng

Ta

sks

Ex

it &

Fo

llo

w-U

p

Re

ferr

al

In

Su

ita

bil

ity

Re

ferr

ed

Ou

t

Co

mp

leti

on

s

public

1 INTAKE

5 EXIT and

FOLLOW-UP

4 TRAINING

3 LEARNER

PLAN

2 ASSESS

Goal Paths Activities

17

Service Quality Standard (SQS)

18

Core Measures Table

No data.1. Customer Satisfaction

2. Service Coordination

3. Suitability4. Learner Progress

5. Learners Served

Why is this core measure important?

What data integrity issues must be anticipated related to this core measure?

What strategies could the service provider implement to increase performance in this area?

2

19

Service Quality Standard (SQS) Calculation

PerformUnit 2 - Exercise 2

‘YTD All Participants - Actual’ X ‘Service Quality Weight’ X 10

(to the maximum SQS value allowed)

20

Exercise 2 Answers

SERVICE QUALITY ‘Actual’ column value for:

Customer Satisfaction: 100% X 15% X 10 = 1.50Service Coordination: 83% X 25% X 10 = 2.08Suitability: 26% X 20% X 10 = 0.52Learner Progress: 100% X 30% X 10 = 3.00Learners Served: 175% X 10% X 10 = 1.75 MAX 1.00

Overall SQS value = 8.10

21

Unit 2 Objectives Review

Clients, linear flow through programSQS supported by core measuresSQS calculationData integrity affects report dataConnect input to reports:

Table matched inputs to reports Report user guides Data Dictionary

22

Unit 3

Data Integrity

23

Unit 3 Objectives

Definition of ‘data integrity’Business owners of the dataImportance of timely data collectionManagement support of staffValue of orderliness

24

What is Data Integrity?

Includes:- collection of data- entry into EOIS-CaMS- paper documentation

Regarding each of the above:- complete, consistent, timely, and accurate

25

Business Owners of the Data

Service providers as owners:o verification and integrityo value of data is only as good as the inputo link cause and effect

26

Two Groups use the Data of the PMS

MTCUService providers

27

Data Collection

Service providers:o collect and report data (EOIS-CaMS)o maintain quality assurance systemso support data with documentation

MTCU and service providers:o analyse data

28

Orderliness

Consistent and clear processes

=

Resources used efficiently

29

Unit 3, Exercise 1‘Four Unusual Numbers’

LBS CM#

SERVICE QUALITY CORE MEASURES

LBS Prov.

Target

Perf Com Ann.

Target

YTacipants

NumYTD al Participants

DenYTD all Participants

Actual % YTD of

Target

Learner 2 11 18%

2 Service Coordination 50% 50% 0 45 0% 0%

3 Suitability 30% 30% 57 45 13% 42%

4 Progress 60% 60% 64 64 100% 167%

30

Unit 3 Objectives Review

Data Integrity: complete, consistent, timely, accurate data collection and entry into CaMS

Service providers are business owners of site data (collect quality data)

Timely data collection supports evidence-based decision-making

Staff: collect and enter data (CaMS)Management: stress data integrityOrderliness increases efficiency & quality

31

Unit 4

Report Analysis

32

Unit 4 Objectives

Reports availableAnalyse LBS dataWhat drives continuous improvementSupport of staff: report analysis and

continuous improvement

33

Learners Served – Pro-Rated Denominator

‘YTD All Participants – Actual’ for Learners Served

Performance Commitment Annual Target = 105 YTD All Participants – Numerator = 65Months in the report dated December 31 = 9 months

Annual Target - Pro-Rated:

105 X (9/12) = 79 65 / 79 = 82%

34

Learners Served – Pro-Rated Denominator (1)

35

Learners Served – Pro-Rated Denominator (2)

36

Continuous Improvement (2)

Client Path Diagram (re-visited)37

secondary school credit

Info

rma

tio

n a

nd

Re

ferr

al

Inta

ke

as

ses

sme

nt

Le

arn

er

Pla

n C

rea

tio

n

Se

rvic

e P

lan

(E

OIS

-Ca

MS

)

Le

arn

ing

Ac

tivi

tie

s

Mil

es

ton

es

Cu

lmin

ati

ng

Ta

sks

Ex

it &

Fo

llo

w-U

p

Re

ferr

al

In

Su

ita

bil

ity

Re

ferr

ed

Ou

t

Co

mp

leti

on

s

public

1 INTAKE

5 EXIT and

FOLLOW-UP

4 TRAINING

3 LEARNER

PLAN

2 ASSESS

Goal Paths Activities

38

Report 61 SORTED by: 1. Program Status; 2. Program Duration

39

Report 61 SORTED by:Program Status

40

Report 61 SORTED by:1. Program Status; 2. Referred In

41

Report 61 SORTED by:1. Program Status; 2. Goal Path

42

YTD All Participants Actuals

Report 64 DSQR

Calc 1 - Excluding ‘3. Suitability’ and ‘5. Learners Served’:

YTD All Participants Num / YTD All Participants Den Calc 2 - ‘3. Suitability’:

YTD All Participants Num / YTD All Participants Den / 10

Calc 3 - ‘5. Learners Served’:

YTD All Participants Num / Perf Com Ann. Target (pro-rated YTD)

43

‘% YTD of Target’ Calculation

Report 64 DSQR

YTD All Participants Actual /

Performance Commitment Annual Target

44

Continuous Improvement (3)

45

Unit 4 Objectives Review

‘Reports Available’ sectionReport analysis:

Calculations; Report 61; exercisesContinuous improvement:

Accurate reports Understand results

Support of staff: Participant Guide; user guides, Data Dictionary

46

Summary

•Training Summary

•Evaluation

•Exercise

47

End