Institute for Citizen-Centred Service COMMON MEASUREMENTS TOOL

IPAA NSW 2012 State ConferenceAugust 9, 2012

Presenter:Nicholas Prychodko – Director of Research and International Relations, ICCS

0

Overview of the Common Measurements Tool (CMT)

1

What does the ICCS do?What does the ICCS do?

The mission of the ICCS is to support public-sector organizations in achieving high levels of citizen and business satisfaction with public services.

The ICCS lines of business include:

pub c se ces

• Managing inter-governmental research agenda (Citizens First and Taking Care of Business)

• Supporting and developing the Common Measurements Tool (CMT)Managing the Certified Service Manager (CSM)• Managing the Certified Service Manager (CSM) Program

2

What is the CMT?

The CMT is designed to collect feedback relating to service experience across all channels ofexperience across all channels of service delivery (telephone, in-person, web, mail, email, and kiosk).

The CMT was created for and has been used in both external and internal client surveys at all levels ofinternal client surveys at all levels of government. Approximately twenty percent of data in the ICCS Benchmarking Database comes fromBenchmarking Database comes from internal client surveys.

3

CMT: A Foundation in Research

DRIVER CMT QUESTIONDRIVER CMT QUESTIONExtra Mile Staff went the extra mile to make sure I got what I

needed. How much do you agree with the statement?

Knowledge Staff were knowledgeable and competent. How much do you agree with the statement?

Fairness I was treated fairly. How much do you agree with

Five drivers account for nearly 80% of variance in client

the statement?

Outcome In the end, did you get what you needed?

Ti li Overall how satisfied were you with the amount

4

variance in client satisfaction

Timeliness Overall, how satisfied were you with the amount of time it took to get the service?

Elements of the CMTElements of the CMT

CMT Question Bank• Contains a set of core questions based on the key drivers ofContains a set of core questions based on the key drivers of

satisfaction• Has nearly 150 additional questions designed to meet users’ specific

requirements and organized according to headings based on service dimensionsdimensions

• Allows for customization and integration with pre-existing surveys

CMT User ManualCMT User Manual• Supports the use of the CMT• Provides step-by-step guidance to new users and suggests

additional analytical options to more advanced clients• Includes sample surveys

5

CMT BenchmarkingCMT Benchmarking

A central database for storing CMT d t h b b ilt ll iCMT data has been built, allowing organizations to anonymously compare results against peer organizations.

Benefits include:The CMT can be used as a stand-alone client satisfaction

Benefits include:• Comparative benchmarking• Identification of best

survey tool, or enhanced value can be leveraged through benchmarking

Identification of best practices

benchmarking.

6

CMT BenchmarkingAll tiers of government are represented, guaranteeing meaningful peer comparison for public sector organizations

CMT Benchmarking

for public sector organizations. Federal Government

25%Local

Government

Provincial/ State

Government26%

Government49%

Out of the over 400 survey units in the ICCS database, several distinct ‘analytical lenses’ have been developed to facilitate ‘apples-to-apples’ cohort comparison. Also, to ensure robustness only independent survey units are permitted in the lenses

26%

to ensure robustness, only independent survey units are permitted in the lenses.

Existing Comparative Lenses• Overall

Emerging Service Area Lenses• Regulatory / Compulsory

• International• Single-Window

S i l S i

g y p y• Relational• Internal

I di

7

• Social Services• Financial Services & Payments

• Indirect• Client-facing

Leveraging Best PracticesLeveraging Best PracticesIn line with the ICCS’ stated goal of becoming a repository of information about best practices the CMT Benchmarking Reports offer examples ofabout best practices, the CMT Benchmarking Reports offer examples of actions taken by different organizations in connection with specific drivers of satisfaction.

WHAT LEADERS ARE DOING . . . • Veterans Affairs Canada has shifted from a program-centred to a client-centred

approach by first identifying the needs of individual clients and then enabling easy access to the right services by coordinating departmental and community resources. Learn more… Th G t f O t i i d ibilit b ki lti l i il bl• The Government of Ontario increased accessibility by making multiple services available over multiple channels. Many of them have extended hours of service. Learn more…

• The Region of Peel’s Contact Centre has improved accessibility by ensuring that its customers do not experience busy signals, extended periods on hold or inadequate automated systems. Learn more…

Each example is linked to online content providing further illustration of best practices

illustration of best practices.

8

ICCS RoleICCS Role

As the custodian of the CMT, the ICCS is mandated to promote, license and assist with the use of the instrument across the public sector in Canada and internationally.y

To fulfill this mandate, the ICCS offers tools, direct consultative services and publicly validated standards of practice for:services, and publicly-validated standards of practice for:

• Survey Design• Aligning Measurements with Strategic Initiatives• Analysis of Results• Helping an organization build internal survey standards and

capabilities• Identifying and applying best practices• Nurturing outreach and collaborative communities of practice

9

Cross-Jurisdictional Collaboration

Building a Community of Practice

The ICCS is increasingly embracing theThe ICCS is increasingly embracing the move beyond simple comparative benchmarking, towards a culture of

ll b ti k l d h icollaborative knowledge-sharing.

Alongside many Canadian jurisdictions,Alongside many Canadian jurisdictions, an increasing number of governments around the world are looking to participate in a collaborative framework of this typein a collaborative framework of this type. This includes governments from Australia, New Zealand, the UAE, the UK, and the US

10

US.

S CMT USome CMT Users

Examples of jurisdictions and organizations using the CMT and, in most cases, the ICCS Benchmarking Service include:

City of OttawaAccess HaltonP i f B iti h C l biProvince of British ColumbiaProvince of ManitobaSaskatchewan Workers’ Compensation BoardpGovernment of New ZealandRoyal Canadian Mounted PoliceVeterans Affairs CanadaVeterans Affairs CanadaProvince of OntarioDepartment of Human Services AustraliaGovernment of Abu Dhabi and municipalities in the UAE

11

Ontario Public Service Case Study

12

Ontario Public Service (OPS) – Key FactsOntario Public Service (OPS) Key Facts

68 000 bli t• 68,000 public servants• 86% of workforce

unionizedunionized• 28 ministries/central

agenciesagencies• 1,800 locations • Serving 12 7 million• Serving 12.7 million

Ontarians • Annual Budget - C$96 2Annual Budget C$96.2

billion

13

The OPS Transformational JourneyThe OPS Transformational Journey

YEAR VII The emergence of an integrated organization fostering Knowledge

Working Together: An Integrated YEAR VI2000 01

YEAR VII2001-02

organization, fostering Knowledge Management, Pride & Recognition, Renewal & Retention

Learning Organizations - Quality Improvements - Adapting to Change

Organization, E-Gov’t, Service & Policy Clusters YEAR V

1999-2000

2000-01

Implementing Quality Service: Citizens First, Common Standards, enterprise wide supports.

Improvements Adapting to ChangeYEAR IV1998-99

YEAR IIIRedefining the OPS Service & New Delivery Mechanisms

Build the vision of future OPS &YEAR II1996 9

YEAR III1997-98

Aggressively address fiscal challenges

Build the vision of future OPS & initiate transformation

YEAR I1995-96

1996-97

FOCUS ON CORE SERVICE

OPS VISION NEW SERVICE DELIVERY

QUALITY SERVICE MEASUREMENT & ACCOUNTABILITY

LEARNING ORGANIZATION

WORKING TOGETHER

INTEGRATEDORGANIZATIONS

14

Adopting a Citizen Centred Approach

M i ti

Adopting a Citizen-Centred Approach

Many organizations are preoccupied with inside-out management. They approach, understand and act in relation to their

Government Throne Speech:“Today Mr. Speaker, I am understand and act in relation to their

environment in terms that make sense from internal divisions and perspectives…Other organizations try t b ild f th “ t id i ” th

oday Spea e , apleased to announce that the government will measure its own performance, including

to build from the “outside in” - they use the views and needs of customers and other key stakeholders as a mirror in which they see and understand their

p gthat of its senior management, based on service delivery and y

own strengths and weaknesses, and they act on these insights to shape their relations with the environment. The Honourable David Tsubouchi

customer satisfaction.”

Gareth MorganImaginization: The Art of Creative Management Professor of Organizational

Chair, Management Board of CabinetApril 23, 2001

Professor of Organizational Behaviour/Industrial Relations

15

OPS Quality Model

ProcessesD l t f

PartnersPeople• Development of

Processes • Monitoring and

improvement of P

• Focus on the Customer

• Selection & Partnering

• Human Resource Planning• A Learning Organization• Participation in the Organization

Processes

Planning• Setting Policy & Strategy

Customers• Service Delivery• Measurement of• Setting Policy & Strategy

• Dev’l & Implementation of Operational Plans

• Measurement & Analysis of Performance

Measurement of Customer & Citizen Satisfaction

Leadership

of PerformanceResults• Customer & Citizen• Operational• Employee• Culture of Excellence

• Strategic Direction• Leadership Involvement

• Employee• Partners, Delivery Agents &

Suppliers• Financial

Leadership

16

The Common Measurements ToolThe Common Measurements Tool

The CMT is the OPS standard for customer surveying:surveying:

• Includes “core” questions that reflect the key drivers of customer satisfaction

• Provides a consistent set of response scales and over 150 survey questions

• Enables comparison of results with like-organizations, apply external benchmarks, and share best practicesD t b d i i t d b th ICCS t d l• Database administered by the ICCS; reports and analyses available upon request

• Can measure satisfaction with any service delivered by, and can be administered through any channel (in-personcan be administered through any channel (in person, telephone, mail and electronic)

17

Introducing the CMT in the OPSIntroducing the CMT in the OPS

AdoptionAdoptionIn 1998, the OPS adopted the CMT as its standard for customer surveying to:

Support and promote its citizen/client-centric service philosophyEnsure a rigorous and empirically-based methodologyEnsure a rigorous and empirically based methodology

Preparation1998-2000 the OPS researched and piloted the introduction of the CMT

15 ministries participated in a review of its introduction (researchers economists 15 ministries participated in a review of its introduction (researchers, economists, statisticians, HR, service delivery managers, business plannersPiloted in 4 ministries

LaunchLaunch2000-2004 the OPS centrally mandated and monitored the adoption of the CMT

All ministries and agencies required to submit bi-annual client survey plansAll i d i i d l t i t ti i All surveys required prior review and approval to ensure appropriate questionnaire design & survey methodology167 surveys planned for 2002-2004 period

18

Corporate Support for CMT Surveying

• Quality Service LeadsQuality Service Leads• Delegated representatives from all

ministries (CAO offices)

• Client surveying resources– Manual– Training coursesTraining courses– Web site– Support office

19

Institute for Citizen-Centred ServiceInstitute for Citizen-Centred ServiceThe OPS has looked to the ICCS for…• Central database for storing CMT data from the

OPS and other jurisdictions• Ability to anonymously compare results among

peer organizations• Allow OPS Ministries to apply external

benchmarking and share best practices

www.iccs-isac.org20

Current CMT Status in the OPSCurrent CMT Status in the OPSEmbedded in the culture of the organizationEmbedded in the culture of the organization

2005 onwards - principle of “modern controllership appliedMinistries responsible and accountable for implementation of client surveying using CMTCentral support in form of advice and guidance rather than control and approvals

Developed customized OPS “Client Survey Tool”2008 - OPS Service Directive adopted by OPSp y

All ministries and agencies required to establish program-specific service standardsRequirement to measure and report on achievement of service standards to clientsAnnual OPS-wide public report on service standards initiated in 2012Component added to CMT to accommodate measurement of service standards “CST”Component added to CMT to accommodate measurement of service standards – CST

Current corporate frameworkDelegated Service Excellence Community of PracticeDelegated Service Excellence Community of PracticeSupported by Centre for Employee Engagement & Client SatisfactionRange of tools, templates and trainingPartnership with Open for Business and other corporate stakeholders

21

OPS Service Quality Index ResultsOPS Service Quality Index Results

Ontario’s Service Quality Index has improved significantlyOntario’s Service Quality Index has improved significantly since 1998

Ontario Service Excellence Index*

6972

70

80

626261

60

70Performance

Index(0-100)

501998 2000 2003 2005 2008

Year

*The index tracks Ontario citizens’ perceptions of the quality of a basket of OPS services based on the series of Citizens First studies.services based on the series of Citizens First studies.

22

The CMT & Performance Measurement DashboardOrganizational Excellence

Key Performance Indicator (KPI) for 

Organizational Excellence Framework Component

( )the Component

Status of KPI

The PM Dashboard reports Key Performance Indicators reflecting the components of the OPS OE Framework - including CSat based on CMT measures

23

Drill-down of the CMT-derived CSat KPIMore detailed CSat measures using the CST are provided through “drill-down” views

P ibl i f th Cli t S ti f ti C t D hb dPossible views for the Client Satisfaction Component Dashboard:1. Longitudinal approach 2. Client satisfaction driver approach3. Organizational level approach 4. Client segmentation approach5. Service area approach 6. Channel segmentation approach5. Service area approach 6. Channel segmentation approach7. Focussed corporate priorities approach

24

Ontario’s Keys to Success• Organizational commitment to Client

Satisfaction and a citizen-centric cultureSatisfaction and a citizen centric culture• Support of senior leadership as part of a

corporate strategy• Commitment to performance measurement

as a way of doing business and promoting organizational excellenceg

• Allocation of central resources to lead and support CSat surveying as part of a planned approachapproach

• Assignment of delegated “champions” in program areas or organizational units

• Maintenance of CMT protocols and standards

• Active links with the ICCS and communities• Active links with the ICCS and communities of practice

25

How to contact us…

Website: www.iccs-isac.org

General Inquiries: info@iccs-isac.org

Common Measurements Tool: michal.dziong@iccs-isac.orgor 416-212-4795

THANK YOU!THANK YOU!

26