41
Utility Customer Experience Measurement – Closing the Loop April 12, 2016 Shaikat Sen, Senior Vice President, Blueocean Market Intelligence Larry Simpson, Research Principal, Pacific Gas and Electric Company © 2016 Blueocean Market Intelligence 1

Customer experience measurement in the utilities industry – closing the loop

Embed Size (px)

Citation preview

Utility Customer Experience Measurement – Closing the Loop April 12, 2016

Shaikat Sen, Senior Vice President, Blueocean Market Intelligence

Larry Simpson, Research Principal, Pacific Gas and Electric Company

© 2016 Blueocean Market Intelligence 1

Agenda

© 2016 Blueocean Market Intelligence 2

From Rating Scales to Rate Cases: The case for CEx Measurement in the Utility Industry

CEx 360 – A Holistic Approach

Identifying Touchpoints/Pain Points

Monitor/Track Performance

Segmentation/Modeling

Cadence

The Utility Perspective

Final Thoughts

From Rating Scales to Rate Cases: The Case for CEx Measurement in the Utility Industry

© 2016 Blueocean Market Intelligence 3

Higher Levels of Customer Satisfaction Positively Impact Credit Ratings, Operating Margins, and ROE

© 2016 Blueocean Market Intelligence 4

Higher Customer Satisfaction

In 2005, S&P clarified that their methodology for determining risk and credit ratings for utilities would include the customer satisfaction score, along with a host of other metrics

Credit Ratings

In 2011, J.D. Power found “regulated US Utilities that achieve the highest customer satisfaction in J.D. Power studies also reported the highest rates of net operating margin to FERC”

Operating Margins

J.D. Power and Associates also found that higher customer satisfaction scores were correlated with more favorable rate case decisions

Return on Equity

Expectations are Increasing, but Meeting Expectations Can Strengthen the Future

© 2016 Blueocean Market Intelligence 5

• Unprecedented impact of social media

- Customer satisfaction has risen to the top of consumer consciousness

• Public exposure to benchmarking studies (J.D. Power)

- Customers are seeing how other utilities are being perceived by their customers

• Customer satisfaction as a competitive differentiator

− Critical when seeking growth through the development of new unregulated products and services, or protecting the core business from disruptive entrants

• Better customer experiences

− Cornerstone for positive outcomes around co-created energy efficiency solutions

Utilities Ranked 9 Out of Top 14 Industries on Customer Satisfaction, Trust in Utilities is on the rise.

In a worldwide study conducted in 2014, Accenture found that trust in utilities had increased by 13% to 37% in 2014…up from 24% in 2013.

© 2016 Blueocean Market Intelligence 6

Expectations from Utility Companies (2014)

8.59

8.52

8.41

8.35

8.28

8.20

7.89

7.88

7.82

7.73

7.71

7.71

7.66

7.55

Customer Engagement | McKinsey on Marketing & Sales

1 Brokerage

2 Auto Ins

3 Retail

4 Bank

5 Hotel

6 Health Providers

7 Mobile Phone

8 Airlines

9 Utilities

10 Phone

11 Postal

12 Health Inc

13 Internet

14 Pay TV

Rank 2012

Industry CSAT

average score1

2011-2013 trend

1 Customer satisfaction was measured on a scale of 1-10; includes top 3 companies per industry per respondent. Consistency of experience operationally defined as standard deviation, that is, consistency of perception across surveyed demographic

Simple home improvement products and materials to

save energy

66%

Home energy audits/consultations to help save

energy

62%

Home energy generation products 58%

Devices or services to automate home energy

management

56%

Installation and/or maintenance services for home

energy devices

56%

Backup energy storage or power generator in case

energy goes out

53%

Warranty and/or financing plans for home energy

devices

52%

Source: The New Energy Consumer: Architecting for the Future, Accenture, 2014.

CEx 360 – A Holistic Approach

© 2016 Blueocean Market Intelligence 7

Components of an Effective 360 CEx Measurement Program

© 2016 Blueocean Market Intelligence 8

Insights

Information Sources to Tap Into

© 2016 Blueocean Market Intelligence 9

• Qualitative • Explore customer journeys • Identify touchpoints • Assess pain points

• Quantitative • Relationship surveys • Transactional surveys

• Blogs/forums/discussions

• Complaint/review web sites

• Facebook/Twitter/other sites

• Segmentation data

• J.D. Power results

• Other benchmarking studies

• Rate plan participation

• Bill payment history

• Energy usage

• Demographics

• Call center transcripts

• Customer emails

• Customer/field service agent updates

Mapping Information Sources to the 360 CEx Cycle

© 2016 Blueocean Market Intelligence 10

Implement Changes

Communicate with Customers

Segmentation/ Modeling

Transactional Data/ 3rd Party Data

Identify Touch/ Pain Points

Qualitative Research/ Unstructured

Feedback/Social Media

Monitor/Track Performance

Quantitative Surveys/ Benchmarking Surveys

Identifying Touchpoints / Pain Points

© 2016 Blueocean Market Intelligence 11

Harvesting Information…

© 2016 Blueocean Market Intelligence 12

Enumerate, identify and map touchpoints through customer focus groups and journey mapping sessions with internal stakeholders

Monitor relevant industry news through clipping services, RSS feeds, etc.

Systematically listen to and harvest information from as many customer information sources as possible

Recordings Transcripts Emails Field Notes

Adopt one or more social media monitoring tools to “scrape” data from a multitude of social media websites

Invest in text analytics solutions to analyze unstructured data streams

…And Harnessing It

© 2016 Blueocean Market Intelligence 13

Siphon all feedback to a dedicated “Listening Intelligence Team”

Code and monitor incoming information on an ongoing basis

Develop a standardized code frame to organize unstructured information across all data sources

Generate ongoing weekly/monthly reports highlighting top customer issues

Visualization Examples of Unstructured Data

© 2016 Blueocean Market Intelligence 14

Theme Comment

Overall Service/ Feedback (+)

• No problems/very few power outages/service is good/happy with service

• Good company/honest/give back to community

Overall Service/ Feedback (-)

• Frequent outages/Service is not reliable

• Service is average

• Not aware of any other services that are provided

• There is always room for improvement/no one is perfect

• Do not like company

• Heard negative things about company

• Not happy or dissatisfied/neutral

Customer Service/ Field Service(+)

• Customer service (friendly, helpful, knowledgeable, etc.)

• Respond quickly to service issues

Customer Service/ Field Service (-)

• Customer service (hard to reach reps, not helpful, etc.)

• Do not respond quickly to service issues

Billing/ Website mention (+)

• Accurate billing/timely

• Online billing available

• Easy to pay bills/easy to read bills

• Can see/compare energy usage

• Website mentions/easy to use

Billing/ Website mention (-)

• Billing issues

• Website issues (confusing, no online billing)

Rates/Discounts (+) • Lower rates/discounts

Monitoring/Tracking Performance

© 2016 Blueocean Market Intelligence 15

Value of Quantitative Measurement

© 2016 Blueocean Market Intelligence 16

Quantitative Assessment is the Backbone of a Strong Measurement Program

• Results are representative of your overall customer base

• Identifies specific areas where you are doing well and those needing attention

• Helps to prioritize your organizational focus to get the best ROI

• Perception is reality – you need to keep your finger on the pulse of your customers’ perceptions

Components of a 360 Program

• Transactional surveys

• Relationship surveys

• Competitive benchmarking (optional)

Part 2: Develop transactional surveys associated with each customer journey

Part 1: Understand the customer journey associated with each transactional touchpoint

Transactional Surveys – Best Practices

• Conduct customer journey focus groups

− Internal multi-functional stakeholder brainstorming sessions

− Customer focus groups

• Enumerate as many customer journeys that reflect how your customers interact with your organization

− New customer sign-up

− Tree trimming

− Emergency service response

− Other

• Prioritize customer journeys and develop a strategy and timeline to address each journey

• Keep the survey short

• Focused on the specific customer journey

• Include questions that tie back to high level customer satisfaction metrics

• Deploy after each transaction is complete

© 2016 Blueocean Market Intelligence 17

Relationship Survey Design – Best Practices

© 2016 Blueocean Market Intelligence 18

Characteristics of High Performing Utilities

Positive customer experience

High levels of customer engagement in products and services offered

Strong communication with customers

High levels of customer trust in the brand

Overall Experience

Overall Trust

Overall Communication

Engagement Levels

Ease of Doing Business

Overall Satisfaction

Likelihood to Recommend

Likelihood to Switch

Relevant Key Metrics to Consider

Experience, Trust, and Communications – Example Metrics

© 2016 Blueocean Market Intelligence 19

Billing

Payment options

Billing statement clarity

Billing statement accuracy

Meter accuracy

Pricing

Fair price

Education - rate plans

Timely change notifications

Custom rate plans

E-Efficiency

Education - saving energy

Energy efficiency incentives

Competence

Process excellence

Well-managed

Leader

Corporate Values

Keeps promises

Take responsibility

Open communications

Cares about employees

Customer First

Values customers

Extra mile

Ease of doing business

Understands needs

Customer Experience Customer Trust

Communications

Other Communications

Web site assessment

Specific marketing collaterals

Advertising recall

External News

Exposure to news

Favorability/un-favorability of news

Word of Mouth

Discussions with others

Positive/Negative opinions shared

Prioritizing Actions – Best Practices

• Carefully plan the structure and sequence of the questionnaire

• Determine where you want the key metrics to be:

© 2016 Blueocean Market Intelligence 20

• Keep attributes and related dimensions together, but randomize and rotate dimensions and attributes to prevent order bias

• Decide on an appropriate rating scale (5-pt., 7-pt., 10-pt.) and stay consistent

• Establish a theoretical model of how attributes might relate to each dimension, and how each dimension relates to the key outcome measure

• Responses sensitized by preceding assessments

• Statistical models tend to be stronger

• But, results may not reflect reality

…or at the end

At the beginning…

• Responses tend to reflect top-of-mind perceptions

• Results may be more closely aligned with real life

• However, statistical models tend to be weaker

Values customers

Hypothesizing Expected Relationships

© 2016 Blueocean Market Intelligence 21

Overall Satisfaction

Trust

Experience

Customer First

Environment

Competence

Billing

E-Efficiency

Reliability

Management

Leadership

Payment options

Statement clarity

Statement accuracy

Extra mile

Ease of doing business

Education on saving energy

Energy efficiency initiatives

Process

Driver Modeling – Best Practices

© 2016 Blueocean Market Intelligence 22

Standard regression doesn’t work well with attitude and perception data

New techniques are available today to get around these challenges

• Shapley Value Analysis or Dominance Analysis or Relative Weight Analysis

• Impractical to use before because of the enormous amount of computing power required

• Today’s computers are able to run the required hundreds of thousands of regressions in a matter of hours

• Traditional techniques can fail when predictor variables are correlated with one another or have skewed distributions

• Attitude and perception data are almost always correlated and skewed

Relative Importance of Drivers

© 2016 Blueocean Market Intelligence 23

Standard Regression Model Shapley/Dominance Analysis

R2= 60%

Var. Important %

Predictor 1 = 15%

Predictor 2 = 20%

Outcome - dependent var.

Predictor 1 , Predictor 2 - Independent var.

Predictor 1

Predictor 2

Outcome

40%

15% 25%

20%

Multicollinearity

Outcome - dependent var.

Predictor 1 , Predictor 2 - Independent var.

Predictor 1

Predictor 2

Outcome

40%

15%

20%

Multicollinearity

25%

21% 4%

R2 = 60%

Var. Important %

Predictor 1 = 36%

Predictor 2 = 24%

2%

7% 11%

21%

25%

34%

Field ServiceCustomerService

BillingE-EfficiencyReliabilityPricing

Relative Importance

10%

16% 16% 18%

23%

26%

CorporateValues

EnvironmentCompetenceCommunityCustomerFirst

Safety

Relative Importance

Overall Satisfaction

Customer Trust 40%

Customer Experience 60%

R2=80% R2=70%

© 2016 Blueocean Market Intelligence

Relative Importance of Dimensions

24

Customer Experience 60%

Pricing E-Efficiency Reliability Billing Customer Service

7% Field Service

11%

14%

16%

17%

19%

25%

Customer Service Representative

Problem resolution

Overall time

CSR Competence

CSR Communication

CSR Courteousness

Phone system navigation

Phone system navigation

Overall time

Problem resolution

R2 = 78% R2 = 80%

Live (70%) Automated (30%)

31%

30%

39%

Automated CustomerService

© 2016 Blueocean Market Intelligence

Relative Importance of Attributes

25

Sampling – Best Practices

© 2016 Blueocean Market Intelligence 26

Target Audience

• Residential surveys – head of household/spouse/partner, etc.

• SMB surveys -

One account can have multiple sites; same company can have multiple accounts

Ideally, your population should be defined as an account and site combination that has a unique account contact

• Minimum recommended sample size for a segment is about 400

• However, you have to consider all relevant segments

• Recommended minimum sample size for each sub-segment is 100 – or more – if budget allows

Sample Size Sample Sizes and Error Margins

Margin of error (Precision)

Confidence Level

90% 95% 99%

3.0% 747 1067 1849

5.0% 269 384 666

10.0% 67 96 166

Data Collection Options

© 2016 Blueocean Market Intelligence 27

Advantages Disadvantages

Channels

Outbound Phone

Inbound Phone/ SMS Surveys

Email

Regular Mail

Web-based

App-based

• Expensive • Cellphone prevalence making it more and

more difficult and expensive • Refusal rates are increasing • Tends to skew towards older, women

• IVR programming required (inbound phone) • Need to automate survey offering to

prevent bias (inbound phone) • Surveys need to be very short

• Response rates are very low • Need to have email addresses

• More expensive than email • Very low response rates • Usually takes longer in field

• Requires internet awareness and may skew sample to a younger demographic

• Customer has to come to the web site to get an opportunity to participate

• Still in infancy, but gaining popularity • Low uptake for older generation

• Easy to complete • Easy to control • Relatively high response rate

• Immediate response – feedback is more accurate

• Generally good response rates • Inexpensive after initial set-up

• Relatively cheap and easy to set up • Cost effective to run continuously • 80-90% of responses are in within 72 hours

• Generally cheaper than telephone • Can capture good, detailed feedback

• Often used to ‘rate’ content • Useful for web site assessments • Typically used in conjunction with email

surveys

• Emerging approach • Cost effective to run continuously • “In”

Data Collection – Best Practices

© 2016 Blueocean Market Intelligence 28

Mixed/hybrid surveys (landline, cellphone, and email) may be optimal

– Acceptable for ongoing tracking programs

– Allows you to capture a more representative sample of your customers

– Happy medium in cost and ability to leverage existing customer information

Customer Experience

60

65

70

75

80

Q2 '14 Q3 '14 Q4 '14 Q1 '15 Q2 '15 Q3 '15 Q4 '15

Phone CONTROL Phone TEST Online TEST Combined TEST

• Ensure sample composition matches distribution of landline, cell phone, and email addresses available in the customer database.

Segmentation and Modeling

© 2016 Blueocean Market Intelligence 29

Leveraging Transactional/Syndicated Data to Profile, Understand and Predict Customer Behaviors

© 2016 Blueocean Market Intelligence 30

Utilities have an enormous amount of data on their customers

Allows you to associate satisfaction with factors like energy use, monthly bills, household composition, and so on

Matching back quantitative survey data to customer base can yield deeper insights about attitudes and behaviors

Attitude and perception data allows modeling of behaviors and segmentation to maximize produce development and communication opportunities

Developing a Customer Engagement Metric

© 2016 Blueocean Market Intelligence 31

(selects EE programs & renewable programs)

Management of energy usage

Adoption of new technology

Takes advantage of offers

Interactions with Utility

(signs up for MyAcct, eNotifications, surveys)

(selects TOD programs, specific eNotifications)

(campaign respondents, enrolls in BPCC programs)

“Dimension-level”

engagement score

Overall “Engagement

Score” - weighted average

of dimensional engagement

scores

50.30%

12.90% 17.60% 19.20%

Adopt NewTechnologies

Interaction withUtility

Manages energyusage

Takes Advantagesof Offer

Composite dimension weights

The Customer Engagement Continuum

© 2016 Blueocean Market Intelligence 32

Focus Areas

.00 13.90 41.70 69.51 97.31 125.11 152.91 180.70 208.52 236.32 264.12 291.93 319.73 347.53 375.33 403.14 27.80 55.60 83.41 111.21 139.01 166.81 194.62 222.42 250.22 278.02 305.83 333.63 361.43 389.23 417

Poorly engaged Moderately engaged Highly engaged Very Highly engaged

Poorly engaged

Moderately engaged

Highly engaged

Very highly engaged

Interaction with Utility

Managing Energy

Takes Advantages of offers

Adopt new technology

Low Medium High

2% 13%

26%

54%

5%

Very highlyengaged

customers

Highly Engagedcustomers

Moderatelyengaged

customers

Poorlyengaged

customers

Non-engagedcustomers

Segment Size

Type of Dwelling (Single Family)

Definite Home Owner

Household Age (>45 yrs.)

Months at Residence (>82 mths.)

House Square Footage (>1920 sqft)

Pool at home

Pool Area (>100 sqft)

Home Construction quality (good)

Customer tenure (10+ yrs.)

HH Income Range ($75K+)

Married (Husband & wife) with/without child

Number of People in HH (4+)

Demographics, Attitudes, Energy Use

© 2016 Blueocean Market Intelligence 33

Ho

me

92%

84%

71%

63%

49%

32%

32%

31%

69%

49%

48%

24%

Total

77%

64%

61%

41%

36%

18%

18%

18%

52%

32%

33%

17%

Ho

use

ho

ld

1%

1%

2%

3%

4%

4%

6%

7%

9%

10%

12%

19%

22% Green Investors

Payback Investors

Green Activists

Show-me Participants

First Costers

Hands-On Believers

Pragmatists

Unplugged

Creatures of Comfort

Budget Watchers

Living in the Now

Tech Frontiersmen

Tech to Live

22%

26%

34%

40%

62%

65% Affordability level

Green affinity

Info action orientation

Investment capacity

Technology propensity

Comfort consumption

13%

12%

9%

11%

17%

5%

4%

10%

4%

5%

6%

2%

5%

40%

52%

26%

23%

26%

15%

Ener

gy C

on

sum

er D

ynam

ics

Axc

iom

Var

iab

les

Total

1440.3 1159.5 274.1 $161.1

Total (avg.) OffPeak (avg.) OnPeak (avg.) Enrgy Bill (avg.)

kWh

kWh

1231 kWh 902 kWh 170 kWh $142 Total

Last 12 Month Energy Use and Bill

Total (avg) Off Peak (avg) On Peak (avg) Energy Bill (avg)

Engagement Segment Profiling

© 2016 Blueocean Market Intelligence 34

Very Highly Engaged Average Engagement Score: 318/1,000

Higher income, likely to spend more and invest more, stay in larger houses with pools. Innovators by nature and pro-active. Display high affinity for comfort consumption and environmental friendliness.

Size: 1.8%

Highly Engaged Average Engagement Score: 191/1,000

Likely to be home-owners who are financially stable and have high disposable incomes. Tend to be older customers. In general they also have an affinity for comfort consumption and environmental friendliness.

Size: 13%

Moderately Engaged Average Engagement Score: 110/1,000

Average income, middle aged families with children staying with more no. of people in HH. Have a pragmatic approach and high info/action orientation. Tend to be innovative, green friendly, and prefer comfort consumption.

Size: 26%

Poorly Engaged Average Engagement Score: 43/1,000

Tend to be newer customers with less energy consumption having poor credit histories, staying in smaller apartments. Mainly young single individuals, tech savvy and mobile, yet to be anchored in long term housing . Generally have lower household incomes.

Size: 54%

Not Engaged Average Engagement Score: 0/1,000

Single wanderers – prefer living in apartments/town houses and usually not for more than 2 years. Financially cautious with a lower propensity for technology and green friendly initiatives. Very low on comfort consumption.

Size: 5%

Rules to Predict High Engagement Levels

© 2016 Blueocean Market Intelligence 35

Cadence

© 2016 Blueocean Market Intelligence 36

Customer Experience Measurement - Cadence

© 2016 Blueocean Market Intelligence 37

Month

1 2 3 4 5 6 7 8 9 10 11 12

Ongoing

Unstructured (text) analysis

Transactional Surveys

Relationship Surveys

Ad Hoc Deep Dives/Drill Downs

Behavioral Connections/Modeling

Dip-stick Quantitative Surveys

Qualitative Deep Dives/Ethnographics

Competitive Benchmarking Survey

The Utility Perspective

© 2016 Blueocean Market Intelligence 38

Utility Perspective

© 2016 Blueocean Market Intelligence 39

Evolve research to changing environment while maintaining essential trending

Adapt to measure emerging customer needs and obtain feedback about major events

Complement syndicated research with proprietary surveys

Own survey content, manage changes to methodology, identify respondents for data appends and segmentation, and holistically measure gas and electric satisfaction

Partner with data science to set key goals and forecast performance

Process has grown from analyzing trend data and using management judgement to employing advanced statistical modeling and Monte Carlo simulation

Supplement perception research to add clarity

Deep dive into customer touch points using transactional surveys, qualitative research, and operational data

Utilize key research to inform the Customer Experience (CX) team

Collaborate to measure and analyze data, identify and prioritize friction points, work cross-functionally, and inform strategic decision making

Final Thoughts

© 2016 Blueocean Market Intelligence 40

Leverage existing assets (harness all available unstructured/internal data)

Identify a relevant and comprehensive set of attributes to measure

Ensure that the instrument is designed to allow for statistical analysis of key drivers of overall satisfaction

Create enthusiasm and accountability within your organization

Deploy transactional surveys to monitor customer journeys

Develop a tracking survey instrument that can remain stable over a period of one to three years

United States | United Kingdom | India | United Arab Emirates www.blueoceanmi.com

Thank You

Shaikat Sen

Senior Vice President

[email protected]

Larry Simpson

Research Principal

Pacific Gas and Electric Company

© 2016 Blueocean Market Intelligence 41