14
Switched-on research summary 5 th June 2009

Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Embed Size (px)

Citation preview

Page 1: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Switched-on research summary

5th June 2009

Page 2: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Contents

• Summary of activity to date• Research findings and recommendations• Call centre activity update

Page 3: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Summary of activity• From November 2008 through to May 2009 approx 75K lines across

13 areas in South Wales have been migrated• During these migrations we have tested the distribution method and

the impact each has on the levels of customer recall • Research was also used to evaluate customers’ understanding of

the communications themselves. For instance;– Was it clear what was going to happen and what they had to do– Did they understand what was going to happen– Reassured about the process

• To follow are the research findings and recommendations for the next VPOTS migrations due to commence January 2010.

Page 4: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Research findings

Page 5: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Recap:Research Objectives

• BT intends to upgrade to 21CN without the affected end users noticing any disruption to their service.

• The overall research objective is the measurement of recall of the switched-on communications programme.

• The research was therefore designed to specifically measure:– Level of customer recall of the leaflets– Customer perceptions of leaflet (level of information, usefulness)

• Ferndale and Tonyrefail, and later, Roath D&E – were used to measure the comparative effectiveness of Door Drop and Direct Mail.

Page 6: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Recap:Research Methodology

• Earlier migrations were researched by either telephone interview or, later, face-to-face interview. To ensure accuracy, the sample size was kept to at least 5% of the migrated population where possible.

• Face to face research followed the following procedure:– 500 five-minute face-to-face interviews, divided equally between

door drop and direct mail recipients– Respondent contact details provided by switched-on– The interviewees represented every end user available in Roath,

excluding SMEs

• These results were then collated and analysed for recall, clarity and any other comments.

Page 7: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

DM & PR Direct Mail Only DM/Door Drop

Results Recap

• The core metric of this study is recall

• Prior to the Roath migration our current insights were:

• PR & DM in Wick & Bedlinog generated highest recall. However, the intensity of the PR was very high in this case in a small area.

• A comparison between DM in Nelson/Penarth, and door drop in Ferndale/Tonyrefail suggested decrease of approximately 25% recall

Page 8: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Roath D&E- recall• The 50/50 split in Roath

revealed Door Drop recall of 25%, and DM recall of 22%. With sample sizes of 250 per method, this is not a statistically significant difference.

• When the two methods are averaged, Roath’s recall level is the lowest to date at 23%.

• The results between DM and DD as the most cost effective distribution method is inconclusive

Page 9: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Roath D&E- clarity

• The scores for helpfulness and clarity of the communication by those who recalled it has remained high in all migrations

• However misunderstandings about impact on broadband, BT not being their provider so discarding the communication continue

• There has also been a downward trend in actual complaints

0%10%20%30%40%50%60%70%80%90%

100%

Wick &Bedlinog

Nelson Penarth Ferndale&

Tonyrefail

Roath

Helpful

Understandable

Page 10: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Conclusions & Recommendations – Media

PR• The highest level of recall was achieved using PR and DM.

However these small communities Wick and Bedlinog were saturated and was a one off scenario

Post vs Door drop• Initial tests in Nelson and Penarth (DM) vs Ferndale and

Tonyrefail suggested that DM delivered 25% higher recall• However the Roath 50/50 split test suggests that this conclusion

still needs more robust evaluation• We propose that VPOTS migrations are used to evaluate this

furtherLow cost/no cost media• Test during VPOTS to see if low cost/no cost media can cost

effectively act as a fillip to recall levels.

Page 11: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Conclusions & Recommendations – Communications:

• Some customers that have a CP other than BT are dismissing the leaflet.– We recommend re-evaluating the comms to make it even more

explicit that it affects non-BT service providers, too.• Many people think that this migration will improve their

broadband experience.– We recommend further clarity on the impact of this migration on

broadband and what customers should do to find out more.• As helpfulness and clarity scores remain high, it is the

information that is left out which is causing the issues described above– As long as any new information included is given the same

measure of clarity, the above recommendations will be met with success.

• Further analysis of the ‘verbatim’ will be carried out in June.

Page 12: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Summary of Recommendations

• Update copy and design to reflect some of the issues raised via the verbatims received during the research– CP services running over a BT line– Broadband experience

• As results are inconclusive on the Roath D&E test we propose to continue to evaluate door drop and DM media distribution in greater volumes during the January and February VPOTS migrations

• Test no cost and low cost media to see if recall can be cost effectively increased

• We will then make final recommendations to the Consult 21CN communications working group on distribution method at the end of VPOTS.

Page 13: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Call centre activity update

Page 14: Switched-on research summary 5 th June 2009. Contents Summary of activity to date Research findings and recommendations Call centre activity update

Analysis of activity

• Pathfinder 1 is drawing to a close. We have a period of consolidation before VPOTS in January

• Recommendations for updating the communications assets will be actioned during the summer

• Good opportunity to analyse the call centre activity– Currently it is open Mon- Fri 8-8pm , Sat & Sun 9-5pm, IVR and live

operator• Top line findings

– 80% of all calls are received between 8am and 5pm– 3% of calls are received at the weekend– Approx 65% of calls are managed through the IVR facility– Between 1-2% of migrated customers contact the call centre

• Based on these findings we will be making recommendations on how the call centre can better reflect customer behaviour.