2/10/05 Slide 1 SBC Laboratories User Needs and User Profiling

2/10/05 Slide 1SBC Laboratories

User Needs and User ProfilingUser Needs and User Profiling


High PerformanceLow Performance

Target“Expert” Level

Target“Novice” Level

User PopulationUser Population


Any single view of the User is incomplete, whether it is the novice, average, or expert

» Each view only captures a limited number of users » The implemented design may actually hurt the performance of the other non-represented

users.

Representing all of the Users individually is not cost effective.

A reasonable number of views (3 to 5) has high leverage.

Small enough to be definable, but large enough for coverage.

“Single View”“Single View”


Basic Call + Up Selling = Total Revenue

Rep 1 : + =

Rep 2: + =

Behaviors - StrategiesBehaviors - Strategies


Modeling Approach:

» The first step is to categorize the service representatives’ behaviors into clusters.

» The second step is to objectively and quantitatively describe the service representatives’ behaviors and strategies.

» The third step is to create a model for each of those clustered behaviors.

» Simulate the models to determine impact.

User ModelsUser Models


0

5

10

15

20

0.0 0.5 1.0 1.5 2.0 2.5Average Cross-Selling Offers per Call

Ave

. Min

utes

per

Cal

l F

ew

Man

y

YellowOrange

Green

Blue

None Many

Purple

Agents, GroupedAgents, Grouped


High

Low

Few Many

Offers per Call

Monthly Sales

“Blue” “Yellow”

“Orange” “Purple”

Performance by GroupPerformance by Group


Blue Group» Very few cross sell attempts» Efficient call takers, shortest call duration» Most likely do only what the customer requests» High Revenue

Yellow Group» Moderate number of cross sell attempts» Moderate call duration» High Revenue

Qualitative User ModelsQualitative User Models


Large, diverse user-populations can be categorized into distinctive groups.

Top performers may use different strategies and those behaviors need to be understood.

Models reflect factors that are important in achieving business goals.

Models can be constructed of those behaviors.

Models should be integrated with interface design and operational improvement.

CommentsComments


Customer-Centric Interface DesignCustomer-Centric Interface Design


Customer-Centric approach takes the customer’s view or perspective in accomplishing tasks.

Frequently, organizations focus on the business, department, or process perspective. Organizations concentrate on the processes, people, and technologies that must interact to address the customer’s request and ultimately accomplish the customer’s task.

The goal is to have the Customer routed to the most appropriate location (agent) where the customer can on their own, or with our assistance, accomplish their desired task with the highest level of satisfaction at the least amount of cost.

Customer-CentricCustomer-Centric


Customer’s Contacts

How many? 1,000,000s

How many?10

“Orders”

“Billing”

“Want Caller ID”

“Don’t understand my bill”

“Need a 2nd line”

“Reconnect phone”

Customer’s Expressions

Task Completion

“Collections”

How many?100,000s

MENU

For Service Order Center, press 1.

For Billing Inquiry Center, press 2.

For Revenue Management Center, press 3.

Interface

?

Customer has Mapping BurdenCustomer has Mapping Burden


Task Categories


“Order Service”

“Get Bill Information”

“Reconnect Services”


How many?100

How many?10

“Orders”

“Billing”






Task Completion

“Collections”

How many?100,000s

MENU

To order service, such as caller id or a second line, press 1.

To get billing information, press 2.

To reconnect your services, press 3.

Interface

System has Mapping BurdenSystem has Mapping Burden


Action- Object


“Acquire-Optional Service”

“Inquire-Bill”

“Reconnect- Basic”


How many? 100 How many? 20

“Orders”

“Billing”






Task Completion

“Collections”

How many?100,000s

RoutingTable

Customer has Mapping BurdenCustomer has Mapping Burden


Customer-Centric approach uses the customer task as the primary influence on interface design.

– Menu items would directly match tasks that customers are trying to accomplish.

– Menu items would be grouped and ordered by frequency of customer tasks.

– Menu items would be worded in the language of the customer.

Customer-Centric DesignCustomer-Centric Design


All interface technologies have limitations. A given interface technology (e.g. IVR systems) can achieve a certain level of customer satisfaction and a certain level of performance (e.g. accomplish their task). All interface technologies will have less than perfect customer satisfaction and performance.

Customer-Centric approach will specify a design that maximizes the performance of a given interface technology. In other words, Customer-Centric approach delivers the best interface possible for a given interface technology.

Performance RealitiesPerformance Realities


• Understand customer goals

• Design from user’s perspective

• Lab tests & field evaluations

• Reduce misdirects• Increase customer satisfaction

1. Customer Task Frequency Table

2. Customer Perceptions for Interface Structure

3. Interface Design

4. Customer Usability Test

5. Refine Design for Implementation

• Understand customer preferences

5 Step Design Approach5 Step Design Approach


Why do customers contact us ? Customers want to accomplish a task. What are those tasks ?

Gather data. Customer’s opening statement provides an excellent snapshot of

their (first, primary) task. Gather sufficient number of tasks to be statistically valid. (e.g.

2,500) Categorize tasks into Customer-Centric groupings. Tabulate results and construct Frequency Table.

Step 1: Customer TasksStep 1: Customer Tasks


REGION TYPE # RECORDS

Southwest Billing / Orders 14,227

West Billing / Orders 10,038

Midwest Billing / Orders 8,947

East Billing / Orders 5,949

Midwest Credit & Collections 4,239

Southwest Credit & Collections 9,224

West Credit & Collections 6,466

Total = 59,090

Customer Task DataCustomer Task Data


# of Words per Customer Statement

0%

2%

4%

6%

8%

10%

12%

14%

16%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Words per UtteranceWords per Utterance


Item Action-Object Percent Cumulative

1 Pay Bill 10.2% 10.2%2 AskAction-Billing 8.6% 18.9%3 Inquire-Bill_DM 7.4% 26.3%4 Acquire-Basic_DM 6.7% 33.0%5 Agent 6.5% 39.5%6 Inquire-Balance 5.0% 44.5%7 Acquire-Basic 4.8% 49.2%8 Fix-Basic 4.3% 53.5%9 Cancel-Basic_DM 3.7% 57.2%

10 Inquire-Bill 3.4% 60.6%11 Move-Basic 3.0% 63.6%12 AskAction-Service 2.8% 66.4%13 Cancel-Basic 2.2% 68.5%14 Fix-Basic_DM 2.1% 70.7%15 Acquire-OptionalService 1.8% 72.5%

Example Call Center

Customer Task Frequency TableCustomer Task Frequency Table


0%

2%4%

6%8%

10%12%

14%16%

18%

Customer Tasks

Fre

qu

ency

Task FrequenciesTask Frequencies


Task Mapping = pair of words that captures the caller’s task concisely and unambiguously.

Task Mapping = “Action” / “Object” format. Example Actions: pay, acquire, inquire. Example Objects: bill, dsl, long_distance.

“I would like to pay my bill” = Pay-Bill “Get new phone service to my home” = Acquire-Basic

Task Mapping is based upon a business model. shop, purchase, pay, maintain, terminate

Task MappingTask Mapping


Customer Statement Action-Object

“I want to get a new phone line.” Acquire-Basic

“I would like to pay my phone bill.” Pay-Bill

“Change my billing address.” Change-Bill

“I need to move my service to a new address.” Move-Basic

“I want to have a phone installed.” Acquire-Basic

“I need to have somebody discuss the bill with me.” Inquire-Bill

“I want to know my balance.” Inquire-Balance

Customer Tasks = Action-ObjectsCustomer Tasks = Action-Objects


Customer Task

1 I want to pay my bill.

2 We are moving to the new address.

3 I made appointment, but the technician did not show up.

4 I want to add CallNotes.

5 How much is my bill?

6 I want to change my long distance carrier.

7 What does Caller ID cost?

8 What do I owe on phone bill?

9 I deserve a credit on my bill.

10 I have a question on my bill.

Discussion: Action-ObjectsDiscussion: Action-Objects


Card Sort Method: Customer’s perception of how customer tasks should be

grouped into menu items. They are given the most frequent customer tasks and asked to categorize them into logical groups.

Procedure: Sort (numbered) task cards into groups based on the

similarity of tasks. Rank groups in order of importance.

The more tasks are linked, the more they should be in the same menu structure.

Step 2: Customer PerceptionsStep 2: Customer Perceptions


How often do the customers visit the interface?

Frequent daily use. Infrequent “walk up and use”.

Key behavior - do the customers remember the interface from one use to the next?

Frequency of UseFrequency of Use


1 2 3 4 5 6 7 8 9 10 11 121 94 10 9 8 5 8 4 8 13 29 30 62 10 94 25 32 59 7 63 22 27 5 13 173 9 25 94 25 22 5 19 44 21 4 13 314 8 32 25 94 26 14 28 7 50 7 5 75 5 59 22 26 94 7 67 32 26 3 17 256 8 7 5 14 7 94 7 8 6 5 14 117 4 63 19 28 67 7 94 26 25 5 15 228 8 22 44 7 32 8 26 94 10 5 22 519 13 27 21 50 26 6 25 10 94 7 11 10

10 29 5 4 7 3 5 5 5 7 94 14 511 30 13 13 5 17 14 15 22 11 14 94 1812 6 17 31 7 25 11 22 51 10 5 18 94

Customer Perception of StructureCustomer Perception of Structure


Customer Perceptions of StructureCustomer Perceptions of Structure


2. Add Optional Service

3. New Service/Open Account

4. How to Use

5. Disconnect Optional Service

7. Change Optional Service

8. Disconnect Services/Close Account

12. Move Service

22. Reconnect Service

24. Schedule a Move


1. Get Info about Bill

10. Schedule a Payment

13. Information on Account

14. Discuss Bill

15. Get Information on Payment

17. Where to Make a Payment

18. Give Information on Payment

19. Make a Payment




Customer-Centric approach: – Customer Task Frequency Table identifies what topics

should be included on the menu.

– Card Sort identifies how to group menu items and how customers would structure the interface.

– Customer Opening Statements identify how to word or what language to use in the menu items.

Step 3: DesignStep 3: Design


0%

2%4%

6%8%

10%12%

14%16%

18%

Customer Tasks

Fre

qu

ency

Customer-Centric Design

Business-Centric Design

Customer-Centric vs Business-CentricCustomer-Centric vs Business-Centric


Rank order of design styles (best to worst):

– Specific Action & Specific Object. (e.g. “order CallNotes”)

– “No” Action & Specific Object. (e.g. “for CallNotes”)

– Specific Action & General Object. (“order a service”)

– General. (e.g. “speak to someone”)

“Action-Object” in Design“Action-Object” in Design


Examples of Business-Centric menu items: For orders, press 1. For repair, press 2.

Examples of Customer-Centric menu items: To get information about your account, or our services

and prices, press 1. To get new service, move, change, or disconnect your

service, press 2. To report a problem with your service, press 3.

Customer-CentricCustomer-Centric


Business-CentricMenu

Customer-CentricMenu

# of Menu Options11 19

Pseudo-ProblemPseudo-Problem


5 10 15 201

Announcement

User Selection

Time in seconds

Ann. A

1

Ann. B

32 + 2 = 4

5 10 15 201

Announcement

User Selection

Time in seconds

Ann. A

9

Ann. A

32 + 10 = 12

5 10 15 201

Announcement

User Selection

Time in seconds

Ann. A

1- 3 + (-1) = -4

Ann. B

3

CRT Score

CRT Score

CRT Score

“Cut Through”“Cut Through”


-4

0

4

8

12

Customer-Centric Business-Centric

Cumulative Relative TimeCumulative Relative Time


Testing reduces risk. No matter how well the interface has worked in the past or on other applications, testing is a relatively quick inexpensive approach to reducing the risk.

We set up a controlled experiment to ensure that implementation surprises are minimized.

Approximately 100 customers (actually 96) perform tasks using the interface in order to achieve statistically valid results.

Normally, more than one design (usually four) is tested and evaluated.

Step 4: Usability TestingStep 4: Usability Testing


I I I I I IV1 B4 D2 A3 C12 A1 C3 B2 D43 D3 B1 C4 A24 C2 A4 D1 B3

5 A2 D4 C1 B36 C4 B2 A3 D17 B1 C3 D2 A48 D3 A1 B4 C2

9 D2 B1 A4 C310 A3 C4 D1 B211 B4 D3 C2 A112 C1 A2 B3 D4

13 A2 B4 D1 C314 B3 A1 C4 D215 C1 D3 B2 A416 D4 C2 A3 B1

17 A4 D1 B3 C218 D2 A3 C1 B419 C3 B2 D4 A120 B1 C4 A2 D3

21 A1 B4 C2 D322 D4 C1 B3 A223 B2 A3 D1 C424 C3 D2 A4 B1

Experimental DesignExperimental Design


Customer-Centric vs Business-Centric

0%

4%

8%

12%

16%

20%

Willing To Use Menu Relevant Menu Confidence in Menu

Pe

rce

nt

Imp

rove

me

nt

Customer SatisfactionCustomer Satisfaction


MARKETCUSTOMER

SATISFACTION CALL ROUTING

Consumer A 19% improvement 49% improvement

Small Business 27% improvement 37% improvement

Consumer B 22% improvement 26% improvement

Customer-Centric ResultsCustomer-Centric Results


Performance Matrix – Provides a view of two important performance indicators

(i.e. customer satisfaction and correct call routing). – Used to evaluate the performance of specific customer tasks.

High CustomerSatisfaction

Low CustomerSatisfaction

High Correct Call Routing

Low Correct Call Routing

Area of Good Design

Step 5: Refine DesignStep 5: Refine Design


Accomodated IVR Performance Matrix:PacBell Study

High Freq

Med Freq

Low Freq





(Business) Performance Matrix(Business) Performance Matrix


Customer-Centric IVR Performance Matrix:PacBell Study

High Freq

Med Freq

Low Freq





(Customer) Performance Matrix(Customer) Performance Matrix


Post Implementation DataPost Implementation Data


Assume reported data is wrong. Or, at minimum, the data is not what it appears. Lesson: always validate and verify data before performing analysis.

Key Step: Compare predicted to observed.

CommentsComments


No Example: Welcome to SBC. I’m here to help you reach the right place. First, Please tell me the purpose of your call and then press the pound key.

Example: Welcome to SBC. I’m here to help you reach the right place. First, Please tell me the purpose of your call and then press the pound key. You can say things like, “What’s my account balance?” or “I’d like to get some rates for long distance.” So, how can I help you?

Prompts with ExamplesPrompts with Examples


No Example Example

Account Balance 14% 86%

Long Distance Rates 16% 84%

Key WordsKey Words


Use Examples

Use Examples From High Frequency Tasks (or, from “problem” tasks)

Word Examples Carefully

CommentsComments


-3

-2

-1

0

1

2

3

0 1 2 3 4

Number of Times the Caller Had to Repeat His Response

Very Satisfied

Very Dissatisfied

Customer SatisfactionCustomer Satisfaction


“What words are customer words?”

Customer’s LanguageCustomer’s Language


RANK FOR

SPANISHCODE CUSTOMER TASK

SPANISH

FREQUENCY

ENGLISH

FREQUENCY

1 B4 Get information about a bill 26.3% 17.8%2 I4 Find out how to use a service 9.3% 6.7%3 P3 Schedule a payment 9.2% 3.9%4 B6 Discuss / dispute bill 8.8% 2.1%5 A1 Get new service / open an account 7.3% 9.0%6 F1 Report a problem with service 6.0% 6.2%7 D2 Disconnect optional service 5.9% 6.6%8 C2 Change optional service 4.9% 6.0%9 A4 Get information about an add 4.5% 0.4%10 D1 Disconnect service / close account 2.7% 5.3%11 M1 Move service 2.4% 3.4%12 A2 Add optional services 1.9% 9.3%13 I1 Get information on cost of services 1.9% 4.8%14 P1 Make a payment 1.9% 0.8%15 P5 Give information about a payment 1.6% 0.9%16 P4 Get information on payment 1.2% 1.7%17 C1 Change account information 1.1% 3.8%18 I2 Get information on account 0.9% 2.7%

English vs SpanishEnglish vs Spanish


0

5

10

15

20

25

30

0 5 10 15 20 25 30

Frequency by Task for Spanish

Frequency

by T

ask

for

Englis

h

"Task B4"

"Task P3"

"Task B6"

"Task A4"

"Task A2"

English vs SpanishEnglish vs Spanish


Word/ Phrase Language English Translation# of

Callers% of

CallersPoner Spanish, standard Put in, install...new service,

phone line, a phone18 32.7%

querer* Spanish, standard Want... 6 10.9%Ordenar Spanish, standard Order... 5 9.1%Regresar Spanish, standard Return, switch back to PB 4 7.3%Tener Spanish, standard Have, get... 4 7.3%Conectar Spanish, standard Connect... 3 5.5%Instalar Spanish, standard Install... 3 5.5%Pedir Spanish, standard Ask for, request... 3 5.5%Abrir Spanish, substandard

usageOpen... 2 3.6%

Volver Spanish, standard Return, switch back to PB 2 3.6%necesitar* Spanish, standard Need... 1 1.8%Establecer Spanish, standard Establish, set up... 1 1.8%Requerir Spanish, misusage Require (meant request) 1 1.8%Entrar Spanish, misusage Enter 1 1.8%Bajar Spanish, awkward,

ambiguous usageDrop a line to a new address 1 1.8%

Total 55 100.0%

“Order New Service”“Order New Service”


Word/ Phrase Language English Translation# of

Callers% of

Callersquitar Spanish, standard remove, drop, take away 20 33.4%cancelar Spanish, standard cancel 19 31.6%desconectar Spanish, standard disconnect 12 20.0%cortar Spanish, standard cut off 3 5.0%hacer unacancelación

Spanish, standard "make a cancellation,"cancel

2 3.3%

remover Spanish, misusage; falsecognate w/ remove -----

2 3.3%

retirar Spanish, standard withdraw, remove, takeaway

1 1.7%

cerrar Spanish, awkward close... a line 1 1.7%Total 60 100.0%

“Disconnect Service”“Disconnect Service”


Word/ Phrase LanguageEnglish

Translation# of

Callers% of

CallersBill English ---- 61 31.8%Cuenta Spanish, standard bill, account 51 26.6%cuánto + deber Spanish, standard how much + to owe 19 9.9%Cobrar Spanish, standard to charge 11 5.7%Balance Spanish, standard Balance 10 5.2%Factura Spanish, standard bill, invoice 9 4.7%Cargos Spanish, standard Charges 8 4.2%Cobro Spanish, awkward Collection 7 3.6%Recibo Spanish, standard receipt, bill 6 3.1%Saldo Spanish, standard Balance 4 2.2%Carta Spanish, awkward Letter 1 .5%estado de cuentas Spanish, standard Account statement 1 .5%estatement de cuenta "Spanglish" Account statement 1 .5%lo del teléfono Spanish, awkward the phone thing 1 .5%Monto Spanish, standard Amount 1 .5%Reporte Spanish, standard Report 1 .5%

Total 192 100.0%

“Bill Amount”“Bill Amount”


Customer’s choice of words is important and should not be assumed.

Solution: Customer-Centric (data driven) approach.

Comments Comments


Enterprise A-O Model

0%

20%

40%

60%

80%

100%

Inq

uire

-

Acq

uir

e-

Ca

nce

l-

Ca

nce

l-

Acq

uir

e-

Bill

ing

Bill

ing

Op

-Sv

s

LD G

rp 2

LD G

rp 6

Acq

Grp

3

Acq

Grp

7

Inte

rnet

Rat

eP

lan

Acc

t G

rp 1

Cin

gu

lar

Enterprise Action-Object ModelEnterprise Action-Object Model


Unique, High Frequency Action-Object

- Acquire-Basic

- Acquire-Basic_DM

- Acquire-DSL

- Acquire-LD

- Acquire-OptionalService

- Cancel-Basic

- Cancel-Basic_DM

- Change-OptionalService_DM

- Fix-Basic

- Fix-Basic_DM

- HTU-OptionalService

- … …

Count = 20 covering 68% of the A-O customer calls.



Action-Objects in alphabetical order (equal to or greater than ~1% frequency) West region Southwest region Midwest region Action-Object Action-Object Action-Object 1 Acquire-Basic Acquire-Basic Acquire-Basic 2 Acquire-Basic_DM Acquire-Basic_DM Acquire-Basic_DM 3 Acquire-DSL Acquire-DSL Acquire-DSL 4 Acquire-LD Acquire-LD Acquire-LD 5 Acquire-OptionalService Acquire-OptionalService Acquire-OptionalService 6 AskAction-Bill AskAction-Bill 7 AskAction-Billing AskAction-Billing AskAction-Billing 8 AskAction-LD AskAction-LD AskAction-LD 9 AskAction-OptionalService AskAction-OptionalService AskAction-OptionalService

10 AskAction-Service AskAction-Service AskAction-Service 11 Cancel-Basic Cancel-Basic Cancel-Basic 12 Cancel-Basic_DM Cancel-Basic_DM Cancel-Basic_DM 13 Change-Bill Change-Bill Change-Bill 14 Change-Listing Change-Listing Change-Listing 15 Change-OptionalService_DM Change-OptionalService_DM Change-OptionalService_DM 16 Check-AcquisitionStatus Check-AcquisitionStatus Check-AcquisitionStatus 17 Fix-Basic Fix-Basic Fix-Basic 18 Fix-Basic_DM Fix-Basic_DM Fix-Basic_DM 19 HTU-OptionalService HTU-OptionalService HTU-OptionalService 20 Inquire-Account Inquire-Account Inquire-Account 21 Inquire-Account_DM Inquire-Account_DM Inquire-Account_DM 22 Inquire-Balance Inquire-Balance Inquire-Balance 23 Inquire-Bill Inquire-Bill Inquire-Bill 24 Inquire-Bill_DM Inquire-Bill_DM Inquire-Bill_DM 25 Inquire-LD_DM Inquire-LD_DM 26 Move-Basic Move-Basic Move-Basic 27 Pay-Bill Pay-Bill Pay-Bill 28 Reconnect-Basic (#35 0.5%) Reconnect-Basic Reconnect-Basic 29 Reconnect-Basic_DM Reconnect-Basic_DM Reconnect-Basic_DM 30 Schedule-Payment Schedule-Payment Schedule-Payment

Count >= 1%: ~29 Count >= 1%: ~29 Count >= 1%: ~30 82% – 85% 82% – 85% 82% – 85% Acquire-RatePlan (#21 1.1%)



Enterprise & Midwest Action-Object Model

0%

20%

40%

60%

80%

100%

Inqu

ire-

Acq

uire

-

Can

cel-

Can

cel-

Acq

uire

-

Bill

ing

Bill

ing

Op-

Svs

LD G

rp 2

LD G

rp 6

Acq

Grp

3

Acq

Grp

7

Inte

rnet

Rat

ePla

n

Acc

t G

rp 1

Cin

gula

r



Customer Channel Preferences

0%

20%

40%

60%

80%

100%

Custo

mer

Pre

fere

nces

Agent Self-Service

Willingness of Self-ServiceWillingness of Self-Service


10/566,437System and Method for an Automated Collection of Data for Grammar Creation

10/188,152 System and Method for Automated Categorization of Statements

10/188,152Method and System for Automating the Creation of Customer-Centric Interfaces

10/263552Dynamic and Adaptable System and Method for Selecting A User Interface Dialogue Model

Intellectual PropertyIntellectual Property


Questions?Questions?

Documents

2/10/05 Slide 1 SBC Laboratories User Needs and User Profiling