View
216
Download
1
Embed Size (px)
Citation preview
Website Evaluation in the Tourism IndustryEvaluating Quality of Customer Service in Airline B2C Systems
Prof. Niv Ahituv, Gil Amoray
' כנס , ן נטוויז מכון הגלובלי בכפר ואינטרנט , 013תיירות האינטרנט 8.6.09לחקר
Based on a Research performed for M.Sc. Thesis in Management Sciences - Technology and Information Systems, 2006
2
Research Introduction, Objectives and Scope
Assumptions and Hypotheses
Methodology
Results and Best Practices
Agenda
3
WWW introduced new method of communication:
Research Introduction, Objectives and Scope
In-Store Service/Sales
Phone Service/Sales
Web-Based Service/Sales
Companies and non-profit organizations have recognized the
benefits and values of online communication channels
Reducing operating costs
Increasing sales opportunities
Improving customer service by:
Reducing wait times and expanding service hours (24/7/365)
Enabling self-sufficiency of customers
Research Introduction
4
Define criteria for the evaluation and measurement of the
Quality of Service (QoS) in airline B2C systems
Demonstrate the applicability of those criteria through an
empirical study
Substantiate applicability of evaluation tool (AWET-- Airline
Website Evaluation Tool) in other industries
Research Objective
Research Introduction, Objectives and Scope
5
Website Evaluation
Highlights of Literature Review
Highlights of Literature Review
Interface & Design
18 evaluation criteria:
• Graphic design and multimedia
• Style & text
• Flexibility
20 evaluation criteria
• Logical structure
• Ease of use
• Search engine
• Help functions
• Navigation necessities
Navigation
15 evaluation criteria
• Prod/Service-related content
• Company info
• Information quality
• Interactivity
Reliability
28 evaluation criteria:
• Customer profile
• Order process
• After-sale support
• Customer service
Technical Aspects
12 evaluation criteria
• Speed
• Security
• Software and database
Content
6
Assumptions: tested using non-statistical sample
A1 – Streamlined process and improved website design are the
main reasons for users to favor a website
User group had to identify dominant criteria in favoring websites:
Website design, Process, Content, Technical Aspects, Other
A2 – Users do not consider technical aspects to be a significant
factor in the quality of service provided by airline websites
Technical aspects were not mentioned as a contributor for an award-winning
website in previous studies
Research Assumptions
Assumptions and Hypotheses
7
Hypotheses: tested by evaluating 33 airline websites
H1 – Web-based QoS level in the airline sector is higher in N.A
compared to Europe and Israel
Percentage of online sales: 40% in N.A vs. 14% in Europe and lower rate in IL
Purchasing patterns are expected to drive higher standards of service
QoS incurs costs, therefore investment decisions seek value and ROI
H2 – Web-based QoS of competitors working in the same region and
industry will be similar
High competition is expected to result in resemblance of websites w/i industry
H3 – QoS in websites of “low-fare” airlines is superior comparing to
QoS of “Major” Airlines’ websites
Direct sales (e.g.: online) are expected to impel outstanding service levels
Research Hypotheses
Assumptions and Hypotheses
8
Hypotheses Analysis
Results & Best Practices
H1 – Web-based QoS level in the airline sector is higher in N.A
compared to Europe and Israel
Validated: Average total score of 67% vs. 64% vs. 59% respectively
H2 – Web-based QoS of competitors working in the same
region and industry will be similar
Could not be categorically validated or refuted: airlines differ by category
H3 – QoS in websites of “low-fare” airlines in superior
comparing to QoS of “Major” Airlines’ websites
Rejected: Average total score of 67% for major airlines vs. 63% for low-fare
Possible explanation: better capabilities to make significant investment by
major airlines
X
9
Innovative multi-phased approach, embracing various techniques:
Methodology
Methodology
Phase 1 Phase 2 Phase 3
Implementation• Literature review
• Identification of evaluation criteria
• Identification of participating websites
• Preliminary questionnaire
• Industry characteristic mapping
• Development of AWET, v1
• SME review and feedback
• Development of AWET, v2
Phase 4
Analysis• Self evaluation of
33 websites• Results analysis
• Hypotheses validation
Evaluation Tool DevelopmentPreparations
10
Interface and Design
Navigation
Content
Technical
Reservation Process
Usability
Customer Support
Evaluation Categories in AWET
Methodology
Total of 73 Questions in 7 Categories
30 Yes/No Questions
43 Descriptive Questions
11
Implementation
Self evaluation of 33 airline websites
Methodology
Major Airlines Continental
United
American Airlines
Delta
U.S. Airways
Air Canada
Northwest Airlines
Alaska Airlines
Hawaiian
Alitalia
British Airways
Air France
Lufthansa
Iberia
Olympic
Swiss
El Al
“Low-Fare” Airlines
America West
JetBlue
Southwest
Spirit
ATA
Ted
Song
USA3000
Frontier
Independence Air
EasyJet
Sky Europe
Virgin Atlantic
Air One
Arkia
Israir
12
AWET DemonstrationEnter Search Parameters (Reservation Process)
Methodology
1
2 3
4
5 6
7
8
9
10
• 2 adults, 1 child (age: 12)• NY (LGA) to San Diego• Prefer non-stop• Date and time specified
• Flexible plans for better deal• Assign seats together• Obtain flight info
11
12
13
13
19 questionnaires sent to non-random B2C system users
Return rate = 68% (13 surveys)
Total of 26 evaluations, for 21 airline websites:
12 American airlines (46%)
6 European airlines (23%)
5 Israeli airlines (19%)
3 Canadian airlines (12%)
User group members average 11 trips per year
Performing 58% of airline reservations online
Results: Preliminary QuestionnaireUser Group Demographics
Results & Best Practices
14
Process and Website Design are most important for online QoS
Average number of evaluation criteria mentioned by users per group:
Results: Preliminary QuestionnaireDominant Criteria Groups and Validation of Assumptions
AirlinesWebsite design 2.38Navigation 1.69Process 3.08
Contents 1.31Technical 0.54Avg # of criteria mentioned 9.00
Assumptions A1 & A2 have been validated:
A1 – Streamlined process and improved website design are the
main reasons for users to favor a website
A2 – Users do not consider technical aspects to be a significant
factor in a quality of service provided by airline websites
Results & Best Practices
15
Results: Self-Evaluation of 33 Airline WebsitesTotal Score
Total Score - All Websites(Max Score = 365)
180
190
200
210
220
230
240
250
260
270
280
290
300
310
Overall Avg = 237.27
(65%)
N.A Major Carriers
Group Avg = 252
Group Avg = 236
N.A Low-Fare Carriers
Group Avg = 233 Group
Avg = 229
Group Avg = 214
EU Major Carriers
EU Low-Fare Carriers
Israeli Carriers
81% - American Airlines
52% - Arkia
Results & Best Practices
Average Total Score:
237 Points = 65%Best Performer
Worst Performer
69% 65% 64% 63% 59% Avg GroupScore
16
Evaluation Results: Relative Ranking
Relative Score
60%
65%
70%
75%
80%
85%
90%
95%
100%
AA
Ha
wa
iian
Jet B
lue
No
rth
we
st
Am
eri
ca W
est
Co
ntin
en
tal
Vir
gin
Sw
iss
Sp
irit
Alit
alia
So
ng
Un
ited
Air
Fra
nce
US
Air
Ala
ska
Ind
ep
en
de
nce
El-
Al
Fro
ntie
r
Ibe
ria
Bri
tish
Ea
syJe
t
Sky
Eu
rop
e
So
uth
we
st
AT
A
De
lta
Air
Ca
na
da
Oly
mp
ic
Isra
ir
Te
d
Lu
ftha
nsa
US
A 3
00
0
Air
On
e
Ark
ia
% o
f B
est
Sco
re
Results reveal significant performance gaps:
17
Evaluation Results: Distribution
Results are in proximity to Normal Distribution:
0.1%
13.6%
34.1%
2.1%
0.1%
2.1%
13.6%
34.1%
0.0%
3.0%
15.2%
30.3%
0.0%
3.0%
12.1%
36.4%
Normal Distribution
Study
-2s-3s -1s 0s 1s 2s 3s
18
Based on evaluation results, best performance values have been
established for every question within AWET
Best Practices
Results & Best Practices
Question Criterion Measurement Best Practice By1 Fast home page loading
speedSeconds to fully load homepage 3 seconds Air France
2 Fun and matching colors Best dominant colors Blue 79% of websites3 Number of photos Could not be clearly
determinedN/A
Type of photos Hawaiian beach Hawaiian Airlines4 Font type Arial 45% of websites
Font size (headers) Above 8 88% of websitesFont size (main text) Above 8 58% of websitesNumber of words in home page 132-226Number of pictures and banners 2-5Spacing score (1-to-5) 3-5
6 Site avoids vertical and horizontal scrolling
Percentage of home-page presented on screen (w/o scrolling)
100% 18% of websites
7 Amount of menu options and links
Number of menu option and links 34-35 Spirit, British Airways
8 Yes 85% of websites9 Yes 39% of websites
Easy to read typefaces (on homepage)
Photos that inspire vacation atmosphere
United, AA, Spirit, Alitalia
5 Page load
One-click to most menu and sub-menu options, by rolling over the text
Featuring reservations from homepage
Example of Best Practices for Questions 1-9
21
AWET (1/12)
AWET: Airline Website Evaluation Tool
Website:
Evaluation Date:
Section 1 - Website Entry Type the address of the website you wish to evaluate.Please measure time from clicking “go” to full loading of home page (in seconds).
Not at all
Very Much
1 2 3 4 5 Question Answer
1 Fast home page loading speed Time to fully load home page (seconds)
Section 2 - Home Page BrowsingLook at the home page, and answer the following questions:
Not at all
Very Much
1 2 3 4 5 Question Answer
2 How fun and matching are the colors? What are the dominant colors?
Number of photos
Describe photos
Font type (1)
Font size – headers (1)
Font size - main text (1)
Number of words in home page (1), (2)
If you rated 1 or 2 - do you think amount of home page text is lacking or excessive?
Number of pictures and banners
Is the spacing sufficient?
5
To what extent the photos used inspire vacation atmosphere?
How easy to read are the typefaces on the home page?
3
Comments
Comments# Criterion
RatingDescription
# Criterion
RatingDescription
4
How appropriate is the page-load (amount of text and pictures, spacing)
22
AWET (2/12)
AWET: Airline Website Evaluation Tool
6Site avoids vertical and horizontal scrolling?
Estimated percentage of home page presented on screen (w/o scrolling)
Number of menu option and links (including banners)
If you rated 1 or 2 - do you think amount of home page text is lacking or excessive?
(1) Copy sample of text from home page and paste to Word; If page features more than one size / type, refer to the dominant size / type(2) Count number of words by looking at “File >> Properties >> Statistics”.
# Yes No Comments8
9
10
11
Section 3 - Site Structure & Sub-Pages Browse through the website and answer the following questions:
# Yes No Comments12
13
14
15
16
17
18
19
20
21
Criterion
Site using consistent font size across pages
Site featuring easily accessible site map
Home page icon is consistent across all site pages
Is the location of menus consistent across all pages?
Did the site feature easily-accessible flight status information (departure, arrival, gate)?
Site using consistent colors across pages
All the following contact methods are available: email, fax, phone, mail
Availability of Frequently Asked Questions (FAQ)
Site available in foreign languages
7To what extent is the amount of menu options and links appropriate?
"Contact Us" information easily accessible from the home page
Site featuring reservations from home page
Home page features specials on the home page
One-click to most menu and sub-menu options, by rolling over the text
Criterion
Site featuring consistent frame
23
AWET (3/12)
AWET: Airline Website Evaluation Tool
Not at all
Very Much
1 2 3 4 5 Question Answer
22Site structure is clear and straightforward
What are the main menu options?
23How interactive is the route map(1 = not existing, 5 = interactive)
What type of map is featured?
24
Does the site offer cars, hotels and packages?(1 = none, 3 = just one, 4 = both separately, 5 = packages)
What add-on service is featured?
25 Fast sub-pages loading speed (3) Time to fully load sub-page (3)
(3) Average loading time of three random pages
Section 4 - Reservation
Note: Please read instructions & questions to all phases prior to starting the reservation process.
You wish to make flight reservations for your family trip to San Diego. - Your wife, 10 year-old daughter and you would like to leave NY on Thursday, August 11, 2005 and return on Thursday, August 18, 2005. - Price is a major consideration in your plans.- You prefer leaving NY between 7 and 10 am, but would consider leaving earlier or later if the price difference is significant (over $50 per tickets). - Preferred return time would be anytime after 12:00 noon, but could be earlier or later if necessary.- You prefer flying out of LaGuardia airport. Second priority is JFK airport. - You prefer non-stop flights, but would consider maximum of one connection if cheaper by $50 or more. However, you would only agree to one connecting flight as long as the stop-over is not longer than 3 hours in each direction.- You wish to assign all three seats together.- You would like to know what meals are served during the flights.- You would like to know how many miles you can earn for the round-trip.
DescriptionCommentsCriterion
Rating
#
This reservation section is comprised of seven phases Basic search, View results, Modify search, Selecting flights, Reservation support, Error handling and Reservation summary
24
AWET (4/12)
AWET: Airline Website Evaluation Tool
Phase I - Enter Basic Search RequirementsPlease return to the website home page. Record the current time: Start entering your flight requirements. Count the number of mouse clicks / fields filled. If needed, continue to the reservation page
Time to complete phase I : 26 Total duration of phase I (in seconds): 27 Number of clicks (phase I):
Please answer the following questions:
Not at all
Very Much
1 2 3 4 5 Question AnswerDid the site feature calendar for date search?
Did the site feature type-in fields for date search?
Did the site feature day and month fields for date search?
Default search dates are in the future?
Return date automatically updated based on departing flight?
Site allowing part-of-day entry for time (e.g.: morning, evening)
DescriptionComments
28
# Criterion
Rating
How easy to operate did you find the entry for dates and hours of travel?
Stop this phase upon completion of your basic flight requirements - just before you click on "Search Flight" (or any equivalent button).
25
AWET (5/12)
AWET: Airline Website Evaluation Tool
Did the site feature type-in fields for airports?
Did the site feature drop down list of airports?
Did the site feature search for airport codes?
Did the site feature an interactive map to select the departure/desination airports?
Did the provide search of nearby airports?
# Yes No Comments30
31
32
33
34
35 Website offers ability to navigate across fields using the tab and arrow keys
Search entry differentiates passengers: adult, children and seniors
Upon moving to reservation page, could you avoid data re-entry?
Criterion
Website offers ability to search flights based on number of connectionsWebsite offers ability to search flights based on price or schedule
Could you complete the reservations from the home page (without continuing to the reservation page)?
29How easy to operate did you find the entry for airport or route?
26
AWET (6/12)
AWET: Airline Website Evaluation Tool
Phase II - View Search ResultsRecord the current time: Click on "Search Flight" (or "Go", or any equivalent button).
Time to load search results (in seconds) (4)
Please view results and answer the following questions:
Not at all
Very Much
1 2 3 4 5 Question AnswerOutbound and inbound flight presented on one page?
Outbound and inbound flights are paired (as opposed to showing separately)?
Did site feature matrix for combinations of departing and returning flight?
37 Fast loading speed (4) Time to present flight search results (as
recorded above) (4)
Total number of itineraries presented (5)
Number of itineraries fully accommodating your request
Percentage of matching itineraries (calculated from the data above)
Did the site clearly indicate when the no matches were available?
(4) If inbound and outbound flights presented on separate pages, consider the first search only(5) If inbound and outbound flights presented on separate pages, count the number of possible combinations (Outbound x Inbound)
CommentsCriterion
RatingDescription
#
Presentation of all possible itineraries was clear and comfortable
36
38Abundance of flights was sufficient and appropriate
27
AWET (7/12)
AWET: Airline Website Evaluation Tool
# Yes No Comments39
40
41
42
43
44
45
Identify the least expensive flights accommodating your request (please do not select flight yet)Total Price (two adults and one child):
Pricing information available in U.S. dollars
Site features ability to filter results based on number of connections
Site features ability to sort results by price
Did the site feature clear presentation of departure and arrival times?
CriterionRequested itinerary (route, time) showing next to search results
Did the site feature clear child pricing for your flight?
Did the site feature meal information during the flight?
28
AWET (8/12)
AWET: Airline Website Evaluation Tool
Phase III - Modify Search
Not at all
Very Much
1 2 3 4 5 Question AnswerSearch results page features a link for flexible flights
Flexible search suggested alternative dates and time of day
Site providing fields to enter range of possible travel dates (e.g.: +/- 3 days)
Did you have to browse back and forth in order to search your flights?
Did you have to re-enter airports (from/to)?
Did you have to re-enter number of passengers?
Identify the least expensive flights accommodating your request Total Price (two adults and one child):
46
47Site prevented unnecessary data re-entry
You are still not sure you got the best deal, and would like to search for cheaper flights…You might be more flexible with your travel dates if you would get better prices.Please try to find cheaper flights for a 7-day vacation on the same route
# Criterion
Rating
Search for cheaper flights for flexible dates was easy and comfortable
DescriptionComments
29
AWET (9/12)
AWET: Airline Website Evaluation Tool
Phase IV - Selecting Your Flights
(6) If the website does not allow placing reservation on hold, please write "N/A" in question 52 and other relevant questions
Please answer the following questions:
Not at all
Very Much
1 2 3 4 5 Question Answer
48 Comfortable entry of passenger details Number of field requiring entry (per person, not including optional fields)
Price information shows breakdown by person and total
Price information shows breakdown of price, surcharges and taxes?
50Clear and straightforward presentation of Terms & Conditions
Terms & conditions clearly presented (not hidden) prior to purchase?
51Site allows alternative methods of payment
Number of credit cards accepted
# Yes No Comments52
53
54
55
56
Seat selection available online
Site allows you to return to previous process step (without an error)
Criterion
Help available for process-related questions
Description
Stop this phase upon placing your flights on hold, or prior to entering credit card information (6)
Comments#
Rating
Seat availability clearly presented prior to purchase
Site allow “Hold” on tickets prior to purchaseCriterion
Clear presentation of all order charges 49
30
AWET (10/12)
AWET: Airline Website Evaluation Tool
Phase V - Reservation Support
Select your preferred flights. Record the current time: Try to contact a customer service / sales representative online, to ask them if you could change your reservation within 2 hours
# Yes No Comments57
58
Send an email regarding your question via the websiteDate and time email was sent:
Time you received response to your question (7):
(7) Refers to "final" response to your question (not to immediate response promising future follow-up)
Not at all
Very Much
1 2 3 4 5 Question Answer
User-friendly contact form (via email) Number of required fields
Are all required fields are applicable for your enquiry?
Did you receive immediate response informing you that your enquiry has been received (within 5 minutes from sending your question)?
Did the website (or immediate email response) specify expected length to reply to your question?
Time to reply to your email enquiry (in hours)
Business hours to reply (assuming 8 hours/day)
Did final response meet time specified?
Were you satisfied with the answer provided to you?
Reponse was courteous?
61
59
# Criterion
60
Site offering online, in-person support?
Quick reply to email enquiries
Email response was courteous, accurate and relevant
DescriptionComments
Rating
Looking up contact info was available without losing my reservation info?
Criteria
31
AWET (11/12)
AWET: Airline Website Evaluation Tool
Phase VI - Error Handling
Return to the reservation page / home page. Enter departure date of Aug 11, 2005 and return date of Aug 8, 2005 (same route)
# Yes No Comments
62
63
Return to the reservation page / home page. Enter departure date of Aug 11 (morning) and return date of Aug 11 (afternoon)
# Yes No Comments64
Return to the reservation page / home page. Enter departure city "Nwe York" (deliberately misspelled)
# Yes No Comments65
CriterionSystem alerts for possible error when return time is adjacent to departure time.
System provides error warning message when return date is earlier than departure date.
System provides instructions for fixing the error
Criterion
CriterionSystem suggests ways to handle spelling mistakes?
32
AWET (12/12)
AWET: Airline Website Evaluation Tool
Phase VII - Reservation Summary
Please answer the following questions regarding the reservation process you just experienced
Not at all
Very Much
1 2 3 4 5 Question Answer
66The reservation process was transparent, easy and clear
Was progress of the reservation process indicated?
67 Task was fully completed
Percentage of the completion, based on the following five tasks: - Found desired flight? - Found seat assignment? - Found child pricing? - Found mileage information? - Found meal information?
68Task completed without errors and broken links
Number of error messages
69 Is the transaction perceived as secure?Was a security alert or message containing security presented prior to entering credit card info?
70Avoidance of unnecessary security alerts
Number of security alerts during visit
Section 5 - Post Sale and Summary
Please answer the following questions:
# Yes No Comments7172
73
(8) Assuming use of same computer, to eliminate impact of different settings.
Site allows the user to change an existing reservation
Site allows the user to cancel reservation
Criterion
RatingDescription
Comments
You were able to avoid pop-ups during the visit (8)
Criterion
#