Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
https://lib.uliege.be https://matheo.uliege.be
How Artificial Intelligence impacts the customer experience
Auteur : Knidiri, Hiba
Promoteur(s) : Dessart, Laurence
Faculté : HEC-Ecole de gestion de l'Université de Liège
Diplôme : Master en sciences de gestion, à finalité spécialisée en international strategic marketing
Année académique : 2020-2021
URI/URL : http://hdl.handle.net/2268.2/13565
Avertissement à l'attention des usagers :
Tous les documents placés en accès ouvert sur le site le site MatheO sont protégés par le droit d'auteur. Conformément
aux principes énoncés par la "Budapest Open Access Initiative"(BOAI, 2002), l'utilisateur du site peut lire, télécharger,
copier, transmettre, imprimer, chercher ou faire un lien vers le texte intégral de ces documents, les disséquer pour les
indexer, s'en servir de données pour un logiciel, ou s'en servir à toute autre fin légale (ou prévue par la réglementation
relative au droit d'auteur). Toute utilisation du document à des fins commerciales est strictement interdite.
Par ailleurs, l'utilisateur s'engage à respecter les droits moraux de l'auteur, principalement le droit à l'intégrité de l'oeuvre
et le droit de paternité et ce dans toute utilisation que l'utilisateur entreprend. Ainsi, à titre d'exemple, lorsqu'il reproduira
un document par extrait ou dans son intégralité, l'utilisateur citera de manière complète les sources telles que
mentionnées ci-dessus. Toute utilisation non explicitement autorisée ci-avant (telle que par exemple, la modification du
document ou son résumé) nécessite l'autorisation préalable et expresse des auteurs ou de leurs ayants droit.
HOW ARTIFICIALINTELLIGENCE IMPACTS
THE CUSTOMEREXPERIENCE
Jury:
Promoter:
Laurence DESSART
Readers:
Maarten BOSMA; Florian PETERS
Dissertation by
Hiba KNIDIRI
For Master’s degree in Management
specializing in International Strategic Marketing
Academic Year 2020/2021
TABLE OF CONTENTS
ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
CHAPTER 1. GENERAL INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Research aim and objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
CHAPTER 2. LITERATURE REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.1 Customer Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2 AI-enabled Customer Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3 Theoretical framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.4 Proposed model and hypotheses development . . . . . . . . . . . . . . . . . . . . 14
CHAPTER 3. STUDY 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1.1 Participants and experimental procedure . . . . . . . . . . . . . . . . . . . 213.1.2 Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2 Analyses and results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.2.1 Pre-test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.2.2 Preliminary results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.2.3 Manipulation checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.2.4 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.2.5 Relationship between variables . . . . . . . . . . . . . . . . . . . . . . . . 31
3.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2
CHAPTER 4. STUDY 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.1.1 Experiment and stimulus development . . . . . . . . . . . . . . . . . . . . 334.1.2 Participants and experimental procedure . . . . . . . . . . . . . . . . . . . 354.1.3 Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.2 Analyses and results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.2.1 Pre-test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.2.2 Preliminary checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.2.3 Manipulation check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.2.4 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414.2.5 Relationship between variables . . . . . . . . . . . . . . . . . . . . . . . . 44
4.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
CHAPTER 5. GENERAL DISCUSSION . . . . . . . . . . . . . . . . . . . . . . . . . . . 485.1 Short Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485.2 Theoretical Implications of this study . . . . . . . . . . . . . . . . . . . . . . . . 495.3 Managerial Implications of this study . . . . . . . . . . . . . . . . . . . . . . . . 505.4 Limitations and suggestions for further research . . . . . . . . . . . . . . . . . . . 51
APPENDIX A. Confound checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
APPENDIX B. Normality checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
APPENDIX C. Descriptive statistics of control variables . . . . . . . . . . . . . . . . . . . 62
3
ACKNOWLEDGMENTS
The present work has been prepared with great care and attention to detail. But it would not
have been possible without the people who have contributed to its production. It is only natural to
express my gratitude to these people who have also made it a growth experience.
First, I would like to thank my academic advisor, Mrs. Laurence Dessart, for her guidance through-
out the process.
Additionally, I would like to acknowledge Mr. Maarten Bosma and Mr. Florian Peters for the time
spent reading this thesis and for their valuable feedback.
I owe a great debt of gratitude to my friend Mohammed Abdelbaki. Thanks to his development
skills, we were able to turn this academic research into a work experience.
I would like to express my sincere gratitude and appreciation to my entire family of HEC Liege
who made my journey to HEC possible.
Last but not least, I would like to thank my friends and beloved family in Morocco for their un-
conditional love. Their love and care during this difficult time is the greatest encouragement for all
my efforts.
One last big thank you to everyone who took 5 minutes to fill out my thesis questionnaire!
4
ABSTRACT
The increasing use of AI technologies such as chatbots by retailers is leading academics and
practitioners to recognise their impact on the customer experience. Although research shows that
consumers generally prefer to deal with humans rather than robots, it has also shown that embed-
ding virtual agents into online retail and service websites leads to positive customer outcomes. In
this paper, we investigate the presence of intelligent, disembodied virtual agents (i.e., AI chatbots)
in an online shopping context and extend the study to include human-like cues such as conversa-
tional style and avatar that can improve the customer experience. We conduct two online exper-
iments (N = 77 and 143) with real chatbot conversations in a simulated online store to test the
premises of our study. Study 1 showed no differences between the presence and absence of a chat-
bot on participants’ customer experience. The interaction effect in Study 2 (conversation style x
avatar type) also proved to be non-significant. Nonetheless, in line with previous studies, signifi-
cant evidence of perceived humanness, personalization, and social support was found, opening the
door for further research. The paper concludes with practical implications for retailers using AI
chatbots for their customers.
Keywords:
AI Chatots – Online Customer Experience – Interaction Style – Avatar – Online Shopping
5
CHAPTER 1. GENERAL INTRODUCTION
This chapter provides an overview of the topic chosen for this thesis, its implications and scope.
1.1 Context
With the immense growth of digitization, artificial intelligence is the game changer that com-
panies are banking on to boost their organizational performance. According to ? recent survey on
the topic, artificial intelligence is on the rise, with 37% of companies reporting that they are already
using artificial intelligence. In practice, AI applications and tools are widely used by companies in
all industries and are therefore ubiquitous. They are not only changing business processes but also
customer behavior. Ameen et al. (2021) recently published an article that provides examples of
common AI applications and how they improve and enhance the customer experience. For exam-
ple, intelligent recommendation agents as AI-powered tools can improve customers’ convenience
by providing personalized recommendations that presumably profile their preferences (e.g., Net-
flix, Spotify). In retail, AI technologies for virtual fitting are also a case worth mentioning, as
they help reduce and limit errors in online clothing purchases, from size to color to fashion pref-
erences (Yang et al., 2020) . One example of the AI technology broadly used by by businesses
to enable more efficient interactions with customers is chatbots1, also known as (disembodied)
conversational agent.
Chatbots are usually defined as computer programs with text or voice interface, based on natu-
ral language and specifically designed to provide faster and more natural access to information to
facilitate and enhance user interaction (Jain et al., 2018). The chatbot revolution is in full swing,
1The term chatbot and conversational/virtual agent are not identical in meaning but will be used interchangeablyhenceforth.
1
appearing and disappearing from the limelight depending on how good their capabilities are. How-
ever, advances in artificial intelligence, natural language processing (NLP), and machine learning
are the reason why chatbots are making a comeback and becoming more adept and intelligent
(Gnewuch et al., 2017) . Given these improvements, the use of chatbots has expanded and can be
found in various industries and sectors such as healthcare, tourism, and e-commerce. AI chatbots
make it possible to access customer-related data, interpret it correctly, and adapt it in a flexible
manner (Kaplan and Haenlein, 2019). By gaining such insights, companies can improve their po-
tential knowledge of customer needs and optimize their decision-making in response,making AI
chatbots an innovative way to add value not only for marketers but also for customers.
There is a plethora of AI chatbot use cases in marketing and sales. These often include answer-
ing frequently asked questions, handling customer service requests, recommending new offers,
capturing customer inquiries, etc. Therefore, AI chatbots can be considered as interaction brokers,
i.e., a point of contact between businesses and customers. Customers interact with chatbots to
get information in the pre-purchase phase, to get assistance during the purchase phase, and finally
to get their questions answered in the post-purchase phase. Previously, this required customers
to either fill out forms, write formal emails, or call hotlines with often long queues.In summary,
chatbots are touchpoints that allow customers to interact effortlessly and save time and money
throughout the purchase phase (Gnewuch et al., 2018; Adam et al., 2019; Moriuchi, 2020; Pren-
tice et al., 2020) .In line with research, the internal and subjective responses customers receive
from similar interactions are referred to as customer experience. Therefore, it can be inferred that
chatbots have an important impact on customer experience.
In today’s retail service environment, where customers are more in control than ever, mar-
keting scholars and practitioners agree that a largely pleasant customer experience represents a
significant long-term competitive advantage for any business. It follows that CX is a key aspect of
differentiation and engagement, making it one of the most important areas of research in marketing.
And although customer experience is increasingly shifting to technology-based online touchpoints,
2
there are only few studies investigating this new phenomenon (De Keyser et al., 2020; Becker and
Jaakkola, 2020) . This growing attention has led to calls for research to better understand this trend
in science and practice.
1.2 Research aim and objectives
As mentioned earlier, AI-based chatbots’s omnipresence in customer-business encounters is
pushing both scholars and practitioners to understand how these technologies impact customer
experience in particular Jain et al. (2018) , but little is known about how customers respond to their
use. The technology behind chatbots is growing, but understanding of chatbot interactions is not
growing at the same rate. The development of a successful chatbot largely depends on these two
factors, i.e. technological and social factors. In addition, while chatbots provide faster and more
natural access to information, they often fail to meet customer expectations if they do not provide
interactions as engaging, entertaining, and complex as interpersonal conversations in messaging
apps Pérez et al. (2020). This means that more knowledge about the interaction between customers
and chatbots is needed to achieve positive results for customers.
Based on these insights, the purpose of this study is to understand the impact of introduc-
ing an intelligent chatbot that performs search support functions in an online shopping context.
More specifically, this study examines the difference between AI conversational user interface
(i.e.,chatbot) and graphical user interface (website) in assisting customers to find the right products.
In other words, we want to evaluate the difference between the presence and the absence of an AI
chatbot in online shopping context in terms of customer experience ( Study 1). Despite the differ-
ent types and designs of chatbot interfaces (e.g., embodied, avatar, etc.), scholars and practitioners
still disagree on which elements should be considered or disregarded when designing chatbots to
achieve a better conversation with customers. These factors are related to human-computer inter-
action and are crucial for a positive customer experience. The study (2) addresses conversational
3
and visual anthropomorphic cues in chatbots and examines their impact on customer experience,
which is very relevant given the increasing prevalence of chatbots.
1.3 Contribution
The literature review of this paper does not include a study that examines the direct impact
of AI chatbots and their features on customer experience. This gap motivates the author of this
paper to provide meaningful insights to marketing scholars and managers and hopefully suggest
directions for future research. Furthermore, it is not intended to provide definitive and conclusive
answers regarding the causality of the variables studied, but rather to explore the research ques-
tions. Thus, the relevance of this work is firstly at the functional level for developers of chatbots
by uncovering the potential impact of design elements on the experience. Secondly, at the man-
agement level for marketers testing chatbots on their landing pages to create a better and smoother
online experience. And finally, at the academic level, adding to existing scientific knowledge and
laying the groundwork for future studies on chatbots, such as understanding the underlying factors
that might influence chatbot adoption.
Chatbots tend to be disembodied CAs in that they interact with customers primarily via messaging-
based interfaces through verbal (e.g., conversational style) and nonverbal cues (e.g., three animated
dots) that enable real-time dialogue, with generally a static profile picture. Apart from two excep-
tions that focused on verbal ADCs (Araujo, 2018; Go and Sundar, 2019), this study is, to our
knowledge, one of the first to examine virtual agents in the form of AI chatbots (as opposed to
other forms, such as interactive avatars) and to incorporate visual conversational cues to investi-
gate their impact on the online customer experience in an online shopping environment (Adam
et al., 2019).
4
1.4 Approach
This paper begins with a literature review explaining the concepts of the study (i.e. AI chatbots,
customer experience). This section serves to enhance the understanding of the topic by allowing
the researcher to map and analyze the current intellectual terrain and define the research question
to add to the existing body of knowledge. Next, the major theories on the topic are highlighted
and incorporated into the development of the hypotheses and the creation of the conceptual model.
Finally, the research design and data collection process for this thesis are defined and the results
are revealed and addressed to determine the managerial and scientific significance of this analysis
and its potential implications for further study.
5
CHAPTER 2. LITERATURE REVIEW
In this chapter, a literature review important to establishing the research questions and
assumptions that underpin the study design is presented to explain and clarify all constructs, terms
and core theories that will form the basis for the development of the corresponding hypotheses
and conceptual model for this thesis.
For expositional ease, we use the terms “interaction style” “conversational style” interchangeably
throughout the paper.
2.1 Customer Experience
Holbrook and Hirschman (1982) were the first to challenge the traditional consumer decision-
making process that assumed the customer was purely rational (Bhattacharaya), and instead rec-
ognized the "experiential" dimension of consumption, which includes symbolic, hedonistic, and
esthetic factors. Later, in Pine and Gilmore’s "experience economy", experience becomes an im-
portant offering by firms to customers, similar to goods, commodities and services. However, the
first serious discussion and analysis of customer experience in marketing emerged with a major
study by Schmitt (1999), which extended the work of Pine and Gilmore. Schmitt (1999) presented
a distinction between traditional marketing and experiential marketing based on five strategic ex-
periential modules (SEMs) in the form of "sense", "feel", "think", "act" and "relate", which he used
to argue that experiences occur when a consumer encounters and lives through things that convey
relational, emotional, sensory, behavioral and cognitive values(Verhoef et al., 2009)
Since the conceptual focus of SEMs developed by Schmitt (1999), customer experience has re-
ceived growing attention in academia with numerous conceptual and empirical studies on the topic.
6
Various definitions and conceptualizations have emerged, most of which share some elements with
the model proposed by Schmitt (1999) (Berry et al., 2002; Gentile et al., 2007; ?; Pine 2nd. and
Gilmore, 1998) . However, despite their contrasting perspectives, these studies agree that customer
experience is a complex, multifaceted concept defined differently depending on the approach and
context, that involves a person (subject) interacting with a firm or its offerings (object) at different
levels.(De Keyser et al., 2020; Jain et al., 2018) Customer experience is defined as a state that
is evoked in an individual in response to a stimulus Poulsson and Kale (2004) . For Becker and
Jackola, customer experience is the ”subjective response or interpretation of any direct or indirect
contact with the elements of service, such as the provider, offering, brand, environment, or pro-
cess”, while (Gentile et al., 2007, p.397) defines CX as ” set of interactions between a customer
and a product, a company, or a part of its organization which provoke a reaction ).Against these dif-
ferent definitions of customer experience (Lemon and Verhoef, 2016; Becker and Jaakkola, 2020)
, there seems to be some agreement that customer experience reflects the customer’s internal re-
action and subjective response to all direct and indirect encounters with a company’s products,
service, or brand (Gentile et al., 2007; Becker and Jaakkola, 2020) . Indirect contact usually in-
cludes unplanned encounters with representatives, word of mouth, advertisements, news reports, or
product reviews, while direct contact occurs in the course of a customer’s purchase, use, or service
and is usually initiated by the customer.
These internal responses and subjective reactions are a set of sensations and mental states
that can be described as the dimensions of the customer experience construct Rose et al. (2012).
Notwithstanding the varying number of CX dimensions that exist depending on the context and
the research setting, recent studies, in line with Smith’s work, have extended the dimensionality
approach from two dimensions, i.e. affective and cognitive (Rose et al., 2012; Danckwerts et al.,
2019), to other dimensions such as sensory, behavioral and relational (Gentile et al., 2007; Schmitt,
1999). CX is thus a multidimensional construct composed of individuals’ cognitive, affective, sen-
7
sory/physical, social/relational, and behavioral/pragmatic responses to a service at every possible
point of contact, i.e., touchpoints outside and within the control of the retailer or company.
All in all, a more dynamic perspective on customer experience states that CX is the amalga-
mation of small, incremental, bounded experiences that evolve throughout the customer journey
Lemon and Verhoef (2016) .In similar vein, De Keyser et al. (2020) presents a comprehensive con-
ceptualization of CX: the customer experience is formed through interactions with "touchpoints"
( firm-controlled vs. non-firm-controlled and direct vs. indirect) within "phases" (pre-purchase,
purchase, post-purchase) conditioned by a broader "context" (individual, social, environmental,
etc.) and characterized by a set of "qualities" (strength, duration, valence, etc.) that together result
in a value judgment by the customer.
The different conceptualizations of customer experience in terms of scope and dimensions,
antecedents and consequences also complicate the operationalization of the construct in a holistic
manner. Despite the associated concerns about measurement validity and results generalizability,
scholars recommend that customer experience measurement tools should be adapted to fit the con-
text (Waqas et al., 2021) .That being said, two influential scales are extensively used in CX research
in view of their applicability in multiple, wide-ranging contexts in marketing (e.g., tourism,retail).
These scales include the brand experience scale (Josko, 2009) and Experience quality scale (EXQ)
(Klaus et al., 2013; Maklan and Klaus, 2011) The Brand Experience Scale follows the conceptual
focus on SEMs developed by Schmitt (1999) to measure customer experience with a brand, using
four dimensions: intellectual, sensory, affective and behavioral. The Experience Quality Scale
(EXQ), on the other hand, includes four dimensions (product experience, outcome focus, moments
of truth, and peace of mind) based on evaluative judgments about the service. The latter approach
has been criticized by scholars considering that evaluative concepts such as perceived service qual-
ity, satisfaction, and motivational concepts such as engagement should be distinguished from the
definition of CX and that this link deserves further investigation.(e.g.,) (Becker and Jaakkola, 2020;
De Keyser et al., 2020; Lemon and Verhoef, 2016)
8
2.2 AI-enabled Customer Experience
Due to technological advancements and the proliferation of digital services, customer inter-
actions are shifting to online channels and touchpoints. As a result, companies are expected to
provide better service in the online environment and maximize the capabilities of each digital
touchpoint to enhance the customer experience in the online context. However, it is becoming
increasingly difficult to create a superior online customer experience (OCE) as customers become
more aware of the competition. The main advantage of online stores is that retailers can operate
their business 24/7 and customers can access the store from anywhere in the world, but one of the
biggest disadvantages is the naturalness of the interaction with the retailer: when a customer enters
the website, they should have the same feeling as when they enter a physical store. Therefore, the
online customer experience should resemble the offline environment, where products and services
are easy to find and help is always nearby.
Moreover, due to the lack of natural connection between retailers and online customers, most
online stores resemble vending machines rather than real stores. Online stores, based only on
graphical user interfaces, do not allow retailers to persuade potential customers to buy products
and do not give customers the opportunity to ask questions and learn more details about products
than they would with a human salesperson. There is no doubt that communication is crucial to
attracting, serving and retaining customers. One of the most beneficial ways to engage customers
anywhere, anytime, and provide them with an easy and natural interaction is to use a conversational
user interface, i.e. chatbots.
To clarify, chatbots are computer programmes that simulate human-like interaction by under-
standing user queries and performing a limited number of tasks without undue delay, much as
consumers would if they were talking to a real person. Chatbot systems have become much more
sophisticated thanks to significant advances in artificial intelligence (AI). When supported by AI-
related technologies such as Natural language understanding (NLU), chatbots can better discern
9
the intent behind users’ input and learn more complex ways to simulate human conversations, such
as asking open-ended questions, interpreting users’ free-text responses, and prompting for answers
when needed (Hussain; Klopfenstein et al., 2017) . These technologies can be trained to learn
from each interaction with a customer to improve their performance in the next one and eventually
become more intelligent.
Virtual agents (i.e. AI chatbots) are needed wherever assistance is needed during or after a
purchase: They are used to assist customers in online transactions on websites by providing them
with additional information, personalized advice and recommendations (Araujo, 2018; De Cicco
et al., 2020; Verhagen et al., 2014) and also technical support such as shipping or product returns.
Therefore, these agents can be used strategically to enable companies to interact with their cus-
tomers on a personal level and support them on a full-time basis. With this in mind, it can be
concluded that chatbots aim to provide customers with an effortless online experience that is time
and cost efficient (Gnewuch et al., 2018; Adam et al., 2019; Moriuchi, 2020; Prentice et al., 2020).
Empirical research on online customer experience started to emerge with seminal work of
Novak et al. (2000). During this time, researchers investigated the phenomenon of customer ex-
perience in Internet environments (Waqas et al., 2021) . Only recently, with the rapid growth of
online retailing, is research on digital touchpoints catching up (Bleier et al., 2019; Rose et al.,
2012; Kaatz et al., 2019) . Nevertheless, a review of the extant literature shows that research is
still sparse and at an early stage, and that more in-depth research on online customer experience
in different online environments, e.g. shopping environments, including the measurement of this
experience and its impact, is needed.
Earlier, the online customer experience was studied by Novak from a cognitive perspective and
defined as "a cognitive state experienced during online navigation", i.e. "flow" (Novak et al., 2000).
While the author focused only on the antecedents of online flow (i.e., telepresence, challenge, skill,
and interactive speed), later studies extended this work and incorporated affective state into the
conceptualization of OCX with new hedonic variables as antecedents of OCX (Rose et al., 2012) .
10
Consistent with these studies, the literature review shows that most research conducted in the field
of OCX have described customer experience as consisting of two dimensions (Danckwerts et al.,
2019). In contrast, other research based on previous notable work in the field of offline experiences
has considered additional elements of a customer experience in an online environment, such as
sensory and physical dimensions that include technology-related features such as a user-friendly
interface and clear design (Ameen et al., 2021; Bleier et al., 2019; Waqas et al., 2021) . Similarly,
social elements were captured that relate to the influence of others online, such as online forums,
reviews, and virtual agents (e.g., avatars, chatbots) (Chattaraman et al., 2012) .
Since the context of analysis in this paper is online shopping, CX is inherently determined by
the direct interaction between customers and the online environment. Therefore, a more compre-
hensive measurement such as Service Experience Quality EXQ (Klaus et al., 2013), which captures
CX in three stages, or Brand Experience Scale (Josko, 2009; Schmitt, 1999), which focuses only
on the "brand-related stimuli" (Lemon and Verhoef, 2016, p.70), cannot accurately explain CX
in the online environment (Kuppelwieser and Klaus, 2021) . As mentioned earlier, there is no
work that attempts to develop an all-encompassing measurement tool to capture all CX qualities
holistically and dynamically across the customer journey, but work that tailors CX measurement to
specific contexts (Waqas et al., 2021). Therefore, this paper analyzes CX as it is lived in the online
environment using the operationalization of Bleier et al. (2019).
As recommended by most CX researchers(e.g.) (Lemon and Verhoef, 2016; Schmitt, 1999;
Verhoef et al., 2009) , Bleier et al. (2019) based the OCX scale on the four dimensions of experience
most commonly used in the literature: cognitive, affective, social and sensory, omitting by that
the physical and behavioral dimensions. Indeed, several studies claim that responses to sensory
stimuli are closely related to clients’ physical well-being, implying that the sensory dimension is
inextricably linked to the physical dimension. On the other hand, according to the definition of
Kaatz et al. (2019). the behavioral component is: "the flexibility of customers to enter the store at
any time and from any place", which is a given condition for the success of the experiments of this
11
research. Accordingly, this study will limit the conceptualization and operationalization of OCX
to that proposed by Bleier et al. (2019).
Online experiences are created through the internal reactions (cognitive, affective, social, sen-
sory) that occur when customers perceive and interpret online stimuli, in this case, the nature of the
interface (Study 1) and its features, i.e. anthropomorphic cues (Study 2). Respectively, cognitive
dimension of OCX or "Informativeness" referred to by Bleier et al. (2019) . captures the functional
aspect and value of the experience Verhoef et al. (2009), and is defined as "the extent to which a
website provides consumers with helpful and useful information Bleier et al. (2019); Li and Unger
(2012) .Customer interactions with the online environment can also elicit affective responses: The
affective dimension or"Entertainment" reflects the immediate enjoyment of the experience, inde-
pendent of its ability to facilitate a particular shopping task. The social dimension, i.e. 3Social
Presence3, refers to the warm, social and human contact that a website provides . Finally, the
sensory dimension, "Sensory Appeal", refers to "the representational richness of a mediated envi-
ronment as defined by its formal features" (Bleier et al., 2019) . Sensory responses in the online
environment can be elicited for example by visual design elements (e.g., colors, photos, videos).
2.3 Theoretical framework
In the context of human-computer interaction, social-response theory assumes that individuals
respond to technology endowed with human-like features such as speech, voice, and interactivity
with anthropomorphic impressions and socially desirable behavior (Nass and Moon, 2000) . The
well-established social response theory has paved the way for several studies in digital contexts
showing how humans apply social rules to anthropomorphically designed computers. Although
many studies have been conducted on chatbots and conversational user interfaces, most of them
have focused on their technical elements, such as improving natural language processing algo-
rithms, Conversely, the understanding of chatbot interaction is not growing at the same rate. In
12
line with HCI, chatbots have the ability to mimic interpersonal interactions and can therefore be
considered as social and not just functional technologies (Følstad and Brandtzaeg, 2020).
Among the few research papers that have examined how users respond to social cues from chat-
bots and other systems with conversational user interfaces (e.g., embodied conversational agents)
are those that examine the effect of visual cues like avatar (virtual representations of real persons)
attractiveness (Holzwarth; Jin and Bolebruch, 2009), Other types of social cues have also been
found to influence users’ perception of a chatbot, such as interaction style (Chattaraman et al.,
2019) and the degree of interactivity (Schuetzler et al., 2014). Actually, according to many, inter-
activity is a social cue that elicits social responses and can be explained by user’s perception of
interpersonal interaction and sense they are in the presence of a social other (Wang et al., 2007;
Kim and Sundar, 2012). Along the findings of previous authors we argue that the presence of
virtual agents elicits higher perceptions of interactivity.
Consistent with Zhang et al. (2020) , this study defines social support as a multidimensional
construct consisting of informational and emotional support. Informational support refers to the
cognitive feelings triggered by an online material in the form of recommendations, instructions,
or useful advice that help customers overcome difficulties. Emotional support, on the other hand,
refers to the affective experience of emotional concerns such as caring, understanding, and empa-
thy. Indeed, a chatbot can provide the customer with response feedback (i.e., Nice to you meet
you), social acknowlegments (i.e.,Thanks, i’m glad i could help) and deal with user excuses (i.e,
no worries, want to start over). Similar to Chattaraman et al. (2019) we believe that social support
through procedural and navigational instructions provided by a conversational user interface that
simulates human-like conversations has the potential to give customers the feeling or experience
of being cared for, helped, and responded to compared to a graphical user interface (Zhang et al.,
2020).
Personalization is the customer’s perception of the flexibility of the website to meet their pref-
erences in an online environment. According to Komiak and Benbasat (2006), personalization has
13
a positive effect on the adoption of recommendation agents, even if they’re rudimentary. Further-
more, Verhagen et al. (2014) argue that CAs can reduce the lack of interpersonal interaction in
online environments by evoking perceptions of social presence and personalization (Adam et al.,
2019) . Finally, in line with previous authors, personalization of CA has been shown to positively
affect cognitive experience state and affective experience state. In particular, it’s worth noting that
intelligent chatbots can collect a range of textual data to increase personalization, which signifi-
cantly improves customers’ virtual experiences. For example, they can remember the user’s name
and address him with it throughout the interaction, remember the user’s choices and provide him
with personalized information based on his needs and profiles. In this sense, we can say that con-
versational agents provide a higher level of personalization than a website when shopping online.
2.4 Proposed model and hypotheses development
2.4.0.1 Study 1
Websites present an environment of increased cognitive load when performing search tasks
(Chattaraman et al., 2019). In this context, customers strive to obtain information about a product
or service, compare alternatives, or find a better price (Bleier et al., 2019). According to cognitive
load theory, individuals select the most relevant information from a variety of sources during a
learning process. In line with this theory, more and more customers are using omnichannel ser-
vices to complete the shopping process. However, recent research has shown that incongruence
between channels leads to the need to complete multiple tasks simultaneously, which increases
the cognitive effort required to switch from one channel to another. Information channels may
therefore naturally become contrained with with diminished attentional capacity and cognitive re-
sources, which impairs performing digital tasks like navigating through pages (Gao et al., 2021;
Kaatz et al., 2019). Instead, Ciechanowski et al. (2019) point out the importance of recognizing
how "interactive interfaces mediate the redistribution of cognitive tasks between humans and ma-
14
chines." In other words, the cognitive effort required to find a product is reduced by the presence
of a virtual agent, and the efficiency of the customer’s shopping process is appealed to the intellect.
We therefore propose that the virtual agent assists the consumer in the purchase decision, which in-
volves reasoning, conscious mental processing, and usually problem solving (Gentile et al., 2007;
Bleier et al., 2019; Josko, 2009).Consequently, we propose the following hypotheses:
H1a : Chatbot (vs website) interface will yield a better cognitive customer experience in an
online shopping environment.
Qiu and Benbasat (2010) show in a laboratory experiment that the use of recommendation
agents with verbal and nonverbal anthropomorphic design cues in online stores strongly influences
consumers’ perception of social presence. This increased customers’ trust, their enjoyment of
the virtual agent, and ultimately their intention to use it as a decision-making tool. In another
study by Chattaraman et al. (2019), it was found that perceived enjoyment of the robot increased
when the virtual assistant provided social cues, i.e., social conversation, and conveyed the desired
information in a personable manner, making the shopping process entertaining and enjoyable for
the consumer. This also corroborates the findings of Zhang et al. (2020) who established that
entertainment or playfulness can influence the success of human-computer interactions and thus the
user experience. Entertainment or affective experience, according to Bleier et al. (2019) involves an
appreciation for "spectacle" on the website, incorporates fun and play into online purchases, and
offers more than just performance and goal-oriented purchase decisions. Consistent with Bleier
et al. (2019) conceptualization, the above empirical studies suggest that virtual agents can stimulate
customers’ affective states and influence their online experience. Hence, it is hypothesized:
H1b : Chatbot (vs website) interface will yield a better affective customer experience in an
online shopping environment.
Despite the absence of a real human presence, the use of a conversational agent capable of
simulating a two-way conversation, i.e., understanding and responding to online customers in the
form of natural language dialogs, can create a sense of sociability during the interaction by assum-
15
ing the role of a sales representative in online stores, thus enhancing the sense of social presence.
Social presence is described as the extent to which the online environment makes customers feel
that there is a personal, sociable and intimate human contact (Bleier et al., 2019). Research on
online agents in marketing has shown that simulated interactions facilitated by virtual agents can
enhance perceptions of social presence. Social cues on a website can evoke feelings of social pres-
ence. However, since the chatbot can reduce social distance and is imbued with more social cues
(e.g., interactivity, personalization, social support), as mentioned earlier, we propose in line with
the results:
H1c : Chatbot (vs website) interface will yield a better social customer experience in an online
shopping environment.
Unlike conventional brick-and-mortar retail, which provides opportunities for social connec-
tions (salespeople and peer customers) and physical experiences (browsing, touching, and feeling
products), customers evaluate products online not through physical interaction but through verbal
and visual stimuli on product pages. The customer performs cognitive and affective processing
of incoming sensory information from the online environment, resulting in the impression being
created and stored in the customer’s memory (Josko, 2009; Rose et al., 2012). Sensory appeal,
the sensory dimension, refers to the way a website stimulates the senses (Bleier et al., 2019), and
regardless of the limited scope of sensory experiences in the online environment, sensations can be
evoked by images (e.g., pictures, videos), although colors, shapes, fonts, and designs usually lead
to sensory experiences. These sensory experiences can be enhanced through social interactions
with textual and visual content and personalized greetings (Hassanein and Head, 2005) supported
by virtual agents.Thus, we propose the following hypothesis:
H1d : Chatbot (vs website) interface will yield a better sensory customer experience in an
online shopping environment.
From the formulated hypotheses, we posit:
16
H1: Chatbot (vs website) interface will yield a better online customer experience in an online
shopping environment.
2.4.0.2 Study 2
Regardless of their physical or virtual embodiment, the ability of virtual agents to communicate
with users linguistically is a crucial human-like feature sufficient to evoke a sense of social presence
and perception of humanness. The common basis of research in this area is that cues as minimal
as response delays (Gnewuch et al., 2018), chatbot typos, capitalization of words in responses
(Westerman et al., 2019), and adaptive responses to user input are sufficient to simulate social
schemas in users’ minds and thus evoke perceptions of humanness. Wirtz et al. (2018) hypothesize
that the acceptance of CAs is generally dependent on perceived humanness. This is evidenced in
research by the positive outcomes when perceptions of humanness are high. Jin and Bolebruch
(2009) show that human-like visual features in a 3D virtual environment positively influence CAs’
likability and enjoyment of the experience. In this sense, previous research by Holzwarth et al.
(2006) on attractive features of CAs shows that anthropomorphism of virtual sales agents helps to
increase entertainment, information value, and customer satisfaction with the retailer. On the other
hand, Araujo (2018) proves that the linguistic style of a chatbot should be calibrated to optimize the
experience, claiming that a text-based CA would benefit from a stronger perception of humanness.
Similarly, De Cicco et al. (2020) suggests that a human-like CA with humanized conversational
qualities, i.e., social-oriented can increase social presence, which in turn leads to a more satisfying
user experience among a young population. These findings were confirmed in a previous study by
Chattaraman et al. (2019) on an older population, in which a socially-oriented interaction style of
CAs that emphasized empathy, personality, and friendliness led to better social outcomes than a
task-oriented interaction style that used formal language and focused solely on functional goals.
Accordingly, we hypothesize that a human-like CA will improve the experience, specifically :
17
H2: The effects of the Chatbot interface on customer experience will be stronger when the
chatbot uses contextual (vs scripted) conversational style and human (vs robot) visual identity.
Based on these hypotheses, a graphical model ( see Figure 2.1 ) was developed to provide a
visual representation of the research questions. In brief, this work aims to investigate the impact
of an independent variable, the type of type of interface, on a four-dimentional dependent vari-
ables: online customer experience an online shopping context. In addition, the study is extented to
examine particular chatbot modalities ( interaction style x avatar type) effect on online customer
experience.
Figure 2.1 Research model
18
CHAPTER 3. STUDY 1
Study 1 investigated the impact of interacting with an AI chatbot that performs navigation
support functions on the customer experience compared to traditional website navigation in the
context of online shopping. For this purpose, an AI chatbot was developed on the open-source AI
platform Rasa and connected to a mock website displaying real natural cosmetic products.
3.1 Method
This section presents the methodological scheme followed to answer the research questions of
this study. In the following, we describe our experimental design, sample and the measures used
in the post-experiment questionnaire.
In order to test the developed hypotheses and evaluate relationships between variables or, no-
tably, an impact that one variable has on another, it is conventional to use causal research design
(Malhotra et al., 2017; Eisend and Kuss, 2019) .Causal research is also clearly defined and highly
structured in design. Moreover, since causal research aims to demonstrate causal relationships
between variables as well as the nature of this relationship between the causal variables and the
effect to be predicted, research shows that experimental designs are the best way to evaluate causal
hypotheses.
Amongst the large array of experimental designs available (i.e., pre-experimental, true exper-
imental, quasi experimental, statistical). Post-test only control group, was judged to be the most
relevant for this study. Post-test only control group involves two groups. An experimental group
is exposed to a particular treatment and a control group that serves for comparison (also called the
testing effect) with no pretest measure to consider. This type of design is useful when participants
19
are not available for a pretest and for a follow-up, which was the case for this study. At first,
the researcher manipulates the independent variable, to create the two groups, and accordingly,
randomly assigns participants to the experimental and the control group. The groups are then post-
tested and compared in terms of their scores on the dependent variable after the experimental group
has received the experimental treatment condition. The randomization of the different treatments
allows the control of all extraneous variables and ensures that there are no systematic bias between
the groups (Malhotra et al., 2017).
Thus, the manipulated variable of this study is the "type of interface" with an experimental con-
dition (chatbot interface) and a control condition (website interface) that serves as an established
baseline measure from which we can monitor and highlight the effects of the treatment, in this case
the absence of the chatbot. In our online experiment, the stimuli, i.e., the type of interface (chatbot
vs. website), were developed and mimicked the design of many modern chat and web interfaces.
The website interface that hosted the chatbot was identical for both conditions for each page of the
shopping task, including the home page, product list page, product page, shopping cart, shipping,
and "return to questionnaire".The chatbot was developed using the Rasa platform, a conversational
AI platform that provides developers with the necessary AI-based functional capabilities for natu-
ral language processing, understanding, as well as dialogue management, allowing them to design,
script, and train conversational assistants. The chatbot is designed to support both search and nav-
igation for online shopping using natural language. Once the user’s input is sent, processed and
understood, the virtual agent responds in a few seconds in the same natural way. Given the focus of
this study, the agent did not have a profile picture and interacted with participants using text only.
As previously noted, both the control and experimental conditions provide the same informa-
tion about the products. How users will access that information remains different.The control
condition is regarded as a static delivery of information where users must click buttons, filters and
tags to functionally navigate on a web page. Whereas, in the treatment condition, users can ac-
cess content and services by the use of natural language in interaction with the virtual agent.The
20
experience provided in the former is considered to be more interactive than the static access to the
information. Also, the chatbot as compared to the website offers a functional support to users’
specified query and a social support by assisting users and reducing their struggles. Moreover,
the chatbot assists the users during their online shopping experience and reduces their struggles
by asking relevant questions to tailor the corresponding fit to their input criteria. Accordingly, the
treatment condition offers social and personalized support to the users in contrast with the control
condition. Thus, the chatbot is regarded as a more interactive interface that offers higher levels of
personalization and social support.
In an attempt to match real life experiences, real products of a young start-up specialized in
natural cosmetics were displayed on the mock website. All products were rebranded to match the
language of the research. The product database used for both interfaces was re-built from scratch,
and a new design and packaging were used to better match the context of online shopping. No
knowledge base was found with which to train the chatbot . Then, a series of sample dialogs with
professionals and knowledgeable customers were conducted and validated with the conversation
design fundamentals proposed by Rasa. Finally the conversations were tested with wizard of oz
process to find the best conversational flow and translate natural speech into structured data, make
the virtual assistant more helpful and natural. ( see key conversational skills)
3.1.1 Participants and experimental procedure
A total of 101 people participated in the study. Only the cases with valid data for all the
variables in the model were retained for hypothesis testing. Therefore, the final sample size for
hypothesis testing was 77, including 46 males and 29 females (response rate = 76 %), with each
experimental condition containing more than 20 observations Hair et al. (2006).The unequal dis-
tribution across the two conditions was due to the random assignment of participants by our ex-
perimental platform that did not account for participants who did not complete the study. The
distribution of the demographic characteristics of participants is included in table 3.1.1.
21
Descriptive statistics of demographics
Characteristics %
Gender Female 37.7%
Male 59.7%
Other 2.6%
Age Under 20 years old 3.9%
20 - 40 87%
40 - 60 7.8%
Over 60 1.3%
Level of education High school graduate 7.8%
Bachelor’s degree 15.6%
Master’s degree 64.9%
Doctorate degree 11.7%
The survey was created using the Qualtrics software program provided by HEC Liège. The
survey of this study was slightly personalized using images, which could be beneficial for the
psychological preparation of respondents (Malhotra et al., 2017) The survey offered alternative
fixed-response questions that required respondents to select from a set of predetermined answers
to control for variability in responses and simplify both data analysis for the researcher and the
survey experience for respondents.
The main purpose of the experiment was not disclosed to the participants. Instead, participants
were told that the study aimed to investigate people’s knowledge about Moroccan cosmetics. Par-
ticipants were randomly assigned to one of two conditions (i.e., treatment or control).Participants
read a set of instructions describing the task they were to perform. In these instructions, partici-
pants in the chatbot condition were told to select one of two products and engage with the virtual
agent to find the product, get more information about it, add it to the shopping cart, and click
22
"Return to questionnaire." Participants in the website condition, on the other hand, were asked to
follow the same instructions by navigating the website. The manipulation check is asked directly
after the last dependent variable measurement. Lastly attention check questions and demographic
questions are kept for the end. An extensive explanation of the experimental setup, summarized in
Table : experimental setup.
Table : Experimental setup
Group Treatment group Control group
Manipulation Chatbot Website
Shopping task Adapted from
Chattaraman et al. (2019)
Both Argan oil and Prickly Pear Seed oil are becoming ever
popular treatments thanks to their unparalleled benefits for
men and women.
Argan oil is a an excellent conditioner, mostly used to pre-
vent hair breakage and hair loss. Prickly pear seed oil is
an exceptional moisturizer that regenerates the skin and re-
duces ageing signs.
Choose 1 of these oils and :
1- Look up its benefits and its price. 2- Look up information
about shipping. 3- Add this product to cart. 4- Click on
"Return to questionnaire".
Manipulation The participant is asked to in-
teract with the chatbot to ob-
tain the information described
in the instructions
The participant is asked to ob-
tain the information described
in the instructions using the
content on the pages of the
website.
23
3.1.2 Measures
3.1.2.1 Independent variable
The independent variable in this study is the “type of interface” which is used as a stimuli in
the experiment. The independent variable takes two conditions : (1) the chatbot condition and (2)
the interface condition serving of a control group to the experiment.
3.1.2.2 Manipulation checks
To asses the assumed disparity between the two conditions A 7-item scale adapted from Chat-
taraman et al. (2012) was used to measure social support perceived by the participants from both
conditions, perceived personalization was appraised with a scale from Danckwerts et al. (2019),
and finally, perceived interactivity was measured using a 6-item interactivity scale with two sub-
scales measuring two-way communication and synchronicity adopted from Chattaraman et al.
(2019)
3.1.2.3 Dependent variable
The dependent variable of this research is "customer experience". In line with the work of
Bleier et al. (2019), customer experience is composed of four dimensions: The affective (enjoy-
ment), the cognitive(informativeness), the social (social presence) and the sensory experience (sen-
sory appeal). In order to get an impression of the overall customer experience, all these components
are assessed using the scales proposed by Bleier et al. (2019)
3.1.2.4 Control variables
To ensure the validity of our results, we accounted for some possible confounding variables in
the questionnaire. First, after answering our construct-related questions, participants self-reported
24
how familiar they were with the brand, which could potentially influence their response to a stim-
ulus with familiar elements and thus their answers.
Next, familiarity with natural products and Moroccan natural cosmetics was examined, because
if respondents are already well acquainted with the products, their answers may differ significantly,
and the findings can’t be generalizable to a population consisting of less knowledgeable individu-
als.Both control questions were measured on a 5-point Likert scale ranging from "very bad 1" to
"very good = 5".
Participants’ proximity to technology and virtual assistants could also influence the relation-
ships examined in the model. In this case, two effects are expected: greater familiarity could
enhance the interaction experience in the study, or it could impair the interaction due to the nov-
elty effect, which decreases as familiarity increases. Technology propinquity was measured and
chatbot use was measured using the same scale going from "very bad" to "very good".
In addition, we assessed participants’ Perceived Degree of Realism. For this scale, participants
were asked to rate how realistic the website was, using the two items "I could imagine an actual
web page looking like the one I just saw" and "I believe this website could exist in reality" on a
7-point Likert scale with anchors ranging from "strongly disagree" to "strongly agree." We also
asked them to indicate which device they used for the experiment, as the online environment and
the size of the screen of a PC is not comparable to that of a smartphone.
The general profile of the respondents was portrayed thanks to a question related the age,
gender and education of the participants. It is well-established that younger people are more open
to new technology and that cosmetics’ consumption is higher among female population. And
finally, participants’ English proficiency was also assessed using a 5-point rating scale ranging
from "very bad" to "very good" .we asked participants to report the device (see A)
25
3.2 Analyses and results
This chapter describes the statistical approaches and analysis used to evaluate the hypotheses
posed using the empirical data collected.The result are presented and commented.
3.2.1 Pre-test
Prior to the main experiment, a pretest was conducted to ensure that the manipulation was
effective. Participants were randomly assigned to one of the conditions and 25 responses were
retained for data analysis. The extent to which interaction with the chatbot was perceived as more
personalized, interactive and offers more social support than the website interface. T-test results
reveal that both personalization and social support were successfully manipulated whereas the
interactive variable was not successfully manipulated. Table : Manipulation checks pre-test results
shows the results.
Manipulation checks pre-test results
condition nP. Personalization P. Interactivity P. Social Support
Mean Mean Mean
Website37 2.805 4.902 4.166
(Control)
Chatbot40 5.333 5.307 5.604
(Treatment)
Test statistic t = -6.54, p = .000 t = -1.001, p = .342 t = -3.115, p = .005
3.2.2 Preliminary results
First of all, we performed a series of confound checks to control for the possibility that differ-
ences in our control variables like "Natural cosmetics familiarity" and "virtual agent usage" could
26
have been equally distributed among the four conditions. A one-way ANOVA revealed no sig-
nificant differences in terms of the control questions between the two groups (p>0.05). Next, we
performed complementary robustness tests on RASA to check whether participants in the treatment
group used and interacted indeed with the chatbot. Here, we verified the number of conversations
against the number of our valid data-set.
Before turning to the research questions and starting the hypothesis testing, there are often
several preliminary analyses to conduct (Sreejesh et al., 2014). The quality of the data and the
measures used are to be appraised first. Checking the normality assumption is a prerequisite to
many statistical tests. In this instance, A Shapiro-Wilk’s test (p <.05) and a visual inspection of the
histograms showed that the data significantly deviate from a normal distribution. However, our data
are approximately normally distributed, in terms of skewness and kurtosis with values between -2
and 2 for all the items forming our measurement scales ??, which is considered acceptable in order
to assume normal distribution (Hair et al., 2006).
To examine the constructs’ discriminant and convergent validity, we performed factor anal-
ysis with an oblimin rotation. The analysis performed on all the measures identified their uni-
dimensionality. The analysis suggests the second item of the sensory appeal scale (sens2) to be
removed, as it indicates a small loading of 0.47 that falls below the recommended threshold of 0.6
Hair et al. (2006) . This removal had no significant impact on the reliability of the scale.For a more
accurate check, we computed the AVE of each contruct, with all results above 0.5 which is con-
sidered acceptable Hair et al. (2006). Similarly, the remaining items displayed adequate construct
reliability and internal consistency. A complete list of variables, factor loadings and scale relia-
bilities is provided in Table : study 1 - factor analysis, with an adequate reliability demonstrating
strong Cronbach’s alphas (ranging from 0.86 to 0.94).
27
Study 1 : Factor analysisFactor
Construct loading
Informativenessα = 0.86; AVE= 0.68Information obtained from the Interface is useful 0.744I learned a lot from using the Interface 0.675I think the information obtained from the Interface is helpful 0.875
Entertainmentα = 0.94; AVE= 0.59Not Fun → Fun 0.942Not enjoyable → enjoyable 0.950Not entertaining at all → Very entertaining 0.954
Social Presenceα = 0.93; AVE= 0.66There is a sense of human contact in the interface 0.712There is a sense of human warmth in the interface 0.811There is a sense of human warmth in the interface 0.910
Sensory AppealAVE = 0.73The product presentation on this interface is lively 0.613This interface contains product information exciting to senses 0.610
Perceived Personalizationα = 0.87 ; AVE = 0.68The Interface can provide me with relevant product recommendations. 0.820The Interface can provide me with product recommendations tailored to my preferences. 0.841The Interface can provide me with personalized product recommendations 0.818
Perceived Social Supportα = 0.91; AVE = 0.58I would have enjoyed interacting with this service agent 0.728This Interface really tries to help me 0.791This Interface gives the help and support needed for shopping on it 0.884This Interface is like a salesperson who is around when I am in need while shopping 0.728This Interface is comforting to me 0.687This Interface is comforting to me 0.766This Interface is willing to help me make decisions 0.762
28
3.2.3 Manipulation checks
The manipulation checks were used in this study to make sure that the two treatments were
correctly manipulated. The effectiveness of the manipulation relied on whether participants were
led to perceive the chatbot as a more personalized interface and which provides higher levels of
social support compared to the website. The manipulation check measures used (i.e., perceived
personalization and perceived social support) were placed directly after the experiment with the
hope of minimizing potential biases. Accordingly, we checked for the homogeneity of variance
and tested for a significant difference between both conditions using Student’s t-tests. Our results
in Table reveal that there is a significant difference in perceived personalization and perceived
social support between the two conditions with a x̄Chatbot=4.82 > x̄Website=3.54) which means that
the chatbot interface higher levels of personalization and social support as against the website
interface. Thus, the treatments were successfully manipulated. ( see Table : Manipulation checks
results
Manipulation checks results
condition nPerceived Personalization Perceived Social Support
Mean SD SE Mean SD SE
Website37 3.549 1.108 .182 3.899 .872 .143
(Control)
Chatbot40 4.825 1.393 .220 4.803 1.283 .202
(Treatment)
Test statistic t = -4.420, p = .000 t = -3.637 , p = .001
29
3.2.4 Hypotheses
Hypothesis 1 stated that the chatbot interface will yield a better customer experience than the
website interface.That is, a better cognitive(a), affective(b), social (c) and sensory (d) experience.
The expected mean difference is to be significant, with reported means for each customer experi-
ence component higher for the chatbot interface group than for the website interface. Therefore,
We conducted an independent samples t-test to determine the effects of the presence (versus ab-
sence)of the chatbot. The results in table show that there is no significant difference between the
groups as for the cognitive(a), affective(b), social (c) and sensory (d) experience. Altogether, there
is no significant difference in the participants’ customer experience between the two conditions.
Hence, we conclude that H1 is not confirmed. ( See Table : Descriptive results and test statistics
for both conditions )
Descriptive results and test statistics for both conditions
condition nInformativeness Entertainment Social Presence Sensory appeal
Mean SD Mean SD Mean SD Mean SD
Website37 5.126 1.202 4.882 1.432 4.342 .266 5.00 1.974
(Control)
Chatbot40 5.150 1.186 5.116 1.600 5.00 .233 4.67 1.671
(Treatment)
Test statistic t = 1.125, p = .264 t = -.824, p = .412 t = -.673, p = .503 t = -.888, p = .930
Hypothesis H1a not confirmed H1b not confirmed H1c not confirmed H1d not confirmed
30
3.2.5 Relationship between variables
Multiple regressions is a relevant method of analysis relationships between a metric dependent
variable and one or more independent variables.Here, to uncover some potential interrelations
between the different variables and constructs the significant models are to be commented in the
following.
First, "Perceived Social Support" and "Perceived Personalization" together were found to have
significant impact on all four customer experience dimensions (p<0.05). Overall the models were
significant, most of which explain more than 10% of the variance. Indeed perceptions of interac-
tivity and personalization positively explain the informativeness, enjoyment, social presence and
sensory appeal of an experience online.
3.3 Discussion
In relation to the manipulation variable, the results suggest that the AI-driven digital assistant
is perceived as a more personalised and "socially supportive" tool when shopping online. Existing
literature has shown that the use of anthropomorphic cues with utilitarian value, such as providing
real-time dialogue, and hedonic value, such as tailored communication and online assistance, pos-
itively influence the shopping experience (Qiu and Benbasat, 2014; Verhagen et al., 2014; Roy and
Naidoo, 2021) , which wasn’t the case in our study. That’s, the online experience of using a chatbot
to navigate a shopping environment and arrive at a desired piece of information wasn’t perceived
as more efficient or enjoyable than the website. Specifically, the chatbot wasn’t able to provide a
better cognitive, affective, social, and sensory experience. This could be due to the fact that the cus-
tomers’ evaluation was altered by the stimulating effect of the website attributes. In other words,
the website where the interaction takes place in the absence of the chatbot and the customers reach
the information by clicking, scrolling or swiping is interspersed with sensory and social informa-
tion, i.e. aesthetic images of different social environments. In line with the literature, the presence
31
of human images (i.e. the human element) is sufficient to arouse senses and create perceptions of
social presence among users (Hassanein and Head, 2007) . Thus, reduce the difference between
the two interfaces (website vs. chatbot) in terms of online customer experiences.
32
CHAPTER 4. STUDY 2
Study 2 is an extension of Study 1, which aimed to investigate the impact of chatbot modality on
customer experience. Specifically, the effects of conversational and visual anthropomorphic cues,
i.e. conversational style and avatar type. The experiment examined conversational style
(social-oriented/contextual vs. task-oriented/menu-based) x avatar type (human vs. robot) in the
context of online shopping.
4.1 Method
This section presents the methodological scheme that will be used to answer the research ques-
tions of this study. The experimental design, sample, and measures used in the post-experiment
questionnaire are described below.
4.1.1 Experiment and stimulus development
To test the hypotheses of Study 2, we conducted an online experiment with a 2 (interaction
style of the digital assistant: social vs. task-oriented) ) × 2 (avatar type: human vs. robot-like)
between-subjects design, where the interaction between the conversational style of the assistant
and the avatar type was the focus of the current study. Participants were randomly assigned to
one of four groups. Accordingly, four simulated virtual agents were designed based on the same
algorithm and interaction scripts as in Study 1, but with human and robotic avatars, respectively,
and with button (menu-based) or contextual (text) interaction styles. A total of 143 participants
were recruited, representing more than 30 subjects in each treatment group. Hair et al.
33
As mentioned above, the stimuli were similar to those in Study 1. The chatbot was set up to
interact with users, answer their questions, and address their concerns in natural language. The
main changes involved manipulations of the avatar and interaction style used in Study 1. First,
instead of using a logo image, we attempted to use avatars that would be perceived as high and low
in humanness. In this case, the avatar was manipulated by either a human-like (high) or a robot-
like (low) image. The human-like treatment photos were selected from an online photo database.
We chose a female human avatar because it is perceived as more competent than the male human
avatar (Pfeuffer, 2019). For the robot-like avatar, we chose a common chatbot avatar as used on
websites.
In contrast to previous operationalizations of virtual agent interaction style in experiments
Chattaraman et al. (2019); Go and Sundar (2019), the development of this manipulation was based
on Chattaraman et al. (2019) with a task-oriented chatbot that uses buttons instead of natural lan-
guage. In other words, to achieve a stronger manipulation of humanness, the task-oriented (menu-
based) virtual agent was designed to interact with the participants using a formal computer-like
language, which means that participants use buttons instead of natural language. The menu-based
chabot focuses on goal achievement, speaks in a goal-oriented manner, and structures the conver-
sation in contrast to the contextual (socially-oriented) bot, which uses informal language, has a
human name (Anika), initiates and concludes the interaction using informal and friendly conver-
sational cues (e.g., "hey there !" "my bad, I’m afraid I didn’t learn that yet") (Chattaraman et al.,
2019).
In summary, the experiment was a conversation style (low and high) × avatar type (low and
high) design. Thus, the interaction style and avatar type served as stimulus material and experi-
mental manipulation in this study. The manipulations were made to affect the perception of hu-
manness. The conversational style was manipulated by using either a social-oriented/contextual
(high) or a task-oriented/menu-based (low) interaction, and on the other hand, the avatar type was
34
manipulated by using a human-like (high) or robot-like (low) image. The four chatbots desgined
for this study are shown in 4.1
4.1.2 Participants and experimental procedure
In all other respects, the procedure was identical to Study 1. Before beginning to interact
with the chatbots, participants read the same set of set of intsructions (study1: chatbot condition).
143 participants (out of 203) took part in the experiment and were randomly assigned to one of
the four experimental conditions. In an identical environment, they were asked to complete the
same shopping task as in Study 1 (searching for one of two cosmetic products) with the help of
Anika (virtual assistant). After subjects were exposed to the chatbot manipulation, they returned
to the questionnaire and answered a different manipulation check question than in our first study.
Subsequently, subjects completed the questions on key variables - cognitive, affective, social, and
sensory experience - which were measured with similar items as in study1.( see Experimental
conditions)
Characteristics %
Gender Female 49.0%
Male 50.3%
Non-binary 0.7%
Age Under 20 years old 2.1%
20 - 40 92.3%
40 - 60 5.6%
Level of education High school graduate 2.1%
Bachelor’s degree 21%
Master’s degree 60.8%
Doctorate degree 16.1%
35
Figure 4.1 Four experimental conditions
36
4.1.3 Measures
4.1.3.1 Independent variable
As mentioned earlier, the independent variable "chatbot modality" was controlled within the
survey by randomising the four different chatbot interfaces (i.e., contextual with human-like avatar;
contextual with robot-like avatar; menu-based with human-like avatar; menu-based with robot-like
avatar).
4.1.3.2 Manipulation check
To assess the hypothesised disparity between the four conditions, Perceived humanness was
measured on a 9-point semantic differential scale adapted from Gnewuch et al. (2018) and executed
the scales developed by , which consist of a nine point semantic differential scale, ranging from
extremely nonhuman to extremely human. This was used to measure the degree of humanness
participants perceived in each of their assigned conditions.
4.1.3.3 Dependent variable
As previously stated, the dependent variable is the same as in Study 1. Customer experience
(affective (enjoyment), cognitive ( informativeness ), social (social presence), and sensory (sensory
appeal)) was assessed using the scale developed by Bleier et al. (2019). The scales are listed in the
table.
4.1.3.4 Control variable
All control measures used in study 1 are used in study 2. Namely, participants’s Technology
proximity, English Proficiency, Familiarity With Natural Cosmetics, Familiarity With Moroccan
Natural Cosmetics, Chatbot Usage A
37
4.2 Analyses and results
This chapter explains the statistical procedures and analyses used to test the hypotheses against
the empirical data collected, as well as the results and comments.
4.2.1 Pre-test
Before the main experiment, we conducted a pretest to ensure that the manipulation was effec-
tive (Chattaraman et al., 2019; De Cicco et al., 2020) We randomly assigned 35 students to one of
the two conditions. We randomly assigned 35 students to one of the conditions. The extent to which
the interaction with the chatbot was perceived as human was measured by asking participants how
much they thought the chatbot was human-like,skilled,polite,thoughtful,responsive,engaging from
Gnewuch et al. (2018) on a 9 point differential scale. The responses were averaged to create a
variable score .As expected, findings revealed that the chatbot was perceived more human in high
conditions with significant main effect and interaction effect ( see table).
4.2.2 Preliminary checks
Several analyses of variance confirmed the successful random assignment to the different ex-
perimental conditions: We did not observe any significant differences in terms of participant’s
Gender, Age, Technology propinquity and Natural cosmetics familiarity across the groups (all
p>0.05), suggesting that these (control) variables did not confound our dependent variables.
Before turning to the research questions and starting the hypothesis testing, there are often
several preliminary analyses to conduct (Sreejesh et al., 2014) . The quality of the data and the
measures used are to be appraised first. Checking the normality assumption is a prerequisite to
many statistical tests. In this instance, A Shapiro-Wilk’s test (p <.05) and a visual inspection of the
histograms showed that the data significantly deviate from a normal distribution. However, our data
are approximately normally distributed, in terms of skewness and kurtosis with values between -
38
2 and 2 for all the items forming our measurement scales(see Appendix), which is considered
acceptable in order to assume normal distribution (George Mallery, 2010) (see table)
Several preliminary analyses are frequently required before moving on to the research topics
and hypothesis testing. (Sreejesh et al., 2014). First, the quality of the data and the measures
used must be examined. Checking the normality assumption is a prerequisite for many statistical
tests. In this case, a Shapiro-Wilk test and visual inspection of the histograms showed that the data
deviated significantly from a normal distribution (p<0.05). However, our data are approximately
normally distributed in terms of skewness and kurtosis, with values ranging from -2 to 2 for all
items making up our measurement scales (see Appendix), which is considered acceptable for the
assumption of a normal distribution (George Mallery, 2010) ( See table ).
Factor analysis conducted for all measures revealed their unidimensionality, with all scales
having acceptable values except for the perceived humanity scale. One item (ph1: extremely
inhuman/extremely human-like) was removed because it had a low loading of 0.53, which is below
the recommended threshold of 0.6. This removal helped to achieve a better reliability of the scale
of 0.91. The AVE calculated thereafter also showed acceptable values above 0.5 for construct
validity. All items showed adequate construct reliability and internal consistency, with all scores
having strong Cronbach’s alphas (ranging from 0.82 to 0.91). The list of variables, factor loadings,
and scale reliabilities with adequate reliability can be found in the Appendix.
4.2.3 Manipulation check
As mentioned earlier, manipulation checks were used in a pilot study to ensure that treatments
were manipulated correctly. The efficacy of the manipulation depended on the extent to which
participants perceived the chatbot as more human-like in the high conditions and less human-like
in the low conditions. As reported in the pretest, there is a significant main effect of avatar type
39
Study 2 : Factor analysisFactor
Construct loading
Informativenessα = 0.92; AVE = 0.66Information obtained from the Interface is useful 0.853I learned a lot from using the Interface 0.782I think the information obtained from the Interface is helpful 0.815
Entertainmentα = 0.91; AVE = 0.76Not Fun → Fun 0.844Not enjoyable → enjoyable 0.840Not entertaining at all → Very entertaining 0.913
Social Presenceα = 0.92; AVE= 0.75There is a sense of human contact in the interface 0.843There is a sense of human warmth in the interface 0.940There is a sense of human warmth in the interface 0.835
Sensory Appealα = 0.82; AVE = 0.67The product presentation on this interface is lively 0.862I can acquire product information on this interface from different sensory channels 0.812This interface contains product information exciting to senses 0.789
Perceived Humannessα = 0.91; AVE = 0,73Extremely unskilled → Extremely skilled 0.846Extremely unthoughtful → Extremely thoughtful 0.837Extremely impolite → Extremely polite 0.896Extremely unresponsive → Extremely responsive 0.867Extremely unengaging → Extremely engaging 0.884
40
and conversational style, and a significant interaction effect of the two factors. This means that the
level of conversational style depends on the level of avatar type. In the main study, the same results
were obtained, with all effects significant at a p <0.05. Thus, we conclude that the treatments were
also successfully manipulated. ( See Figure 4.2 )
Figure 4.2 Perceived humanness interaction effect
4.2.4 Hypotheses
The influence of Conversational style and Avatar type:
A series of two-way analyses of variance (ANOVAs) and pairwise comparisons shown in the
table were conducted to compare the effects of contextual vs. menu-based conversational style
and human-like vs. machine-like avatar on (a) cognitive experience (b) affective experience, (c)
social experience and sensory experience(i.e., customer experience).
41
4.2.4.1 Main effect of Conversational Style
When looking exclusively at the conversational style, graphically, we could see some differ-
ences from the low (menu-based) to the high (contextual) conditions. However, when comparing
the Contextual chatbot with the Menu-based chatbot for each component of CX separately us-
ing Pairwise comparisons with Bonferroni adjustment the differences are not significant with all
p>0.05. Therefore, the main effect for conversational style was not significant for customer expe-
rience.
4.2.4.2 Main effect of Avatar Type
The main effect for type of avatar (human vs. machine-like) was not significant for all compo-
nents of customer experience, despite the differences shown in the graphical representations.Pairwise
comparisons with Bonferroni adjustment indicate that, when comparing the human with the machine-
like separately (human-vs. machine-like agent), the differences are not significant (all p> 0.05).
Thus, the avatar type has no main effect.
4.2.4.3 Interaction effect
Beyond the reported results of main effects, we did not register any effect of a factor that is
contingent upon the level of the other, with all p> 0.05.Although, the interaction between Conver-
sational style * Avatar type was almost nearly significant for the social experience, F=3.4,p=0.07.
Thus, eventually, we conclude that there are no interaction effect for any of the manipulations used
(i.e., Conversational style and Avatar type).
42
Study 2 : Pairwise comparison
Dependant variable Avatar
Chatbot Interaction Style
Interaction effect p-valueTask-oriented Social-oriented
M M
Cognitive experience Robot 5.19 5.60.368
(Informativeness) Human 5.12 5.15
Affective experience Robot 4.74 4.94.724
(Entertainment) Human 4.73 5.03
Social experience Robot 4.12 4.55.074
(Social presence) Human 4.58 4.16
Sensory experience Robot 5.06 5.02.911
(sensory appeal) Human 4.98 4.98
43
Figure 4.3 Perceived humanness interaction effect
4.2.5 Relationship between variables
A series of two-way analyses of covariance (ANCOVAs) were conducted to compare the ef-
fects of human-like vs. robot-like agent and social vs. task oriented interaction style on cognitive,
affective, social and sensory experience with gender, age and chatbot usage were included as co-
variates. the main effects and the other interaction effects results were evaluated. No main effects
emerged for the independent variable, digital assistant interaction style, on any of the outcome
measures. Further, no main or interaction effects emerged all p> 0.05.
Although non-hypothesized, a series of simple linear regression were conducted to explore
the effects of the manipulation variable "Perceived Humanness" on the online customer expe-
rience measures. the results show that perceived humanness significantly impacts informative-
ness (F=11.080,p<0.05), entertainment (F=13.028,p<0.05),sensory appeal (F=8.293,p<0.05). This
44
mean that the more humanized the chatbot seems to people, the better their cognitive, affective and
sensory experience is. Interestingly, the effect of perceived humanity on social presence was al-
most significant (p: 0.052), implying that the perception of how human-like the chatbot is doesn’t
imply that there’s a sense of presence in the interaction environment.
4.3 Discussion
The results of our online experiment suggest that Avatar type ( human-like vs robot-like) and
conversational style positively affect customers perceptions of disembodied virtual agents in the
form of chatbots.The findings, in line with response theory, support the assumption that the more
the CA displays social cues the more "humanized" it will be perceived have a positive effect on
the experience (Verhagen et al., 2014; Danckwerts et al., 2019; Araujo, 2018) . However, the
thought-provoking finding of Study 2 is the fact that perception of humanness of the chatbot do
not necessarily trigger social presence in constrast with the findings of (Jin and Bolebruch, 2009;
Etemad-Sajadi, 2016; Holzwarth et al., 2006; Etemad-Sajadi, 2016). Actually, in comparable re-
search, social presence (i.e., automated social presence) was reported to have better consumer
outcomes (i.e.,conflict mitigation) when the avatar design is low in attractiveness. Along this line,
De Cicco et al. (2020) in his study ascribes the ineffectiveness of avatars in enhancing feelings of
social presence to the chat platform where avatars are not highly visible.Verhagen et al. (2014) sug-
gest that a change in physical appearance does not elicit more social responses and that an increase
in anthropomorphism from robot-like to human-like agents might be too small to find variance.On
the other hand, higher conditions of interaction style was always reported to significantly generate
feelings of social presence Roy and Naidoo (2021).Interestingly, the current research cannot sup-
port this statement. This may partially be explained by the manipulation of task-oriented chatbot
which has taken the form of button rather than a limited formal interaction style as operational-
45
ized in the literature but mainly by the limited contingency (i.e., exchange of responses) of the
conversations as instructed in the experiment.
A
46
.
47
CHAPTER 5. GENERAL DISCUSSION
5.1 Short Summary
In this study, chatbots are considered as a form of ubiquitous AI-powered application that
disrupts the online customer experience. To understand this phenomenon, it is equally necessary
to consider a key element: Anthropomorphism. In a self-developed web environment (website and
chatbot), we conduct two different experimental studies. First, to study the impact of the presence
(or absence) of an AI conversational agent on the customer experience, and second, to deepen
the understanding of which anthropomorphic design cues (interaction style and avatar type) are
more appreciated and improve the customer experience. With this in mind, Study 1 used a post-
test design with a control (website) and a treatment (chatbot) condition, and Study 2 used a 2x2
between-subjects design with 4 experimental conditions (social or task-oriented interaction style,
with either a human or robot-like avatar). Participants were conveniently recruited online and in
both studies and randomly assigned to one of the respective conditions in each study.
In accordance with Chattaraman et al. (2019) and De Cicco et al. (2020), a pre-test was first
conducted to see if the conditions were effectively manipulated. This resulted in some minor
changes to the constructs used to check the manipulations, but also some adjustments to the exper-
imental instructions to improve subjects’ understanding before the main study questionnaire was
created.
The results of Study 1 show that the AI conversational agent provides the same level of in-
formativeness, entertainment, social presence, and sensory appeal as the website. This suggests
that customers respond similarly to a conversational user interface as they do to a graphical user
interface, although there are a number of differences between them. Similarly, no differences
48
were recorded between the four conditions in study 2. The customer experience remain invariable
irrespective of the visual or conversation cue used in chatbots.
Both results could be explained by either (1) the artificial environment in which the shopping
task was performed despite the control of the environment: If customers want to buy something
online, the need must come from themselves and not from an instruction. (2) Customers do not
care how they get to the shopping goal as long as they reach it. (3) Selection and participant bias:
only interested subjects participate and end up giving inaccurate answers These results lead to
implications and suggestions for similar research studies.
5.2 Theoretical Implications of this study
First and foremost, the theoretical and empirical findings acknowledge that despite the emerg-
ing discourse of AI in the field of marketing, the research is still scarce and decentralized with most
theoretical and empirical work on AI and chatbots only addressing the role of anthropomorphism
of digital assistants. The theoretical basis for this is social response theory, which states that hu-
mans mindlessly adopt the same social behaviors when interacting with computers by displaying
social cues, such as interacting with others using natural language. Therefore, no theory has yet
been developed to examine the elements that underline the interaction between users and chatbots
specifically.
Second,both studies conducted on conversational agents and online customer experience have
shown that there’s no significant relationship between the two: The presence of a chatbot doesn’t
improve online customer experience, nor does changing chatbot features and design (anthropomor-
phic cues). This is in contrast to most of the findings in the literature, where virtual agents in their
various forms and in different environments, e.g., Animated avatars (Liew et al., 2017), interactive
3D avatars in virtual environments (Lin et al., 2021), and human-like animated customer service
agents that mimic real salespeople (Verhagen et al., 2014), affect affective, social, cognitive, and
49
functional outcomes, but consistent with these studies, the chatbot actually conveys perceptions of
social support and personalization (Study 1) and humanness (Study 2) that positively impact the
online experience holistically. These mixed results are suspected to originate from differences in
the measurements used or the type of environment. As can be seen from the literature, trust is
an important variable to explore when researching conversational assistants, as users have privacy
concerns (Danckwerts et al., 2019).
Finally, customer experience is one of the most debated topics at the moment. Online customer
experience, however, still lags behind in terms of both conceptualization and operationalization.
Therefore, due to its significant impact on a company’s competitive advantage and the increasing
tendency of customers to use digital channels, OCX should be given more attention by practitioners
and scholars.
5.3 Managerial Implications of this study
The main findings need further discussion, both in terms of the business implications for mar-
keters and the practical implications for AI developers and chatbot designers. The current results
suggest that adding an AI conversational interface to an online store does not lead to better cus-
tomer experiences. This is the case if the website is already interactive and has sensorial and social
stimuli (colors, images, videos). We suggest that the AI interface needs to provide a higher level
of personalization and social support to differentiate itself from other interfaces.The exponential
amount of consumer data makes marketing a natural beneficiary of these evolving technologies
especially as customers’ buying processes increasingly shift to online channels where they can
compare different options (Jarek and Mazurek, 2019).A more proactive use of AI would allow
marketers to leverage rich contextual consumer data, to customize future conversations and pro-
vide frictionless experiences. On the other hand, a better understanding of customer perceptions
of chatbots use will allow chatbot developers and designers to design interfaces that could have
50
a greater impact on the experience, i.e., more testing needs to be done to find out how customers
ascribe meaning to the different anthropomorphic design cues and what kind of responses that
elicits.
In summary, digital assistants are brilliant technologies that provide ongoing support to cus-
tomers and enable businesses to build and maintain relationships with them. AI-based digital
assistants can open up new ways for businesses to reach out to customers, interact with them, and
customize how they communicate with them. However, as artificial intelligence technologies con-
tinue to evolve, the marketing landscape is likely to fundamentally change. Therefore, marketers
need to start developing marketing strategies that engage customers and change their perception of
chatbots and therefore add value to the interactions between businesses and customers.
5.4 Limitations and suggestions for further research
Our current work has several limitations that may stimulate future research. First, we focus
on the study of natural cosmetics and in particular Moroccan natural cosmetics (e.g., argan oil,
prickly pear seed oil). These products are often considered gender-specific and are more popular
among women in general and Moroccan women in particular. In addition, natural cosmetics are
characterized by properties that are difficult to evaluate (e.g. category, benefits, application). Future
research should therefore investigate the impact of AI conversational agents as shopping assistants
in a different context. Since the chatbot was newly developed, it could not be sufficiently trained,
so we could not perform a robustness test to check in which conversations the chatbot failed and
provided wrong answers, so we suggest that more control variables need to be considered when
studying chatbot interactions (e.g. e.g. time of interaction, number of messages exchanged).We
could also add, as mentioned earlier, the fictional task of shopping, which is quite dissimilar to a
pre-purchase experience, and here we claim that similar experiments should be conducted in a less
artificial way. Regarding the large discrepancies found in responses to control variables such as
51
Product Involvement, self-reported measurement should be done with more accurate scales: When
asked about technology affinity, a software engineer and a cell phone user might both receive the
same ratings and thus biased responses.Finally we contend that these type of type of experiments
should be conducted on different generational populations as there will likely be strong differences
in how consumers react to these technologies according to age as it has been shown that age impacts
consumer acceptance and use of information technology (Khatri et al., 2018)
52
Bibliography
Adam, M., Wessel, M., and Benlian, A. (2019). AI-based chatbots in customer service and theireffects on user compliance. page 19.
Ameen, N., Tarhini, A., Reppel, A., and Anand, A. (2021). Customer experiences in the age ofartificial intelligence. Computers in Human Behavior, 114:106548.
Araujo, T. (2018). Living up to the chatbot hype: The influence of anthropomorphic design cuesand communicative agency framing on conversational agent and company perceptions. Com-puters in Human Behavior, 85:183–189.
Becker, L. and Jaakkola, E. (2020). Customer experience: fundamental premises and implicationsfor research. J. of the Acad. Mark. Sci., 48(4):630–648.
Berry, L. L., Carbone, L. P., and Haeckel, S. H. (2002). Managing the Total Customer Experience.MIT Sloan Management Review, page 8.
Bleier, A., Harmeling, C. M., and Palmatier, R. W. (2019). Creating Effective Online CustomerExperiences. Journal of Marketing, 83(2):98–119. Publisher: American Marketing Association.
Chattaraman, V., Kwon, W.-S., and Gilbert, J. E. (2012). Virtual agents in retail web sites: Benefitsof simulated social interaction for older users. Computers in Human Behavior, 28(6):2055–2066.
Chattaraman, V., Kwon, W.-S., and Gilbert, J. E. (2019). Should AI-Based, conversational digitalassistants employ social- or task-oriented interaction style? A task-competency and reciprocityperspective for older adults. Computers in Human Behavior, page 16.
Ciechanowski, L., Przegalinska, A., Magnuski, M., and Gloor, P. (2019). In the shades of theuncanny valley: An experimental study of human–chatbot interaction. Future Generation Com-puter Systems, 92:539–548.
Danckwerts, S., Meißner, L., and Krampe, C. (2019). Examining User Experience of Conversa-tional Agents in Hedonic Digital Services – Antecedents and the Role of Psychological Owner-ship. SMR, 3(3):111–125.
De Cicco, R., e Silva, S. C., and Alparone, F. R. (2020). Millennials’ attitude toward chatbots: anexperimental study in a social relationship perspective. IJRDM, 48(11):1213–1233.
53
De Keyser, A., Verleye, K., Lemon, K. N., Keiningham, T. L., and Klaus, P. (2020). Moving theCustomer Experience Field Forward: Introducing the Touchpoints, Context, Qualities (TCQ)Nomenclature. Journal of Service Research, 23(4):433–455.
Eisend, M. and Kuss, A. (2019). Research Methodology in Marketing: Theory Development,Empirical Approaches and Philosophy of Science Considerations. Springer International Pub-lishing, Cham.
Etemad-Sajadi, R. (2016). The impact of online real-time interactivity on patronage intention: Theuse of avatars. Computers in Human Behavior, page 6.
Følstad, A. and Brandtzaeg, P. B. (2020). Users’ experiences with chatbots: findings from aquestionnaire study. Qual User Exp, 5(1):3.
Gao, W., Li, W., Fan, H., and Jia, X. (2021). How customer experience incongruence affectsomnichannel customer retention: The moderating role of channel characteristics. Journal ofRetailing and Consumer Services, 60:102487.
Gentile, C., Spiller, N., and Noci, G. (2007). How to Sustain the Customer Experience:. EuropeanManagement Journal, 25(5):395–410.
Gnewuch, U., Morana, S., Adam, M. T. P., and Maedche, A. (2018). FASTER IS NOT AL-WAYS BETTER: UNDERSTANDING THE EFFECT OF DYNAMIC RESPONSE DELAYSIN HUMAN-CHATBOT INTERACTION. page 18.
Gnewuch, U., Morana, S., and Maedche, A. (2017). Towards Designing Cooperative and SocialConversational Agents for Customer Service. page 14.
Go, E. and Sundar, S. S. (2019). Humanizing chatbots: The effects of visual, identity and conver-sational cues on humanness perceptions. Computers in Human Behavior, 97:304–316.
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., and Tatham, R. L. (2006). MultivariateData Analysis: Pearson Prentice Hall. Upper Saddle River, NJ, pages 1–816.
Hassanein, K. and Head, M. (2005). The Impact of Infusing Social Presence in the Web Inter-face: An Investigation Across Product Types. International Journal of Electronic Commerce,10(2):31–55.
Hassanein, K. and Head, M. (2007). Manipulating perceived social presence through the webinterface and its impact on attitude towards online shopping. International Journal of Human-Computer Studies, 65(8):689–708.
54
Holbrook, M. B. and Hirschman, E. C. (1982). The Experiential Aspects of Consumption: Con-sumer Fantasies, Feelings, and Fun. Journal of Consumer Research, 9(2):132–140. Publisher:Oxford University Press / USA.
Holzwarth. The influence of avatars.
Holzwarth, M., Janiszewski, C., and Neumann, M. M. (2006). The Influence of Avatars on OnlineConsumer Shopping Behavior. Journal of Marketing, 70(4):19–36.
Hussain, R. Personalization, Customer Experience, and AI. page 4.
Jain, M., Kota, R., Kumar, P., and Patel, S. N. (2018). Convey: Exploring the Use of a ContextView for Chatbots. In Proceedings of the 2018 CHI Conference on Human Factors in ComputingSystems - CHI ’18, pages 1–6, Montreal QC, Canada. ACM Press.
Jarek, K. and Mazurek, G. (2019). Marketing and Artificial Intelligence. CEBR, 8(2):46–55.
Jin, S.-A. A. and Bolebruch, J. (2009). Avatar-Based Advertising in Second Life: The Role of Pres-ence and Attractiveness of Virtual Spokespersons. Journal of Interactive Advertising, 10(1):51–60.
Josko, B. (2009). Brand Experience: What Is It? How Is It Measured? Does It Affect Loyalty?page 18.
Kaatz, C., Brock, C., and Figura, L. (2019). Are you still online or are you already mobile? –Predicting the path to successful conversions across different devices. Journal of Retailing andConsumer Services, 50:10–21.
Kaplan, A. and Haenlein, M. (2019). Siri, Siri, in my hand: Who’s the fairest in the land? Onthe interpretations, illustrations, and implications of artificial intelligence. Business Horizons,62(1):15–25.
Khatri, C., Venkatesh, A., Hedayatnia, B., Gabriel, R., Ram, A., and Prasad, R. (2018). AlexaPrize — State of the Art in Conversational AI. AIMag, 39(3):40–55.
Kim, Y. and Sundar, S. S. (2012). Anthropomorphism of computers: Is it mindful or mindless?Computers in Human Behavior, 28(1):241–250.
Klaus, P., Gorgoglione, M., Buonamassa, D., Panniello, U., and Nguyen, B. (2013). Are youproviding the “right” customer experience? The case of Banca Popolare di Bari. Intl Jnl of BankMarketing, 31(7):506–528.
55
Klopfenstein, L. C., Delpriori, S., Malatini, S., and Bogliolo, A. (2017). The rise of bots: A surveyof conversational interfaces, patterns, and paradigms. In Proceedings of the 2017 conference ondesigning interactive systems, pages 555–565.
Komiak and Benbasat (2006). The Effects of Personalization and Familiarity on Trust and Adop-tion of Recommendation Agents. MIS Quarterly, 30(4):941.
Kuppelwieser, V. G. and Klaus, P. (2021). Measuring customer experience quality: The EXQ scalerevisited. Journal of Business Research, 126:624–633.
Lemon, K. N. and Verhoef, P. C. (2016). Understanding Customer Experience Throughout theCustomer Journey. Journal of Marketing, 80(6):69–96.
Li, T. and Unger, T. (2012). Willing to pay for quality personalization? Trade-off between qualityand privacy. European Journal of Information Systems, 21(6):621–642.
Liew, T. W., Tan, S.-M., and Ismail, H. (2017). Exploring the effects of a non-interactive talkingavatar on social presence, credibility, trust, and patronage intention in an e-commerce website.Hum. Cent. Comput. Inf. Sci., 7(1):42.
Lin, Y.-T., Doong, H.-S., and Eisingerich, A. B. (2021). Avatar Design of Virtual Salespeople: Mit-igation of Recommendation Conflicts. Journal of Service Research, 24(1):141–159. Publisher:SAGE Publications Inc.
Maklan, S. and Klaus, P. (2011). Customer Experience: Are We Measuring the Right Things?International Journal of Market Research, 53(6):771–772. Publisher: SAGE Publications.
Malhotra, N. K., Nunan, D., and Birks, D. F. (2017). Marketing research: an applied approach,5th ed. Pearson Education Limited. Accepted: 2020-10-22T01:13:40Z Journal Abbreviation:Revised edition of Marketing research, 2012.
Moriuchi, E. (2020). Engagement with chatbots versus augmented reality interactive technologyin e-commerce. page 16.
Nass, C. and Moon, Y. (2000). Machines and Mindlessness: Social Responses to Computers. JSocial Isssues, 56(1):81–103.
Novak, T., Hoffman, D., and Yung, Y.-F. (2000). Measuring the customer experience in onlineenvironments: A structural modeling approach. Marketing Science, 19(1):22–42.
Pine 2nd., B. and Gilmore, J. (1998). Welcome to the experience economy. Harvard businessreview, 76(4):97–105.
56
Poulsson, S. and Kale, S. (2004). The Experience Economy and Commercial Experiences. ERA -Social, Behavioural and Economic Sciences, 4.
Prentice, C., Weaven, S., and Wong, I. A. (2020). Linking AI quality performance and customerengagement: The moderating effect of AI preference. International Journal of Hospitality Man-agement, 90:102629.
Pérez, J. Q., Daradoumis, T., and Puig, J. M. M. (2020). Rediscovering the use of chatbots ineducation: A systematic literature review. Computer Applications in Engineering Education,28(6):1549–1565. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/cae.22326.
Qiu, L. and Benbasat, I. (2010). A study of demographic embodiments of product recommendationagents in electronic commerce. International Journal of Human-Computer Studies, 68(10):669–688.
Qiu, L. and Benbasat, I. (2014). Evaluating Anthropomorphic Product Recommendation Agents:A Social Relationship Perspective to Desig. page 39.
Rose, S., Clark, M., Samouel, P., and Hair, N. (2012). Online Customer Experience in e-Retailing:An empirical model of Antecedents and Outcomes. Journal of Retailing, 88(2):308–322.
Roy, R. and Naidoo, V. (2021). Enhancing chatbot effectiveness: The role of anthropomorphicconversational styles and time orientation. Journal of Business Research, 126:23–34.
Schmitt, B. (1999). Experiential Marketing. Journal of Marketing Management, 15(1-3):53–67.
Schuetzler, R., Grimes, M., Giboney, J., and Buckman, J. (2014). Facilitating Natural Conversa-tional Agent Interactions: Lessons from a Deception Experiment. Human Computer Interaction,page 17.
Sreejesh, S., Mohapatra, S., and Anusree, M. R. (2014). Business Research Methods. SpringerInternational Publishing, Cham.
Verhagen, T., van Nes, J., Feldberg, F., and van Dolen, W. (2014). Virtual Customer ServiceAgents: Using Social Presence and Personalization to Shape Online Service Encounters. JComput-Mediat Comm, 19(3):529–545.
Verhoef, P., Lemon, K., Parasuraman, A., Roggeveen, A., Tsiros, M., and Schlesinger, L. (2009).Customer Experience Creation: Determinants, Dynamics and Management Strategies. Journalof Retailing, 85(1):31–41.
Wang, L. C., Baker, J., Wagner, J. A., and Wakefield, K. (2007). Can A Retail Web Site be Social?Journal of Marketing, 71(3):143–157.
57
Waqas, M., Hamzah, Z. L. B., and Salleh, N. A. M. (2021). Customer experience: a systematic lit-erature review and consumer culture theory-based conceptualisation. Manag Rev Q, 71(1):135–176.
Westerman, D., Cross, A. C., and Lindmark, P. G. (2019). I Believe in a Thing Called Bot:Perceptions of the Humanness of “Chatbots”. Communication Studies, 70(3):295–312.
Wirtz, J., Patterson, P. G., Kunz, W. H., Gruber, T., Lu, V. N., Paluch, S., and Martins, A. (2018).Brave new world: service robots in the frontline. Journal of Service Management, 29(5):907–931. Publisher: Emerald Publishing Limited.
Yang, G., Ji, G., and Tan, K. H. (2020). Impact of artificial intelligence adoption on online returnspolicies. Ann Oper Res.
Zhang, J., Oh, Y. J., Lange, P., Yu, Z., and Fukuoka, Y. (2020). Artificial Intelligence ChatbotBehavior Change Model for Designing Artificial Intelligence Chatbots to Promote Physical Ac-tivity and a Healthy Diet: Viewpoint. J Med Internet Res, 22(9):e22845.
58
APPENDIX A. Confound checks
Study 1 Study 2
Gender t= 1.621 ; p=0.109 F= 0.389 ; p=0.761
Age t=-0.781 ; p=0.437 F=0.177 ; p=0.912
Education t=0.368 ; p=0.714 F=0.614 ; p=0.607
English t=0.608 ; p=0.545 F=0.557 ; p=0.644
New Technologies t=0.080 ; p=0.937 F=0.189 ; p=0.904
Natural Cosmetics t=-1.421 ; p=0.159 F= 1.343 ; p=0.263
Moroccan Cosmetics t=-1.018 ; p=0.312 F=1.370 ; p=0.255
Chatbot Usage t=0.941 ; p=0.350 F=2.527 ; p=0.060
Brand Recognition t=0.794 ; p=0.430 F=1.176 ; p=0.321
Note : The mean difference between the two treatments in Study1
and across the four conditions in Study2 is not significant (p > .05)
59
APPENDIX B. Normality checks
Study 1
Scale ItemsShapiro-Wilk
Skewness KurtosisStatistic p-value
Informativeness inf1 .809 .000 -1.473 1.205
inf2 .842 .000 -1.258 1,573.
inf3 .845 .000 -1.323 1.372
Entertainment ent1 .916 .000 -.537 -.190
ent2 .917 .000 -.664 0.69
ent3 .929 .000 -.501 -0.17
Social Presence sp1 .908 .000 -.657 -.264
sp2 .897 .000 -.657 -.232
sp3 .920 .000 -.577 -.191
Sensory Appeal sa1 .891 .000 -.843 .460
sa2 .922 .000 -.546 .211
sa3 .921 .000 -.627 .297
Perceived Humanness ph1 .809 .000 -.289 -.583
ph2 .961 .000 -.224 -.028
ph3 .963 .001 -.185 -.198
ph4 .950 .000 -.421 -.237
ph5 .956 .000 -.313 -.511
ph6 .959 .000 -.291 -.501
60
Study 2
Scale ItemsShapiro-Wilk
Skewness KurtosisStatistic p-value
Informativeness inf1 .874 .000 -.916 .629
inf2 .935 .001 -.407 -.345
inf3 .873 .000 -.944 .771
Entertainment ent1 .909 .000 -.713 .316
ent2 .896 .000 -.629 -.313
ent3 .901 .000 -.775 .284
Social Presence sp1 .918 .000 -.515 -.596
sp2 .919 .000 -.497 -.640
sp3 .922 .000 -.480 .642
Sensory Appeal sa1 .903 .000 -.763 .066
sa2 .921 .000 -.284 .650
sa3 .908 .000 -.467 -.623
Perceived Personalization per1 .924 .000 -.380 -.798
per2 .928 .000 -.221 -1.014
per3 .927 .000 -.916 -1.024
Perceived Social Support soc1 .942 .002 -.160 -1.024
soc2 .933 .001 -.095 -.818
soc3 .923 .000 -.326 .679
soc4 .943 .002 -.513 -.101
soc5 .946 .003 -.127 -.871
soc6 .942 .002 -.037 -.761
soc7 .921 .000 -.480 -.587
61
APPENDIX C. Descriptive statistics of control variables
Controls Study 1 Study 2
N=77 N=143
Mean SD Mean SD
Proficiency in the English Language 4.55 0.68 4.55 0.64
Familiarity with New technologies 4.51 0.73 4,51 0,70
Familiarity with Natural Cosmetic Products 3.29 1.22 3.62 1.05
Familiarity with Moroccan Cosmetic Products 3.13 1.30 3.41 1.17
Chatbot Usage 2.82 1.33 2.78 1.09
62
Technique Synopsis Example
Response
Feedback
Acknowledge a user’s responses
and provide positive feedback
to encourage information disclo-
sure
User: "I find all of this very interest-
ing"
Anika: "Thank you. We really appre-
ciate the interest."
Response
Prompting
Ask for more information User: "Do you have hair products ?"
Anika: "Would you be interested in a
specific category of products ?"
Anika: "Such as Serum , Styler or
maybe Shampoo ?"
Response
Probing
Probe answers to elicit deeper
information
User: "Please add sesame oil to my
bag"
Anika: "Sesame Oil comes in 2 sizes :
50ml and 100ml"
Anika: "Which one should i add to
your bag ?"
Social
Acknowledgement
Acknowledge a user’s input to
convey understanding and ver-
balize emotion
User: "I am doing great"
Anika: "Happy to hear that"
Handle
conversation
flow
Probe answers to elicit deeper
information
User: "Can we restart ??"
Anika: "Alright. Sorry if i’m being
confusing."
Anika: "Shall we start over ?"
Handle User
Excuses
Handle a user’s excuses not to
answer a question and encourage
continuation
User: "I’m sorry"
Anika: "Oh no it’s fine"
Anika: "May i still be of help ?"
63