Using Affective Embodied Agents in Information Literacy Education

Yan Ru Guo, Dion Hoe-Lian Goh, Brendan Luyt Wee Kim Wee School of Communication and Information

Nanyang Technological University 31 Nanyang Link, Singapore 637718

{w120030, ashlgoh, brendan}@ntu.edu.sg ABSTRACT This study aims to evaluate the impact of affective embodied agents (EAs) on students’ learning performance in an online tutorial that teaches academic information seeking skills. A hundred and twenty tertiary students from two major universities participated in the between-subjects experiment. The results suggested that the use of affective EAs significantly increased students’ learning motivation and enjoyment, compared to neutral-EAs or text-only conditions. However, there were no significant differences in knowledge retention between the three groups. This study paves the way for a better understanding of embedding affective EAs in online information literacy (IL) education. Furthermore, the improvement in students’ learning motivation and enjoyment can serve as a basis for future research in this context.

Categories and Subject Descriptors J.0 [Computer Applications]: General.

K.3.1 [Computers and Education]: Computer Uses in Education - Computer-assisted instruction (CAI)

General Terms Measurement, Performance, Design, Reliability, Experimentation, Human Factors, Theory, Verification.

Keywords Affect, affective agents, embodied agents, emotions, enjoyment, information literacy, information seeking, Information Search Process (ISP), knowledge retention, motivation

1. INTRODUCTION In the information society, it has become commonplace to accept that IL is a basic skill that people should equip themselves with. With the proliferation and accessibility of information made possible by the Internet, there is an increasing need for IL in order to effectively find, use, and evaluate information to meet specific needs. The fast speed of communication and easy access to information brings about the issue of information accuracy and reliability, where people need to carefully assess the varying quality of content before putting it into use [1]. Therefore, the ability to seek, acquire, navigate and evaluate information is

critical to knowledge workers, students and the general public alike, as such skills can help people navigate vast amounts of information, and effectively make use of content that is relevant.

At the individual level, IL has been progressively recognized for its value not only in academia, but also in everyday life, as such skills are indispensable for obtaining new information and constructing new knowledge. At the societal level, IL has been recognized as central to the practice of democracy and citizenship, and to the mission of developing lifelong learners [2].

There is today a growing consensus on the need for IL and a sense of urgency about its implementation [3]. IL education has become the shared responsibility of all educators and information providers. The Association of College and Research Libraries identified six tasks that an information literate person is able to do, namely, to determine the extent of information needed, to access the needed information effectively and efficiently, to evaluate information and its sources critically, to incorporate selected information into one’s knowledge base, to use information effectively to accomplish a specific purpose, and to understand the economic, legal, and social issues surrounding the use of information, and access and use information ethically and legally. Among the six tasks, the present study will focus on the aspect of accessing the needed information effectively and efficiently, in other words, information seeking. Information seeking refers to the purposive and intentional effort to acquire information, in order to satisfy some need, and it is a basic activity in which everyone participates on regular basis [4].

The need for IL education is especially important for university students, as they encounter situations where they need to search for information from multiple sources for various knowledge-intensive tasks more frequently. Librarians, among all professionals, have always been the ones that teach students and the general public about IL skills. However, it has been argued that people who grow up with digital technologies are unwilling to initiate interaction with librarians when searching for academic resources. They prefer to use email, social media and search engines instead, despite the uncertain quality and reliability of such information [5].

The above problem does not pertain only to IL education, but to education in general. The differences in the expectations between students and their teachers had already existed in 1967, when [6] pointed out that the generation gap is a fundamental gap between “a generation bred on the book and a generation bred on the tube and related forms of electronic communication” (p. 21). This observation is truer today than it was in 1967 as the young generation of today grows up with digital technologies, which have become their primary means of living and communicating. Their pervasive use of digital technologies has led them to expect

978-1-4799-5569-5/14/$31.00 ©2014 IEEE.

all forms of communication, including learning, to involve immersive experiences that engage all their senses. This puts an increasing challenge on educators about how to educate young people in today’s classrooms. At the same time, it also presents new opportunities to engage the youth.

It has been observed that in conventional learning environments, expert tutors pay as much attention and spend as much time to help students achieve affective and emotional goals when tutoring, as they do to help them achieve of cognitive and informational ones [7]. Because of the important role that emotions play in learning, researchers have, over the last few years, attempted to factor in the learners’ affective states when designing educational systems [8, 9]. This echoes recent research in affective computing, where scholars have called for the design of systems and devices that can recognize, interpret, process, and stimulate human affect, as well as express affects [10]. One of the common ways to achieve this is the use of EAs in interface design.

In the computing literature, the term “agent” refers to an autonomous computer program that can “act” on its own [11]. An EA thus refers to a life-like agent, i.e., one with a physical face and body [12]. In the last decade, the notion of embodiment exerted a great influence on computer science as researchers found that the “disembodied” approach of logic and abstract symbols in computer systems was unable to achieve desirable outcomes. In view of this, the subsequent shift towards embodiment, including embodiment of agents, has brought great success to computer science [13]. EAs are appraised to have transformed our experience of interacting with computers by explicitly referencing human interactions [14].

It is also believed that EAs’ ability to detect and express affective states is crucial for improving their believability, eliciting emotions in the users, as well as contributing to more entertaining interactions [15]. Accordingly, an affective EA is defined as one that is capable of eliciting certain emotional experiences from users through multiple modalities such as speech, facial expressions and body gestures [13]. The embodied agent REA that was designed by [13] is one example of an affective EA.

Studies have found that using EAs can make the interactions between computers and humans more natural [16]. The manipulation of an EA’s affective states can significantly influence learners’ learning motivation and self-efficacy [17]. Another function of affect associated with failed expectations is to direct attention to the proceeding or accompanying events as important lessons to be learnt [18]. Therefore, the timely intervention from instructors could greatly enhance the learning experience and effectiveness. In particular, the use of affective EAs in a pedagogical role such as an instructor, mentor, assistant, and companion, has been found to not only increase students’ learning motivation and perceived self-efficacy as mentioned above, but also help students overcome negative emotions such as boredom or frustration during learning process [19].

However, despite the initial spurt of research on affective EAs in education, a number of research gaps remain. They motivate the present study, and are presented as follows.

First, although much research has been conducted on the use of EAs in online educational systems to teach lower order thinking skills, facts, concepts, and procedures, there has been little on higher order thinking skills, such as how to apply, analyze, or evaluate knowledge [20]. Among the limited initiatives to design

educational systems that teach higher order thinking skills, many were not brought to completion, with some projects getting jettisoned, and others prototyped, tested, but finally terminated. IL education involves higher order thinking skills; it teaches important procedural knowledge that synthesizes complex level of thinking and knowledge [21]. In addition, with the proliferation of search engines, social media, and other communication tools, methodologically rigorous research to understand exactly what is involved when searching for information has become more important, yet more confusing than ever. It is now recognized that the information seeking process is not the well-defined, rules-driven process it has been portrayed to be, but is in fact, a lot messier [22].

Next, most research on EAs has focused on their design and technical implementation, as well as theoretical postulations about their benefits. Little empirical evidence to support their positive impact on students’ learning performance has been reported so far [23, 24]. Additionally, research findings have not conclusively shown that the use of affective EAs can improve learning performance [25], with some reporting positive results and others, negative. For example, [26] reported that students’ attitudinal knowledge can be enhanced when agents have affective facial expressions. On the other hand, [27] reported that the presence of affective EAs did not make any difference in students’ learning outcomes or perceived motivation. In addition, methodologically, there is a lack of research to evaluate affective EAs against neutral ones [9].

Based on the research gaps identified above, the objectives of this study are two-fold:

1. To design an online tutorial to teach IL skills to university students, by incorporating affective EAs; and

2. To evaluate the impact of the affective EAs on the learning performances of students.

The paper proceeds as follows. First, we will review extant literature on the use of affective EAs in the online educational context. Next, a three-phase research method will be presented. The first phase involved the design of an information seeking tutorial. In the second phase, two sets of EAs, each consisting a teacher EA and a student EA, were created and integrated into variant tutorials: affective-EAs, neutral-EAs (no affective expressions), in addition to the control condition with no EAs (text-only). In third phase, we evaluated the effectiveness of using EAs in online learning, where participants were recruited to view one of the tutorials and complete a survey questionnaire. The paper concludes with a discussion of the results.

2. LITERATURE REVIEW 2.1 Information Literacy Education The notion of IL has been around for many decades. However, only in the 1990s has it caught public attention when American Library Association released the influential “Presidential Committee on Information Literacy” report. This called for more attention to IL [21]. The association suggested that people not only need a knowledge base, but also need skills for exploring it, connecting it to other knowledge bases, and making practical use of it. In other words, people need IL skills. The emphasis and standards of IL have been growing over the decades. For example, an early definition given by [28] referred to IL as “the ability to use techniques and skills for the wide range of information tools as well as primary sources in molding information-solutions” (p. 2). In 2000, the American Library Association extended the

standards to emphasize that an information literate person is able to exert greater control over his/her own learning, which echoes the call to life-long learning [36].

While most IL definitions apply to the general public, there are additional requirements specifically for students, for whom the quality of the information they find determines the quality of the scholarship they produce. For example, it is expected that information-literate students not only be proficient in reading, thinking, learning, and communicating skills, but also to be motivated and responsible [29]. Due to the widespread use of the Internet among the younger generation, [30] expanded information literacy to include electronic searching and information retrieval skills as well. In addition, students, especially university students, frequently face the task of writing academic papers, and therefore, the ability to search for reliable sources of information in various academic databases and journals, to narrow down the search focus and select an appropriate topic based on available information, to summarize different ideas and synthesize to a paper within limited time, are of great importance to their academic performance. It also involves extending their own knowledge base to create new perspectives.

Similarly, [31] likened the collapse of conventional schooling as the fall of Berlin Wall, and cautioned that traditional linear schooling is progressively being replaced by new digital learning environments. This new environment poses new challenges and requires students to learn new skills. They need “the ability to pick out reliable sources from an overwhelming heap of misinformation, to find relevant material amid an array of options, to navigate the shifting ethics of creative commons and intellectual property rights and to present conclusions in a manner that engages modern audiences” [32].

Although most young people today, the so-called digital natives, are adept at sending emails, downloading music or watching movies online, the majority have little idea on how to effectively seek credible sources of information [1]. In a report named Researchers of Tomorrow, which is UK’s largest study on the research behaviors of Generation Y (born between 1982 and 1994) to date, revealed that they are insufficiently trained to take full advantage of the latest opportunities in the digital information environment, such as the rich digital library collections [34]. From both observation and literature, although libraries regularly organize workshops, student turnouts have been very low, perhaps because students want personally-customized, just-in-time assistance [21].

All these developments have created new demands on librarians, but concurrently, new opportunities and new directions are available for librarians to deliver IL instruction. Among many models in use for IL instruction, the ISP stands out as a user-centered approach as it begins with statements of the informational need articulated by information seekers [22]. This model is also one of the few theoretical models that has been empirically verified by numerous studies [21]. The next section will review the original study on ISP model by [35], as well as follow-up studies by others.

2.2 Information Search Process Model The role of emotions and affect has become increasingly important in the research of information seeking behavior [36]. As learning partly involves searching for information, and making sense of the new material encountered, it is believed that the understanding of the learning process will be greatly enhanced if

the learners’ emotional and cognitive states are better understood [8]. One of the seminal works on the role of emotions in information seeking is the ISP model.

Kuhlthau’s ISP model takes into consideration the information seeker’s affective states. It predicts that information seekers could experience positive affective states such as confidence and assurance, but also negative affective states such as anxiety and frustration. For these reasons, the ISP model deserves special attention and is highlighted here. According to [18], negative emotions could arise when the results deviated from our expectations. This is the case with information seeking process. Inspired by the personal construct theory, [35] regarded information seeking as a constructive, vigorous process, which involves not only thoughts and actions, but also feelings.

The ISP model claimed that in the early stages of searching, negative feelings are common, especially when the user has little knowledge of what was available or when the search problem was not clear [35]. However, as the search progresses, and the awareness of the process increases, there is a corresponding improvement in the level of satisfaction and confidence. At the end of the search process, the seeker will feel a sense of relief or satisfaction when the required information is found, or disappointment and anxiety when it was not. The ISP model also asserted that the search process is iterative, and information seekers can repeat previous steps to ensure that the materials retrieved are relevant as their requirements become clearer. A number of studies have shown that the level of anxiety and discomfort will increase up when people seek information, especially for novice information seekers. In fact, [35] regarded anxiety as an integral part of the information seeking process, and such anxiety is often associated with uncertainty and confusion.

However, Kuhlthau’s main studies were carried out in the 1990s and early 2000s, when most information seeking happened in offline environments. With the proliferation of online search engines and digital databases, there is a need to investigate the information seeking behaviors in online environment as well. Kuhlthau’s work has inspired others to take on the task.

For example, [37] investigated the cognitive, physical and affective searching behaviors of young children on a specific search task when using the Yahoo! search engine. The children experienced positive feelings such as enjoyment and confidence, as well as negative feelings such as confusion and frustration. Further, the study found that the children’s searching process was nonlinear, and they frequently shifted between different searching techniques, looped various hyperlinks, and backtracked within the websites. In another study by [22], participants were required to write a research paper. The post-test suggested that problems are associated not only with the cognitive tasks to search for information, but also the emotional aspects of searching. The study found that students experienced a rise in anxiety level during the information seeking process, but a reduction when the searching was concluded.

In summary, the ISP model has been empirically verified in various studies over the course of thirty years, in both online and offline settings, in both library context and other informational setting, and can serve as a holistic model for librarians to understand and teach information seeking skills [22]. For the purpose of this study, the ISP model sheds light on a possible means of easing negative emotions that are associated with information searching. To the best of our knowledge, there has been no attempt to apply the ISP model in an online IL

educational system context. This may be an important way forward in the teaching of IL as online education holds special attractions to young students.

2.3 Affective Embodied Agents Studies have found that people tend to interact with computers in the same way as they do with people [38]. EAs, which are “animated or static entities that are displayed on a computer screen and attempt to interact with users in some way” [9], could make the interaction between humans and computers more interesting and engaging. Facial expressions, speech, and deictic gestures such as pointing with arms or hands and nodding of the head are common ways through which EAs communicate with users. Extending the definition of affective computing [10], we define affective EAs to be those that can detect and express emotion.

Explorations into EAs have become a burgeoning research area. By providing visual clues of their operation, well-designed EAs could enrich one’s learning experience, and make it easier to attract people’s attention [13]. Interfaces that used EAs have been praised as the ultimate interface [13]. Reeves and Nass also contributed to the fundamental understanding of EAs in computer-mediated communication [38]. They contended that our interaction with computers could evoke a sense of inter-subjectivity, encouraging us to respond to computers in fundamentally social ways, just like in human-to-human communication [38]. Their argument can also be applied to learners’ interactions with EAs, as learners could interact with EAs as in a natural communication context. A growing number of research papers have been written on the use of affective EAs to promote interactive learning [12, 24, 39].

In one study, a Wakamaru robot was used as an affective EA to provide feedback to users in a speed-reading test [19]. The experiment employed a 2 (no agent vs. agent) × 2 (no praise vs. praise) × 3 (no comparison vs. positive comparison vs. negative comparison) factorial design, with 192 participants. The results showed that participants who received positive feedback not only played significantly more rounds of the test, but also reported significantly higher ratings of intrinsic motivation (an indication of their interests and persistence), than participants who received no positive feedback. In addition, participants who were given the test with the affective EA reported significantly higher levels of motivation than those who were given the test without the EA. As seen from the affective EA design, even a character as rudimentary as a Wakamaru robot succeeded in generating a significant effect on motivation. This is an indication of the potential for incorporating affective EAs as motivational and persuasive tools.

Yet other studies have been conducted to investigate the impact of providing other affective expressions, such as politeness and empathy, on students’ motivation and performance. These have provided more evidence that students can benefit from pedagogical characters that express emotions. For example, [40] investigated the effect of politeness in pedagogical agents to enhance communication and students’ learning performance. They created a polite EA in a factory modeling and simulation system to teach students product inventory and management. Seventeen students were divided into two groups with different politeness strategies from the agent: direct and polite. The polite tutor encouraged students by emphasizing on success, and suggested collaboration with students when the student failed. The experimental results suggested that politeness in the EA can lower

students’ perception of difficulty of the learning material and make the presence of a tutor less intimidating. This finding indicates the importance of politeness in designing an EA to facilitate learning experience.

More importantly, it has been found that the manipulation of an EA’s affective states in educational systems can significantly influence learning motivation and self-efficacy [17]. In particular, the use of affective EAs in a pedagogical role such as an instructor, mentor, assistant, and companion, has been found to not only increase students’ learning motivation and perceived self-efficacy as mentioned above, but also help students overcome negative emotions such as boredom or frustration during learning process [19, 41]. In [41], an affective-support EA was designed to actively support users to recover from negative emotional states. Behavioral results suggested the use of affective EAs can ease users’ negative emotional states, compared with text-only conditions. The ability of affective EAs to reduce negative emotions is especially important in the context of this study, which investigates the use of affective EAs to ease potential negative emotions arising from information seeking.

As shown above, there are two major gaps in current literature on EAs in the online educational context. First, extant literature focuses mainly on the use of affective EAs to teach factual knowledge in the areas of mathematics, history, and science. There is little research done to investigate the use of affective EAs to teach procedural knowledge, which is more difficult to teach than factual knowledge [20]. In our work, we contend that the information seeking process is considered important procedural knowledge that should be taught to students. Second, a majority of the literature has focused on the design and implementation of EAs. There is a lack of empirical evaluation of their effectiveness, especially on the way EAs influence learners’ performance. Therefore, this study addresses the two gaps by designing and developing an online tutorial that teaches the information seeking process, and evaluates students’ learning performance. This is done by comparing affective-EAs with neutral-EAs and text-only condition.

3. METHODOLOGY In order to shed light on the mixed findings on the effectiveness of EAs, we first embarked on creating an online tutorial using Adobe Flash, to teach IL to university students, and to gather user data to fulfil the objectives of this study. Although online learning systems that teach mathematics, history, or computer literacy have been available for some time [12, 23], the notion of affective EAs in online learning is relatively unknown to IL education. In fact, the notion of affective EAs is almost alien IL education research. In addition, [42] argued that although studying existing systems can lead to understanding of the system structure, it is essential to build them, in order to theorize about potential features beyond what has already been achieved. Therefore, the study embarked on creating a new online tutorial that teaches information seeking skills.

3.1 Designing an Information Seeking Tutorial First, we created a 15-minute 2D online tutorial featuring a novice female student learning how to search for academic information for assignments. Given the differences in individual reading and learning speeds, forward and backward buttons are provided in the tutorial for users to control the progress of the tutorial. There are two EAs in the tutorial (see Figure 1), one representing a young female student who is new to academic information seeking, and

another representing an experienced female teacthe approach taken by [43]. A female agenrepresent the teacher as female agents are perceiinfluence on students, especially in academic enThe female teacher in this tutorial instructs lecope with an academic search situation as a novserves as a “coping model” [39]. The agent’s most important design feature as it dictatperception of the agent as a virtual social modedeictic gestures like pointing and nodding are imemotions for the affective agents [12]. Hencedesigned in such a way that they have pleasant (e.g., smiling, calming, encouraging), and apgestures (e.g., pointing to the website for informa

Figure 1. Affective teacher and studNext, the content employed the ISP model as thtutorial. Based on the ISP Model, the tutorial wastages, namely, Task Initiation, Topic SelExploration, Focus Formulation, InformationSearch Closure. Each stage is associated with dstrategies, emotions on the part of the studinstructions from the teacher.

At the start, the student was presented with twriting a literature review on a topic selectedstudent became aware of a lack of knowledge, need for information. In the literature, this awareaccompanied by feelings of uncertainty and aThe teacher acted as a learning guide. Next, the further explore information on the selected topicmarked by feelings of confusion, uncertainty,teacher displayed empathic emotions and emwords to encourage the student. Additionally,introduced some useful skills to help the studrelevant information, such as using Booleanperforming backward and forward chaining. stage of information collection involves tinformation related to the topic. As the studentsense of direction, her confidence level increasethe assignment neared completion, the student fsense of satisfaction. The task in Search Closureto the search, and to prepare oneself to write or findings [35].

Given the overview of the tutorial above, we wilPrefocus Exploration as an example to illustratewere used (see Figure 2). At this stage, the studea focus for the assignment. To do this, the“knowledge sharing through storytelling” into thsearch bar, intending to retrieve a list of articles

cher. This mirrors nt was chosen to ived to exert more nvironments [44].

earners on how to vice. She therefore appearance is the tes the learner’s el [9]. In addition,

mportant to convey e the agents were facial expressions ppropriate deictic ation).

dent EA he structure for the as divided into six lection, Prefocus

n Collection, and different searching dent, as well as

the assignment of d from a list. The

and recognized a eness is frequently apprehension [35].

student needed to c, and this stage is , and doubt. The

mployed empathic , the teacher also

dent retrieve more n operators, and Furthermore, the

the gathering of t now had a clear

ed accordingly. As felt relieved and a e is to bring an end

otherwise use the

ll take the stage of e how the two EAs ent needed to find e student entered he Google Scholar . According to the

ISP model, this is the stage in which inintimidated by the huge amount ofconfused by some contradicting opinioabout what to do next. Therefore, in ththe teacher tried to give the student coencouraging words, such as “don’t wnormal to feel frustrated”, and so on.

Figure 2. Screenshot of Tutori

Figure 3. Screenshot of Tutori

3.2 Experimental Design anThe experiment used a between-subjewere randomly assigned to one of threeneutral-EAs, and text-only. This apprwork of other studies, e.g., [24, 2condition (see Figure 2 and Figure 3)and one teacher not only maintained athe tutorial, but also displayed affecfrustration and confusion from encouragement and support from thneutral-EAs condition (see Figure 4), tha visual presence throughout the tutorino affective expressions. For these twand instructions were presented in sponly condition, the two EAs were

nformation seekers might be f retrieved information, or ons. They might be doubtful he next step (see Figure 3), onfidence by offering some worry”, “don’t panic, it is

ial I (Affective-EAs)

ial I (Affective-EAs)

nd Materials ects design, and participants e conditions: affective-EAs, roach is consistent with the 27]. In the affective-EAs ), the two EAs, one student a visual presence throughout ctive expressions, such as the student agent, and

he teacher agent. For the he two EAs also maintained ial; however, they displayed

wo conditions, conversations eech bubbles. For the text-absent, with only textual

instructions, presented in square dialog boxes.

Figure 4. Neutral teacher and stude The posttest survey questionnaire was adaptliterature to suit this study’s purpose. It compriThe first two sections focused on students’ leaand enjoyment. All question items that were these two constructs were formulated based on[45, 46]. The questions were measured on a 5-pranging from 1 (strongly disagree) to 5 Furthermore, the third section aimed to tesknowledge that students attained from the penultimate section, participants were asked forthe tutorial, and for suggestions to improve information was collected in the final sectionfurther explained below.

• Motivation. Motivation is an importanlearning, and it can be a catalyst in learachieve one’s goals [45]. Designed by [45]Material Motivational Survey was used motivational support provided by instrucwhich is very suitable in this study. subconstructs, which includes: attention something interesting that got my attention”“The content is relevant to my study”), conftutorial was too difficult”), and satisfactioenjoyed learning in the tutorial”).

• Enjoyment. Enjoyment is the active paexperience, and it has been found to be a stechnology use intention [46]. It can be othree levels: affective enjoyment, which isinvest emotionally in the experience (e.g., “attached to the tutorial”), cognitive enjoymwillingness to develop skills and solve problthis tutorial is a good way of learning infskills”), and behavioral enjoyment, which iparticipate on a kinaesthetic level (e.g., “I wuse the application to accomplish my designa

• Knowledge Retention. Although it has beethe instructional effectiveness of online ebased on its novelty, rather than increretention by students, knowledge retention ismeasure, because it is the most direct and from the intervention [12]. Therefore, tascertained the amount of knowledge retaafter watching the tutorial, in the form oquestions and sentence completion questionquestions include: “Which of the followininformation sources for academic informat

ent EAs

ted from existing ised five sections.

arning motivations used to measure

n extant literature point Likert scale, (strongly agree). t the amount of

tutorial. In the r their opinions of

it. Demographic n. Each section is

nt component to rning in order to , the Instructional to measure the

ctional programs, It includes four (e.g., “There is

”), relevance (e.g., fidence (e.g., “The on (e.g., “I really

articipation in an strong indicator of operationalized at s a willingness to “I feel emotionally ment, which is a lems (e.g., “I think formation seeking s a willingness to was able to easily ated tasks”) [47].

en pointed out that education is often eased knowledge s still important to immediate result the third section ained by students f multiple choice

ns. Some example ng are the typical ation?”, “Citations

are important in academic writing different stages are there in a typsearch process?”

• Subjective Feedback. This secsubjective feedback of the tutorasked on what they liked and didtutorial, and suggestions on how to

• Demographic Data. Informdemographic data such as age, geperceived computer knowledge, wfrequency of computer usage was c

3.3 Participants and ProceFirst, a pilot study was carried out wiorder to gather feedback for the onlinequestionnaire. Based on their commquestions was adjusted, and some amade clearer. Next, mass emails and flmajor universities in Singapore to hundred and twenty students (both unparticipated in the study. Controls werthe sample was more representativexample, students from various disuniversities, including both undergradrecruited. The demographics of the sam

Table 1. Demographics of

Gender Male Female Age Less than 20 21-25 26-30 Educational background Natural Science Arts, humanities, and social scieEngineering

The experiment was conducted in a coParticipation in this study was voluntainformation that could be used to idewas collected. After signing an onlinewere directed to the study’s website. participants were provided with instrwere to evaluate an online tutorial. tutorial, participants were asked questionnaire (described earlier). Tapproximately 25 minutes. As a toparticipant was given $5 upon complet

4. Results Table 2 shows the means and staconstruct from the questionnaire. Nanalyses of variance (ANOVAs) isignificant differences with respect top<0.01]; Satisfaction [F (2,117)=[F(2,117)=3.20, p<0.05]; Cognitive [FBehavior [F(2,117)=3.33, p<0.05]. Tsignificant differences between Cop=0.11]; Relevance [F(2,117)=1.67, Retention [F(2,117)=1.76, p=0.18].

because”, and “How many pical academic information

ction gathers participants’ rial. Three questions were d not like about the online o improve it.

mation on participants’ ender, computer experience,

ways of computer usage, and collected.

dure th 22 university students in

e tutorial and to improve the ments, the sequence of the

ambiguous questions were yers were distributed in two

recruit participants. One ndergraduate and graduate) e put in place to ensure that

ve of the population, for sciplines from two major duates and graduates were

mple are shown in Table 1.

Sample (n=120)

n % 37 30.83 83 69.17 38 31.67 79 65.83 3 2.50 15 12.50

nce 92 76.67 13 10.83

ontrolled laboratory setting. ary and anonymous, and no entify individual participant e consent form, participants Upon entering the website,

ructions, and told that they At the completion of the to complete an online

The whole study lasted oken of appreciation, each tion of the questionnaire.

andard deviations of each ext, results from one-way indicated that there were o Attention [F(2,117)=5.97, 3.96, p<0.05]; Affective

F(2,117)=3.20, p<0.05]; and There were no statistically onfidence [F(2,117)=2.24, p=0.20]; and Knowledge

Table 2. Mean and SD of dependent variables

Tutorial Type Text-Only

(n=39) Neutral-EAs

(n=40) Affective-EAs

(n=41) Variable Mean SD Mean SD Mean SD

Motivation Attention** 28.65 2.88 29.37 3.07 30.85 2.72 Relevance 32.86 3.40 32.30 4.70 33.78 3.24 Confidence 27.76 2.56 28.54 2.37 28.90 2.35 Satisfaction* 29.24 3.30 29.14 3.90 31.10 3.20 Enjoyment Affective* 9.59 2.94 9.54 2.96 11.02 2.94 Cognitive* 10.84 2.77 11.23 2.80 12.29 2.36 Behavioral* 14.97 2.57 14.26 3.11 15.78 2.02 Knowledge Retention 4.77 1.61 5.51 2.13 5.02 1.45

Notes: *Statistically significant differences at p< 0.05. **Statistically significant differences at p< 0.01.

Post-hoc comparisons using Tukey’s test were then conducted. The results are summarized below (see Table 3).

Motivation • Attention. Participants had better attention towards the

affective-EAs (M=30.85) than text-only condition (M=28.65), and this difference was statistically significant. However there were no statistically significant differences between text-only and neutral-EAs, or neutral-EAs and affective-EAs condition.

• Relevance. There were no statistically significant differences between pairwise comparisons among the three groups. This suggests that participants in the three groups had similar levels of relevance when going through the tutorial.

• Confidence. Likewise, pairwise mean differences between text-only, neutral-EAs and affective-EAs were insignificant.

• Satisfaction. In terms of satisfaction achieved after viewing the tutorial, the affective-EAs (M=31.10) was significantly higher than the neutral-EAs (M=29.14). Though the affective-EAs condition gave better satisfaction to participants than the neutral-EAs, they were no differences between the text-only and affective-EAs or neutral-EAs condition.

Enjoyment

• Affective Enjoyment. As with satisfaction, the affective-EAs (M=11.02) had a significant difference over the neutral-EAs (M=9.54). Similarly there were no differences between the text-only and affective-EAs or neutral-EAs condition.

• Cognitive Enjoyment. The post-hoc tests for cognitive enjoyment showed similar results as that of attention. The affective-EAs condition (M=12.29) had a statistically significant difference over the text-only condition (M=10.84). Likewise, there were no statistically significant differences between the text-only and neutral-EAs, or neutral-EAs and affective-EAs condition.

• Behavioral Enjoyment. Like satisfaction and affective enjoyment, the affective-EAs (M=15.78) had a significant difference over the neutral-EAs (M=14.26). There were no differences between text-only and affective-EAs or neutral-EAs condition.

Knowledge Retention. Pairwise mean differences between text-only, neutral-EAs and affective-EAs conditions were insignificant.

Table 3. Post-hoc test of dependent variables

Variable Type (1) Type (2) Mean

Difference (1) - (2)

Motivation

Attention Text-only Neutral-EAs -.723 Text-only Affective-EAs -2.205* Neutral-EAs Affective-EAs -1.482

Relevance Text-only Neutral-EAs .665 Text-only Affective-EAs -.916 Neutral-EAs Affective-EAs -1.580

Confidence Text-only Neutral-EAs -.786 Text-only Affective-EAs -1.146 Neutral-EAs Affective-EAs -.360

Satisfaction Text-only Neutral-EAs .100 Text-only Affective-EAs -1.854 Neutral-EAs Affective-EAs -1.955*

Enjoyment

Affective Text-only Neutral-EAs .052 Text-only Affective-EAs -1.430 Neutral-EAs Affective-EAs -1.482*

Cognitive Text-only Neutral-EAs -.391 Text-only Affective-EAs -1.455* Neutral-EAs Affective-EAs -1.064

Behavioral Text-only Neutral-EAs .716 Text-only Affective-EAs -.808 Neutral-EAs Affective-EAs -1.523*

Knowledge Retention

Text-only Neutral-EAs -.745 Text-only Affective-EAs -.253 Neutral-EAs Affective-EAs .492

Note: *Statistically significant differences at p< 0.05.

5. Discussion Taken together, these results suggest that the use of affective EAs had a positive effect on learning motivation and enjoyment for students. For example, the participants praised the use of affective EAs as an innovative way of teaching information seeking process. A comment from a participant lends support to this view, “The animations and illustrations made a rather dry topic slightly more interesting. There were encouraging comments after each slide to reinforce a positive attitude in the student.”

5.1 Motivation Participants who interacted with the affective EAs paid more attention and were more satisfied with the tutorial. This finding is consistent with [12, 16, 23, 41], and lends further support to the argument that the use of affective EAs is able to improve students’ learning performance. One theoretical perspective that helps explain the positive impact of affective EAs on learning is social agency theory [12, 23], which was derived from [38]. This theory asserts that the social cues provided by computers can promote users to interact with the computers as if in human-to-human interactions, therefore facilitating the students’ motivation in the learning process. Further, the social responses evoked by affective EAs will also increase the level of enjoyment during the interactions. For example, some positive comments include: “It is visually appealing as it has a variety of graphics.”, which showed that the tutorial was able to hold participants’ attention; “It is very informative and insightful at the same time in an engaging way I learnt something about what would have been really dry.”, which indicated participants’ satisfaction with the tutorial. Another

example is “It can be applicable to my academics.”, which showed that the tutorial was of relevance to their studies.

5.2 Enjoyment Participants who interacted with the affective EAs derived more enjoyment from the tutorial, affectively, cognitively, and behaviorally. This is predicted by the social agency theory [23]. Some participants remarked that “it was very different from a plain powerpoint slide, which was nice to see.”, and “I like that the design was simple and that it was rather interactive (there were pop ups and buttons to click etc.) / I also liked that there was some freedom given to the user (like choosing the ‘topic’ I wanted to do), therefore making it more personalized.” This result is in concurrence with [43]. Two theories are of relevance as a lens for understanding the results here.

The first one is transportation theory [48]. Transportation theory uses the analogy between media transportation and physical transportation. As in physical transportation, users goes out of their original world as a result of being exposed to media, and when they come back, they are somewhat changed by the journey. In other words, transportation theory claims that the experience of being immersed and the consequences of the immersion in a narrative world will lead to enjoyment in users. Here, the consequences include connections with characters in the media. Therefore, in the context of this study, participants created a connection with the EAs in the tutorial, and this experience resulted in the increase in enjoyment level.

Another useful theory that can also account for this result is social cognitive theory [49]. This theory contends that by observation of other’s behaviors, users could develop rules to guide their own subsequent actions, or be prompted to engage in previously learnt behaviors. In addition, the positive reinforcement of behaviors performed by media characters will also increase the level of user enjoyment in the media presentation. Thus we believe that the behaviors of the EAs in the tutorial not only prompted participants to engage in learning, but also contributed to the increased enjoyment level.

5.3 Knowledge Retention There were no statistically significant differences in knowledge retention in the three groups. This result contradicts those from [23]. In particular, all three groups had low scores. This is probably because of the large amount of content information and the complexity of the learning material in the tutorial, which prevented the students from extracting the most useful information in a short period of time. This explanation is also confirmed by the subjective comments from participants, who remarked that there was too much text and information presented in the tutorial. For example, one participant said: “Maybe just highlight the keywords, instead of showing chunks of words.” Another likely reason which is also mentioned in [23] is that probably the novelty of incorporating EAs in IL instruction caused students to engage in superfluous cognitive activities that did not contribute directly to learning, thus decreasing the amount of knowledge retained. Furthermore, a highly likely reason is that there was a lack of audio cues. It has been found that in order to understand the materials, the learners had to hold auditory information in addition to the textual information on the screen [50].

However, some participants wrote that they benefited from the tutorial. For example, one participant mentioned: “The tutorial answered some questions I have about finding references. New

information is also learned on top of what I’ve known, such as the backward reference search (i.e. reference list of the paper of interest) and the forward reference search (i.e. which papers cited the paper of interest)”.

A few interesting conclusions may be gleaned from the participants’ comments. First, the use of affective EAs can ease and relieve the negative emotions experienced during information seeking process. This result also lends further support to the findings from [41, 43]. For example, some students liked the presentation of information in the form of personal narratives as this made the learning process less stressful for them. This also suggests that when carefully designed, an online tutorial with affective EAs can successfully imitate the social human-human interactions. In addition, the use of the ISP model to structure the tutorial was received favorably by participants. They found the tutorial was well organized, which further shows the necessity of using a theoretical framework to scaffold the online tutorial. This can be reflected from comments by participants, who said that “The content is structured well, in order of the steps to be taken when doing a literature paper.”, and “I liked that it broke down the process of academic writing into smaller, more manageable steps. It was easy to understand the programme”.

6. CONCLUSION The present work investigated the impact of affective EAs on learning motivation and enjoyment in an information seeking tutorial. A between-subjects experiment was used in the study, where students were divided into three conditions: affective EA, neutral-EAs, and text-only. The results have shed light on several issues. The study supported the general prediction that students would benefit from the added use of an affective EA in educational systems. It showed that affective EAs significantly increased students’ learning motivation and enjoyment, compared to neutral-EAs or text-only instructions. However, there was no significant difference in knowledge retention. The incorporation of both positive and negative affective states in the EAs, which are common during the information search process, can enhance students’ learning motivation and enjoyment.

This study has several implications for the design of IL instruction. First, the tutorial used in this study was constructed from rudimentary elements of a graphical user interface, such as dialog boxes, buttons, and text. This shows that it is possible to create useful, affective EAs without the need for complex user interface elements. This is important as libraries may not have the resources for sophisticated software development projects. Our study showed that it is possible to create affective EAs without employing expensive technologies and still be effective, illustrated by the improvement in user motivation and enjoyment. The study thus paves the way for a better understanding of embedding EAs in IL instruction. Second, the improvement in students’ learning motivation and enjoyment can serve as a basis for future research; for example, for developing a full-fledged online IL educational system, which contains more interactive elements and more content. Further, given the success of digital game-based learning, future research can be done to marry digital game-based learning and affective EAs to teach IL. Last but not least, the ISP model was effectively used as the conceptual framework for the online tutorial, as indicated by the favorable comments from participants. To our best knowledge, this is one of the few efforts in structuring an affective online library tutorial on a theoretically sound framework, which could guide practitioners to base their practice on grounded principles rather than depending solely on intuition or individual experience.

Although this study has yielded useful insights, it has some limitations that could be addressed by future research. While participants in this study were undergraduates and graduates in two major local universities, it would be instructive to carry out follow-up work with other age groups with more diverse backgrounds for better generalizability. Furthermore, although Kuhlthau’s ISP model sheds light on users’ affective states in information seeking, there are no explanations on how to ease negative emotions. Therefore, the emotions displayed by the teacher EA in this study are based on personal experience, as well as a preliminary study by [8]. This might lead to inaccurate treatment of emotions, especially negative ones.

This study also offers opportunities for future research directions. First, the tutorial could be further improved. One common suggestion from participants is that the text should be further reduced to make it easier to read and understand. Such suggestions will be taken into consideration in the next stage of the research. Next, there is a need for future studies to conceptualize the key elements in this online tutorial into a reusable model. Such a model could also assist researchers in differentiating those elements that are essential in producing successful outcomes and those that are not. Lastly, more research could be conducted to find out how negative affective states will influence the learning performances. For example, experiments could be done to compare the difference between the presence of negative affective states and the absence of negative affective states.

7. REFERENCES [1] J. Brockman, Is the Internet Changing the Way You Think?:

The Net's Impact on Our Minds and Future. New York: HarperCollins, 2011.

[2] J. Lilburn, "You’ve Got to Know and Know Properly”: Citizenship in Kazou Ishiguro’s Never Let Me Go and the Aims of Information Literacy," ed: Information Literacy and Social Justice: Radical Professional Praxis.–Sacramento, CA: Library Juice Press, 2013.

[3] X. Zhang, S. Majid, and S. Foo, "Environmental scanning: An application of information literacy skills at the workplace," Journal of Information Science, vol. 36, pp. 719-732, 2010.

[4] T. D. Wilson, "Human information behavior," Informing science, vol. 3, pp. 49-56, 2000.

[5] P. Corbett, "What about the" Google Effect"? Improving the Library Research Habits of First-Year Composition Students," Teaching English in the Two-Year College, vol. 37, pp. 265-277, 2010.

[6] J. W. Carey, "Harold Adams Innis and Marshall McLuhan," The Antioch Review, vol. 27, pp. 5-39, 1967.

[7] M. R. Lepper and R. W. Chabay, "Socializing the intelligent tutor: Bringing empathy to computer tutors," in Learning issues for intelligent tutoring systems, ed: Springer, 1988, pp. 242-257.

[8] B. Kort, R. Reilly, and R. W. Picard, "An affective model of interplay between emotions and learning: Reengineering educational pedagogy-building a learning companion," in Advanced Learning Technologies, 2001. Proceedings. IEEE International Conference on, 2001, pp. 43-46.

[9] R. Beale and C. Creed, "Affective interaction: How emotional agents affect users," International Journal of Human-Computer Studies, vol. 67, pp. 755-776, 2009.

[10] R. W. Picard, Affective computing. Cambridge, MA.: MIT press, 2000.

[11] M. Haake, A. Silvervarg, and B. Sjödén, "Building a Social Conversational Pedagogical Agent-Design Challenges and Methodological Approaches," in Conversational Agents and Natural Language Interaction: Techniques and Effective Practices, A. Gulz, M. Haake, A. Silvervarg, B. Sjödén, and G. Veletsianos, Eds., ed Hershey, PA.: IGI Global, 2010, pp. 128-155.

[12] R. K. Atkinson, "Optimizing learning from examples using animated pedagogical agents," Journal of Educational Psychology, vol. 94, pp. 416-427, 2002.

[13] J. Cassell, Embodied conversational agents. Cambridge, MA.: The MIT Press, 2000.

[14] J. Zimmerman, E. Ayoob, J. Forlizzi, and M. McQuaid, "Putting a face on embodied interface agents," 2005.

[15] P. Rizzo, "Why should agents be emotional for entertaining users? A critical analysis, Affective interactions: towards a new generation of computer interfaces," ed: Springer-Verlag New York, Inc., New York, NY, 2001.

[16] R. J. E. Lee, C. Nass, S. B. Brave, Y. Morishima, H. Nakajima, and R. Yamada, "The Case for Caring Colearners: The Effects of a Computer‐Mediated Colearner Agent on Trust and Learning," Journal of Communication, vol. 57, pp. 183-204, 2007.

[17] B. Perez, Warren, Park, Shen, "The impact of frustration-mitigating messages delivered by an interface agent," Artificial intelligence in education: supporting learning through intelligent and socially informed technology, vol. 125, p. 73, 2005.

[18] G. H. Bower, "How might emotions affect learning," The handbook of emotion and memory: Research and theory, pp. 3-31, 1992.

[19] J. Mumm and B. Mutlu, "Designing motivational agents: The role of praise, social comparison, and embodiment in computer feedback," Computers in Human Behavior, vol. 27, pp. 1643-1650, 2011.

[20] D. Charsky, "From edutainment to serious games: A change in the use of game characteristics," Games and Culture, vol. 5, pp. 177-198, 2010.

[21] N. P. Thomas, S. R. Crow, and L. L. Franklin, Information literacy and information skills instruction: applying research to practice in the 21st century school library. Santa Barbara, CA.: ABC-CLIO, 2011.

[22] J. Kracker, "Research anxiety and students' perceptions of research: An experiment. Part I. Effect of teaching Kuhlthau's ISP model," Journal of the American Society for Information Science and Technology, vol. 53, pp. 282-294, 2002.

[23] Q. Dunsworth and R. K. Atkinson, "Fostering multimedia learning of science: Exploring the role of an animated agent’s image," Computers & Education, vol. 49, pp. 677-690, 2007.

[24] R. E. Mayer and C. S. DaPra, "An embodiment effect in computer-based learning with animated pedagogical

agents," Journal of Experimental Psychology: Applied, vol. 18, pp. 239-252, 2012.

[25] N. C. Krämer and G. Bente, "Personalizing e-learning. The social effects of pedagogical agents," Educational Psychology Review, vol. 22, pp. 71-87, 2010.

[26] A. L. Baylor and S. Kim, "The effects of agent nonverbal communication on procedural and attitudinal learning outcomes," in Intelligent virtual agents, 2008, pp. 208-214.

[27] L. Lin, R. Atkinson, R. Christopherson, S. Joseph, and C. Harrison, "Animated agents and learning: Does the type of verbal feedback they provide matter?," Computers & Education, 2013.

[28] P. G. Zurkowski, "The Information Service Environment Relationships and Priorities. Related Paper No. 5," 1974.

[29] D. V. Loertscher, "A Farewell Challenge," School Library Media Quarterly, vol. 24, pp. 192-194, 1996.

[30] K. C. Montgomery, Generation digital: Politics, commerce, and childhood in the age of the Internet: MIT Press Cambridge, MA, 2007.

[31] P. W. Cookson Jr, "What would Socrates say," Educational Leadership, vol. 67, pp. 8-14, 2009.

[32] M. Moran, "Young learners need librarians, not just Google," Forbes. com, 2010.

[33] I. F. Rockman, "Integrating Information Literacy into the Higher Education Curriculum: Practical Models for Transformation (The Jossey-Bass Higher and Adult Education Series)," 2004.

[34] J. Carpenter, L. Wetheridge, N. Smith, M. Goodman, and O. Struijvé, Researchers of Tomorrow: A Three Year (BL/JISC) Study Tracking the Research Behaviour of'generation Y'Doctoral Students: Annual Report 2009-2010: Education for Change, 2010.

[35] C. C. Kuhlthau, "Seeking meaning: A process approach to library and information services Libraries Unlimited," Westport, CT, 2004.

[36] D. Nahl and D. Bilal, Information and emotion: The emergent affective paradigm in information behavior research and theory: Information Today, Inc., 2007.

[37] D. Bilal, "Children's information seeking and the design of digital interfaces in the affective paradigm," Library Trends, vol. 54, pp. 197-208, 2005.

[38] B. Reeves and C. Nass, The Media equation: how people treat computers, television, and new media: Cambridge University Press, 1997.

[39] A. L. Baylor, "The design of motivational agents and avatars," Educational Technology Research and Development, vol. 59, pp. 291-300, 2011.

[40] N. Wang, W. L. Johnson, P. Rizzo, E. Shaw, and R. E. Mayer, "Experimental evaluation of polite interaction tactics for pedagogical agents," in Proceedings of the 10th international conference on Intelligent user interfaces, 2005, pp. 12-19.

[41] J. Klein, Y. Moon, and R. W. Picard, "This computer responds to user frustration:: Theory, design, and results," Interacting with computers, vol. 14, pp. 119-140, 2002.

[42] M. Mateas and A. Stern, "Build It to Understand It: Ludology Meets Narratology in Game Design Space.," presented at the Digital Games Research Association, University of Vancouver, 2005.

[43] H. Maldonado, J.-E. R. Lee, S. Brave, C. Nass, H. Nakajima, R. Yamada, et al., "We learn better together: enhancing eLearning with emotional characters," in Proceedings of th 2005 conference on Computer support for collaborative learning: learning 2005: the next 10 years!, 2005, pp. 408-417.

[44] A. L. Baylor and S. Kim, "Designing nonverbal communication for pedagogical agents: When less is more," Computers in Human Behavior, vol. 25, pp. 450-457, 2009.

[45] J. M. Keller, "Manual for instructional materials motivational survey (IMMS)," Unpublished manuscript, Tallahassee, FL, 1993.

[46] E. P. P. Pe-Than, D. H.-L. Goh, and C. S. Lee, "Enjoyment of a Mobile Information Sharing Game: Perspectives from Needs Satisfaction and Information Quality," in The Outreach of Digital Libraries: A Globalized Resource Network, ed: Springer, 2012, pp. 126-135.

[47] R. L. Nabi and M. Krcmar, "Conceptualizing media enjoyment as attitude: Implications for mass media effects research," Communication Theory, vol. 14, pp. 288-310, 2004.

[48] M. C. Green, T. C. Brock, and G. F. Kaufman, "Understanding media enjoyment: The role of transportation into narrative worlds," Communication Theory, vol. 14, pp. 311-327, 2004.

[49] A. Bandura and J. Bryant, "Social cognitive theory of mass communication," Media effects: Advances in theory and research, vol. 2, pp. 121-153, 2002.

[50] H. J. Jeung, P. Chandler, and J. Sweller, "The role of visual indicators in dual sensory mode instruction," Educational Psychology, vol. 17, pp. 329-345, 1997.