Cognitive support embedded in self-regulated e-learningsystems for students with special learning needs

K. Chatzara & C. Karagiannidis & D. Stamatis

# Springer Science+Business Media New York 2014

Abstract This paper presents an anthropocentric approach in human – machineinteraction in the area of self-regulated e-learning. In an attempt to enhance communi-cation mediated through computers for pedagogical use we propose the incorporationof an intelligent emotional agent that is represented by a synthetic character withmultimedia capabilities, modelled to imitate human behaviour. The agent is aiming toprovide cognitive support to users with learning difficulties and attention disorders andis designed to accommodate self regulated learning elements. We review the basicprinciples of self regulated learning which, in turn, act as a basis for designing andimplementing our system. Kolb’s learning cycle is used to provide a framework uponwhich agents’ pedagogical behaviour is constructed. A study between 24 students fromhigher education with learning difficulties and attention disorders is presented. Thelearning particularities of this special group that contradict with the principles of selfregulated learning are reported. The study refers to students in higher education, in thedomain of information technology. The analysis of results indicates that emotionalagents improve communication between users of the particular learning group andlearning environments by providing cognitive support through behavioural communi-cation, compared to agents with neutral behaviour.

Keywords Adaptive learning . Emotional pedagogical agents . Learning difficulties .

Affective computing . Instructional learning

Educ Inf TechnolDOI 10.1007/s10639-014-9320-1

K. Chatzara (*)Department of Special Education, University of Thessaly & NOUS Institute of Digital Learning &Communication, Argonafton & Filellinon, 38221 Volos & Filikis etaireias 7, 54621 Thessaloniki, Greecee-mail: k.chatzara@i-nous.org

C. KaragiannidisDepartment of Special Education, University of Thessaly, Argonafton & Filellinon, 38221 Volos, Greece

D. StamatisDepartment of Informatics, Alexander T.E.I. of Thessaloniki, Greece, P.O BOX 141, GR T.K 57 400,Thessaloniki, Greece

1 Introduction

The lack of the physical presence of the educator may cause problems in distancelearning as communication between users and educators is radically distracted (Picard2011; Palincsar et al. 2001). Common problems identified in the literature (Kinnebrewand Biswas 2011; Kienle and Wessner 2006) are the limited interaction between usersand educators and the limited communication between them. This bereaves the sense ofbelonging to a team which shares common goals in learning, faces common problems(in comprehending knowledge) and shares common emotions related to learning. Usersfrom special learning groups, such as users that face learning difficulties (LD) andattention disorders (AD) find it often more difficult to learn in such environments due totheir particular characteristics that oppose to self-regulated learning (Klassen 2010;Palincsar et al. 2001).

To resolve this lack of communication, researchers used graphical representations ofagents that communicate in a pedagogical manner with users via spoken or writtenconversation. (Beale and Creed 2009; Burleson and Picard 2007; Dehn and VanMulken 2000).

They are developed for a variety of educational domains including maths and showthat they can be effective support for learners (Lee et al. 2007; Beale and Creed 2009;Dehn and Van Mulken 2000).

In an attempt to establish emotional communication between computer systems andusers, Picard and her team from MIT introduced affective computing (Burleson andPicard 2007). Agents are enriched with the emotional component of communication(Howe 2009) and they use facial expressions and body movements to communicate.The use of multimedia (Maldonado et al. 2005; Conati and Maclaren 2009; Burlesonand Picard 2007) and the plurality of expressions help in the plausibility of the agent’sgraphical representation. They can talk or write and have the ability to express emotionsrelevant to the learning procedure. The expressiveness of agents may create a sense oftrust and communication (Lee et al. 2007). They are used in various applications forvarious purposes and often used as interface agents playing the role of personalassistants for their users (Beale and Creed 2009; Dehn and Van Mulken 2000).

Researchers are looking into designing agents that identify as precisely as possibleuser’s emotional stage and their impact in different domains (Picard 2011). In thispaper, we propose an Intelligent Emotional Agent (IEA) that “understands” user’semotional states and acts accordingly by producing realistic behaviour. The agentproposed is designed under the framework of Kolb’s learning cycle to act in apedagogical manner. IEA’s role is to support users to obtain the characteristics neededfor more productive self regulated learning.

Further on, we identify the problems resulting by the lack of physical presence of theeducator in e-learning environments and state our research questions. As this paperfocuses on educational environments for specific user groups (people with learningdifficulties and attention disorders) we will indicate the characteristics of these learninggroups that contradict with self regulated e-learning and record a set of principles ofself-regulated theory that applies to this specific group. Based on these principles wespecify and implement IEA’s behaviour that supports students in their learning.

A comparative study between 24 students with LD and AD is recorded.Methodology and statistical analysis of results is presented that support our conclusion

Educ Inf Technol

that the effectiveness of IEA’s, improves communication between learners by providingaffective cognitive support that result in a more successful learning outcome.

2 Self-regulated e-learning

New rules and behaviours need to be employed in order to study successfully in e-learning settings. E-learners must rely on their individual abilities to direct theirlearning, that is, employ SRL strategies. As Lee et al. (2007) and Pintrich (2004)suggest successful e-learning requires self-regulated learning.

Self regulated learning theory argues that learners need to follow certain proceduresto complete successfully their learning tasks when studying without support and focuson an individual’s ability to organize their learning. They need to control their learningtime by monitoring and managing their performance and reach an effective outcome(Kinnebrew and Biswas 2011).

Literature suggests (Bartolomé et al. 2011; Pintrich 2004; Usher and Pajares 2008;Caprara et al. 2008; Kember 2001) that in education (both traditional and distance one),learners should have the following self-regulatory characteristics:

(1) Motivation in terms of goal orientation and self-efficacy: Motivation and self-efficacy is a crucial factor in academic success of children, adolescents, and adults(Caprara et al. 2008; Kember 2001). Self-efficacy theory (Bandura 1993) per-ceives self-efficacy as beliefs in one’s abilities to proceed in certain actions. Iflearners believe that they can manage, they are more likely to succeed in theirtasks (Caprara et al. 2008). Low self-efficacy beliefs may block performance; incontrary, high self-efficiency beliefs improve task engagement, effort and perfor-mance (Pintrich 2004). These self-beliefs are the result of four different domains(Klassen 2010); (a) academic achievements on previous similar learning tasks, (b)modelling or being present when other perform similar tasks, (c) feedback fromteachers, and (d) physiological and emotional reactions related to certain tasks (forexample, nervousness, anxiety, eagerness).

(2) Time and environment management: In the duration of teaching, the educatorapplies the tempo of learning and repeats, when needed, instructions or tips toachieve satisfactory results. In asynchronous learning individuals need to regulatetheir time without supervision and manage their environment through applicationinterfaces which often are not tailored to suit to their individual’s needs andpreferences. It is like you are sitting on a desk and the books are placed at a veryinconvenient place. At traditional classrooms you may put them anywhere youlike but in virtual environments the system has to be able to accommodate theseprocedures (Pintrich 2004).

(3) Help seeking: In systems that do not give learners the ability to seek help,students may become frustrated and this can result negatively on the achievementof their educational goals (Kember 2001). The potentiality to ask for help andclarification instructions could be accommodated partly in e-learning even inasynchronous environments. Distance learning research suggests that peoplewho ask for help when facing difficulties may be more likely to achieve learningoutcomes (Newman 2002)

Educ Inf Technol

Bell and Akroyd (2006) have reported characteristics with a cognitive and socialbasis that students must possess in order to be able to self-regulate their learning. Theseare as follows; (a) being intrinsically motivated to reach goals, (b) expecting that one’sefforts to learn will result in positive outcomes, (c) expecting to succeed in one’slearning, (d) being confident in one’s ability to perform and complete an academic task,(e) monitoring one’s progress toward goal completion, (f) controlling one’s effort andattention, and (g) managing time and place resources for learning and studying.

3 Pedagogical agents

To assist users in the learning procedure without the physical appearance of educators,researchers introduced pedagogical agents that were assigned different roles such aslearning companions, tutors, trainers, virtual consultants and coaches. They are oftenused as basic elements in digital learning environments in a variety of educationalapplications including mobile applications. They can be a tool that promotes activerather than passive learning by involving students’ communication with the agents.Their appearance varies; they are virtual cartoon characters or they have realistic facesand they implement pedagogical features such as presentations, probing questions,suggestions, explanations and monitoring. Their purpose is to create a social learningenvironment, providing motivation, information and guidance to students during thelearning experience. They give feedback to students in quizzes, test, exercises andproblem solving.

At the most basic level, pedagogical agents can be classified into two categories;interactive and non interactive (Steptoe et al. 2013). They can demonstrate complextasks, perform important functions such as visual simulation of user gestures andmotion interaction. Through these they may improve human machine interaction andhelp students to focus their attention on the learning procedure. It is reported in theliterature that people can recognize and correctly identify emotional expressions byembodied agents (Maldonado et al. 2005; Beale and Creed 2009). This inspiredresearchers to create agents with the ability to convey emotional responses andaccommodate a behavior that makes them appear knowledgeable, attentive, helpfuland concerned, etc.

Pedagogical agents may:

& Provide the physical representation of the user in a virtual learning environment.& Create the impression to a user that they communicate with real people during the

interaction with the system. They have the ability to induce emotions such ascommunicating with body language, expressing emotions through facial expres-sions and facial movements or gestures.

& Give the impression to the user that they are not alone as they interact with thesystem. This encourages learners to be more concerned about their progress throughthe empathy that they stimulate for users. (Beale and Creed 2009).

Agents have also the potential to cater for the requirements of different groups ofusers and for this purpose they have been used in many adaptive (learning) applications(Maldonado et al. 2005). Users have different profiles and are responding to different

Educ Inf Technol

learner’s models. Researchers argue (Baylor and Kim 2003) that maybe the use of agentscan assist users and cater for their individual characteristics in their learning.

4 Learning difficulties and attention disorders

As indicated in the introduction, people with LD and AD have characteristics that oftenoppose to self-regulated learning. Learning difficulties (LD) and attention disorders(AD) are often met together (McGillivray and Baker 2009; Katz et al. 2011). Differenttypes of LD can affect the uptake and processing of information. Common disordersaffect the ability to use spoken and written language, to make mathematical calcula-tions, coordinated movements and to maintain attention (Panteliadou 1995; Barneset al. 2007). Although they usually appear in a very early age, these disorders areusually not recognized until the child reaches school age. Results from Klassen’s recentresearch work suggest that adults’ internalizing problems are similar to those found instudies of children and adolescents with LD and attention disorders (Klassen et al.2013).

Both learning groups often show lower academic achievements and have lower self-efficacy beliefs than their typical peers (Klassen 2010; Gans et al. 2003; Katz et al.2011). Poor school performance often leads to bad behaviour in the classroom. Badbehaviour in the classroom leads to difficult interaction with teachers and classmates.This in turn leads to the development of distorted beliefs in relation to school perfor-mance, which adds to the existing ones that are caused from the learning difficultyitself. These false and distorted views contribute to greater behavioural problems andattention deficit and a cycle of failure is created (Palincsar et al. 2001); low self-esteemcauses lower academic achievement, and lower academic achievement causes low self-esteem (Gans et al. 2003). It is supported by researchers (Wong 1985) that maybe dueto often failures in the learning procedure, students with LD posses lower level ofprocedural knowledge than their typical peers. Their poor academic performance leadsto doubts about their ability to succeed and this builds on their motivation difficulties.In a recent study Klassen (2010) reported results of a study he contacted in order toexamine the self-efficacy for self-regulated learning of 146 early adolescents with andwithout LD. He reported that the group with LD rated their self-regulatory efficacylower than the group without LD.

Many students with learning difficulties are characterized with serious problemswith attention in learning. This is often due to unsuccessful teaching for this group(Palincsar et al. 2001) and does not constitute a disorder. To be defined as such,inattention and/or hyperactivity needs to be more frequent and severe than is typicallyobserved, met in at least two different environments (e.g. school and home) and appearbefore the age of seven. In these cases we are referring to attention deficit hyperactivitydisorder (ADHD). Students that may experience only one behaviour (inattention orhyperactivity) are still diagnosed as having ADHD, with each behaviour occurringalone.

Attention disorders affect an individual’s daily life in all respects and areas and aredefined as conditions in which executive functions are not sufficiently developed topermit adequate self-management. They can result in poor school performance, de-creased self esteem, as well as a student’s confidence in their learning abilities

Educ Inf Technol

(Mangione Walcott and Landau 2004). They need special help in order to manage theeducational procedures. Often, they face failure not because of their incompetence tocomprehend knowledge but because of the system’s failure to accommodate suchcapabilities that could reinforce their learning. Very often the most dramatic andcatalytic element of attention disorders is not the difficulty as such but the affect thatthis situation has on learner’s confidence. They need approval and encouragement inorder to increase their self esteem and confidence (Klassen 2010). In conclusion,learners with LD and attention disorders share common characteristics that create adifferent student profile than their typical peers. This profile is altered by their specificimpairment but is affected from social factors and leads to;

& low self-esteem,& low self-confidence& Insufficient self-management.

These characteristics contradict with the qualities that a learner needs to posses inorder to regulate learning on their own, without an educator’s affective behaviour that isabsent in e-learning environments. In order to stimulate this affective behaviour wecreated an agent and its graphical representation whose role is to provide users withcognitive support to minimize the effects of their particularities that might raiseobstacles in the learning procedure.

5 Materials and methods

We adopted the principles of self regulated learning to our system to help usersaccommodate SRL techniques. An embodied agent, Sophia, is created whose role isto communicate with the user by portraying the suitable agent’s graphical representa-tions. Sophia is a 3d agent, made in software I-Clone. A photograph of a real teacher isused to simulate the agent. She is able to portray gestures, facial expressions and bodymovements. She is programmed to understand user’s actions and presume user’semotional state in order to show the appropriate behaviour. Sophia “understands”user’s emotional state by collecting messages, from the e-learning environment, whichare the result of user’s interaction and categorizes them as sequences of eventsperceived from the user. These messages are generated either through a user’s directinput or by recording a user’s actions; keystroke, mouse movement. A user’s perfor-mance on learning tasks is recorded too. For example, the user is completing a taskunsuccessfully; if the user tries to complete the task again and he/she is unsuccessfulthis is recorded too and adds to the particular events sequence.

Sophia incorporates the basic principles of self-regulated theory, giving specialemphasis on learner’s characteristics with LD and attention disorders. The mappingof basic principles of self regulated learning, of particularities of learners with LD andattention disorders and the way Sophia is reacting to resolve the difficulties arise issummarized in Table 1:

As Sophia needs to act in a pedagogical manner, her behaviour depicted in the abovetable is designed under the framework of experiential learning. Kolb’s model (1984) isused that refers to the process by which individuals or groups perceive and comprehend

Educ Inf Technol

their experiences and modify their behaviour based on these experiences. Kolb’slearning cycle defines educator’s actions towards learners and perceives learning as aprocess whereby knowledge is created through transformation of experience. In orderfor the system to fulfil the learning objectives and accommodate the above pedagog-ically we defined and distribute Sophia’s behaviour in the four stages of learning(experience, reflection, abstraction and experimentation) that are described in themodel. The inner cycle of Fig. 1 represents the four stages of learning defined byKolb. Around them externally we represent educator’s appropriate behaviour and mapagent’s and user’s performance accordingly.

The four stages of actions that are described by Kolb are adopted, in our work in twodifferent perspectives; to represent user’s and agent’s performance. We stimulateeducator’s role through agent’s actions and we perceive an agent as a learner too.Our agent “learns” user’s behaviour in order to respond accordingly.

The model delimits agent’s behaviour towards users. User’s behaviour (as a se-quence of events), in conjunction with user profiles, are feed into user behaviourmanager which decides about a user’s emotional state (Fig. 2). This emotional stateis reported to administrative module which decides agent’s appropriate behaviour forthe specific user’s action by looking at learning objectives and behaviour rules. Theadministrative module passes this information to the emotion database which in turncommunicates with the synthetic characters database and peaks the correct facial

Table 1 Basic principles of self-regulated theory and relations to LD and Attention Disorders. Sophia’saffective behaviour mapped accordingly to resolve the difficulties arise

Characteristics of learnerswith LD and attentiondisorders

Basic principles of selfregulated learning

Sophia (our agent)

Low self-esteem Being intrinsically motivated toreach goals

Sophia motivates studentsthrough her behavior and instructions

Expecting that one’s efforts tolearn will result positiveoutcomes

Sophia is encouraging learners to have apositive attitude

Expecting to succeed in one’slearning,

Sophia is encouraging learnersto succeed in the learning tasks

Low self-confidence Being confident in one’s ability toperform and complete anacademic task

Sophia’s role is to increase user’s selfesteem and confidence by providingemotional support that is aiming to theuser believe that he can complete theassignments.

They need special help inorder to manage theeducationalprocedures.

Monitoring one’s progresstoward goal completion

Sophia is monitoring user’sperformance and “reminds” user of hisoutcomes

Controlling one’s effort andattention

Sophia by recording user’s actionscontrols (in a way) user’s performance

Managing time and placeresources for learning andstudying.

Sophia acts like a “driver” in thelearning procedure and pointsout to user time delays and resourcesavailable

Educ Inf Technol

expression, movement, gesture, and tone of voice. This corresponds to a particularemotional state that is chosen to be portrayed.

Fig. 1 Adapting Kolb’s learning cycle to map agent’s and user’s behavior

Fig. 2 IEAs model

Educ Inf Technol

Sophia has rich multimedia capabilities that serve for body movements, facialexpressions and gestures (Fig. 3).

We chose 36 different emotions to be portrayed in Sophia’s personality (Fig. 4). The36 different emotions are defined in Aist’s pioneer work (Kort et al. 2001; Aist et al.2002) as emotions relevant to learning. They categorised them as pairs of oppositeemotions, namely: Anxiety-Confidence, Ennui-Fascination, Frustration-Euphoria,Dispirited-Encouraged, Terror-Excitement and Humiliated-Proud. For every such pairof opposite emotions, i.e. frustration-euphoria, a number of in between emotions areused to indicate a more exact emotional state represented, i.e. discomfort (Table 2).

Sophia was placed in an e-book (Fig. 5), that we created in adobe flash specificallyfor this study, by using actions script 3 programming language. E-books’ subject matteris referring to the Internet’s basic principles for students who attend a higher educationcourse in Information Technology. The emotional reactions are represented by acollection of production rules that specify a class of external situations that will berecognized as the ones appropriate for turning on that emotion.

We contacted 184 participants with the experiment but our results are referring to asubset of those; 24 students with learning difficulties and attention disorders. The studytook place in the University department with students aged 18–20 years old. The firsttime we announced a call for the students to participate in the experiments we calledonly people with learning difficulties and attentions disorders. Not many students showup to participate. We then announced a second call but we ended up with thesame poor attendance. The third time, we decided to announce a call for allstudents but we indicated that we were particularly interested in special learninggroups. On that occasion 184 students attended and the 24 of them had learningdifficulties. The application ran online. In the beginning, the students had to fillin a form with personal data (age, gender, disabilities, learning difficulties).Then the system assigned the two different versions randomly to two groups.We made sure that the students with LD and AD were divided in two equalgroups by including a variable in the online form to define the two groups(with and without LD). As the whole procedure was contacted on line, thestudents were not aware of their fellow student’s difficulties as this wasreported on line and all 184 students tried the application.

The results below are referring to the 24 students with attention disorders and LD.Twelve students (group A) used the application with graphical representation of theagent that portrays Emotional Affective Behaviour (EAB) and 12 (group B) withSophia that portrays Neutral Behaviour (NB). When we are referring to neutralbehaviour we mean that the agent had no emotional responses to the user i.e. the agent

Fig. 3 Sofia’s body movements

Educ Inf Technol

had a neutral expression during the learning process and the messages from the agentsdid not show empathy or care. We used three different evaluation methods; question-naires, tests and observation. Users could send messages to the agent (by choosingthem from a predefined messages list) and leave messages for other users as tipsreferring to difficulties they faced within the application and the way they resolvedthem.

& Questionnaires, Scales: Questionnaires which comprise open and closed questions.Students could answer with yes or no, choose one of the given answers, or indicatethe degree of their agreement to an answer using a scale. The questionnaires weregiven to students on paper.

& Tests: The tests were a set of questions or exercises with predetermined answers. Theanswer required a choice between a response to other (multiple choice, true / false).The tests used were a type of recurrence and were part of the e-learning application.They were presented at the end of the application after the theoretical part and werepart of the application. All test’s answers were covered in the theory part.

& Observation: The observation of student’s behavior was realized within the appli-cation itself by recording a user’s behaviour in the computer system. By monitoringdependent variables such as time spent in a theory or practice module, intensity ofmouse clicks and repletion of tasks after failure. Observation was systematic andreferred to the recording of predefined behavior in respect to the frequency,intensity and duration of user’s actions.

Fig. 4 Sofia’s facial expressions

Table 2 Emotions relevant to learning (Aist et al. 2002)

Anxiety-confidence Anxiety Worry Discomfort Comfort Hopefulness Confidence

Ennui-Fascination Ennui Boredom Indifference Interest Curiosity Fascination

Frustration-Euphoria Frustration Puzzlement Confusion Insight Enlightment Euphoria

Dispirited-Enthusiasm Dispirited Disappointed Dissatisfied Satisfied Thrilled Enthusiasm

Terror-Excitement Terror Dread Apprehension Calm Anticipatory Excitement

Humiliated-Proud Humiliated Embarrassed Self-conscious Pleased Satisfied Proud

Educ Inf Technol

We used Henri’s (1992) model of evaluation of online discussion groups. The modelis based on the level of participation and interaction of online groups and on theanalysis of the content of the messages that are transmitted during the learningprocedure. The model refers to a participation - social - interactive axis.

6 Results and discussion

The following factors were examined:

& Number of attempts to contact the agent& Positive or negative emotions that arose from the interaction with the agent& Success in the learning tasks& Success in the learning tasks after support from the agent& Number of tries after failure& Duration of attempts& Nature and quantity of the messages sent from the user to the agent

Results showed the following:

& Group A attempted to contact the application more than group B in all fields, social,technical assistance issues, subject matter issues. (The application gave users theability to send messages to the application (the agent) by selecting them from aspecific list of messages). The messages were divided into three categories (assis-tance on technical matters, assistance on subject matter, social nature).

& Group A formed basically positive feelings while group B neutral feelings. (Datacollected by the social messages concerning the user’s emotional state and byquestionnaires completed by users).

Regarding the cognitive - metacognitive axis results showed the following:

& The correct answers were more in group A. At the end of the application users wereasked to complete a test. (type of test: Repeat).

& Participants in group A had tried more than participants in group B (the computersystem recorded the efforts of each user, frequency, duration)

Fig. 5 Sofia in the learning environment

Educ Inf Technol

& After a failed user’s attempt to complete correctly the test and then receivebehavioural support from Sophia, group A had more correct answers than theparticipants of group B (the computer system recorded user’s behavior after a failedeffort).

For statistical analysis we used Pearson’s Chi-square (χ2) test and when certainrequirements were not fulfilled we used Fisher’s exact test and Monte Carlo’s test.Statistical analysis showed the following:

& Students communicated more with the application with Sophia with emotionalaffective behaviour (EAB) than the application with Sophia with neutral behaviour.

& Sophia with EAB is the catalyst that makes students communicate more with theapplication.

& Sophia affects users’ success in the learning tasks.& Sophia’s behaviour with EAB makes users try more times to complete the learning

tasks.& Sophia with EAB provides cognitive support and helps students overcome failure.

The percentage of social nature messages and help messages (help fortechnical issues and messages referring to the learning subject matter) isdifferent for the two e-learning systems. Fifty eight messages were categorizedand used for the analysis. As the formula 5nIJ> is not satisfied and thus control×2 is not powerful, we used Fisher’s exact test to statistically analyzed theresults. We found that p=0.041 therefore the condition that requires that p<0.05is satisfied thus we dismissed the null hypothesis in that the two variables (usercommunication through messages and teaching method) are independent from

Fig. 6 Messages send to e-learning system (to application with and without IEAs)

Educ Inf Technol

each other. Therefore Sophia’s behaviour is the catalyst that makes studentscommunicate more with the application (Fig. 6).

A factor that is examined is the nature of emotions created during the use of the e-learning application. The students from group A felt more positive emotions thanstudents from group B who mostly had neutral emotions.

One hundred and twenty four emotion instances were reported and used forthe analysis. Since p<0.05 there is evidence that the two variables of emotionsand teaching method are connected and that Sophia helps students to createpositive emotions during operation of the e-learning application (Fig. 7).

We also ran the positive emotion’s test created and laboratory tested by BarbaraFredrickson. It is consisted by 20 questions; ten of them are referring to positiveemotions and ten of them to negative emotions. The test was taken by users beforeand after operation of the program. The positivity ratio was increased the second timethe test was taken. Before the test the average positive ratio was 0,33. After operation ofthe e-learning system the ratio increased to 0,47.

To examine the effectiveness of Sophia, we asked participants to answer to10 questions referring to the subject matter in the end of the e-learningapplication. Each group had 120 responses (wrong, correct and without an-swer). Since p=0.18 there is indication that Sophia affects students’ success inanswering the questions and consequently in comprehending the lesson giventhrough the e-learning application (Fig. 8).

Users had the opportunity to try three times to give the correct answer. The test thatwas presented at the end of the application had ten questions. Group A tried overall 222times and group B tried 185 times. The frequency of responses after failing to give thecorrect answer and providing cognitive support by Sophia was analyzed. Thirty fiveanswers are correct in case of the learning system with IEA (Sophia with emotionalaffective behaviour) and 27 in the case of the learning system without IEA (Sophia with

Fig. 7 Nature of emotions created during the use of the e-learning application

Educ Inf Technol

neutral behaviour). Since p=0.046 there is indication that Sophia with EAB helpsstudents overcome failure (Fig. 9).

Students without LD and AD found to have slightly lower results fromstudents with LD but the statistical factor is not strong enough (p>0.05) tosignify difference between this learning group and their typical peers.

Fig. 8 Correct and wrong answers in the two e-learning systems

Fig. 9 Frequency of successful completion of learning tasks, in case of failing the first time and after supportfrom agent

Educ Inf Technol

7 Conclusions

The plurality of research projects and reports shows a high interest in the field ofIntelligent Emotional Agents and their use in e-learning applications. Research results(Lee et al. 2007; Beale and Creed 2009; Picard 2011) regarding the use of emotions incomputer systems, are promising.

The researchers may be able to conclude that this emotion-enhanced avatar wentsome way to support some of these students.

Our study concluded that our agent Sofia may enhance human computer interactionand learning experience, resulting in a more efficient, productive and enjoyable appli-cation that increases a learner’s performance. We conducted an experiment with 24participants with LD and attention disorders that used a learning system with an agentthat had emotionally affective behaviour versus a system with an agent with neutralbehaviour. Results showed that the presence of the agent’s empathic responses im-proved communication between users and learning environments and this lead to amore successful user’s performance in two respects; completion of learning tasks anduser satisfaction of the system.

Our results complement research work from neuroscience which state that theemotional charge of visual stimuli significantly modulates automatic attention inADD (Carretie et al. 2004).

IEAs might contribute to the improvement of communication between learners andmachines and help in designing intelligent learning environments that accommodatecognitive support to serve as educational tools for learners and educators. In the future,we might need to operate long test beds that will treat the relation between emotionalagents and users as an ongoing process that needs feedback from both participants andprovide evidence for the possibility of predicting a user’s emotions.

We might need to examine and access the efficiency and productiveness of thisprocedure, in a context sensitive manner, in specific e-learning applications. There arestill open research questions that need to be answered. Are agents independent from thelearning domain they are used for? Does this emotional intelligence have the sameresults for all learning groups? Other factors such as the design of the learningenvironment and storytelling techniques do play a role in enhancing such a systemwith emotional intelligence?

References

Aist, G., Kort, B., Reilly, R., Mostow, J., & Picard, R. W. (2002). Experimentally augmenting an intelligenttutoring system with human-supplied capabilities: Adding human-provided emotional scaffolding to anautomated reading tutor that listens. Proceedings of the International Conference on MultimodalInterfaces (Pittsburgh, PA, USA, October 14–16, 2002), pp. 483–490.

Bandura, A. (1993). Perceived self efficacy in cognitive development and functioning. EducationalPsychologist, 28(2), 117–148.

Barnes, M. A., Fletcher, J., & Fuchs, L. (2007). Learning disabilities: from identification to intervention. NewYork: The Guilford.

Bartolomé, A., Bergamin, P., Persico, D., Steffens, K., & Underwood, J. (Eds.). (2011). Self-regulated learning in technology enhanced learning environments: problems and promises.Proceedings of the STELLAR-TACONET conference, Universitat de Barcelona, Oct.1, 2010.Aachen: Shaker.

Educ Inf Technol

Baylor, A. L., & Kim, Y. (2003). Three pedagogical agent roles: designing, developing, and validating agentas expert, motivator, and mentors. Annual Conference of EDMedia: Honolulu, Hawaii. June 23–28.

Beale, R., & Creed, C. (2009). Affective interaction: how emotional agents affect users. International Journalof Human-Computer Studies, 67(9), 755–776.

Bell, P. D., & Akroyd, D. (2006). Can factors related to self-regulated learning predict learning achievement inundergraduate asynchronous Web-based courses? International Journal of Instructional Technology andDistance Learning, 3(10), 5–16.

Burleson, W., & Picard, R. W. (2007). Evidence for gender specific approaches to the development ofemotionally intelligent learning companions. IEEE Intelligent Systems, Special issue on IntelligentEducational Systems, 22(4), 62–69.

Caprara, G. V., Fida, R., Vecchione, M., Del Bove, G., Vecchio, G. M., Barbaranelli, C., et al. (2008).Longitudinal Analysis of the role of perceived self-efficacy for self-regulated learning in academiccontinuance and achievement. Journal of Educational Psychology, 100, 525–534.

Carretie, L., Hinojosa, J. A., Martin-Loeches, M., Mercado, F., & Tapia, M. (2004). Automatic attention toemotional stimuli: neural correlates. Human Brain Mapping, 22, 290–299.

Conati, C., & Maclaren, H. (2009). Empirically building and evaluating a probabilistic model of user affect.User Modelling and User-Adapted Interaction, 19(3), 267–303.

Dehn, D., & Van Mulken, S. (2000). The impact of animated interface agents: a review of empirical research.International Journal of Human–Computer Studies, 52(1), 1–22.

Gans, A., Kenny, M., & Ghany, D. (2003). Comparing the self-concept of students with and without learningdisabilities. Journal of Learning Disabilities, 36(3), 287–295.

Henri, F. (1992). Computer conferencing and content analysis. In A. R. Kaye (Ed.), Collaborative learningthrough computer conferencing: the Najaden papers (pp. 117–136). Berlin: Springer.

Howe, K. (2009). Anthropomorphic Systems: An Approach for Categorization, IDGD '09 Proceedings of the 3rdInternational Conference on Internationalization, Design and Global Development: Held as Part of HCIInternational 2009, (SanDiego, CA,USA, July 19-24, 2009), pp 173–179. Springer-VerlagBerlin: Heidelberg.

Katz, L., Brown, F., Roth, R., & Beers, S. (2011). Processing speed and working memory performance inthose with Both ADHD and a reading disorder compared with those with ADHD alone. Archives ofClinical Neuropsychology, 26(5), 425–433. Oxford University Press.

Kember, D. (2001). Beliefs about knowledge and the process of teaching and learning as a factor in adjustingto study in higher education. Studies in Higher Education, 26(2), 205–221.

Kienle, A., & Wessner, M. (2006). The CSCL community in its first decade: development, continuity,connectivity. International Journal of Computer-Supported Collaborative Learning (ijCSCL), 1(1), 9–33.

Kinnebrew, J., & Biswas, G. (2011). Self-regulated learning in teachable agent environments. Journal of e-Learning and Knowledge Society - EN, 7(2), 19–35.

Klassen, R. (2010). Confidence to manage learning: the self-efficacy for self-regulated learning of earlyadolescents with learning disabilities. Learning Disability Quarterly, 33(1), 1–12. Publisher: Council forLearning Disabilities.

Klassen, R., Tze, V., & Hannok, W. (2013). Internalizing problems of adults with learning disabilities: a meta-analysis. Learning Disabilities, 46(4), 317–327.

Kolb, D. A. (1984). Experiential learning: experience as the source of learning and development. EnglewoodCliffs: Prentice-Hall.

Kort, B., Reilly, R., & Picard R. (2001). An Affective Model of Interplay between Emotions and Learning:Reengineering Educational Pedagogy-Building a Learning Companion, pp.0043, Second IEEEInternational Conference on Advanced Learning Technologies (ICALT’01).

Lee, T. Y., Chang, C. W., & Chen, G. D. (2007). Building an interactive caring agent for students in computer-based learning environments. In Proceeding of International Conference on Advanced LearningTechnologies (Niigata, Japan, July 18–20, 2007), pp. 300–304.

Maldonado, H., Lee, J. R., Brave, S., Nass, C., Nakajima, H., Yamada, R., et al. (2005). We Learn BetterTogether: Enhancing e-Learning with Emotional Characters. In Proceedings, Computer SupportedCollaborative Learning, Taipei, Taiwan.

Mangione Walcott, C., & Landau, S. (2004). The relation between disinhibition and emotion regulation inboys with attention deficit hyperactivity disorder. Journal of Clinical Child & Adolescent Psychology,33(4), 772–782.

McGillivray, J. A., & Baker, K. L. (2009). Effects of comorbid ADHD with learning disabilities on anxiety.Depression, and aggression in adults. Journal of Attention Disorders, 12(6), 525–531.

Newman, R. S. (2002). What do I need to do to succeed… when I don’t understand what I’m doing!?:developmental influences on students’ adaptive help seeking. In A. Wigfield & J. Eccles (Eds.),Development of achievement motivation (pp. 285–306). San Diego: Academic.

Educ Inf Technol

Palincsar, A., Magnussen, S. J., Marano, N., Ford, D., & Brown, N. (2001). Designing a community ofpractice: principles and practices of the GIsML community. Teaching and Teacher Education, 14(1), 5–19.

Panteliadou, S. (1995). Students’ knowledge about the training people with learning difficulties, The.Educationals, 37–38, 148–158.

Picard, R. W. (2011). Emotion research by the people, for the people. International Society for Research onEmotion, SAGE Publications. Emotion Review, 2(3), 250–254.

Pintrich, R. A. (2004). Conceptual framework for assessing motivation and self-regulated learning in collegestudents. Educational Psychology Review, 16(4), 385–407. Publisher: Springer.

Steptoe, W., Steed, A., & Slater, M. (2013). Human tails: ownership and control of extended humanoidavatars. IEEE Transactions on Visualization and Computer Graphics, 19(4), 583–590.

Usher, E. L., & Pajares, F. (2008). Self-efficacy for self-regulated learning: a validation study. Educationaland Psychological Measurement, 68, 443–463.

Wong, B. Y. L. (1985). Metacognition and learning disabilities. In D. L. Forrest-Pressley, G. E. MacKinnon, &G. T. Waller (Eds.), Metacognition, cognition, and human performance, Vol. 2. Instructional practices(pp. 137–180). New York: Academic.

Educ Inf Technol