Grimace Docu

7/28/2019 Grimace Docu

1/31

Grimace project documentation

March 17, 2009

Oliver Spindler Thomas Fadrus

0100611 / 066 935 0075129 / 033 532

Vienna University of TechnologyInstitute for Design and Assessment of Technology

Supervisor: Ao.Univ.-Prof. Dr. Peter Purgathofer

1


2/31

Contents

1 Introduction 3

1.1 Emotion model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Related work 4

3 Design 6

3.1 Graphical approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.2 The Uncanny Valley . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.3 Our approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.4 Face model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4 Development 11

4.1 Selected technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.2 Iterative development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5 Technical details 13

5.1 Muscles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5.2 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5.3 Emotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

5.4 Splines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5.5 Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

5.6 Stroke styles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

5.7 Facedata file format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5.8 Deployment and use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5.9 Class diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

6 Results 24

6.1 Conclusion and future directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

References 29

2


3/31

1 Introduction

Emotions are important guides in our lives. Neglected since Plato as something to be overcome by

rational thought, newer research shows emotions play a key role in problem-solving and sense-

making (e.g. Ekman 2003, Norman 2004). Emotions are an aspect of affect, which is an umbrellaterm for cognitive phenomena apart from rational thought. Other kinds of affect include moods

or personality traits.

Information entities, be it words, objects, ideas, music pieces or photographs, carry meaning.

An entitys meaning can be seen as the sole reason for us to get interested in it; meaningless

and pointless are synonyms. Meaning is frequently seen to be comprised of two complemen-

tary aspects, denotation and connotation (e.g. Ogden et al. 1969; Garza-Cuarn 1991; Barthes

1996). Denotation is the actual, or, intended meaning of something; the meaning of words you

find in dictionaries, the plot of a film, the score of a music piece. Affective connotation is the

emotion or feeling which we believe to be communicated by an entity, or which is aroused in us

when exposed to the entity. With the rise of experimental psychology, this aspect has received

more attention. An important step was the work of Osgood (1957), who quantitatively measured

the affective connotation of words and from their results inferred a semantic space or affective

space. Nowadays, connotation usually refers to this kind of meaning.

Seeing the importance emotional information plays in our understanding of the world, it

should be natural to use this kind of information in the description of content on the web. How-

ever, content description is currently focused on denotation, while there are only few attempts

to target affective connotation.

A notable exception is the common usage of emoticons in textual communication. It is no

coincidence that we use emoticons, which are symbolic abstractions of facial expressions, toexpress our emotional state and the affective connotation of our textual messages. Animals and

humans across all cultures express emotions through facial expressions. This hardwired relation

between emotions and facial expressions makes information about emotions an ideal candidate

for describing affect on the web.

Emoticons show that detailed reproduction of facial expression is not necessary to convey

emotions unambiguously. Reduced detail and complexity allows to focus on those features

which are necessary to effectively display a specific facial expression. Our hypothesis is that

concentrating on relevant facial features shows emotions more clearly.

1.1 Emotion model

The nature of emotions is an ongoing scholarly debate. Over the years, many different expla-

nations have been put forward, resulting in various emotion models. Most models are based

on one of two major approaches. In the dimensional approach, emotions are being described by a

small number of independent dimensions usually two or three. For instance, the circumplex

model (Russell, 1980) describes emotion via the dimensions valence and arousal.

The categorical approach, on the other hand, assumes a finite number of basic emotions, which

describe innate emotional reactions. These basic emotions are believed to have developed evo-

lutionary, guiding human behaviour in a world of unforeseeable events (Ekman, 1999, Sloboda

and Juslin, 2001).

3


4/31

Grimace was based on the book Making Comics by Scott McCloud (2006), a manual for artists

on how to draw comics. Chapter 2 of the book deals with how to convicingly draw facial

expressions. In this chapter, McCloud develops an emotion model which follows the categorical

approach, postulating 6 basic emotions:

Anger Joy Surprise Disgust Sadness Fear

Table 1: The basic emotions defined by McCloud (2006)

This list is, in turn, based on the research of Paul Ekman. In a series of cross-cultural exper-

iments, Ekman et al. (1972) showed photographs of facial expressions to members of differentcultures. Since the posed expressions could be judged accurately, he inferred the universality of

several specific emotions. While the existence and the number of basic emotions is still debated,

his results show that these 6 emotions can be judged correctly.

Artists who want to draw convincing portraits of humans need to be expert observers of facial

expression. The Artists Complete Guide to Facial Expression by Gary Faigin (1990), an excellent

guide to drawing detailed facial expressions, is based on a categorical model. McCloud (2006)

takes visual cues for his model from Faigins work and shows how to depict emotion through

facial expressions in the world of comics, offering the ideal framework for our work.

In McClouds model, these basic emotions can be blended to achieve complex emotions. He

compares this process to the way arbitrary colours can be mixed from three primary colours.

Accordingly, he calls the 6 basic emotions primaries, and blendings of two basic emotions secon-

daries. He gives example depictions and names for all primary and secondary emotions, which

we used as the basis for our work. He asserts that mixtures can occur in arbitrary intensity and

might even include three emotions.

1.2 Goals

The goal of the project Grimace was to build a facial expression display using web technology

which effectively conveys emotions as depicted by McCloud (2006, p.83-85). This includes any

primary (or basic) emotion and any secondary emotion (blendings of two basic emotions) in

arbitrary intensity. The result should be a free component which can be easily integrated into

other projects to enable the addition of emotional expressiveness to interactive systems.

2 Related work

Realistic approaches

The development of dynamic face models is an important research area. Most of this work is

being undertaken in the field of affective computing, which aims to enhance interactive systemswith affective awareness and expressiveness. In a principal work for this research area, the

4


5/31

goal was defined as making machines less frustrating to interact with. (Picard, 1997, p. 214)

Interactive systems should recognise the affective state of the user and adapt their output and

behaviour accordingly.

One commonly proposed solution is the use of embodied conversational agents (ECA). In such

an environment, the system interacts with the user through an avatar with human-like appear-

ance and expressiveness. ECA strive to be believable conversation partners. Therefore, they need

to be able to express their emotions through facial expressions. Since an ECA also speaks, an-

other requirement is appropriate facial animation which supports the impression that the ECA

is actually speaking. Usually, an ECA is modeled as a 3D head.

Wang (1993) undertook an early yet impressive attempt at building a three-dimensional face

model, which predates affective computing. The face consists of a set of hierarchical b-spline

patches, which can be manipulated by simulated muscles.

Pan et al. (2007) focus on the notion that the basic emotions as postulated by Ekman (see

section 1.1) are not the ones which are actually needed for conversations with believable agents.They developed a 3D character which expresses affective states like agreement, interest, thinking

and uncertainty through facial expressions and animated gestures. Ochs et al. (2005), on the

other hand, base their three-dimensional ECA on basic emotions, but allow them to be blended

to achieve more subtle affective states.

Albrecht et al. (2005) depart from the concept of basic emotions and base their agent on a

dimensional approach. Their agent supports a text-to-speech system. The system analyses a

text for certain terms whose coordinates for three affective dimensions are stored in a table.

With these values, the system augments the spoken text with appropriate facial expressions.

Zhang et al. (2007) follow a similar approach. In their system, high-level affective dimensions

are translated into MPEG 4 Facial Animation instructions. This standard is described by Pandzic

and Forchheimer (2002) in detail.

There are publicly available ECA frameworks. The CSLU Toolkit1 is one example. It is a

comprehensive framework which facilitates application development with the included ECA.

Another example is Xface2, which achieves an impressive level of realism. It makes use of the

MPEG 4 Facial Animation standard.

Comic-like approaches

The approaches described above usually employ characters which are designed for a high level

of realism. Such level of realism, however, is not necessary for this project. We aim to unam-biguously express emotions through facial expressions. McCloud (2006) shows that a certain

level of abstraction is possible without any loss of expressiveness (see section 3). There are a few

attempts to develop face models which aim to achieve such abstracted comic-like appearance.

Bartneck (2001) conducted a study on the well-suitedness of different representations of facial

expressions. Three representations at different abstraction levels were compared; photographs

of a real face, an embodied conversational agent from the CSLU Toolkit, and highly abstracted

emoticons (black & white, 10 9 pixels in size). Subjects rated facial expressions of these repre-sentations for convincingness and distinctness. Results showed that the very abstract emoticon

1h t t p : / / c s l u . c s e . o g i . e d u / t o o l k i t /

2h t t p : / / x f a c e . i t c . i t /

5


6/31

was perceived to be as convincing as the photograph, while distinctness was rated as decreasing

with increasing abstraction levels.

Tanahashi and Kim (1999) developed a comic-like face model. Their model is designed to

express four out of six basic emotions as defined by Ekman. They also experiment with exagger-

ation of the features and with the addition of symbols to achieve a higher level of expressiveness.

The symbols employed are highly culturally dependent. Iwashita et al. (1999) also follow a comic

approach in their system for caricature development. An interesting point is the use of a survey

to improve the validity of the system, in which they asked subjects to pick the most expressive

and convincing representation out of a few alternatives.

Schubert (2004) describes a very simple comic-like face model, which only consists of eyes

and mouth. These features are represented by parabolic functions. His model is based on a

dimensional approach. The shape of the mouth indicates valence, the degree to which eyes

are opened represents arousal. The model is used to visualise emotions expressed by music.

Emotions are very short experiences which are not constant over the duration of a song. Themodel expresses these changes in affect.

Discussion

Our goal is not to build a conversational agent; we do not include animation or a speech com-

ponent. The realistic three-dimensional approaches are designed for a purpose quite different

from our goals. Although embodied conversational agents can express emotions through facial

expressions, we believe that expressiveness can be increased further. For instance, the ECA cited

above do not include wrinkles in their design. However, wrinkles are an essential aspect of some

facial expressions (disgust, for example).The comic-like models we have found, on the other hand, reduce the facial complexity to

an extent which results in a loss of expressiveness. Tanahashi and Kim try to counter this with

the addition of symbols. This is something we wish to avoid, because symbols are culturally

dependent, while cross-cultural research has shown that facial expressions on their own are

universally understood.

This brief survey leads to the principles on which we built our system.

3 Design

3.1 Graphical approach

Our main premise was to keep everything simple and minimalist. We also applied this principle

to the visual aspects of Grimace. Inspired by the approach applied in figure 1 by McCloud, we

decided that we need to make our facial representation somewhat comic-like but also natural

enough to be able to stand on its own, because we did not want to use symbols to augment it.

As seen in figure 1, it is not necessary to use an entirely realistic representation to achieve

emotional readability. The first three images all show the same emotion, which can be reliably

identified as an anxious state of mind. In fact, it even gets easier to identify this state when the

head is simplified. However, this cannot be done ad infinitum, because there is a point when

too much of the facial features are omitted and the intended emotion may not be recognized

6


7/31

Figure 1: McCloud (2006, p.96)

any more. In figure1, this point is reached a little bit left of the dotted line. The face has been

stripped of all the wrinkles and even the primary features like the eyes mouth and nose are mere

dots and lines. It is difficult to read the emotion on the face. Symbolic augmentation, in this case

drops of anxious sweat emenating from the face, is needed to make it work again.

We did not want to use symbols in our implementation, because this would make blending

different emotions together far more difficult. Furthermore, symbols arent as easily recognised

across different cultures (see section 1.1).

Omitting all the unnecessary wrinkles and features makes the face easier to recognize and it

would also ease the implementation and aid the performance of the system. Furthermore, if the

face was too humanlike the viewer wouldnt tolerate if something wasnt completely right. The

comic-like face is better suited for our approach because it would allow us to work more freely.

3.2 The Uncanny Valley

Another reason why we opted for a comic-like approach is the notion of The Uncanny Valley.

The concept was introduced in a short essay by robot-researcher Masahiro Mori (1970). In this

essay, he describes the phenomenon that when something (e.g. a robot) looks almost human-

like, it causes aversion rather than sympathy. As it can be seen on figure 2, this only happens at

the very end of the human likeness graph. Mori also states that movement amplifies the aversive

effect. So that something that looks almost like a human and moves in an almost human fashion,

causes a repulsive reaction in an observer.

A highly realistic 3D representation of a human would be very near to the right wall of

the valley, but obviously deep in the eerie area of the uncanny. In contrast, a comical face

would be located somewhere on the left slope of the ascent to humanoid robot, which puts it

comfortably out of the uncanny area. Thats exactly why we chose to use a comic-like appearencefor Grimace, because it is far more difficult to get something to look right if its supposed to

be very humanlike. Small errors in the appearance or movement can have disastrous effects,

because we are so used to seeing human beings around us all day, that slight deviations can

cause a repulsive reaction.

Bartneck et al. (2007) conducted a study in which they tried to find out if Moris predictions

were accurate. They used pictures of robots at different levels of human-likeness, as well as

pictures of real humans. It has to be noted that Bartneck et al. measured the level of likeability in

their study, not the level of familiarity which Mori described. Their findings did not confirm

the predicted raise in likeability. More abstract depictions were perceived as more likeable

than pictures of real humans. They note that knowledge about whether the photo showed a

7


8/31

still

100%50%

zombie

prosthetic handcorpse

human likeness

industrial robot

stuffed animal

healthy

person

uncanny valley

bunraku puppet

humanoid robot

moving

familiarity

Figure 2: Graphical representation of the Uncanny Valley. Taken from Geller (2008), based onMori (1970)

real human or a robot did not have an influence on likeability. Instead, results were highly

dependent on the perceived beauty of the stimulus. From their results, they infer that it might bemore accurate to speak of an Uncanny Cliff.

Geller (2008) gives an up-to-date examination of the concept. He notes that there are a

number of examples which contradict Moris predictions. One important factor which influences

familiarity is the depiction of the eyes. There might be an uncanny threshold; the eyes need to

be very well done to be acceptable. He closes with a recent quote of Mori, in which he, too,

questions the predicted raise in familiarity. Mori now believes that the human ideal is best

expressed by the calm facial expressions of Buddha statues.

3.3 Our approach

So the first step was to find a simplified version for each of the basic emotions laid out in table 1.

To do this, we used the representations of McCloud (2006), omitted some wrinkles and removed

the plasticity (see figure 3). As it was difficult to judge which ones could be left out, it would

have been helpful to conduct an early experiment with these reduced representations, to verify

that all the minimized faces still conveyed the intended emotion.

8


9/31

Figure 3: Simplified faces

Wrinkles that form when muscles in the face are contorted are as important for emotion-

recognition as the features themselves. It was also necessary to take some time and find out

which wrinkles comprise the minimal set to accurately represent all the emotional states and

still be manageable in terms of implementation complexity. As it can be seen in figure4 we

first identified the wrinkles that occur in most of the basic emotions, like the wrinkles in purple

around the mouth or the ones around the eyes in red, blue and green. Than we incrementally

added the ones that help to recognise a certain emotion and added them to the set.

Figure 4: Essential wrinkles

Basically wrinkles are crevices that form when skin is compressed through muscle tension.

To simulate this behaviour of appearing and disappearing wrinkles we used opacity. When a

wrinkle would form on a human face we would slowly raise the opacity of the correspoding

spline.

We realised that most of the facial expressions are fairly symmetrical and decided to model

only half of the face and then mirror all of it over to the other side. This would ease development

and also reduce the redundancy in the system.

3.4 Face model

We believe that closely following the biological process of how emotions result in facial expres-

sions increases the credibility and clarity of the displayed emotion. In humans and animals

alike, facial expressions result from the contraction of facial muscles. Therefore, the face model

was implemented following a muscle-based approach. The muscles are basically a system of

influences. When tension is applied to them they deform the facial features they are attached to.

These muscles are themselves influenced by emotions. These are defined as groups of muscles

which are tensioned if an emotion is applied to the face. The tension is calculated with a mathe-

matical function which was specified to match the non-linear behaviour of human muscles. This

approach is explained in detail in section 5.5.

Facial features had to be translated into graphical elements that could be transformed algo-rithmically. We found the necessary combination of accuracy and flexibility in Bzier splines. All

9


10/31

facial features and muscles are represented by one or more splines. Figure 5 shows the first at-

tempt to represent the facial features with a minimal number of Bzier splines. This early model

proved to be not capable of expressing all necessary facial expressions and was augmented in

later iterations. The shape of Bzier splines is determined by a very small number of control

points. The idea was to connect the virtual muscles to these control points in such a manner that

contraction of the muscles would result in natural looking transformations of the splines.

Figure 5: First attempt to represent facial features via Bzier splines.

If a specific emotion is dialed in, the system checks which muscles are involved and calculates

the corresponding tension through the function specified for this particular set of emotion and

muscle. If necessary, it blends multiple muscle-tensions together if they are attached to the

same feature point. Furthermore, if more than one emotion is applied, Grimace also handles the

interpolation based on a priority assigned to each muscle. After the calculation phase the result

is used to deform the splines which make up the features.

emotions

featuresstatic dynamic

muscles

Figure 6: Face model

10


11/31

4 Development

4.1 Selected technology

Grimace has been developed in Actionscript 3 and is being deployed as a Flash / SWF file.Actionscript 3 is the scripting language used by Adobe Flash and Adobe Flex. Though not

advertised, the language can be used on its own. The technology was selected for several reasons:

Free: The Flex 3 SDK is available open source under the Mozilla Public License. It containsMXMLC, an Actionscript 3 compiler, which allows generation of SWF files entirely through

Actionscript 3 code, without the need for an IDE like Adobe Flash IDE or Adobe Flex

Builder.

Optimised for dynamic vector graphics: Flash originated as a vector-based animation tool andoffers comprehensive vector-drawing functions.

Optimised for the web: Flash is a web-centric technology which delivers small file sizes andcan be conveniently integrated in web projects.

Ubiquity and consistency: The Flash player is available for all major operating systems andhas an extremely high install base. SWF files are displayed in exactly the same way across

platforms and browsers.

4.2 Iterative development

After the visual style had been laid out we started with the technical implementation. As already

stated we tried to keep the face model as simple as possible to keep it managable and also

performant. This approach should be applied throughout the whole system, so we started again

with a minimum set of splines to define the features. First we modeled the eyes, as they are one

of the simpler forms in the human face. They are fairly static and basically just need to open

and close. This is an oversimplified assumption, but it sufficed to take the complexity of the

modelling process away.

In the course of developing the face model, these simple splines where reworked and ex-

tended to cover the whole gamut of facial expressions. This gamut represents the entirety of

facial expressions that can be covered with a certain set of emotions through blending. We had

to model our system so that it would cover as much of this gamut as possible. To achieve this,we iteratively added splines and tried to model every possible facial expression with them. If the

simple spline turned out to be insufficient it was extended and reevaluated until it was sufficient

to represent all of the basic emotions.

The next step was to define the muscles that would influence the features to form the respec-

tive facial expressions. Again it was our premise to keep the muscular system as uncomplex

as possible without sacrificing expressiveness. Muscles were defined and linked to the spline

points of their corresponding feature. Then we had to try and match all the basic emotions with

this setup and when it wasnt possible to mimic all the emotional states it had to be adjusted

and the trial phase had to be repeated until every necessary facial expression could be formed

with the defined muscular system.

11


12/31

0 0.25 0.5 0.75 1

0.25

0.5

0.75

1

Anger

Tension

Polynomial mapping

Sine mapping

Figure 7: Muscle tensions were plotted and interpolated for each emotion.

After that a similar procedure had to be executed to model the wrinkles and tie them to the

muscle system. Unfortunately it wasnt possible to simply connect the wrinkles to the same

muscles that moved the feature splines, because the underlying physics are far more complex

than our minimalist system could reproduce, so we had to define a seperate muscle system

for the wrinkles because we needed very fine-grained control over how they moved. This also

proved to be very helpful at a later stage when it was necessary to blend different emotions

together, because we had separate control over the muscle systems.

When we had verified that the system was able to represent the basic emotions in this static

setup we had to put it all into motion. McCloud (2006) again proved to be a valuable resource

at this stage. He already defined how a face changes with different emotional intensities very

precisely. We manually modelled four gradual steps for every emotion, visually aligning them

with McClouds illustrations of the according state. Then we printed out the corresponding

muscle tensions and put them into a table. After this step we had a numerical representation of

the motion flow of a certain emotion from neutral to a fully expressed state.

For every muscle we dialed these values into Grapher.app and either manually matched amathematical function to the point set or used so called curve fitting the get an interpolated

polynomial function. For example, figure 7 shows the relationship of two muscles with anger.

The indicated forms are approximated by two mathematical functions. These functions were

then implemented via mappings (see section 5.5). Now we could define an emotion as a set

of muscle influences with a mapping, which specifies how the tension changes with different

intensities of this emotion.

After this step had been repeated for every muscle, every emotion and all the wrinkles we

had a pretty good system to dial in a basic emotion in a continous way and get a meaningful

representation of the defined state. The only thing left to do was to make the combination of

emotions work as well. In parts this was automatically achieved by averaging the mathemat-

12


13/31

ical functions that formed the basic emotions, but human emotions arent just mathematical

functions so a mixture of different emotions is hardly ever just an average state between the

corresponding basic emotions. To cope with this a priority was assigned to every muscle. The

ones with higher priority where favored in the blending process. A lot of manual work was

required to determine the priorities so that every mixture resulted in a meaningful facial expres-

sion. This was quite tedious, but our system proved to be very manageable and our commitment

to simplicity helped a lot in this phase.

When the face was capable of expressing any primary or secondary emotion, the component

was adapted for deployment. This included the addition of a JavaScript API, which allows full

control over the faces capabilities, and the construction of a project website.

5 Technical details

Grimace follows a muscle-based approach and thus mimics the way biological faces operate.In human and animal faces, facial expressions result from contraction of facial muscles. Facial

muscles are, unlike muscles in any other part of the body, only fixed to bones at one end, while

the other one is attached directly into the facial skin. This unique property allows the wide range

facial expressions humans are capable of displaying.

Our face model consists of three major components: emotions, muscles and features.

Features, which are the visible elements of a face. Features can be transformed by muscles.Typically, this includes dynamic features like eyes, eyebrows, mouth and wrinkles, as well

as static features like nose and head outline.

Muscles, which are the link between emotions and features. The shape of a muscle isdefined by a spline and when contracted can move an arbitrary number of control points

along its path.

Emotions, which are the high-level concept that influences a number of muscles in an arbi-trary fashion. Each emotion affects specific regions of the face and results in familiar facial

expressions.

Figure 8 illustrates how these three components work together to achieve a facial expression. It

shows the mouth and its surrounding muscles for a neutral expression and four states of anger.

The shape of the mouth is represented by two features, upper lip and lower lip (shownas ). A feature consists of several control points. The mouth is surrounded by several muscles.

A muscle has a defined path (shown as ) and a current tension (the dot along its path).

Each control point can be influenced by multiple muscles (influences are ). When a muscle is

contracted, it moves its tension dot along its path. Any control point which is influenced by the

muscle is then moved, which results in a change of the features shape. Now, when an emotion

is present the example shows the influence of anger several muscles contract simultaneously.

In the following, each of these components and their underlying technologies are being de-

scribed. This is followed by a brief description of how Grimace can be put to use in other

projects. A complete overview of all the classes is given in a UML-style class diagram.

13


14/31

0.0 0.25 0.5 0.75 1.0

Figure 8: Influence of anger on muscles surrounding the mouth.

5.1 Muscles

As explained before, facial muscles are fixed to a bone at one end, and attached to skin at the

other end. When muscles contract, they shorten and thus pull the skin towards the point wherethey are attached to the bone.

We simulate this behaviour. The shape of a muscle is defined by a spline (see section 5.4).

However, unlike real muscles, muscles in Grimace have no width. Thet e n s i o n

parameter of a

muscle corresponds to the position t [0, 1] along the spline. Thus, t = 0 is a completely relaxedmuscle, while t = 1 represents maximum tension. A muscle can be defined with the parameter

i n i t T e n s i o n

, which defines the neutral state for this muscle. This defaults to 0, but in some

cases, a neutral face i.e. no emotion is active results in contracted muscles. An example is

Levator palpebrae, which controls the upper eye lid. Since the eyes are halfway open in neutral

state, this muscle is defined withi n i t T e n s i o n

.

The tension of a muscle, or rather, the distance between the points Q(t = initTension) and

Q(t = tension) influences the position of feature nodes (see section 5.2: Node influences). In

turn, the tension of a muscle is calculated from the emotions which exhibit an influence on the

muscle (see section 5.3).

Finally, Muscles are grouped into instances ofM u s c l e G r o u p

. This grouping is optional, but

currently muscles are divided into feature muscles and wrinkle muscles, defining additional

muscles which simulate the wrinkles resulting from when feature muscles contract. This is a

point where we had to leave an accurate biological representation to achieve the desired facial

expression gamut.

5.2 Features

Features, the visible parts of a face, can be transformed by muscles. TheF e a t u r e

class encapsu-

lates distinct facial features, e.g. the upper lip, an eyebrow or a wrinkle.

A feature is comprised of one or more segments, which are instances of theF e a t u r e S e g m e n t

class. The shape of a segment is defined by a spline. Thus, a feature can take an arbitrary shape

by connecting several segments.

14


15/31

Node influences

A spline has two endpoints and 0 or more control points, referred to as nodes. Every point is

represented by theF e a t u r e N o d e

class and can be influenced by an arbitrary number of muscles.

For every node-muscle influence, aw e i g h t

parameter is stored.For n registered muscles, the position of node N is evaluated in the following way: For each

registered muscle M we calculate the distance between the muscles position resulting from its

current tension v, and the position resulting from its initial tension t. The distance is scaled by

the respective weight factor w. The nodes initial position N0 is then translated by the resulting

vector.

N = N0 +n

i=1

(wi (Mi(v)Mi(t)))

Fills

Features can also be filled arbitrarily, represented by theF e a t u r e F i l l

class. Fills can also be

influenced by muscles, thus adding a way to add animation. For every fill, a FeatureNode

represents a pivot point, which can then be influenced by muscles like any other node and

moves the whole fill when translated.

Alpha mapping

Not every feature is constantly visible. Wrinkles result from tightening of facial skin and thus

only become visible when certain muscles are contracted. To simulate this behaviour, the visibil-

ity of features can be mapped to the tension of a muscle. The relation is not direct but mediatedthrough mappings (see section 5.5). In this way, the feature opacity can be controlled flexibly.

5.3 Emotions

Emotions are the high-level concept which we aim to display via facial expressions. In real faces

as well as in our implementation, the 6 basic emotions we have implemented result in distinct

facial expressions, which have been said to be recognisable cross-culturally (see section 1.1).

The presence of an emotion is represented by a parameter value [0, 1], where value = 0means the emotion is not present, and value = 1 represents maximum influence of an emotion.

When an emotional state is present, it results in simultaneous contraction of a set of muscles.This contraction does not follow

v a l u e

linearly. For instance, some features only start to be

influenced when an emotion is strongly present, while others are continously influenced, but

more strongly in early than in later stages. Therefore, for every emotion-muscle influence, we

have defined a mapping which allows flexible control over how a muscle is contracted for an

emotion state (see section 5.5).

Our emotion model subscribes to the idea that complex emotions are in fact mixtures of

basic emotions. If two or more emotions are present simultaneously, more than one set of

muscles is affected. However, since different emotions sometimes influence the same muscles,

influences have ap r i o r i t y

parameter. When a muscle is influenced by more than one emotion

simultaneously, priority defines the influence of each emotion on the final tension of a muscle.

15


16/31

For instance, a genuine smile not only influences the shape of the mouth, but also results in

squinting of the eye. However, a result of feeling surprised are widely-opened eyes. If joy and

surprise are experienced together, the eyes remain open, because surprise has a stronger influence

on the eyes than joy. This is represented by different priorities.

Given an influence of n emotions, with emotion values vi, influence priorities pi and raw

emotion tensions ti, the final tension of a muscle is calculated as:

t =n

i=1

vi pi ti

1n

i=1(vi pi)

5.4 Splines

Spline is the common term for the use of piecewise parametric curves in computer graphics.

A spline has two endpoints and may have control points in between. With splines, complexshapes can easily described or approximated by very few points. Bzier curves are a form of

parametric curves which are commonly used in vector-drawing and animation software. It is a

notable property of Bzier curves that the curve does not run through the control points but is

merely pulled towards them.

All shapes in Grimace facial features and muscles are based on straight lines and Bzier

curves. They offer an easily understandable way to model the face, and the selected technology

offers native support for these types of splines. Facial features are all visible components of the

face, e.g. the eyes, the mouth or wrinkles. Each feature consists of one or more segments, and

the shape of each segment is defined by exactly one spline. In addition, muscles are also based

on Bzier curves; the shape of each muscle is defined by exactly one spline.

Splines implement theI S p l i n e

interface. The interface defines theg e t P o i n t ( t )

method,

which calculates the location of a point along the spline given the position t [0, 1], where t = 0is the starting point of the spline, and t = 1 is the endpoint.

The following splines are available for muscles and facial features:

Figure 9: Line

i i

Figure 10: Quadratic Bzier

i i

Figure 11: Cubic Bzier

Line

A spline which connects two endpoints with a straight line. Flash offers the native drawing

methodl i n e T o

for this spline type.

16


17/31

Quadratic Bzier

A Quadratic Bzier curve has one control point. Flash offers the native drawing methodc u r v e T o

for this spline type.

The parametric form of a Quadratic Bzier curve is:Q(t) = P0(1 t)2 + P1 2t(1 t) + P2 t2 , t [0, 1]

Cubic Bzier

A Cubic Bzier spline has two control points and offers great control over the curve form. If two

or more Cubic Bzier splines are concatenated, they offer enough flexibility to draw all necessary

facial features, including the mouth, which demands the greatest flexibility.

The parametric form of a Cubic Bzier curve is:

Q(t) = P0(1 t)3 + P1 3(1 t)2t + P2 3(1 t)t2 + P3 t3 , t [0, 1]

Flash does not offer a native drawing method for Cubic Bziers. However, the form canbe approximated by lower-complexity curves like Quadratic Bzier splines. The more lower-

complexity curves are used, the more accurate the form of the approximated curve becomes.

1

2

1

2

1

8

7

8

1

2

1

2

1

2

1

2

3

83

8

C0=Q

0

Q1

R1

S1

T1

Q2=R

0

R2=S

0

S2=T

0

C3=T

2

C1 C

2

Figure 12: Fixed Midpoint approximation

We have selected the Fixed Midpoint approximation method described by Groleau (2002). It

approximates a Cubic Bzier with four Quadratic Bziers and offers a good trade-off between

accuracy and calculation complexity. The approach is illustrated in figure 12. Given the four

points of a Cubic Bzier C to be approximated, endpoints and control points for Quadratic

Bziers Q, R, S and T are calculated in the following way:

17


18/31

H0 =C0+C1

2 +C1+C2

2

2=

C0 + C24

+C12

H1 =

C1+C2

2 +

C2+C3

22 =

C1 + C34 +C22

Q0 = C0 ; Q1 =5C0 + 3C1

8

R1 =7H0 + H1

8; R2 =

H0 + H12

Q2 = R0 =Q1 + R1

2

S0 = R2 ; S1 =H0 + 7H1

8

T1 =3C2 + 5C3

8; T2 = C3

S2 = T0 = S1 + T12

In our implementation, the spline can be used like a regular Cubic Bzier with two endpoints

and two controlpoints, while the approximation is handled internally by the class.

Joiner

For some facial features, i.e. the shape of the mouth or several wrinkles, a single Cubic Bzier

curve does not suffice. In these cases, two or more curves may be joined together to form a curve

with additional flexibility. In these cases, one feature consists of more than one segment.

Parametric continuity Cn

is a description of the smoothness of concatenated parametric curves:

C0: curves are joined.

C1: first derivatives are equal.

C2: first and second derivatives are equal.

Without additional measures, connected Bzier curves only offer C0 continuity. If two connected

splines are to appear as a single and coherent curve, however, at least C1 continuity is necessary.

Q0

Q1 Q

2=R

A

Q3=R

0

C1 continuity

S0=R

3

R1

R2

S1=R

B

S2

S3

Figure 13: Joiner

18


19/31

The Joiner spline is a Cubic Bzier spline whose control points are calculated from the control

points of adjacent splines to achieve C1 continuity. The concept is illustrated in figure 13.

A Joiner spline R is constructed from two endpoints R0, R3 and two additional points RA,

RB. These additional points are used to calculate the necessary control points R1, R2 to achieve

C1 continuity in both endpoints. R1 and R2 lie on the lines formed by R0RA and R3RB. Thedistance of the control points from the respective endpoints on their respective axis is derived

from the distance between the endpoints.

Typically, RA and RB are set to the nearest control points of adjacent splines. For instance, if

Cubic Bzier Q ends in R0, then RA would be set to Q2. Likewise, if Cubic Bzier S starts in R3,

then RB would be set to S1.

The Joiner class is also used for mirroring. Assume a mirror through the vertical axis at

position x = 0, which results in horizontal mirroring. To ensure a smooth curve, R(x = 0) mustbe 0. If R0 = (x = 0,y = y0), this can be achieved by setting RA = (x < 0,y = y0). Then, R0

and RA form a horizontal line, which places R1 at (x > 0, y = y0) and results in zero slope forx = 0. When the curve is now horizontally mirrored at this point, C1 continuity is achieved.

5.5 Mappings

Each emotion influences a different set of muscles. McCloud offers drawings for each basic emo-

tion in 4 intensity levels. These drawings were used as references, which we wanted to match.

For each emotion and each intensity level, muscles were adjusted to match the reference draw-

ing. The values of the muscle tensions were saved for each intensity level. Plots of the muscle

tensions showed that the relationship is a different one for each combination of muscle and emo-

tion. In some cases, the relation is a linear one heightening the level of an emotion increases amuscles tension. More often than not, however, the relation is much more complicated. In order

to achieve credible muscle tensions, this relationship, only indicated by 5 points (neutral and 4

intensity levels for each emotion), needs to be interpolated.

We represent the relationships by a number of mathematical functions, which we call Map-

pings. A Mapping takes a few parameters which influence the resulting function in a flexible

way to approximate the form of the underlying relationship. TheI M a p p i n g

interface is merely a

wrapper for a low-level mathematical function with one parameter and only has one method:

f u n c t i o n y ( x : N u m b e r ) : N u m b e r ;

Every registered emotion-muscle influence is represented by a Mapping. They

-method takes

the current value of an emotion as parameterx

and returns the current tension for the muscle.

Another relation represented by Mappings is the visibility of Features. Some features

wrinkles only become visible when a muscle is contracted. Representing this relationship

through Mappings allows fine-grained control over the opacity.

Three mapping types are currently available:

SineMapping

This form of Mapping is defined by four parameters. The function returns y0 for x < x0, and y1

for x

x1. For x0

x < x1, the curve interpolates between y0 and y1, following the form of a

sine function. This results in a smooth transition between the two states.

19


20/31

0 x0 x1

y1

y0

Figure 14: SineMapping

y(x) =

y0 x < x00.5 sin

xx0x1x0 + 1.5

+ 0.5

(y1 y0) + y0 x0 x < x1

y1 x x1

GaussMapping

This mapping represents the Gaussian function and is used in cases where a muscle is only

contracted for intermediate values of an emotion, but not for low or high values.

0

a0

a1

a2

(a) Influence of scale factor a

0

2=0.005

2=0.01

2=0.02

(b) Influence of variance 2

Figure 15: GaussMapping

y(x) = a 1

2e (x)2

22

The mapping takes three parameters: value = a; mean = ; variance = 2

PolynomialMapping

This is a direct representation of a polynomial function. It can approximate any necessary form

by increasing the order of the polynomial. However, the function is hard to configure manually.

20


21/31

In practice, we used the curve-fitting methods of Grapher.app, which calculates a polynomial

interpolation of desired order for a given point set.

y(x) = anxn + an

1x

n1 +

+ a2x2 + a1x + a0

5.6 Stroke styles

The shape of features is represented by splines. Stroke styles determine how the splines are

visually represented.

If no stroke style is set, the spline is simply stroked by a constant width brush. However, in

many cases, this does not deliver favourable results. Stroke styles implement theI S t r o k e S t y l e

interface. The interfacesd r a w

method supplies the style with the spline to be drawn.

BrushStyle

Currently, BrushStyle is the only stroke style available. It simulates the characteristic form of a

brush; thin lines at the start, getting thicker towards the center, and again thinner towards the

end. This corresponds to the parameterss t a r t W i d t h

,m a x W i d t h

ande n d W i d t h

.

From the spline to be stroked, two splines are derived which define the shape of the stroke.

One spline defines the upper edge, the other one defines the lower edge. In every point of the

base spline, a normal is drawn. On each normal, the positions of the points of the upper and

lower splines are shifted; points of upper spline to the left, points of lower spline to the right.

Thus,m a x W i d t h

does not directly represent the actual thickness of the resulting stroke, but the

distance of the control points. The concept is illustrated in Figure 16.

1

3t=

2

3t=

startWidth

endWid

th

max

Width

maxW

idth

Figure 16: BrushStyle applied to a Cubic Bzier spline

21


22/31

5.7 Facedata file format

Faces are entirely defined through external files which are loaded at runtime. This allows the

development of faces which look entirely different to the standard face we developed. Additional

emotions can also be implemented.A complete set of Facedata defines the following:

Features, which are the visible elements of a face. Features can be transformed by muscles.Typically, this includes dynamic features like eyes, eyebrows, mouth and wrinkles, as well

as static features like nose and head outline.

Muscles, which are the link between emotions and features. The shape of a muscle isdefined by a spline and when contracted can move an arbitrary number of control points

along its path.

Emotions, which are the high-level concept which influence a number of muscles in anarbitrary fashion. Each emotion affects specific regions of the face and results in familiar

facial expressions.

Overlays, which are optional graphical elements added on top of the face to add additionalpersonality to the face. In the standard model, the hairdo is an overlayed vector graphic.

Pixel-based graphics can also be included.

Facedata is an XML-based file format. Currently, no graphical editor is available; Facedata has

to be edited manually. A corresponding DTD is kept up-to-date3 with the current capabilities of

Grimace and allows face developers to validate their files through an XML validation service4.

Since the definitions can become quite large and data have to be edited manually, Facedatadefinitions can be spread across files. The

l o a d F a c e d a t a

API method takes an array of URLs as

parameter, loading the files in the supplied order.

5.8 Deployment and use

Grimace is a self-contained component which enables the addition of facial expressions to soft-

ware projects. Being written in Actionscript 3, the component is deployed as SWF file and can

be opened by Adobe Flash Player 9 and upwards. The component can be downloaded from the

project website and includes detailed instructions and demo files.

Control of the component is offered by an API, which is compatible with JavaScript andActionscript 3. The recommended method is to embed Grimace into web pages and control it

through JavaScript via the API. Through embedding, Grimace can also be controlled via Action-

script 3. Apart from pure AS3, this includes Flex and Flash (from version CS4 upwards). The

AS3 API is basically identical to the JavaScript API but less tested.

Customisation

The download package includes a complete face in the form of a set of Facedata XML files. We

encourage the development of new faces based on these definitions. Currently, no graphical

3The latest version of the DTD can always be found at h t t p : / / g r i m a c e - p r o j e c t . n e t / d t d / l a t e s t . d t d 4e.g. h t t p : / / w w w . v a l i d o m e . o r g / x m l /

22


23/31

editor is available, values need to be edited manually. However, the package also includes

Facemap.swf, the tool we used to develop the face definition. The tool allows to show muscles

and their current tension, include underlayed pictures which offer reference, and allows the

output of the current state of all components.

5.9 Class diagram

Data input

External API

AbstractSpline

ISpline

QuadraticBezierLine CubicBezier

Joiner

IMapping

PolynomialMapping

SineMapping

GaussMapping

Feature

1..*

1

FeatureSegment

Grimace

FeatureController

0..*

1

1..*

1

FeatureNode

IStrokeStyle

0..1

11

1

MuscleGroup

MuscleController

0..*

1

Muscle

0..*1

1

1

Emotion

EmotionCore

0..*

1

0..*

1

1

1

0..*

0..*

XMLFactory

FacedataLoader

XMLDraw

ExternalCommands

JSHandler

ASHandler

Geom

BrushStyle

FeatureFill

0..*

1

Figure 17: Class diagram

23


24/31

6 Results

We have developed a software component which can display all primary and secondary emo-

tions as depicted by McCloud. Furthermore, primaries can be blended in arbitrary intensities,

thus covering states not covered before.The component has been released to the public under a Creative Commons licence. A project

website5 has been implemented. The website features a demo application that allows visitors

to express arbitrary blendings of any two emotions. A download package is available, which

includes the component, demo applications for all supported programming environments and

comprehensive documentation on how to use the component.

Public reactions to the project were notedly positive, shown in a large number of approv-

ing comments. Scott McCloud kindly featured our project on his blog on February 25, 2009,

emphasising that facial expressions should be taught in school, for which our project might be

very useful. We are also very thankful to Mr. McCloud for his encouraging words and useful

comments about our work at an intermediate stage of the project.

The resulting face is shown in figure 18 with a neutral expression. Figure 19 shows the 6

emotional primaries at four intensity levels. In figure 20, any combination of two primaries

(both at 75% intensity level) is shown.

Figure 18: Neutral expression

5h t t p : / / g r i m a c e - p r o j e c t . n e t

24


25/31

Joy

Sadness

Disgust

Anger

Fear

Surprise

Figure 19: Primary emotions in 4 intensity levels

25


26/31

Joy + Surprise

Joy + Fear

Joy + Disgust

Joy + Anger

Joy + Sadness

Surprise + Disgust

Surprise + Sadness

Surprise + Fear

Surprise + Anger

Fear + Sadness

Fear + Anger

Fear + Disgust

Sadness + Anger

Sadness + Disgust

Disgust + Anger

Figure 20: Secondary emotion blendings of intensity level 3

26


27/31

6.1 Conclusion and future directions

The described software component Grimace displays emotions through a comic-like face. We

believe that the display of emotional information is a valuable addition to information resources,

and facial expressions are a natural way of expressing this kind of information. The work ofMcCloud (2006) was used as guide and visual reference throughout the design and development

process. We believe to have found a useful compromise between simplicity and necessary detail.

We include all facial features which are necessary to convey an emotion while omitting the rest.

The component was developed using web technology, which allows easy deployment. We

defined an API which allows convenient integration into other projects without the need for

knowledge about technical details. All configuration data is loaded from external files, which

use an XML-based file format. The file format is fully documented and allows full customisation

of all aspects features, muscles and emotions. The component is stable and ready for use

for the intended purpose. A project website was implemented, from which the component and

documentation can be downloaded.

While we believe that the goal of the project has been achieved in a satisfactory manner, there

are many areas which remain to be addressed, a few of which will be outlined in the following.

First of all, the current face model can be further optimised. We had to add additional

muscles to the principal facial muscles in a few cases to achieve the desired expressiveness.

However, it might be possible to reduce the number of necessary muscles by optimising the

definition of the actual muscles.

Furthermore it would be interesting to see, if the intended emotions are actually recognized

in the facial expressions Grimace produces. In order to verify this an online experiment was

conducted. A website was designed and built to make the survey easily available to a large au-dience. The setup consisted of a number of facial expressions rendered by Grimace, covering the

whole emotional gamut. Next to the face the 6 basic emotions (see figure 1) where listed and the

participants had to specify which emotions they associated with the shown expression. About

200 people from around the world participated in the experiment. The analysis of the collected

data is in progress and the findings may be incorporated in the next iteration of Grimace to

improve the readability of the emotions.

Calder et al. (2000) show that comprehensibility of facial expressions can be increased further

if the characteristic features of an expression are exaggerated. Our model has comic-like appear-

ance, and it might be possible to make our model even more expressive if we allow a certain

level of unrealistic, cartoon-like exaggerated expressions.

Customisation and extension of the current face model would become much easier if a graph-

ical editor was available. First and foremost, such an editor should facilitate customisation of

visible features of a face. Currently, the control points for features need to be entered manually

in XML files. These are the parts which can be exchanged easily. The relationships between

muscles and emotions, however, need considerable attention and are quite tedious to change.

So far, the system can only display facial expressions which represent emotional states. Of

course, humans can communicate much more through their faces, which can be easily observed

by studying the wide range of facial expressions which actors can display. Facial expressions

which cannot be expressed currently include doubt or agreement. The Facial Action Coding

System, or FACS, (Ekman et al., 1978) describes a comprehensive framework of all possible

27


28/31

facial movements. If the range of possible facial expressions was to be extended, this framework

would offer a good basis.

This would also mean a departure from the mirroring of facial features. Right now, facial

features are completely symmetrical. In FACS, asymmetrical movement of features is possible.

Ideally, the system would still mirror those parts that are symmetrical and only consider the

differences to the symmetrical state when necessary.

28


29/31

References

I. Albrecht, M. Schrder, J. Haber, and H. Seidel. Mixed feelings: expression of non-basic

emotions in a muscle-based talking head. Virtual Reality, 8(4):201212, 2005. URL h t t p :

/ / w w w . s p r i n g e r l i n k . c o m / i n d e x / G 4 0 7 T 2 1 7 5 1 T 8 1 1 6 1 . p d f .

K. Balci. Xface: Open Source Toolkit for Creating 3D Faces of an Embodied Conversational

Agent. In Smart Graphics: 5th International Symposium, SG 2005, Frauenwrth Cloister, Germany,

August 22-24, 2005: Proceedings. Springer, 2005.

R. Barthes. Denotation and connotation. The Communication Theory Reader, pages 129133, 1996.

C. Bartneck. How Convincing is Mr. Datas Smile: Affective Expressions of Machines. User

Modeling and User-Adapted Interaction, 11(4):279295, 2001.

C. Bartneck, T. Kanda, H. Ishiguro, and N. Hagita. Is The Uncanny Valley An Uncanny Cliff? In

Proceedings of the 16th IEEE International Symposium on Robot and Human Interactive Communica-

tion, RO-MAN, pages 368373, 2007.

S. Brennan. Caricature generator: The dynamic exaggeration of faces by computer. Leonardo, 18

(3):170178, 1985.

A. Calder, D. Rowland, A. Young, I. Nimmo-Smith, J. Keane, and D. Perrett. Caricaturing

facial expressions. Cognition, 76(2):105146, 2000. URL h t t p : / / l i n k i n g h u b . e l s e v i e r . c o m /

r e t r i e v e / p i i / S 0 0 1 0 0 2 7 7 0 0 0 0 0 7 4 3

.

P. Ekman. Basic emotions. Handbook of cognition and emotion, pages 4560, 1999. URLh t t p :

/ / w w w . v h m l . o r g / t h e s e s / w i j a y a t / s o u r c e s / w r i t i n g s / p a p e r s / b a s i c _ e m o t i o n s . p d f

.

P. Ekman. Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emo-

tional Life. Times Books, 2003.

P. Ekman, W. Friesen, and P. Ellsworth. Emotion in the Human Face: Guidelines for Research and an

Integration of Findings. Pergamon, 1972.

P. Ekman, W. Friesen, J. Hager, and A. Face. Facial action coding system. 1978.

N. Etcoff and J. Magee. Categorical perception of facial expressions. Cognition, 44(3):22740,

1992. URL h t t p : / / w w w . n c b i . n l m . n i h . g o v / p u b m e d / 1 4 2 4 4 9 3 .

G. Faigin. The Artists Complete Guide to Facial Expression. Watson-Guptill, 1990.

B. Garza-Cuarn. Connotation and Meaning. Mouton De Gruyter, 1991.

T. Geller. Overcoming the uncanny valley. IEEE Computer Graphics and Applications, 28(4):1117,

2008.

T. Groleau. Approximating cubic bezier curves in flash mx, 2002. URLh t t p : / /

t i m o t h e e g r o l e a u . c o m / F l a s h / a r t i c l e s / c u b i c _ b e z i e r _ i n _ f l a s h . h t m

.

29


30/31

S. Iwashita, Y. Takeda, and T. Onisawa. Expressive facial caricature drawing. In Fuzzy Systems

Conference Proceedings, 1999. FUZZ-IEEE99. 1999 IEEE International, volume 3, 1999. URL

h t t p : / / i e e e x p l o r e . i e e e . o r g / x p l s / a b s _ a l l . j s p ? a r n u m b e r = 7 9 0 1 4 3

.

S. McCloud. Making Comics: Storytelling Secrets of Comics, Manga and Graphic Novels. Harper-Perennial, 2006.

M. Mori. The uncanny valley. Energy, 7(4):3335, 1970. URLh t t p : / / g r a p h i c s . c s . u c d a v i s .

e d u / ~ s t a a d t / E C S 2 8 0 / M o r i 1 9 7 0 O T U . p d f .

D. Norman. Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books, 2004.

M. Ochs, R. Niewiadomski, C. Pelachaud, and D. Sadek. Intelligent expressions of emotions. In

1st International Conference on Affective Computing and Intelligent Interaction ACII. Springer, 2005.

URLh t t p : / / w w w . s p r i n g e r l i n k . c o m / i n d e x / 7 0 0 7 2 8 0 g t q 4 1 2 j 0 h . p d f

.

C. Ogden, I. Richards, and U. Eco. The Meaning of Meaning: A Study of the Influence of Language

Upon Thought and of the Science of Symbolism . Routledge & Kegan Paul, Ltd., 1969.

A. Ortony and T. Turner. Whats basic about basic emotions. Psychological Review, 97(3):315331,

1990. URLh t t p : / / w w w . c s . n o r t h w e s t e r n . e d u / ~ o r t o n y / p a p e r s / b a s i c % 2 0 e m o t i o n s . p d f

.

C. Osgood. The Measurement of Meaning. University of Illinois Press, 1957. URL

h t t p : / / b o o k s . g o o g l e . c o m / b o o k s ? h l = e n & l r = & i d = Q j 8 G e U r K Z d A C & o i = f n d & p g = P A 1 & d q =

o s g o o d + m e a s u r e m e n t & o t s = R F I 2 _ X N I 8 d & s i g = h v 5 z z k O 6 9 B J W z C I K - 3 7 h S 8 Q o e c U

.

X. Pan, M. Gillies, T. Sezgin, and C. Loscos. Expressing complex mental states through facial

expressions. LECTURE NOTES IN COMPUTER SCIENCE, 4738:745, 2007. URLh t t p : / / w w w .

s p r i n g e r l i n k . c o m / i n d e x / l 8 6 0 7 8 5 4 j t 5 q 2 3 l 9 . p d f

.

I. Pandzic and R. Forchheimer. MPEG-4 Facial Animation: The Standard, Implementation and Appli-

cations. Wiley, 2002.

R. Picard. Affective Computing. MIT Press, 1997. URL h t t p : / / b o o k s . g o o g l e . a t / b o o k s ?

h l = e n & l r = & i d = G a V n c R T c b 1 g C & o i = f n d & p g = P P 1 1 & d q = p i c a r d + a f f e c t i v e + c o m p u t i n g & o t s =

F 1 k 6 r l A a a b & s i g = q x V U 7 L S W n r L 3 X W m O t h w 7 Y X 3 c C - U

.

J. Russell. A circumplex model of affect. Journal of Personality and Social Psychology, 39(6):1161

1178, 1980. URLh t t p : / / c o n t e n t . a p a . o r g / j o u r n a l s / p s p / 3 9 / 6 / 1 1 6 1

.

E. Schubert. Emotionface: Prototype facial expression display of emotion in music. In Proc. Int.

Conf. On Auditory Displays (ICAD), 2004. URLh t t p : / / m u s i c . a r t s . u n s w . e d u . a u / a b o u t u s /

r e s e a r c h / S c h u b e r t / I C A D 0 4 S c h u b e r t E m o t i o n F a c e . p d f

.

J. Sloboda and P. Juslin. Psychological perspectives on music and emotion. Music and emotion:

Theory and research, pages 71104, 2001.

S. Tanahashi and Y. Kim. A comic emotional expression method and its applications. In TENCON

99. Proceedings of the IEEE Region 10 Conference, volume 1, 1999. URL h t t p : / / i e e e x p l o r e .

i e e e . o r g / x p l s / a b s _ a l l . j s p ? a r n u m b e r = 8 1 8 4 1 7

.

30


31/31

C. Wang. Langwidere: A Hierarchical Spline Based Facial Animation System with Simulated Muscles.

PhD thesis, University of Calgary, 1993.

S. Zhang, Z. Wu, H. Meng, and L. Cai. Facial expression synthesis using pad emotional param-

eters for a chinese expressive avatar. LECTURE NOTES IN COMPUTER SCIENCE, 4738:24,2007. URL

h t t p : / / w w w . s p r i n g e r l i n k . c o m / i n d e x / 1 1 8 7 6 6 7 1 7 2 5 6 7 6 6 j . p d f

.

Documents

Grimace Docu