Expressive Generation for Interactive Stories - UCSC ?· NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC…

Embed Size (px)

Text of Expressive Generation for Interactive Stories - UCSC ?· NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB...

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Natural Language and Dialogue Systems Lab

    Expressive Generation for Interactive Stories

    Marilyn A. Walker, Ricky Grant, Jennifer Sawyer, Grace I. Lin, Noah Wardrip-Fruin, and Michael Buell

    The Character Creator Project NSF Creative IT IIS-1002921

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Stories told through dialogue

    What characters say How they say it How they react to what other

    characters say

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Interactive Stories, Role Playing Games

    Role-playing games are a type of interactive narrative game involving a story where the player takes on a role of a character in the story world.

    Role-playing games are one of the most successful types of games

    World of Warcraft, one of the largest role-playing games, has 11.5 million players by recent counts

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Role playing games used to have fixed visuals

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Now games usually dynamically generate animations/scenes

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Dialogue in games is where visuals were 20 years ago

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Authoring is very expensive and limits game play

    Expensive: Script for Baldurs Gate 2 was 3,257 pages long. Planescape: Torment contained nearly a million words of

    dialogue. The Old Republic (2011) had authors developing content since

    2006, with a team of at least twelve full-time writers

    Limiting: Unlike film, games are interactive Exponential growth in authoring: every time a choice point is

    supported, write a new dialogue tree for each branch. Dialogue choices often do not reflect players previous choices If a character is killed, any subplots with that character must be

    removed

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Immersivity, Story, Meaningful Choice

    Are such dialogue choices as meaningful as they might be?

    Do they make us engage with the story?

    Goal: support player agency, narrative choice

    Solution: Write more dialogue and better dialogue trees?

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Procedural Language Generation: A Key Technology

    Provides Parameters & Models Abstract & Modular Interfaces Trainable: Machine Learning Techniques

    => Greater Scalability, More Immersivity, Better Stories

    There is no other way to make it possible to personalize dialogue interaction to an individual and their history playing the game

    Similar issues across all dialogue applications

    9

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Narrative Dialogue Generation Challenges

    Revealing Subtext: key parts of narrative are not explicitly stated Character Personality: I am a friendly

    person. Character Emotion: I am feeling hesitant. Character Motivation: I intend to flatter you.

    Expressing a unique Character Voice Who is this person? More than what we need for other

    dialogue applications

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Natural Language Generation

    Most NLG systems have not focused on expressing character and personality for narrative applications

    Expressive Natural Language Generation (ENLG) focuses on stylistic, social aspects of the linguistic behavior of dramatic characters Politeness theory: Walker et al, 1997, Andre et al,

    2000, Cassell & Bickmore 2003, Wang et al, 2005 Personality: Ball & Breese 2000, Loyall & Bates 97, Isard

    et al, 2006, Mairesse & Walker 2007, 2008 Archetypes: Rowe, Ha & Lester 2009

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Character Creator: Tool for author creativity

    Tool for automatically rendering variations in dialogue

    Learn models of character voice (linguistic style) from film screenplays

    Use the learned models to control the parameters of an expressive NLG engine(PERSONAGE)

    Apply the learned models to control the style of character dialogue in a story

    Test human perceptions of the resulting generated utterances

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Film Corpus

    862 film scripts from IMSDb, as of May 19, 2010 7,400 characters 664,000 lines of dialogue 9,599,9900 tokens

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Scene from Annie Hall: Lobby of Sports Club

    ALVY: Uh you-you wanna lift? ANNIE: Turning and aiming her thumb over her shoulder Oh, why-uh y-y-you gotta car? ALVY: No, um I was gonna take a cab. ANNIE: Laughing Oh, no, I have a car. ALVY: You have a car? Annie smiles, hands folded in front of her So Clears his throat. I dont understand why if you have a car, so then-then wh-why did you say Do you have a car? like you wanted a lift?

    Annie Hall: Getting a lift

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    The Terminator: getting a lift

    Scene from The Terminator: Cigar biker

    TERMINATOR: I need your clothes, your boots, and your motorcycle. CIGAR BIKER: You forgot to say please.

    Terminator hurls Cigar, all 230 pounds of him, clear over the bar, through the serving window into the kitchen, where he lands on the big flat GRILL. We hear a SOUND like SIZZLING BACON as Cigar screams, flopping jerking. He rolls off in a smoking heap.

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    What can we learn from a corpus?

    Reveal Subtext: The way a character says something is one way to reveal subtext and character emotion Short vs. Long turns => friendliness, formality Word choice => level of education Disfluencies, Stuttering => anxiety, hesitation Direct forms vs. indirect forms => extraversion, aggression

    Character Voice: Learning to model specific characters or sets of characters should produce individual character voices

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    4. Generate features reflecting linguistic behaviors

    Pulp Fiction Script

    Vincents Dialogue

    Jules Dialogue

    Others Dialogue

    Jules Dialogue Vincents

    Dialogue

    Jules LIWC results Vincents

    LIWC results

    Jules Tag Question Ratio Vincents Tag

    Question Ratio

    Jules other features

    Vincents other features

    Jules Overall Polarity

    Vincents Overall Polarity

    1. Collect movie scripts from IMSDb

    2. Extract utterances for each character

    3. Select leading roles (dialogue > 60 turns)

    Method

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Jules Learned Model

    Vincents Learned Model

    Generated features

    Vincent in SpyFeet utterances

    PERSONAGE generator

    (ENLG engine)

    Jules in SpyFeet utterances

    Others in SpyFeet utterances

    5. Learn models of character (z-scores)

    6. Generate new utterances using learned models to control parameters of our dialogue generator

    Story domain: SpyFeet utterances

    Method (cont)

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Natural Language and Dialogue Systems Lab

    Background: Personage Generator

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    Automatically producing interesting dialogue

    Parameters: lots of different parameters that produce interesting variations in character voices But which ones?

    Models that control the parameters Tools that let authors control the parameters & models

    Piloted an approach of exposing parameters and models directly to creative writers

    Not natural to creative process to think of character voices in terms of parameters

    But working with examples, and variations on examples, fits better with existing writing practice

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    PERSONAGE Generator: BIG FIVE Theory

    Conscientiousness: Dutiful vs. impulsive Emotional stability: Calm vs. anxious Openness to experience: Imaginative vs. conventional Agreeableness: Kind vs. unfriendly Extraversion: Sociable, assertive vs. quiet

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ 22

    Linguistic Reflexes of Personality: 50 years of studies

    Extraversion (Furnham, 1990) Talk more, faster, louder and more repetitively Fewer pauses and hesitations Lower type/token ratio Less formal, more references to context (Heylighen & Dewaele, 2002) More positive emotion words (Pennebaker & King, 1999)

    E.g. happy, pretty, good Neuroticism (Pennebaker & King, 1999)

    1st person singular pronouns Negative emotion words

    Conscientiousness (Pennebaker & King, 1999) Fewer negations and negative emotion words

    Low but significant correlations

  • NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ

    PERSONAGE Architecture: 67 Parameters

    Realization

    INPUT: Dialog Act, Content Pool

    OUTPUT UTTERANCE

    VERBOSITY RESTATEMENTS

    CONTENT POLARITY

    SYNTACTIC COMPLEXITY

    SELF-REFERENCE

    CONTRAST: e.g. however, but JUSTIFY: e.g.

    so, since

    PERIOD

    EXCLAMATION HEDGES: e.g. kind of,

    rather, basically, you know FILL