Upload
ailani
View
38
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Designing Tools Enhance Interactive Experiences & the development process. Magy Seif El-Nasr College of IST Penn State University. Interactive Entertainment: Industry Impact. Interactive Entertainment: Industry Impact. Interactive Entertainment: Industry Impact. Game industry profits: - PowerPoint PPT Presentation
Citation preview
Designing Tools Designing Tools EnhanceEnhance
Interactive Interactive ExperiencesExperiences
& the development & the development processprocess
Magy Seif El-NasrMagy Seif El-NasrCollege of ISTCollege of IST
Penn State UniversityPenn State University
Interactive Entertainment: Interactive Entertainment: Industry ImpactIndustry Impact
Interactive Entertainment: Interactive Entertainment: Industry ImpactIndustry Impact
Interactive Entertainment: Interactive Entertainment: Industry ImpactIndustry Impact
Game industry profits: 2002: $1 billion more than movie industry 2004: EA reported $1.43 billion quarterly earning 2005: Nintendo $4 billion and expects to sell at
least 2 million Wiis in US alone
Importance of Game Engines + toolsImportance of Game Engines + tools
Entertainment Training
Leaders project, ICTDevil May Cry IIEducation
Virtual Aquarium, NCSU
code
Tools
.radix 16 ;all numbers hexadecimal
start: mov #41,@#8264 ;system variable loop: jsr r4,print1 .asciz "HELLO WORLD! "
br loop print1:jsr pc,@#1248 ;print string (ROM call)
rts r4
System.Writeline (“Hello World”);
Assembly
C#
Learning and Computer literacy ImpactLearning and Computer literacy Impact
Several Researchers believe thatSeveral Researchers believe that
“video games provide an easy lead-in to computer literacy” – Cassell 2000
High Schools students and Modding
Books on Modding for teens gaining popularity
Interactive Entertainment:Interactive Entertainment: Motivation and LearningMotivation and Learning
“I want to learn how to make an awesome video game. I would like to learn everything about technology or at least more than I did”
- middle school girl
“I want to make my characters talk, build a world, and make interesting stories”
- another middle school girl
Impact Impact and and Importance Importance ofof the toolsthe tools
Facilitate development: games, training, health therapy application
Facilitate learning: media, computing literacy
Facilitate Creativity
Two thrusts of my researchTwo thrusts of my research
Tools for building engaging interactive
experiences
Game Engines + Tools as learning
environments
Tools for building engaging interactive
experiences
Game Engines + Tools as learning
environments
Two thrusts of my researchTwo thrusts of my research
Focus: visual design and character design
Based on cinematic and theatric theory
Using CI and AI techniques
My ResearchMy Research
Tools Lighting Camera (in progress) Character (in progress) Dance (in Progress)
Why Lighting?Why Lighting?
Lighting is Everything Lighting is Everything
MiiMii
• BS in Computer Science BS in Computer Science
• Directed in TheatreDirected in Theatre
• Graphics Design in AdvertisementGraphics Design in Advertisement
• MS in Computer Science MS in Computer Science
• Studied Psychology and NeuroscienceStudied Psychology and Neuroscience
• PhD in Computer SciencePhD in Computer Science
• 2 years studied acting & lighting design at 2 years studied acting & lighting design at Northwestern Northwestern
Why Lighting?Why Lighting?
Dramatic Tension
Visual Focus
Style
Feel of the space
Time of Day
Period
visibility
Lighting in Traditional MediaLighting in Traditional Media
Manually setting light layout Manually setting light layout (light maps)(light maps)
Advantages:Advantages: realisticrealistic Controllable Controllable
Disadvantages:Disadvantages: Don’t adapt to variations Don’t adapt to variations
in the environmentin the environment Requires much memoryRequires much memory
Game Lighting Game Lighting Static Lighting Design Static Lighting Design
Images from Lightmaps (static shadowmaps) article written by Kurt Miller from:http://www.flipcode.com/articles/article_lightmaps.shtml
x =
Image from Max Payne
Manually scripting Manually scripting Lighting effectsLighting effects
Dynamic Character Dynamic Character LightingLighting
Real-time ShadowsReal-time Shadows
Dynamic user control of Dynamic user control of some lightssome lights
Game Lighting Game Lighting Static + Dynamic LightingStatic + Dynamic Lighting
Game Lighting Game Lighting Static + Dynamic LightingStatic + Dynamic Lighting
Game Lighting Game Lighting Static + Dynamic LightingStatic + Dynamic Lighting
Advantages:Advantages:More realisticMore realisticAdaptable to changeAdaptable to change
Disadvantages:Disadvantages:Effects are scripted and rely on very careful Effects are scripted and rely on very careful designdesignRestricted Restricted Based on realismBased on realismCompositional Balancing is done at design timeCompositional Balancing is done at design time
Problems: No modulationProblems: No modulation
Screenshot from Mission 21 Devil May Cry
Saturation Graph for Screenshot
Problems (Blade of Darkness): No Problems (Blade of Darkness): No adaptation to game Playadaptation to game Play
Problem 1: unpredictabilityProblem 1: unpredictability
Develop a lighting plot or setup based on:Develop a lighting plot or setup based on:Narrative configuration:Narrative configuration: StoryStory Conflict/dramatic tensionConflict/dramatic tension
Timing and dramatic progressionTiming and dramatic progressionPhysical Configuration:Physical Configuration: Camera orientation Camera orientation
and positionand position Characters positions Characters positions
and orientations and orientations
Requiring dynamic adaptationRequiring dynamic adaptation
depend depend on useron user}
Two GoalsTwo Goals
Dynamic Adaptive Lighting:Dynamic Adaptive Lighting:
Better gameplayBetter gameplay
Facilitate new Interactive modelsFacilitate new Interactive models
High-Level authoring of lighting:High-Level authoring of lighting:
Faster prototyping Faster prototyping
Exploration of design spaceExploration of design space
Developing such as a Developing such as a DesignDesign Tool is Hard Tool is Hard
Technical Issues of Technical Issues of DynamicDynamic Lighting Lighting Artistic Artistic control control for stylefor style low-level balancing of low-level balancing of contextcontext and and lighting parameterslighting parameters but provide but provide continuitycontinuity
Why?
ELE – ELE – Expressive Lighting EngineExpressive Lighting Engine
A lighting system that:A lighting system that:
IntelligentlyIntelligently adjusts adjusts lighting in real-time accommodatelighting in real-time accommodate contextcontext and and effecteffect
Based on Based on cinematic cinematic & theatric & theatric theorytheory
Allow Allow artistartist to to controlcontrol lighting at a lighting at a high-levelhigh-level
ELEELE
The three subsystems:The three subsystems: use optimization to find best solution use optimization to find best solution given context, desired effects, state, and artists’ constraintsgiven context, desired effects, state, and artists’ constraints
Game/Rendering Engine
ELE
Allocation Subsystem
Angle Subsystem
Color Subsystem
Current State
Previous State
LAMP (Lighting Action Message Protocol)
WAMP (World Action Message Protocol)
Artistic Constraints
ELEELE
The three subsystems:The three subsystems: use optimization to find best solution use optimization to find best solution given context, desired effects, state, and artists’ constraintsgiven context, desired effects, state, and artists’ constraints
Game/Rendering Engine
ELE
Allocation Subsystem
Angle Subsystem
Color Subsystem
Current State
Previous State
LAMP (Lighting Action Message Protocol)
WAMP (World Action Message Protocol)
Artistic Constraints
Automatic light allocationAutomatic light allocation
find best allocation find best allocation given given artistic artistic constraintsconstraints:: ModelingModeling Depth Depth VisibilityVisibility Visual ContinuityVisual Continuity Visual FocusVisual Focus Low vs. high keyLow vs. high key
Algorithm for dynamic allocationAlgorithm for dynamic allocation
1.1. Calculate visible area Calculate visible area
2.2. Divide the zone into overlapping areasDivide the zone into overlapping areas
3.3. Allocate a number of lights to areas,Allocate a number of lights to areas,given that given that minimize: minimize:
visibility modeling Visual
continuitydepth
:p L A
arg max( ( ) ( ) ( ) ( ))
opt
v d m vcp
p
V p D p M p VC p
Layout of lightsLayout of lights
ELEELE
The three subsystems:The three subsystems: use optimization to find best solution use optimization to find best solution given context, desired effects, state, and artists’ constraintsgiven context, desired effects, state, and artists’ constraints
Game/Rendering Engine
ELE
Allocation Subsystem
Angle Subsystem
Color Subsystem
Current State
Previous State
LAMP (Lighting Action Message Protocol)
WAMP (World Action Message Protocol)
Artistic Constraints
Lighting angle SelectionLighting angle Selection
Select azimuth, elevation angles, Select azimuth, elevation angles, given given artistic constraintsartistic constraints : :
Visual ContinuityVisual Continuity Motivation of directionMotivation of direction VisibilityVisibility ModelingModeling MoodMood
ELE finds best light angles to meet goalsELE finds best light angles to meet goals
Automating cinematic lighting designAutomating cinematic lighting designChoosing key light azimuth angleChoosing key light azimuth angle
Optimizes:Optimizes:
• kk is the key light angle, is the key light angle, kk-- is angle previous frame is angle previous frame• mm is the mood angle desired, V is visibility is the mood angle desired, V is visibility vv is the cost of deviation from best visibility angle is the cost of deviation from best visibility angle -- is the cost of visual continuity is the cost of visual continuity ll is the cost of deviation from realistic direction is the cost of deviation from realistic direction mm is the cost of deviation from ideal mood angle is the cost of deviation from ideal mood angle
cost( , , , )
(1 ( , )) minv m l ii
k s k m
V k s k k k m k l
visibility & modeling
visual continuity
mood motivation
Evaluation of key light angle Evaluation of key light angle visibility & modelingvisibility & modeling
( , ) sin( )cos( )V k s k sWe described these rules as :
Nose line
Light
Camera
s
k
From: Millerson’s
The technique of lighting for television and Film, 1991
Artist's setting
0
20
40
60
80
100
120
Co
sts
ELE: Angle SubsystemELE: Angle Subsystem
Showing character’s facial Showing character’s facial expressions and gesturesexpressions and gestures
Artist's settings
0
20
40
60
80
100
120
Co
sts
Showing character’s facial Showing character’s facial expressions and gestures, high expressions and gestures, high
tensiontension
ELE: Angle SubsystemELE: Angle Subsystem
Mood Angle = side angle
Artist's settings
0
20
40
60
80
100
120
Co
sts
Emphasizing mood and Emphasizing mood and mysterymystery
ELE: Angle SubsystemELE: Angle Subsystem
Mood Angle = side angle
Artist's settings
0
20
40
60
80
100
120
Co
sts
Emphasizing moodEmphasizing mood
ELE: Angle SubsystemELE: Angle Subsystem
ELEELE
The three subsystems:The three subsystems: use optimization to find best solution use optimization to find best solution given context, desired effects, state, and artists’ constraintsgiven context, desired effects, state, and artists’ constraints
Game/Rendering Engine
ELE
Allocation Subsystem
Angle Subsystem
Color Subsystem
Current State
Previous State
LAMP (Lighting Action Message Protocol)
WAMP (World Action Message Protocol)
Artistic Constraints
Choosing ColorsChoosing Colors
Color (Hue + saturation Color (Hue + saturation
+ intensity)+ intensity)
Compose colors for Compose colors for different areas on the setdifferent areas on the set
Choosing ColorsChoosing Colors
Adjust colors to accommodate desired Adjust colors to accommodate desired artistic artistic constraintsconstraints:: Depth Depth Dramatic IntensityDramatic Intensity Dramatic focusDramatic focus low vs. high key settinglow vs. high key setting Specific author-suggestedSpecific author-suggested
Hue, Saturation, Lightness, color Warmth for Hue, Saturation, Lightness, color Warmth for focus, non-focus, and backgroundfocus, non-focus, and background
Palette restrictions specifying stylePalette restrictions specifying style
+ maintain + maintain visual continuityvisual continuity and and stylestyle
Choosing ColorsChoosing Colors
Optimize:Optimize:
depth
21
2
1
{ , , }
{ , , }
( , ) ( )
contrast ( )
( )
( , )
( ),
t t td
tc
t
t tvc i i
i f n b
ti
i f n b
cost c c D c d
c
p c
E c c
I c
contrast
Palette constraints
Visual continuity
Artist’s desired colorparameters
Choosing ColorsChoosing Colors
2
2
2
2
( )
( )
( )
( )
i
i
i
i
ts i i
th i i
tl i i
tw i i
S c s
H c h
L c l
W c w
( )tiI c
Saturation
Hue
Lightness
Warmth
Calculating Color WarmthCalculating Color Warmth
Based on warmth perceptionBased on warmth perception
Used linear fit to psychophysical data:Used linear fit to psychophysical data:
Warm
Cool0.008
0.0006 0.422
0.0105
TR R
warmth G G
B B
Artist's setting
0102030405060708090
Idea
l V
alu
es
warm tones, warm tones, low color contrastlow color contrast
for low tension, realistic colorsfor low tension, realistic colors
Lightness (NF)Contrast (F – NF)Warmth (NF)
ELE: Color Subsystem ELE: Color Subsystem
cool tones, cool tones, low color contrastlow color contrast
for night scenes,for night scenes,or emphasize characteror emphasize character
Artist's setting
010203040
50607080
Idea
l V
alu
es
ELE: Color Subsystem ELE: Color Subsystem
Artist's setting
0102030405060708090
100
Idea
l V
alu
es
warm, warm, high intensity contrasthigh intensity contrast
for high tensions scenesfor high tensions scenes
ELE: Color Subsystem ELE: Color Subsystem
Two GoalsTwo Goals
Dynamic Adaptive Lighting:Dynamic Adaptive Lighting:
Better gameplayBetter gameplay
Facilitate new Interactive modelsFacilitate new Interactive models
High-Level authoring of lighting:High-Level authoring of lighting:
Faster prototyping Faster prototyping
Exploration of design spaceExploration of design space
Dynamic Intelligent Dynamic Intelligent Cinematic Lighting Cinematic Lighting
can make a difference in gamescan make a difference in games
Use Light as tension meterUse Light as tension meter
Demo (Dynamic Lighting)Demo (Dynamic Lighting)
Demo (Static Lighting)Demo (Static Lighting)
Results of Pilot Study – Results of Pilot Study – during during CHI 2005CHI 2005
Non-players liked the visual representation Non-players liked the visual representation
Liked representation of information through Liked representation of information through lightinglighting
Some players said: it made the game too easySome players said: it made the game too easy
Some gamers said: lighting was disturbingSome gamers said: lighting was disturbing
Adaptive Visual Focus Adaptive Visual Focus
can make a difference in gamescan make a difference in games
Adapting Difficulty of Spotting Adapting Difficulty of Spotting enemiesenemies
Two GoalsTwo Goals
Dynamic Adaptive Lighting:Dynamic Adaptive Lighting:
Better gameplayBetter gameplay
Facilitate new Interactive modelsFacilitate new Interactive models
High-Level authoring of lighting:High-Level authoring of lighting:
Faster prototyping Faster prototyping
Exploration of design spaceExploration of design space
ElevatorElevator
ElevatorElevator
EDISON Edge Detection and Image EDISON Edge Detection and Image SegmentatiON SegmentatiON [Christoudias et al. [Christoudias et al. 2002]2002]
Mean-shift approach Mean-shift approach [Comaniciu and [Comaniciu and Meer 2002]Meer 2002]
Design Galleries Design Galleries [Marks et al. 1997][Marks et al. 1997]
User adjusting Lighting in Games like User adjusting Lighting in Games like The The Sims Sims or or The MoviesThe Movies
A comment from an educator:“This can be used as a way of promotingMedia Literacy”
Classes – Using the ToolsClasses – Using the Tools
400-level interdisciplinary course on Interactive Narrative, tools: C# Flash ELE Interactive Narrative
Engine
Classes – Using the toolsClasses – Using the tools
400-level Game Design, tools:
Unreal + ELE Half Life 2 Torque
Classes – Exploring the theoryClasses – Exploring the theory
400-level Interdisciplinary course on Design of Immersive Environments
Future DirectionsFuture Directions
More on lightingMore on lighting
Lighting: industry partnersLighting: industry partners
Continue on Projects:Continue on Projects: CameraCamera CharacterCharacter DanceDance
Tools for building engaging interactive
experiences
Current ProjectsCurrent Projects
Ambient Intelligence and Dancedesigning tools to allow artists to dictate aesthetic effect
Pressure + Physiological
Sensors
Intelligent Systems
Current ProjectsCurrent Projects
Virtual Actor Training
Comedia dell’arte in Second Life
Character, gesture, and camera tools
Current ProjectsCurrent Projects
Lockheed Martin Lightingvisual designcamera character
Living Hypothesis
Future DirectionsFuture Directions
Use more of the research Use more of the research toolstools
Impact of the use of the Impact of the use of the research toolsresearch tools
Game Engines + Tools as learning
environments
Specifically our work on Games and Modding
Classes – Using Game ModdingClasses – Using Game Modding
Middle/high school Gaming for Girls
Future Work: Camera tool Lighting Character
In ConclusionIn Conclusion
ConclusionConclusionGame Industry is a growing industryGame Industry is a growing industry
Game tools are importantGame tools are important
In my research I develop new tools toIn my research I develop new tools to Enhance design processEnhance design process Enhance interactive experienceEnhance interactive experience
Application of these tools: Education, Application of these tools: Education, Training, and other applicationsTraining, and other applications
Leaders project, ICTDevil May Cry II
Questions?Questions?
http://faculty.ist.psu.edu/http://faculty.ist.psu.edu/seifel-Nasr/seifel-Nasr/
[email protected]@ist.psu.edu
Modding and LearningModding and Learning
TransferTransfer of Computer Science, Math, Art of Computer Science, Math, Art ConceptsConcepts
Different Different engines promote engines promote different different concepts, concepts, and require different and require different pre-reqspre-reqs
Increase self efficacyIncrease self efficacy of high-school female of high-school female studentsstudents
MaterializeMaterialize abstract concepts abstract concepts
Learn something about Learn something about themselvesthemselves
Different engines are Different engines are popular popular among different among different groupsgroups
Concepts Promoted thru ModdingConcepts Promoted thru Modding
Essential ConceptsEssential Concepts AssumptionsAssumptions
of engines need to of engines need to be understood, e.g.be understood, e.g.Parallel processing in Parallel processing in Warcraft IIIWarcraft III
Concepts you need know to Concepts you need know to operateoperate the the engine, engine,
Co-ordinate systemCo-ordinate systemColor, contrast, etc. to use ELEColor, contrast, etc. to use ELE
Assignment-dependant conceptsAssignment-dependant concepts, e.g., e.g. Character ModelingCharacter Modeling
Modding and LearningModding and Learning
TransferTransfer of Computer Science, Math, Art of Computer Science, Math, Art ConceptsConcepts
Different Different engines promote engines promote different different concepts, concepts, and require different and require different pre-reqspre-reqs
GraphicsGraphics: Vector Math, Geometry, Animation: Vector Math, Geometry, Animation ProgrammingProgramming: Parallel processing, event : Parallel processing, event
programming, Object Oriented Programmingprogramming, Object Oriented Programming ArtsArts: Architecture Design, map design, visual : Architecture Design, map design, visual
Composition, Lighting, camera movement, etc.Composition, Lighting, camera movement, etc.
Game Projects involvedGame Projects involved
Project ManagementProject ManagementGroup WorkGroup Workiteration designiteration designPrototypingPrototypingCritiqueCritique
Game Design concepts: Game Design concepts: MechanicsMechanics RulesRules etcetc
What do students learn when they What do students learn when they are building games using the are building games using the
current tools?current tools?