Embed Size (px)
Citation preview
MotionMotionSensors Sensors + Audio Visual + Audio Visual Performance:Performance: Using Kinect to Control Live Visuals and Sound
by Joey Bargsten, Ph.D. Florida Atlantic University • SCMS • MTEn
MotionMotionSensors Sensors + Audio Visual + Audio Visual Performance:Performance: Using Kinect to Control Live Visuals and Sound
by Joey Bargsten, Ph.D. Florida Atlantic University • SCMS • MTEn
Thanks!
http://film-gate.org
http://sweatrecordsmiami.com
Format/Overview:
• Presentation / Demo (noon – 12:40)
• Questions? (12:40 – 12:50)
• Break (10 min)
• Workshop on PureData (1pm – 2pm)
Background on Kinect
Background on Kinect
• Released November 2010
Background on Kinect
• Released November 2010
• Sold 8M units in 60 days
Background on Kinect
• Released November 2010
• Sold 8M units in 60 days
• SDK Drama!
Kinect (Development)Community• Gaming and Virtuality
Kinect (Development)Community• Gaming and Virtuality
• Medicine
Kinect (Development)Community
• Gaming and Virtuality
• Medicine
• Surveillance, Security, Facial Recognition
Kinect (Development)Community• Gaming and Virtuality
• Medicine
• Surveillance, Security, Facial Recognition
• Robotics
Kinect (Development)Community• Gaming and Virtuality
• Medicine
• Surveillance, Security, Facial Recognition
• Robotics
• 3D Scanning and Printing
Kinect (Development)Community
• Gaming and Virtuality
• Medicine
• Surveillance, Security, Facial Recognition
• Robotics
• 3D Scanning and Printing
• Avant-Garde Art, Music, and Performance
Suggestions,ApproachesSystems & Configurations
My Configuration
• Kinect Sensor (2011); Model 1414, EN/ES US HD
(this is the most ‘open’ one!)
• Synapse Middleware (converts skeleton to OSC)
• PureData v. 0.43.4-Extended (incl. MrPeach library)
• Quartz Composer v. 4.0 (+ old FW400 SD
camcorder)
• 2011 MBP, 2.4mHz, running OSX 7.5 “Lion”.
Approaches to Performance• One-Person Band (virtuoso!)
– One person, one controller
– Controls both audio and visual
Video on YouTubehttp://www.youtube.com/watch?v=1ge7RcIBWsc
Approaches to Performance• Division of Labor (conventional
ensembles)
Approaches to Performance• Division of Labor (conventional
ensembles)
“For every new technology that comes along,
make an ensemble out of it!”—JB, EMV (2011)
Approaches to Performance• Division of Labor (conventional
ensembles)
“For every new technology that comes along,
make an ensemble out of it!”—JB, EMV (2011)
• Applied to Kinect:–Multiple controllers –Dedicate separate software to audio and visuals–Multiple performers for audio and visuals
What Do You Want to Control?
How Do You Want To Control It?
• Discontinuous, trigger events
–model: keyboard
• Continuous, gradual events
–model: theremin
• Hybrid Control: multiple parameters
–model: sample player
Kinect Control Parameters
Kinect Kombo™:Quad Theremin + Sample Player
• Built in PureData • Free, cc-by-d-nc
Kinect Kombo™:Skeleton View + Performer Interface
Kinect Kombo™
Kinect Kombo™
Controlling Visuals
Kinect and
Quartz
Composer
TheAugmentedDance:
LiveBody—Kinect— VirtualBody
Variations• Figure
– Simple Camera Figure and Skeleton
– Primitives, 3D Objects, Video Mapped to
Skeleton
• Paint Brush/Image Hose (Effects Mapped to
Skeleton)
• Background (Texture or Stylized Video Controlled by Skeleton)
“My Profundis (2012)”
Video on YouTube http://www.youtube.com/watch?v=gT1lbhR1Qj8
Resources
• OpenKinect.org
• OpenNI.org
• PureData.info
• Synapsekinect.tumblr.co
m
• CreateDigitalMusic.com
• CreateDigitalMotion.com
• V002.info (vade &
bangnoise)
• Cycling74.com
• IdeaChannel ‘Kinect’
Episode
• OffBook ‘Creative Coding’
Episode
• www.fau.edu/scms (MFA -
MTEn)
• www.badmindtime.com
• Melancholalaland™ (my
multimedia opera)
