PowerPoint Presentation
Connect with your
1
When I first saw Kinect2
Its really something. Whether your 6 or 66 if you remember how
to pretend, you can have fun.3
The gaming peripheral may have been the end of the story.. But
hackers being who they are
Johnny Chung Lee and the Adafruit bounty.
4
Exploring what the Kinect can do.5
Art and puppets6
robotics7
HUDs and NUIs8
The really out there stuff9
Wow!
10
Motorized Tilt
RGB Camera
3D Depth Sensors
Microphone Array
11
openkinect.orgwww.openni.org
kinectforwindows.org
Open kinect this march just got skeleton tracking
committer12
Kinect for Windows
Retails for $250. You can develop with the Xbox Kinect, but
youll need this to go live.
But for students, educators, museums and the like can get for
$150.
Also includes near mode. to see objects as close as 50
centimeters in front of the device without losing accuracy or
precision, with graceful degradation down to 40 centimeters
13
Thats not USB
Youll need this
Xbox Kinect
If you got your Kinect in a bundle, youll need this add
on.14
What you Need
SDK
15
Demo 0Hello Tilt
16
BGR32 Formatwidth in pixelsheight in pixels
Stride = Bytes Per Row = bytes per pixel * width = 4 * width
4 Bytes Per pixelByte Array:
17
Demo 1Hello Colorful World
18
Depth data
500mm500mm500mm500mm2000mm2030mm2010mm2000mm500mm500mm500mm2500mm
Our vision is a top down and a bottom up process. We use things
like shadows and colors and what we know about the world to infer
19
Depth dataMax resolution for raw depth data is 640 x 480 2 bytes
per pixel of depth data (short)11011110 pixel pixel
pixel0000111101010000001110101000001depthInMM = pixel >>
DepthImageFrame.PlayerIndexBitmaskWidth000000000
20
Demo 2Depth
21
Depth + Player Data
500mm500mm500mm500mm2000mm2030mm2010mm2000mm500mm500mm500mm2500mm
player 1player 1player 1player 1player 1player 1player
1SkeletonStream.Enable();var player = pixel &
DepthImageFrame.PlayerIndexBitmask; 1101111000001010
22
Demo 3Depth + Player Data
23
Skeletal Tracking
24
skeletonFrame.Skeletons
whereskeleton.TrackingState==SkeletonTrackingState.TrackedskeletonFrame.Skeletons
fromskeletoninskeletonFrame.Skeletons
whereskeleton.TrackingState==SkeletonTrackingState.PositionOnly
25
hand_leftwrist_leftelbow_leftshoulder_lefthand_rightwrist_rightelbow_rightshoulder_righthip_leftheadhip_rightknee_leftknee_rightankle_rightankle_leftfoot_leftfoot_righthip
centerspineshoulder_centerSkeleton.Joints[JointType]Joint.Position
(X,Y,Z) Skeletons are mirroredX, Y, Z in metersOrigin(0,0,0) =
center of camera.
+Z is out +X is to the right+Y is up
26
Demo 3Skeletal Tracking
27
Composites
+Color+
28
Composites
Color
Color
+++MapDepthFrameToColorFrameMapDepthToColorImagePointMapDepthToSkeletonPointMapSkeletonPointToColorMapSkeletonPointToDepth
29
Demo 4Green Screen
30
Kinect Audio
31
32
Echo Cancellation
33
Echo CancellationEchoCancellationMode: None, CancellationOnly,
CancellationAndSuppressionint EchoCancellationSpeakerIndex: Index
of speaker to cancel
34
Noise Suppression
35
Noise Supressionbool NoiseSuppression true, false
36
Automatic Gain Control
37
Automatic Gain Controlbool AutomaticGainControl true, false
38
Beam Forming
39
Beam Forming
-0.875 radians-50 degrees0.875 radians 50 degrees
.0175 radians 10 degrees
40
Demo 5Speech Recognition
41
Audio @!%!#!! GotchasUse Microsoft.Speech not System.SpeechMust
run in an MTAAEC SystemModes crash if no speaker output.Must plug
Kinect into WallDevicePairGuid / Qualitymetrics = crashShould run
in ThreadMode Priority Highest?
42
Demo 6Beam Forming
43
Kinect SDK 1.5!Seated Skeletal TrackingFace TrackingJoint
RotationKinect Studio
44
Creative Commons AcknowledgmentsFlickr by Ian LewisFlickr by
major nelsonFlickr by markodesignFlickr by stephantomFlickr by
Abstract MachineFlickr By Aixo.co.krFlickr By AndresmhFlickr by
Antony BennisonFlickr by Ars ElectronicaFlickr By bm.iphoneFlickr
by campuspartymexicoFlickr by Chris JagersFlickr By ci_polla Paola
SucatoFlickr by creepyhalloweenimagesFlickr by cvanderFlickr by
IMAL.orgFlickr by Institut DouglasFlickr by Joshua and AmberFlickr
by Logon SakaiFlickr by openexhibitsFlickr by Paul_Clarke
Flickr by petyr.rahlFlickr by ptFlickr by StilldavidFlickr by
y3rduaFlickr by yesyesnonoFlickr by yum9meFlickr by Jeff
HowardFlickr by jhscreenwmFlickr by Kyle McDonaldFlickr by Morten
Rand-HendriksenFlickr by Stuart HerbertFlickr by VisiophoneFlickr
by orkomedixMichael Zllner and Stephan Huber at the University of
KonstanzOpenclip by J_AlvesOpenclip by jetxeeOpenclip by
molumenOpenclip by shokuninOpenclip by JhnriOpenclip by
MachovkaAdafruit image of Hector Martin
Thanks!!
45
