Kinect Report

8/3/2019 Kinect Report

1/26

Table Of Contents

1.Brief overview of Kinect

2.History of kinect

3.Why develop kinect?

4.Advantages/problems with kinect5.How kinect works?

6.Kinect for windows

7.How to set up kinect with xbox 360 console

8.Technology in kinect

9.Projects on kinect

10. 1 of the most popular games involving

kinect (star wars)


2/26

KinectBrief overview of kinect

kinect is basically a motion sensing input device developed by Microsoft for the Xbox 360

video game console (and now for PC). Based around a webcam-style add-on peripheral for

the Xbox 360 console, players can navigate around and interact with the xbox 360 console

via hand/body gestures and spoken command, completely eliminating the need of a game

controller.

Kinect was launched in

North America on November 4, 2010

Europe on November 10, 2010

Australia, New Zealand and Singapore on November 18, 2010

Japan on November 20, 2010.

Kinect achieved the Guinness World Record of being the "fastest selling consumer

electronics device" due to 10 million units of the Kinect sensor have been shipped as of

March 9, 2011.

(Note : kinect was first known as Project Natal)


3/26

History of kinect

June 6, 2009Microsoft announces Project Natal at the annual E3 press conference. They

demonstrate the new motion sensing technology they have developed.

September 24, 2009 Microsoft provides more information about the games for Natal andwho will be making them.

June 13, 2010 Microsoft announces that Project Natal will now go by Kinect.

July 20, 2010 Microsoft gives Kinect a price and release date of November 4, 2010 with a

price of $149.99.

November 4, 2010 Microsoft launches Kinect in North America.

November 15, 2010 1 million units of Kinect have been sold less than two weeks after its

North American launch.

November 18, 2010 Kinect is launched in Australia.

November 20, 2010 Kinect is launched in Japan.

November 29, 2010 Microsoft has sold 2.5 million Kinect units in the 25 days since it was

released.

January 5, 2011 8 million Kinect units have been sold.

February 21, 2011 Microsoft announces a plan to release new software for Kinect on

Windows machines in the spring.

March 3, 2011 10 million Kinect units have been sold.

Why develop kinect?

A few important points :

Microsoft wanted people to start using their full bodies, rather than just their hands,while gaming

good way for people to interact without controllers, broadening the typical gamer base

encourages people to be active

released to compete with the Nintendo Wii and the Playstation eye and move.


4/26


5/26

in such a case , users must adjust the lighting condition as the RGB camera

embedded is unable to function in darkness comparable to dark skinned

peoples

How kinect works?

Motion sensor

o Invisible light source illuminates subject

o Sensor chip measures distance and light travel

o Unique embedded imaging software uses depth map to perceive & identify objects

o End user device reacts appropriately

MiddleWare

kinect firmware

can recognize people

distinguishing human body parts, joints, movements and faces

the camera knows who you are, once u step in

Xboxs processor

o translating movement captured by kinect camera to meaningful on screen events
http://www.wired.com/images_blogs/gadgetlab/2010/11/Canesta-howitworks1.jpg


6/26

o context-specific

o easier time recognizing some gestures and postures than others

Skeletal tracking

o creates a digital skeleton of you based on depth data.

o If you move left or right or jump around, the sensor will capture it and put you in the

game.

Facial recognition

o Players have to create a kinect ID (profile)

o This ensure that the kinect will recognize the player in future

Kinect has 2 camera :

A traditional color video camera (light dependent)

o Take pictures and enable video conference

Infrared light sensor (not light dependent)

Digital Skeleton


7/26

o Measure depth position and motion

For facial recognition to work, it uses both cameras

Voice recognition

o Have 4 mics within the sensor to recognize and separate your voice from other

noises

o Allow you to control movies and etc with your voice

(e.g. when playing a movie, u can say play kinect or when pausing a movie, pause

kinect)

Kinect for Windows SDK

Kinect is also window compatible

Need to download some additional drivers specially for kinect (OpenNI, Sensor

kinect drivers, NITE middleware) as the official driver havent been released yet

Able to build kinect applications with C++, C#, or Visual Basic by using Microsoft

Visual Studio 2010

Have the following features

1. Raw sensor streams: Access to low-level streams from the depth sensor, color

camera sensor, and four-element microphone array.

2. Skeletal tracking: The capability to track the skeleton image of one or two people

moving within the Kinect field of view for gesture-driven applications.

3. Advanced audio capabilities: Audio processing capabilities include sophisticated

acoustic noise suppression and echo cancellation, beam formation to identify the

current sound source, and integration with the Windows speech recognition API.

4. Sample code and Documentation.


8/26

How to set up kinect with xbox 360 console ?

Position the kinect sensor

.

Kinect sensor needs to see your entire body.

Place the sensor near the edge on a flat, stable surface.

Position the sensor between 2 feet (0.6m) and 6 feet (1.8m) from the floor. Ideally,

the sensor should be within 6 inches (15 cm) above or below your TV.

Avoid positioning the sensor in direct sunlight or within 1 foot (.3m) of audio

speakers.

Do not manually tilt the sensor, it adjusts automatically.

Be careful not to drop the sensor


9/26

Set up the play space

Kinect needs to be able to see your entire body.

Clear the area between the sensor and the players.

One player: Stand back 6 feet (1.8 m).

Two players: Stand back 8 feet (2.4 m).

Make sure that the play space is at least 6 feet (1.8 m) wide, and not wider or longer

than 12 feet (3.6 m).

Make sure the room has bright, even lighting.

Connect the cables


10/26

Original Xbox 360 console

Plug one end of the cable into the USB port on the back of the console and the other

end into an electrical outlet.

If you have a wireless networking adapter, plug it into the USB port on the front of

the console by using the supplied USB extension cable.

Move all other USB accessories to USB ports on the front of the console.

Xbox 360 S console

If the Kinect sensor is attached to a cable with a power plug, disconnect the power

plug portion of the cable. A power cable is not needed with the Xbox 360 S console. Plug the sensor cable into the AUX port on the back of the console.


11/26

Update Console Software

To update your console software for Kinect

Insert a Kinect game disc and follow the on-screen instructions to update the console

software.

Don't have a Kinect game disc? Sign into Xbox LIVE. If a console update is available,

you will be prompted to update your console software.


12/26

Technology in kinect

Front view

Back view Side view Bottom view

kinect sensor device consisted of 6 components

3D depth sensors

RGB camera

Multi-Array MIC

Motorized Tilt

Kinect sensor

Microsoft software

3D Depth Sensors

RGB Camera

MultiArra MIC

Motorized Tilt

3D Depth Sensor Emitter


13/26

3D Depth Sensors

Consists of infrared laser projection combined with CMOS (complementary metal

oxidesemiconductor) sensor

captures video data in 3D under any ambient light conditions

adjustable sensing range

calibrating the sensor based on gameplay and players physical environment

make facial recognition more accurate

RGB Camera

a video camera that deliver 3 basic color components

found in various webcams and mobile phones

standard 640x480 resolution

operating at 30 frames per second

helps to enable facial recognition and more

depend on visible light

Multi-Array MIC

a microphone that able to locate voice and extract ambience noise.

Enable use of voice command

Able to navigate around with voice Enables chatting online (msn, skype)

Multi-Array Mic consisted of many microphones functioning as a single directional input

device.

Using sound propagation principles, the principal sound sources in an environment can be

located and distinguished from each other

Kinect Sensor

Combination of RGB Camera, 3D Depth Sensor and multiarray microphone

Track full body movement and individual voice, turning you into a controller

Microsoft Software

A proprietary software layer

Differentiates kinect from other technology in the market with its ability to enable

human body recognition and extract other visual noise.

Motorized Tilt


14/26

Necessary as not every room is the same size and not every television has the same

height

Can tilt up or down 30 degree

Zoom function

Projects on kinect

Project which involve hand manipulation

Object Manipulation

In the screenshot above, the man is manipulating the kettle by controlling the movement of

the kettle with his 2 hands.

More information can be found here :http://www.vimeo.com/17045326

Grabbing and letting go
http://www.vimeo.com/17045326http://www.vimeo.com/17045326http://www.vimeo.com/17045326http://www.vimeo.com/17045326


15/26

Need to enter the contours and convexity defects to estimate when a hand opens or closes

in the screenshots above , the person is controlling the virtual hand grabbing the yellow ball

and replacing it at another location with his own hands.

More information can be found here :http://vimeo.com/17333026

Shadow Puppet

In the screenshot above, a program was created in the hacked kinect to recognize the elbow,

forearm, wrist, fingers and thumb to in the bird shape.

The bird movement is controlled by the movement of the hand , by opening and closing your hand

like a beak causes the bird to do the same .
http://vimeo.com/17333026http://vimeo.com/17333026http://vimeo.com/17333026


16/26

More information can be found here :

http://www.popsci.com/diy/article/2010-11/five-hacks-free-microsofts-kinect-xbox?page=6

Making my room dance

In the screenshot above, the person is dancing and as the person dances, the room react to

the dancing movement of the person by shaking and moving with the flow.

More information can be found here :http://vimeo.com/17176078

Manipulating stuffed monkey

In the screenshot above, the person is controlling the monkey with his body movements

http://www.popsci.com/diy/article/2010-11/five-hacks-free-microsofts-kinect-xbox?page=6http://www.popsci.com/diy/article/2010-11/five-hacks-free-microsofts-kinect-xbox?page=6http://vimeo.com/17176078http://vimeo.com/17176078http://vimeo.com/17176078http://www.popsci.com/diy/article/2010-11/five-hacks-free-microsofts-kinect-xbox?page=6


17/26

http://technabob.com/blog/2011/08/11/kinect-hack-stuffed-monkey/

Project which help visually impaired to navigate around

In the screenshot above, the kinect is helping a visually impaired person to navigate around

warning the user about any obstruction or obstacles in his way by a vibrator in the belt, and

a Bluetooth headset which provide verbal feedback.

More information can be found here :http://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigat

e_indoors.html

Projects that involve creativity and drawings

Ghost following you
http://technabob.com/blog/2011/08/11/kinect-hack-stuffed-monkey/http://technabob.com/blog/2011/08/11/kinect-hack-stuffed-monkey/http://technabob.com/blog/2011/08/11/kinect-hack-stuffed-monkey/http://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigate_indoors.htmlhttp://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigate_indoors.htmlhttp://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigate_indoors.htmlhttp://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigate_indoors.htmlhttp://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigate_indoors.htmlhttp://technabob.com/blog/2011/08/11/kinect-hack-stuffed-monkey/


18/26

In the screenshot above, the ghost is draw in the video frame and its designed in such a way

that the ghost is always behind your blind spot (behind you)


http://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtml

Projects which involve virtual effects

Dragonball Z-Style Super Saiyan

In the screenshot above, the person grow out fake virtual hair and has a yellow aura around

his body and he can unleash the super saiyan which consisted of light beam and various

lightning effects.


http://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.ht

ml

Comic book sound effect in fighting scene
http://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtmlhttp://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtmlhttp://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtmlhttp://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.htmlhttp://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.htmlhttp://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.htmlhttp://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.htmlhttp://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.htmlhttp://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtmlhttp://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtml


19/26

in the screenshot above , when a person is being punched/kicked by another person , a

comic book-esque sound effects together with the wordings will be played

more information can be found here :

http://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_b

ook_fight_scene.html

cool lighting effects

In the screenshot above, the kinect sensor is reacting to the movement and gesture of the

lady hand and thus triggering the lighting on the sofa

More information can be found here :http://www.youtube.com/watch?v=ho8KVOe_y08
http://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_book_fight_scene.htmlhttp://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_book_fight_scene.htmlhttp://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_book_fight_scene.htmlhttp://www.youtube.com/watch?v=ho8KVOe_y08http://www.youtube.com/watch?v=ho8KVOe_y08http://www.youtube.com/watch?v=ho8KVOe_y08http://www.youtube.com/watch?v=ho8KVOe_y08http://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_book_fight_scene.htmlhttp://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_book_fight_scene.html


20/26

Interactive stage lighting with kinect

In the screenshot above, hands will be placed in the hole and the kinect sensor inside the

box will read the hand movement and correspond to them

Some of the beautiful lighting effect projected out


http://kinecthacks.net/interactive-stage-lighting-with-kinect/
http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/


21/26

project involve robots

kinect powered robots that could help earthquake victims

In the screenshot above, the robot have sensors which act as eyes to search for victims in

an earthquake zone

They are the 3rd

team of peoples who revamped this rescue robots but using kinect as

shown below
http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/


22/26

In the screenshot above, a kinect powered robot can help to locate earthquake victims and

rescuing them out of the earthquake zone


http://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.h

tml

Kinect assembly robot
http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.htmlhttp://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.htmlhttp://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.htmlhttp://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.htmlhttp://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.html


23/26

step 1 : worker first prepare the materials on the surface

Step 2 : initiate the robot by pressing the start button

Step 3 : robots began to assembly the parts as programmed by the person in charge

More information can be found here :http://www.kinecthacks.com/kinect-assembly-robot/

Surgical robots

Start button
http://www.kinecthacks.com/kinect-assembly-robot/http://www.kinecthacks.com/kinect-assembly-robot/http://www.kinecthacks.com/kinect-assembly-robot/http://www.kinecthacks.com/kinect-assembly-robot/


24/26

in the screenshot above, a kinect camera is used for hand gesture recognition to give

commands to the robots and to browse through radiological images on a nearby computer

screen

Example of hand gesture recognition : cutting with index and middle finger to indicate scissors.

more information can be found here :

http://medgadget.com/2011/02/kinectpowered_purdue_robot_to_replace_scrub_nurses.html

Kinect powered robot (jedibot) duelling

Kinect Sensor
http://medgadget.com/2011/02/kinectpowered_purdue_robot_to_replace_scrub_nurses.htmlhttp://medgadget.com/2011/02/kinectpowered_purdue_robot_to_replace_scrub_nurses.htmlhttp://medgadget.com/2011/02/kinectpowered_purdue_robot_to_replace_scrub_nurses.html


25/26

In the screenshots above, the kinect sensor tracks the location of both lightsabers and has the

robot make an attack or defending move based on their positions.


http://science.kqed.org/quest/2011/07/26/stanford-students-create-kinect-powered-jedibot/

1 of the most popular games involve kinect

Star Wars

Kinect Star Wars allows fans to physically experience training as a Jedi, using the Force and

battling with a lightsaber.

Gameplay Screenshots
http://science.kqed.org/quest/2011/07/26/stanford-students-create-kinect-powered-jedibot/http://science.kqed.org/quest/2011/07/26/stanford-students-create-kinect-powered-jedibot/


26/26

Flashing and slashing of lightsabers

Using the force to move and carry objects

Documents

Kinect Report