JYOTHI ENGINEERING COLLEGE

DEPARTMENT OF IT

SEMINAR ON

SIXTH SENSE TECHNOLOGY

GUIDED BY: Prof. JISNA BASTINE

PRESENTED BY:

JISMI K JACOB S8 IT 18

04/12/2023 1JECC IT Department

CONTENT

• Introduction• History• Components• Applications• Working Principle• Related Technologies• Advantages And Disadvantages• Future Enhancement• Conclusion• References

04/12/2023 2JECC IT Department

INTRODUCTION

• Sixth Sense is a wearable gestural interface.

• Sixth Sense is a gesture-driven computing platform.

• It augments the physical world with digital information.

• It bridges the gap by bringing intangible, digital

information out into the tangible world.

• Device coupled with a projector, camera and a phone.

• All the information stored on the web.04/12/2023 3JECC IT Department

WHY SIXTH SENSE?

Sixth Sense prototype implements several applications that

demonstrate the usefulness, viability and flexibility of the

system.

04/12/2023 4JECC IT Department

Sixth Sense Technology - connecting the digital world

with the physical world seamlessly, eliminating hardware

devices.

HISTORY• Steve Mann is the father of

Sixth Sense Technology who made

of wearable computer in 1990.

• He implemented the neck worn

projector with a camera system.

• It was implemented by Pranav

Mistry, a PhD student in the Fluid

Interfaces Group at the MIT Media

Lab.04/12/2023 JECC IT Department 5

COMPONENTS

• Camera

• Projector

• Mirror

• Smart Phone

• Color Marker

• Microphone04/12/2023 JECC IT Department 6

CAMERA

• Input device of system.

• It acts as a digital eye of the system.

• It captures the scene the user is looking at.

• Captured data is passed to mobile computing device.

• Software program processes the video stream data captured

by the camera.

• It tracks the locations of the colored markers at fingers.

04/12/2023 JECC IT Department 7

PROJECTOR

• Output device of the Sixth Sense system.

• Mobile device provides the data to be projected.

• Projector unit runs on a rechargeable battery.

• It have a tiny LED projector.

• GUI of the selected application, projects onto any surfaces.

• It projects related information of user interacts with

from the Internet just-in-time.

04/12/2023 JECC IT Department 8

MIRROR

• Mirror reflects data from the projector.

• Mirror reflects image on to the desired surface.

• Mirror dangles pointing downwards from the neck.

• User manually can tilt the mirror to change the location

of the projection.

• Mirror overcoming the limitation of the limited projection

space of the projector.

04/12/2023 JECC IT Department 9

MOBILE COMPUTING DEVICE

• Processing device carried in user’s pocket.

• Software program runs on this device.

• Camera, projector and the microphone are

connected to the device using wired/wireless connection.

• Mobile device given data in turn to the projector.

• Mobile computing device is also connected to the Internet.

• A Web-enabled smart phone in the user’s pocket processes

the video data.

04/12/2023 JECC IT Department 10

COLOR MARKER

• Marker placed at the tip of the user’s fingers.

• Red, green, blue and yellow colored tapes

helps webcam to recognize the hand gestures.

• Movements are interpreted as gestures, that acts as the

instruction for the projected application interface.

• Camera tracks the locations of the colored markers.

• Supports multi-touch and multi-user interaction.

04/12/2023 JECC IT Department 11

MICROPHONE

• Optional component of the Sixth Sense.

• Required when using a paper as a interactive surface.

• User clips the microphone to the paper captures the sound

signals of user’s touching the paper.

• Data passed to mobile computing device.

• Sound signal captured by the microphone provides time

information.

04/12/2023 JECC IT Department 12

APPLICATION1. Make A Call

2. Call Up A Map

3. Check The Time

4. Get Flight Updates

5. Drawing Application

6. Zooming Features

7. Check The EMAIL

8. Take Pictures

9. Get Book Information

10. Get Product Information

11. Feed Information On People

12. Create Multimedia Reading Experiences

04/12/2023 JECC IT Department 13

04/12/2023 JECC IT Department 14

1. Make A Call• To project a keypad onto your

hand. • Use that virtual keypad to make

a call.

2. Call Up A Map• we can call up the map of our

choice.• Use thumbs and index fingers

to navigate the map.

04/12/2023 JECC IT Department 15

3. Check The Time

• Draw a circle on your wrist.

• Get a virtual watch that gives

the correct time.

4. Create Multimedia

Reading Experiences

• Programmed to project related

videos onto newspaper articles

you are reading.

04/12/2023 JECC IT Department 16

5. Get Flight Updates

• System will recognize your

boarding pass.

• Display real time information

about your flight over ticket.

6. Feed Information On People

• Display relevant information

about a person.

04/12/2023 JECC IT Department 17

7. Drawing Application

• Tracking the fingertip

movements.

• User draw on any surface.

8. Zooming Features

• User can zoom in or zoom

out using intuitive hand

movements.

04/12/2023 JECC IT Department 18

9. Get Production Formation

• Sixth Sense recognizes

products we pick up, then

feeds us information on those

products.

10. Get Book Information

• The system can project Amazon

ratings on that book.

• Gives relevant information.

04/12/2023 JECC IT Department 19

11. Take Pictures

• Creating a picture frame tells

the camera to snap a photo.

• Desired photos can project

them onto a surface.

12. Check The EMAIL

• Drawing an @ symbol in the air

allows you to check your email.

WORKING PRINCIPLE

04/12/2023 JECC IT Department 20

ImagesPicturesGestures

Info On

Surface

Working Of The Sixth Sense Device

captureSend for processing

Send info

Project image Reflect onto desired location

•Work like mobile wearable interface.

• Camera, mirror and projector connected to a Bluetooth

smart phone in user’s pocket.

• Camera captures hand gestures and sent to the Smartphone.

• Projector projects the output image on to any surface.

• Mirror dangles near to the projector, reflects image onto the

desired surface.

• Digital information is placed in the physical world.04/12/2023 JECC IT Department 21

Contd…..

RELATED TECHNOLOGIES

1. Gesture Recognition

2. Computer Vision

3. Radio Frequency Identification

4. Augmented Reality

04/12/2023 JECC IT Department 22

1. Gesture Recognition

• Capture hand gesture recognition.

• Humans interact with device without

using any external interfacing devices.

• Colored bands are used for hand pose tracking.

• Neural network approaches or statistical templates are the

commonly used techniques. 04/12/2023 JECC IT Department 23

2. Computer Vision

•Extract information from an image.

•It basically includes four processes :

•Recognition: Object have useful data or not.

•Motion Analysis: Estimation of motion where an image

sequence is processed continuously.

•Scene Reconstruction: Recreate a 3D image from the image.•Image Restoration: To remove noise from an given image

using filters to get quality image.

04/12/2023 JECC IT Department 24

3. Radio Frequency Identification

• Transmit identity of an object using tag.

• Reader is needed to retrieve data from tag.

• Reader passes the information in digital form.

• Reader collision -occur when signals from two or more

readers overlap.

• Tag collision -occurs when many tags are present in a

small area.

04/12/2023 JECC IT Department 25

4. Augmented Reality• Virtual experience added over real world.

• AR superimpose graphics for every perception.

• Adds graphics, sounds, haptic feedback and

smell to the natural world as it exists.

Three components of AR :

1.Head mounted Display-View superimposed graphics created

2.Tracking System-Tracks the user’s eye and head movements.

3.Mobile Computer- computing device that fits in user’s hand.04/12/2023 JECC IT Department 26

ADVANTAGES DISADVANTAGES• Portable • Hardware limitations of the devices

• Support Multi touch and Multi user interaction

• Many phones will not allow the external camera feed in real time.

• Saves time spent on searching information.

• Post processing can occur however.

• Mind map the idea anywhere • Failure of any components makes functioning difficult.

• Recognizes user’s freehandgestures and saves electricity.

• It is difficult to wear the interface every time.

• Cost Effective • Number of hardware components.

Sixth Sense Technology

FUTURE ENHANCEMENTS

Imagine the world where Sixth Sense Technology is

applied everywhere.

• Elimination of colored markers.

• 3D gesture tracking and assured security.

• Sixth Sense work for disabled person.

• Use of contact lens and laser projectors.

• It could be helpful in rendering defense services.

04/12/2023 JECC IT Department 28

CONCLUSION

• Sixth sense helps us recognize the world around us.

• It makes the applications user-friendly and portable too.

• It helps in displaying the information automatically.

• Simple hand gestures can help to bring the world close to.

• It keeps us technologically awake and connects us with

the world via this newest invention.

• Sixth sense could make the world magical.

04/12/2023 JECC IT Department 29

REFERENCES[1] P. Mistry, P. Maes. Sixth Sense – “A Wearable Gestural

Interface”, In SIGGRAPH Asia 2009.

[2] P. Mistry. “The thrilling potential of Sixth Sense

technology”, TED India 2009. Mysore, India.

[3] Rao, S.S.. “Sixth sense technology”, Communication and

Computational Intelligence (INCOCCI), on 2010.

[4] S.Pradeep , Pandithurai. “Sixth sense technology”, In

Proceedings of the International Conference on 2010.04/12/2023 JECC IT Department 30

31