( blue eyes – Human operator monitoring system ) · What is blue eye technology ? The The BLUE EYES technology aims at creating computational machines that have perceptual and sensory

Blue eye technologyBlue eye technology

( blue eyes –( blue eyes –

Human operator Human operator monitoring system )monitoring system )

AgendaAgenda

MotivationMotivation What is What is BlueEyeBlueEye technology ? technology ? What is What is BlueEyesBlueEyes ? ? System designingSystem designing System overviewSystem overview DAUDAU CSUCSU Data securityData security IBM researchIBM research ConclusionConclusion

•Motivation

•What isBlueEyes ?

•Physiological foundations

•Systemdesign

•System implementation

•Futureimprovements

•Project worksummary

•Systemdemonstration

MotivationMotivation

Human error - a frequent reason of Human error - a frequent reason of catastrophes and ecological disasterscatastrophes and ecological disasters

•Motivation



•Systemdesign





-temporary indisposition-weariness-oversight

What is blue eye What is blue eye technology ?technology ? The BLUE EYES technology The BLUE EYES technology

aims at creating aims at creating computational machines that computational machines that have perceptual and sensory have perceptual and sensory ability like those of human ability like those of human beings.beings.

How can we make How can we make computers “see” and computers “see” and “feel” ?“feel” ? Blue eyes uses sensing technology to Blue eyes uses sensing technology to

identify a user’s actions and to extract identify a user’s actions and to extract key information.key information.

Information is then analyzed to Information is then analyzed to determine the user’s physical determine the user’s physical ,emotional ,or informational state.,emotional ,or informational state.

What is What is Blue EyesBlue Eyes ? ?

The complex solution for human-The complex solution for human-operator monitoring:operator monitoring:– Visual attention monitoringVisual attention monitoring– Physiological conditionPhysiological condition– Operator’s position detectionOperator’s position detection– Wireless data acquisition using bluetoothWireless data acquisition using bluetooth– Real time user defined alarm triggeringReal time user defined alarm triggering– Recorded data playbackRecorded data playback

•Motivation



•Systemdesign





Doesn’t predict nor interfere with Doesn’t predict nor interfere with operator’s thoughtsoperator’s thoughts

Cannot force directly the operator to Cannot force directly the operator to workwork

What is BlueEyes not?What is BlueEyes not?

App licat ions ofApp licat ions of Blue EyesBlue Eyes

Generic control roomsGeneric control rooms

((System can be applied in every working System can be applied in every working environmentenvironment

requiring permanent operator’s attention)requiring permanent operator’s attention)– Power stationPower station– Captain bridgeCaptain bridge– Flight control centersFlight control centers– Operating theatres – anesthesiologistsOperating theatres – anesthesiologists

•Motivation



•Systemdesign





BlueEy es BlueEy es - benefits- benefits

Prevention from dangerous incidents

Minimization of– ecological consequences– financial loss– a threat to a human life

The reconstructionof the course ofoperator’s work

•Motivation



•Systemdesign





Implementation and Implementation and engineering engineering considerationsconsiderations-THREE groups of users-THREE groups of users OPERATOR -a person whose physiological parameters are OPERATOR -a person whose physiological parameters are

supervisedsupervised

- authorization in the system- authorization in the system

- receiving alerts- receiving alerts SUPERVISOR – a person responsible for analyzing operator’sSUPERVISOR – a person responsible for analyzing operator’s

condition and performance.condition and performance. SYSTEM ADMINISTRATORS- maintains the systemSYSTEM ADMINISTRATORS- maintains the system

deliver tools for addingdeliver tools for adding

- new operator’s to the database- new operator’s to the database

- defining alarm conditions- defining alarm conditions

- configuring logging tools- configuring logging tools

- creating new analyzer modules- creating new analyzer modules

DesigningDesigning

A personal area network for A personal area network for linking all the operators and the linking all the operators and the supervising systemsupervising system

Two major units Two major units - DAU (data acquisition unit )DAU (data acquisition unit )- CSU (central system unit )CSU (central system unit )

System overviewSystem overview

Central System UnitData Acquisition Unit

Atmel 89C52microcontroller

JazzMultisensor

ConnectionManager

Data Analysis

VisualizationModuleBluetooth Bluetooth

Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system.

DataLogger

•Motivation



•Systemdesign





•Objectives

•System overview

•DAU

•CSU

The eye movement The eye movement sensorsensor Off-shelf eye movement sensor – Jazz Off-shelf eye movement sensor – Jazz

multisensormultisensor Supplies raw digital data regardingSupplies raw digital data regarding- Eye positionEye position- Level of blood oxygenationLevel of blood oxygenation- Acceleration along horizontal and Acceleration along horizontal and

vertical axesvertical axes- Ambient light intensityAmbient light intensity

•Motivation



•Systemdesign





Jazz multisensorJazz multisensor

Jazz Multisensor

JazzJazz

DAUDAU

The DAU consists of the following componentsThe DAU consists of the following components ATMEL 8952 microcontrollerATMEL 8952 microcontroller BLUE TOOTH MODULE – supports synchronous voice BLUE TOOTH MODULE – supports synchronous voice

data transmissiondata transmission PCM CODEC –used to transmit operator’s voice and PCM CODEC –used to transmit operator’s voice and

central system sound feedbackcentral system sound feedback UART –communication between bluetooth module and UART –communication between bluetooth module and

microcontroller (115200 bps)microcontroller (115200 bps) MAX232 –level shifterMAX232 –level shifter ALPHAUNUMERIC LCD displayALPHAUNUMERIC LCD display LED indicatorsLED indicators ID CARD interfaceID CARD interface

BlueEyesData Acquisition Unit

ID card

DAU - componentsDAU - components

MC 145483PCM codec

ID cardinterface

Simplekeyboard

Jazz Multisensor

Beeper

LCD display

LED indicators


UART MAX232

UART

Bluetoothmodule

microphone

earphone

Microcontroller & blue Microcontroller & blue tooth moduletooth module

EEPROMS & the PCBEEPROMS & the PCB

Implementation - DAUImplementation - DAU

Bluetooth module

PCM codec board

Batteries andvoltage level monitor

Microcontroller board

•Motivation



•Systemdesign





•DAU

•CSU

DAU - featuresDAU - features

LightweightLightweight Runs on batteries - Runs on batteries -

low power low power consumptionconsumption

Easy to use - does not Easy to use - does not disturb the operator disturb the operator workingworking

ID cards for operator ID cards for operator authorizationauthorization

Voice transmission Voice transmission using hardware PCM using hardware PCM codeccodec

Data Acquisition Unit


JazzMultisensor

Bluetooth

•Motivation



•Systemdesign





•Objectives

•System overview

•DAU

•CSU

CSU - featuresCSU - featuresCentral System Unit

ConnectionManager

Data Analysis

VisualizationModuleBluetooth

Connection management

Data processing

Visualization Data recording Access

verification System maintenance

ConnectionManager

•Motivation



•Systemdesign





•Objectives

•System overview

•DAU

•CSU

DataLogger

CSU – componentsCSU – components

CONNECTION MANAGER – main task to CONNECTION MANAGER – main task to perform low-level blue tooth communicationperform low-level blue tooth communication

DATA ANALYSIS MODULE – performs the DATA ANALYSIS MODULE – performs the analysis of the raw sensor data in order to analysis of the raw sensor data in order to obtain information about operator’s obtain information about operator’s physiological conditionphysiological condition

DATA LOGGER MODULE – provides support DATA LOGGER MODULE – provides support for storing the monitored data.for storing the monitored data.

VISULAIZATION MODULE – provides user VISULAIZATION MODULE – provides user interface for the supervisors interface for the supervisors

CSU - Visualization CSU - Visualization ModuleModule•Motivation



•Systemdesign





•Objectives

•System overview

•DAU

•CSU

Raw and processed data visualization using:

audio and videodata playback

– VU-meters– pie-charts– time series

Data securityData security

Only registered mobile devicesOnly registered mobile devicescan connect to the systemcan connect to the system

Bluetooth connection authenticationBluetooth connection authentication Bluetooth connection encryptionBluetooth connection encryption Access rights restrictionsAccess rights restrictions Personal and physiological data Personal and physiological data

encryptionencryption

•Motivation



•Systemdesign





•Objectives

•System overview

•DAU

•CSU

IBM researchIBM research

BLUE EYE – EMOTIONAL MOUSEBLUE EYE – EMOTIONAL MOUSE sensors in the mouse ,sense the sensors in the mouse ,sense the

physiological attributes which are physiological attributes which are correlated to emotions using correlated to emotions using correlation modelcorrelation model

-by simply touching the mouse ,the -by simply touching the mouse ,the computer will be able to determine a computer will be able to determine a person’s emotional state.person’s emotional state.

BLUE EYE enabled TELEVISION – could BLUE EYE enabled TELEVISION – could become active when the user makes become active when the user makes an eye contact.an eye contact.

ConclusionConclusion

In the near future ,ordinary In the near future ,ordinary household devices- such as household devices- such as television , refrigerators ,ovens television , refrigerators ,ovens may be able to do their jobs when may be able to do their jobs when we look at them and speak to we look at them and speak to them.them.

Future applications of blue eye Future applications of blue eye technology is limitlesstechnology is limitless

ReferencesReferences

www.seminar4u.comwww.seminar4u.com www.howstuffswork.comwww.howstuffswork.com www.cs.put.poznan.comwww.cs.put.poznan.com www.whitepapers.comwww.whitepapers.com www.ibmresearchcenter.comwww.ibmresearchcenter.com www.almaden.ibm.comwww.almaden.ibm.com

http://www.seminar4u.com/

http://www.howstuffswork.com/

http://www.cs.put.poznan.com/

http://www.whitepapers.com/

http://www.ibmresearchcenter.com/

http://www.almaden.ibm.com/

Thank you Thank you

BlueEyesHuman-Operator Monitoring System

•Motivation



•Systemdesign





Documents

( blue eyes – Human operator monitoring system ) · What is blue eye technology ? The The BLUE EYES technology aims at creating computational machines that have perceptual and sensory