Upload
opher-etzion
View
296
Download
1
Embed Size (px)
Citation preview
Speaker: Opher Etzion
Event Processing within the Human Body – Tutorial
2
Event processing within the human body may, in time, be the most important use of this technology and the Internet of Things in general
3
In the human body, we sense, make sense of what we sense, make constant decisions and carry them out. Autonomous systems do the same. When the domain is the human body itself it’s getting interesting….
Sensing
Making sense from the sensing
Real-time decision making
Acting
In this tutorial we’ll talk about current and future applications of IoT and autonomous systems within the human body
OUTLINE
Background: events, autonomous systems, Body
Area Networks
Medical oriented applications of event
processing within the human body
Other applications of event processing within the
human body
The future perspective
4
Topic I
TOPIC II
Topic III
Topic IV
OUTLINE
Background: events, autonomous systems, Body
Area Networks
Medical oriented applications of event
processing within the human body
Other applications of event processing within the
human body
The future perspective
5
Topic I
TOPIC II
Topic III
Topic IV
None of the
authorized drivers
location is near the
car’s location
theft is concluded
Use a built-in car
stopper to slow the
intruder and dispatch
the security company
A person enters a
car and the car
starts moving;
the person does not
look like one of the
authorized drivers
Such applications
become possible
since everything is
connected
6
7
The term “Internet of Things” was coined by Kevin Ashton in 1999.
His observation was that all the data on the Internet has been created by a human.
His vision was: “we need to empower computers with their own means of gathering information, so they can see, hear, and smell the world by themselves”.
8
The world of sensors
1 Acoustic, sound, vibration2 Automotive, transportation3 Chemical4 Electric current, electric potential, magnetic, radio5 Environment, weather, moisture, humidity6 Flow, fluid velocity7 Ionizing radiation, subatomic particles8 Navigation instruments9 Position, angle, displacement, distance, speed, acceleration10 Optical, light, imaging, photon11 Pressure12 Force, density, level13 Thermal, heat, temperature14 Proximity, presence
9
The value of sensors
Kevin Ashton: “track and count everything, and greatly reduce waste, loss, and cost. We could know when things needs replacing, repairing or recalling, and whether they were fresh or past their best”
The value is in the ability to know and react in a timely manner to situations that are detected by sensors
10
Differences between the traditional Internet to the Internet of Things
Topic Traditional Internet Internet of Things
Who creates content? Human Machine
How is the content
consumed?
By request By pushing information
and triggering actions
How content is
combined?
Using explicitly defined
links
Through explicitly
defined operators
What is the value? Answer questions Action and timely
knowledge
What was done so far? Both content creation
(HTML…) and content
consumption (search
engines)
Mainly content creation
Two separate but connected goals: Awareness and Reaction
Awareness Reaction
Event
Detect Derive Decide Do
Detect
Something
that may be
important
The act of bringing into a system’s sphere of understanding knowledge about an event.
The detection is done by sensors, instrumentation and human reports.
Sw
im L
an
e
Trigger Event
Activity
StateChange
13
Intelligent Detection
Determine what is actually been sensed: vision understanding, voice understanding, text understanding.
Derive
The act of becoming aware of events that are not directly detectable by bringing together events with other events, data, patterns and publishing the observation as a derived event.
Raw
eventsRaw
eventsRaw
events
A Person or a computer recognizes the pattern and enters the derived event or just reacts to it directly.
15
Event processing: making sense from what we sense…
Combining data from multi-sensors to get observations, alerts, and actions in real-time gets us to the issue of detecting patterns in event streams
16
Intelligent derivation
Find the causality between events and situations.
Decide
Automated decision by
decision management
system
The act of determining the course of action to do in response to the situation. This includes the background information needed to be collected to make the decision.
Pass through: Sometimes there is no decision. There is only one course of action.
management system that seeks a optimizing quantitative goals.
18
Intelligent Decision
Finding the best decision some times under real-time constraints may require an intelligent process.
DoThe act of performing the course of action that was decided upon.
Notification: Sending a signal of sort to either a person or system. This would include calling a web-service or subscription to alerts.
action.
20
Intelligent Actions
Intelligent actuators
21
Knowledge acquisition for IoT based systems
How do we know how to make sense of all these data?
22
Contemporary
healthcare trends:
Telehealth
23
Contemporary healthcare trends: Personalized Medicine
24
Contemporary healthcare trends: Sensors and actuators
25
Body Area Networks
26
Body Area Networks
Source: Min Chen et al – Body Area Networks: A Survey. Mobile Networks Applications, 16: 171-193, 2011.
27
Body Area Networks
Source: Min Chen et al – Body Area Networks: A Survey. Mobile Networks Applications, 16: 171-193, 2011.
OUTLINE
Background: events, autonomous systems, Body
Area Networks
Medical oriented applications of event
processing within the human body
Other applications of event processing within the
human body
The future perspective
28
Topic I
TOPIC II
Topic III
Topic IV
29
Soldier in Battle (including: policeman, fire fighter…)
Wearables:
Camera
GPS
Biometric Sensors
Goal:
Warning
Orientation
Assessment of readiness by commander.
30
Wearables:
Camera
Motion sensors
Biometric SensorsGoal:
Self feedback
Trainer feedback
Real-time game management for the coach.
Training and Sport
31
Fitbit Surge:
Wristwatch that contains GPS, tracks steps, monitors heart-rate, connects to community
Training monitoring
32
NICU: Pre-mature babies monitoring
Personalized alerts based on collection of monitors: when nurse should be alerted, when physician should be alerted.
There are many false alerts that are ignored, Missing or ignored alert is sometimes fatal
33
iFeel Labs:
Remove stress by bio-feedback. Using games.
Stress control
34
Goal:
Monitor and diagnose sleep disorders .
Sleep disorders monitoring
Source: N. Oliver & F. Flores-Mangas: Health Gear – Automatic Sleep Apnea Detection and Monitoring with Mobile Phone. Journal of Communication, 2(2), 2007
35
Goal:
Monitor and diagnose sleep disorders .
Sleep disorders monitoring
Source: N. Oliver & F. Flores-Mangas: Health Gear – Automatic Sleep Apnea Detection and Monitoring with Mobile Phone. Journal of Communication, 2(2), 2007
36
Pebble time:
Wrist watch that measures sleep, sleep patterns, finds the best time to wake up.
Sleep control
37
Wearables:
Wristband, chest patch and spirometer –
Monitoring both the person and the environment for allergy factors
Goal:
Prevent, predict, monitor asthma attacks .
Monitoring Asthma Patients
Source: https://news.ncsu.edu/2016/06/wearable-tech-asthma-2016/
38
Sensors and actuators –technology still under development
Goal:
to improve insulin replacement therapy until glycemic control is practically normal, and to ease the burden of therapy for the insulin-dependent.
Implants for diabetes patients
Source: https://www.slideshare.net/energexsystems/pancreas-presentation
39
Implantable cardioverter defibrillators (ICDs) are used to detect dangerously fast heartbeats and give a lifesaving shock to correct the heart’s rhythm. A person with an ICD has the equivalent of a paramedic sitting on his shoulder, always watching and ready to give the heart “the paddles,”
Implants for cardiovascular diseases
Source: http://www.webmd.com/heart-disease/implantable-cardioverter-defibrillator
40
Smart pacemaker
A pacemaker is a small device that's placed in the chest or abdomen to help control abnormal heart rhythms. This device uses electrical pulses to prompt the heart to beat at a normal rate.
Implants for cardiovascular diseases
source: https://www.cambridgeconsultants.com/media/press-releases/setting-pace
41
Monitoring for cancer cells, replacing biopsy tests. Monitoring for impact of treatments and medications for personalized treatment.
In the future: also actuators that serve to treat and eliminate
Implants for cancer treatment
42
Closed loop to monitor and regulate DBS Helping Essential Tremor and Parkinson Patients.
Controlling deep brain stimulation
43
Connecting audio signals to hearing nerves.
Implants that bypass damaged senses
https://www.foundationforsightandsound.org/wordpress/?page_id=636
44
Aiding blind people
Implants that bypass damaged senses
45
OrCam MyEye is a tiny camera that uses recognition technology to help the blind and visually impaired identify people, text and things. The portable, intuitive device is comprised of a small camera that can be discreetly affixed to the wearer’s own glasses, plus a lightweight computer processor that’s small enough to be carried in a pocket. OrCam MyEye works by capturing an image of an object and translating it into audio for the wearer. The device is able to read text, identify products, read street signs and even recognize faces.
Implants that bypass damaged senses
46
Almost the entire electromagnetic spectrum is invisible to the human eye. Hyper imaging technology and AI will provide super human vision that will help to spot dangers that are hidden from the view
Source: IBM Predictions for 2022.
Super human vision
47
AI based diagnostic of mental disorders based on behavior, speech, writing etc…
Source: IBM Predictions for 2022.
Detection of mental disorders
48
Personalized aides for elderly to maintain independent life
Motion sensor
Door sensor
ChairSensor
Voice Sensor
Alert family member
Alerts example:Door was not locked within 2 minutes after entranceFalling event detectedVocal distress detectedNo motion for certain time period detected
While much technology exists, it is not widely used. It needs to be more personalized, more affordable, and much simpler…
The research required is multi-disciplinary:
Technology oriented, human oriented, economic oriented and particular domain oriented
49
Family planning andcontraceptives based onwristband that measures andsystems that accumulatespersonal data
OUTLINE
Background: events, autonomous systems, Body
Area Networks
Medical oriented applications of event
processing within the human body
Other applications of event processing within the
human body
The future perspective
50
Topic I
TOPIC II
Topic III
Topic IV
51
Implant in the brain retrieves electric signals.
Brain computer interface
Source: Ed Grabaianowski: How Brain-Computer Interfaces Works: http://computer.howstuffworks.com/brain-computer-interface.htm
52
Brain computer interface
Source: Ed Grabaianowski: How Brain-Computer Interfaces Works: http://computer.howstuffworks.com/brain-computer-interface.htm
53
Microchipimplanted in the body to identify persons.
Identification implants
Applications for security, open doors, payment (activating credit cards) presence and more.
54
Next generation of glasses that connect to the Internet, identify persons, interface for autonomic assitant
Seeing Virtual
55
Automated personal assistant
Sensors that determine the context are linked to active advisors. They understand your context and even listen to your conversations and give you suggestions of what to say (e.g. through google glass).
2018
56
Chemical plants safety
Monitoring people movement and compliance with safety regulations, leak of chemical materials…
57
Summary: Detect-Derive-Decide-Do in the human body level
Detect –
Wearables + implants
Derive -knowledge + learning + personalization
Decide – AI + decision science
Do –Actuators: autonomic and manual
OUTLINE
Background: events, autonomous systems, Body
Area Networks
Medical oriented applications of event
processing within the human body
Other applications of event processing within the
human body
The future perspective
58
Topic I
TOPIC II
Topic III
Topic IV
59
The vision: Human Sapiens 2.0 –next phase of the evolution
60
Reprogramming the body to eliminate aging.
Eternal Life?
61
Kurzweil’s time
Stamp for singularity.
Eternal Life?
62
63
Source: S.A. Dash – Mind Uploading
https://www.slideshare.net/prabin210/mind-uploading
64
Lung on a chip
65
Arthery on a chip
66
Human bodyWith multiple organson a chip
67
Microsoft Biological Computation: Goal to program the body in order to eliminate cancer
By treating cancer like an information processing system, Microsoft researchers are able to adapt tools typically used to model computational processes to model biological ones.
Ultimately, the company hopes to create molecular computers to program the body to fight cancer cells immediately after detection
Andrew PhillipsSource: Christina Farr, Mind and Machine, September 2016, https://www.fastcompany.com/3063835/microsofts-biological-computing-lab-wants-to-fight-diseases-by-reprogrammin
68
Biological computers destroy cancer cells
69
Hacking the DNA – programming the cells
CELLS ARE BASICALLY tiny computers: They send and receive inputs and output accordingly. If you chug a Frappuccino, your blood sugar spikes, and your pancreatic cells get the message. Output: more insulin.
A cell could be programmed, for example, with a so-called NOT logic gate. This is one of the simplest logic instructions: Do NOT do something whenever you receive the trigger…. Biologist Wilson Wong of Boston University, who led the research, refers to these engineered cells as “genetic circuits.”
Applications: diagnostics, disease fighting, in the future: hacking the aging mechanism.
Source: Sophia Chen, Wired, March 2017
https://www.wired.com/2017/03/biologists-made-logic-gates-dna/
70
Biological molecules are used for storing information and transmitting it to other cells
“For example, suppose we could partner with microbes and plants to record events, natural or otherwise, and convert this information into easily observed signals. That would greatly expand our ability to monitor the environment,” Drew Andy, Stanford University
71
Privacy considerationsThe traditional Internet and social networks are already compromising privacy in the virtual world
Wearable devices and implants expose a lot of information about humans. There are already debates about biometric stores.
72
Security considerations
Murder by the Internet
“With so many devices being Internet connected, it makes murdering people remotely relatively simple, at least from a technical perspective. That’s horrifying,” said IID president and CTO Rod Rasmussen. “Killings can be carried out with a significantly lower chance of getting caught, much less convicted, and if human history shows us anything, if you can find a new way to kill, it will be eventually be used.”
EXAMPLES: Turn off pacemakers, Shutdown car systems while driving, stop IV drip from functioning
73
Dangers and challenges
Confusing a sensor
Changing the rules of the game
Abusing an actuator
74
Confusing a sensor
The same as confusing the human eyes. See things that don’t exist, don’t see things that exist, distort picture…
75
Confusing a sensor
Can be used to sabotage, to commit crimes, frauds and more.
76
Confusing a sensor
Example from another domain: the Twitter hoax
77
Changing the rules of the game
The logic is available. The ease of modification can be abused to add/delete/modify rules, change thresholds…
78
Changing the rules of the game
Changing data relevant for the system: maps, pictures, person’s data…
79
Abusing actuators
Deviating from course, shutting down, activating in wrong mode…
80
The Solaria effect
||machines|| >> ||men||
81
The Driverless Car Dilemma as an example
to theautonomous computing dilemmas.
82
The Deep Learning Dilemma
Autonomous systems employ deep learning techniques for self-improvement. The designer loses control over the behavior.
83
Who will move to the next phase of evolution?
84
Social Implications
Spreading the progress or keeping it at the hands of small fraction of the
population
85
Social Implications
Question: What are the implications of long life?
86
Social Implications
Question: What are the implications of Homo Sapiens 2.0 with embedded
computers in the brain and body?
87
Processing human related events is one of the enabling and essential
technologies to the future, and mightbe the area where event processing eventually will have the most impact.