National Biocomputation CenterStanford UniversityNational Biocomputation CenterStanford University

TATRC

US Army Medical Research and

Materiel Command

TATRC

US Army Medical Research and

Materiel Command

Global Integrated Monitoring

Kevin Montgomery, Ph.D.

IntegrationIntegration

04/19/2304/19/23 2

Global Integrated MonitoringGlobal Integrated Monitoring

AnalysisAnalysis

InternetInternetLaptopsLaptops

DesktopsDesktops

In-Situ SensorsIn-Situ Sensors

EnvironmentalEnvironmental

ImageryImagery

TrackingTracking

Medical DataMedical Data

Public HealthPublic Health

Medical Medical DatabasesDatabases

Human InfoHuman Info

Open SourceOpen Source

Field ReportsField Reports

IntelligenceIntelligence

Remote SensingRemote Sensing

Satellite ImagerySatellite Imagery

Aerial ImageryAerial Imagery

GIS DataGIS Data

In-FieldIn-FieldMobile DevicesMobile Devices

PDAsPDAs

DisplayDisplay

04/19/2304/19/23 3

Timeline of an event Timeline of an event (env, bio, disaster)(env, bio, disaster)

• Are conditions right for an event?

• Environmental sensing, prediction and cueing

• Event detection• Triggering, Alerts, Surveillance

• Suspected agent confirmation• Clinical confirmation- in-field, hospital monitoring

• Ground-truth- is it confirmed?• Field reports- worldwide network of personnel on the ground

• Resourcing- what resources can we get to it?• Containment, management of initial response

• Management- how do we deal with it?• Ongoing management

Op

en-

sou

rce

me

dia

/inte

llig

enc

e

Company HistoryCompany History Spun out of Stanford/NASA and UH EPSCoR program in early 2005 to

enable worldwide integrated monitoring of the environment and its inhabitants

Collaborate/play well with others: government, academia, industry

Created a global network of wireless sensors (the InteleNet) that are integrated with many other data sources to enhance understanding of their interrelationships

Currently deployed in multiple sites in Hawaii (Lehua,Kauai, Oahu, Maui, Big Island), continental US (California, Texas, Delaware), Asia (Vietnam, Thailand), Africa (Ethiopia), and Middle East (Iraq)◦ Planning for future deployments in other areas of the Pacific

(Palau, Palmyra, Okinawa), Africa, Asia, and other areas

Corporate office in Honolulu, Hawaii; ◦ Research and Development offices in Silicon Valley◦ Field offices with collaborative partners in each deployment zone:

Hawaii (Waipa, UH), Vietnam (VAST,HSPH), Africa (Ethiopia, Senegal)04/19/2304/19/23 4

04/19/2304/19/23 5

Technology

Server: Integration & Analysis Integration of disparate data sources Algorithms to analyze data Input for future modeling and sim

In-Situ Sensors Multiparameter sensors Remote sensor network Local intelligence

Display & Dissemination GIS-based display over web Custom, secure online views Collaboration tools

External Data Sources

InteleCell / InteleNetInteleCell / InteleNet

Intelesense ServerIntelesense ServerPortals/InteleViewPortals/InteleView

InteleCellInteleCell™™ - Wireless Sensor Device - Wireless Sensor Device

Remotely deployable, rugged smart sensors monitor many locations:■ Sensors: Water, air, weather, soil, video, biosensors, PDAs■ Processor: Small computer, controls devices/actuators■ Wireless: Up to 14mi/20km between stations (40mi/60km dir)

Sensor network can cover hundreds of miles/km■ GPS-Enabled: automatic localization, sensors can be mobile■ Uplink: Local Internet, Cellular, Satellite (deployable anywhere)

■ Network aggregates data to decrease uplink costs- particularly important for satellite

■ Any sensor (analog, digital, serial, …) easy to integrate■ Many labs developing new sensors- we make it easy to deploy/

integrate these devices and get out of the lab and into the field

■ Easy to use: Sensor autodetect, Self-configuring■ Secure transmission: Authentication + Encryption (256-bit AES)■ Future-proof: Remotely upgradeable while deployed in the field■ Frequency: Transmit on-schedule, on-event, on-demand■ Advanced power management:

■ Self-Powered (10+ years deployment with solar recharge)

04/19/2304/19/23 6

04/19/2304/19/23 7

Category Sensors Supported

Water Quality YSI 6000 Series Multi-Sensor Probe (pH, ORP, DO, Temperature, Conductivity, Turbidity), WL 400 Water Level Sensor, EchoTel Stage and Flow Sensor, EnviroTech MicroLab Multi-Sensor

Air Quality Thermo Electron Genesis Multi-Gas Detection Monitor (Cl2, CO, HCN, H2S, O2, NH3, NO, NO2, PH3, SO2), CO2 (K30, Vaisala)

Weather Hobo weather sensors, Vaisala, Davis Instruments

Soil Moisture Hobo Soil Moisture Sensor, EC-5, Vegetronix

Leaf Wetness Hobo Leaf Wetness Sensor

Biotelemetry Biotelemetry Collars, InteleRFID, InteleTracker

Images InteleCam

Motes InteleMote (soil moisture, temp/humidity, …), Crossbow Motes

Video WebCams (various models supported), InteleVideo (rugged, portable videoconferencing system

Misc PDAs/PCs (field reports)

Supported SensorsSupported Sensors

Sensor StationsSensor Stations

04/19/2304/19/23 8

Water sensor station Remote image sensor

GPS-enabled manual field sampling device

Ridge repeater station

Remote weather station

Remote field data collection

InteleNetInteleNet™™ Overview Overview

InteleCell devices form intelligent distributed mesh network◦ Self-configuring, self-organizing, self-repairing◦ Robust, reliable, no configuration necessary◦ Covers hundreds of miles / kilometers

Individual InteleCells acquire and transmit own data and also route data for other InteleCells◦ Novel Propagating Wave Algorithm autoconfigures each

time◦ Simple, robust, extensible, redundantly indestructible04/19/2304/19/23 9

04/19/2304/19/23 10

InteleMote: InteleMote: low-cost local saturation low-cost local saturation sensingsensing

Simple sensor types:◦ ID, switch, soil moisture, temp/relative humidity,…

Short-range:◦ Spec: 1mi/1.6km (new: 6 mi/10km, meshed,

encrypted)

◦ Tested: 2km+ (BORR), 4km+ (Garcia)

Primary power: 2+ yrs, no solar needed Transmits at regular intervals (hourly)

Received by multiple Intelecells◦ Provides redundancy of reception

Low-cost: $350 Applications:

◦ High-density, low-cost saturation sensing◦ Proximity detection, security, trapping

04/19/2304/19/23 11

Applications: Applications: 10 Things You Can 10 Things You Can Do…Do… Monitor rainforest, including in dark/under canopy- carbon

monitoring Monitor the water quality of your stream (InteleCells with

YSI sonde) Setup a flood warning system (InteleCells with stage

sensors and alert trigger) Monitor the weather in your area (weather stations with

InteleCells) Monitor the environment on a remote island (satellite link) Take periodic images over long periods of time of the

same site (InteleCam) Setup a security system (motion-triggered camera) Track animals (collars) Monitor animal traps (Intelemote) Monitor nutrients in a stream/lake/ocean (MicroLAB)

04/19/2304/19/23 12

Applications: Applications: 10 More Things You Can 10 More Things You Can Do…Do… Monitor air quality in any environment, alert if hazardous

to health Track cars, people Record waypoints while hiking/driving Take manual sensor readings, GPS-tagged Map RF propagation Monitor the soil moisture, humidity, and temperature in

your greenhouse Monitor the weather on a vineyard and alert for various

conditions Record and transmit vital signs Setup communications infrastructure in remote places Monitor temperature in your lab and get an alert on your

cell phone if AC fails

ImplicationImplication

We can reliably and securely get information from sensors and

people anywhere in the world in real time with no required

infrastructure

04/19/2304/19/23 13

Worldwide wireless distributed mesh network using Internet as backbone

Integration and Analysis

04/19/2304/19/23 14

■ Examines data across all sensors to assimilate, compare to baseline, integrate many other data sources, and generate alerts

■ Not going to have people watching individual screens of raw data■ Right data, right time to right people in usable, actionable form■ Eliminate real-world sensor variability, background noise

■ Integrates data from other sources automatically:■ Methods: HTTP, FTP, SQL, …■ Intelligent Agents fire at regular intervals- obtain info, analyze, alert

■ Internal XML-based parsing engines■ Satellite imagery: weather, standing water (IR), vegetation, land use■ Internet-accessible Data:

■ Public Health/Medical (syndromic surveillance data, clinical lab info, admissions, pharmaceutical sales), etc■ Weather, Human intelligence, Media (Argus), Absenteeism, etc

■ Easy integration with existing systems and/or other systems■ Modeling/Metaknowledge: hydrology models, …■ Grid/cloud-based Network-centric/SOA architecture

While individual data sources are non-specific and unreliable, integration across many sources yields robust knowledge

Integrated Data SourcesIntegrated Data SourcesIntegrated Data Sources- 4/1/07

Category Data Integrated

InteleNet Real-time weather, water, video, biotelemetry, tracking, medical information, GPS localization

Surface Imagery

Landsat visible & vegetation, USGS maps and urban area Ikonos, Quickbird, daily worldwide MODIS, high-res Landsat, Ikonos

Atmospheric NOAA CO2 sensor network, IR/suspended water vapor, heat flux

Weather Real-time cloud cover, worldwide meteorological station data

Ocean NOAA Ocean Depth data OGS WMS, NOAA buoy data, chemical composition & temp, reefs, ice field maps, sea level, shipping routes

Human Placenames, political boundaries, military bases, population density, languages, aviation routes/bases/beacons, demographics, Shoreland

Disease Argus media reports, ProMed, syndromic surveillance

Seismic USGS real-time Earthquakes, Seismic Hazard zones, volcanoes

Solar Eclipse predicted path, solar irradiance

Disaster Quickbird imagery, Tsunami imagery, NOAA hurricane imagery04/19/2304/19/23 15

Integrated Data SourcesIntegrated Data Sources

Now available and fully operationalFeatures:

◦Massive amount of data available in one place: 315,000+ layers of data now, increasing rapidly User-data upload (KML, ESRI SHP files, imagery)

You own your data: secure to person, group, world

Ability to tap Google KML/KMZ, WMS, imagery Directed and archival satellite imagery- anywhere on planet

◦Updatable, secure, high-performance

04/19/2304/19/23 16

Easy to add new data

All the world’s data at your fingertips

ImplicationImplication

We can integrate data from many sources,

establish baselines (nominal/anomaly), generate alerts,

and provide this information anywhere in the world

in real-time

04/19/2304/19/23 17

PortalsPortals

Having all the world’s data is impressive, but only one part of the need:◦ Google Earth, MS Virtual Earth, NASA WW, so what?◦ Must provide exactly and only what people need◦ Must provide in a domain-specific way that addresses a need◦ Must enable distributed community of interest to collaborate

Therefore, we created domain-specific, community-of-interest portals:◦ Enables groups of similar interests to come together in virtual

location◦ Provides advanced features: web-browser-based visualization, user

data upload/download (GIS data, imagery, etc), user forums/blogs, even videoconferencing/VOIP, Web form-based data input, Excel/CSV/GPX upload

◦ Allows customization and administration by the groups- which layers, functionality, features, content, users, etc

◦ Empowers groups to collaborate

04/19/2304/19/23 18

Portals: examplesPortals: examples

04/19/2304/19/23 19

InteleViewInteleView™™ – – 3D Worldwide 3D Worldwide VisualizerVisualizer

Fast, interactive 3D visualizer- Allows user to "zoom in" to anywhere on the planet and pull down many different types of high-resolution satellite imagery from servers located over the Internet, and "fly around" the terrain

Features:◦ 3D satellite maps: Landsat, Urban Ortho, anything- available

automatically◦ ArcGIS data: Shape-files fully supported- users can upload to server◦ Real-time display of sensor data, annotations and interactive

layer gen◦ Integrative: Data from thousands of sources, all Google data supported◦ Icon-based display of sensor sites, icons link to more detailed information◦ Dynamic Icons (real-time location tracking of people/equipment, status)◦ Advanced visualization features utilizing the 3D view of sensor

locations◦ Collaborative: Real-time collaboration built in: videoconf, annotation◦ Location-based search◦ Open Source - Built on NASA WorldWind

platform, active contributor◦ Cross Platform: PC, Mac, Linux

04/19/2304/19/23 20

Server FacilityServer Facility

Currently online and fully operationalCapabilities:

◦ Network: Up to 10 Gbps peak bandwidth Worldwide dedicated network Top 10 Internet systems in world

◦ Facility: Full battery backup with dedicated generator, 72+ hr offgrid capacity Secure facility, staffed 24x7, video surveillance, card keyed

◦ Servers: Multiple, redundant, load-balanced cluster of servers Dedicated data/imaging servers (16TB user data and imagery,

+48TB soon) High performance, scalable architecture- 1M users via Cloud

Computing

04/19/2304/19/23 21

ImplicationImplication

We can view, interact, and explore data from anywhere in the world

and collaborate with others to develop understanding and

manage an event

04/19/2304/19/23 22

Example DeploymentsExample DeploymentsHawaii Ecological Monitoring Project:

• Ken Kaneshiro/Mike Kido (CCRT)• Kauai (Limahuli, Waipa, Lawai), Oahu (Manoa), Maui (ML&P)

Northern California Deployments:• Blue Oak Ranch Reserve (BORR)• Garcia Forest Reserve

Vietnam Waterborne Illness Project:• Hanoi (Tien Hai, Westlake)• Environmental sensors integrated with public health data• Study water-borne illness, track H5N1, dengue, HIV cases• Partners:

• VAST, HSPH, US DHHS, State Dept Health Attaché• University of Hawaii: Drs Burgess, Wilcox, Gubler• US Embassy: Office of the Ambassador

Ethiopia Clinical Monitoring Project:• PEPFAR/USAID/CDC/TATRC project

Others: DOD, DTRA project work

04/19/2304/19/23 23

Hawaii Environmental Hawaii Environmental MonitoringMonitoringLehua: islet off coastKauai: Limahuli/Waipa, Lawai, MakauahiMaui: Maui Land and PineappleOahu: Manoa, Makaha, Mt KaalaHawaii: Hilo over Saddle Rd

04/19/2304/19/23 24

Kauai DeploymentKauai Deployment

04/19/2304/19/23 25

Integrated water, weather, other sensors with GPS localization over custom wireless voice/data network with secure uplink to Internet-based GIS website

Also supports vector tracking, manual sampling & collection, remote worker tracking

Integrates human knowledge as well: land use patterns, vegetation, manual sampling/surveys, elder/social info

04/19/2304/19/23 26

Other UH Affiliated Other UH Affiliated ProjectsProjects

CIMES –Center for Island, Maritime, and Extreme Env Security◦ DHS-funded Center for maritime situational awareness◦ Integrate sensor and systems data, analyze, visualize

Army – Mt Kaala◦ Real-time monitoring of invasive animal traps, security

video DOFAW – Makiki Valley

◦ Water quality and stream flow monitoring stations◦ GIS data hosting, integration, visualization

IGERT Infectious Disease Project- Thailand◦ JABSOM (Bruce Wilcox, Durrell Kapan, Ron Paik) project for

tracking infectious disease vectors, linked with env data

04/19/2304/19/23 27

CIMESCIMES

04/19/2304/19/23 28

Vietnam DeploymentVietnam DeploymentHanoi area: Tien Hai: (Thai Binh Province- H5N1 affected area)

◦ Environmental sensors (water quality, weather)◦ Integration with health data supplied by HSPH

Easy web-based syndromic surveillance data input Designed for daily use to replace current reporting system Usable for entire country if needed

Generalized syndromic surveillance system: General symptoms, cryptosporidium,

leptosporosis, H5N1, dengue, HIV Easily expandable to other countries/regions

Westlake:◦ Testbed deployment during development

04/19/2304/19/23 29

Tien Hai DeploymentTien Hai Deployment

04/19/2304/19/23 30

Sensors deployed in drinking water supply

Transmit wirelessly to district health office where uplinked via Internet and clinical

data are entered

To servers located at HSPH office in Hanoi for

web-based access

Thailand DeploymentThailand Deployment

IGERT Infectious Disease Project- Thailand◦ JABSOM (Bruce Wilcox, Durrell Kapan, Ron Paik) project

for tracking infectious disease vectors, linked with env data

04/19/2304/19/23 31

Ethiopia ProjectEthiopia ProjectCollaborative Partners:

TATRC, PEPFAR program (CDC, USAID, US Embassy), Defense National University, Jimma University, Govt of Ethiopia

Relevant Technologies: Information access: (patient info, inventory)

◦ BMIS-T: Patient records, blood info◦ Intelesense: Rugged, self-powered wireless networking◦ TATRC-wide: Easy to use, field-deployable clinical devices

04/19/2304/19/23 32

Mexico PortalMexico Portal

04/19/2304/19/23 33

Shoreland Haiti PortalShoreland Haiti Portal

04/19/2304/19/23 34

The Future- The Future- where we’re where we’re headedheaded

We have an amazing tool- let’s use it!Environmental: - preserve environment and culture

◦ Expand scope: NEON et al, international (PS, CCF)◦ Marine: NOAA, IOOS, PACMAN◦ Social information: OHA

Biodefense: - prevent disease and save lives◦ Expanding biosurveillance and travel med◦ Homeland Security: UH CIMES, DTRA◦ Medical Situational Awareness & Logistics◦ Civil Defense and disaster management◦ Remote medical care, awareness, and management

Social: - help people become more◦ Workforce development and social improvement

04/19/2304/19/23 35

SummarySummary• Technology:

• Easy to get data from anywhere in the world• Powerful data integration and visualization• Stable, reliable platform and service

• Company:• Hawaii-based company with worldwide impact on

environment, defense, and other areas

04/19/2304/19/23 36

Explore your worldwww.intelesense.net