A Framework of Intelligent Sensor Network with Video

Camera for Structural Health Monitoring of Bridges

Arslan Basharat*, Necati Catbas** and Mubarak Shah*

*Computer Vision Lab and **Civil Structures Lab,University of Central Florida

Orlando, FL

Outline

IntroductionStructural Health Monitoring (SHM)Framework Details

Network LayoutVibration SensingEvent Detection

Test-bed overview and demonstration VideoConclusion

Introduction

Wireless sensor networks have been used in various areasAdvances in Micro Electro-Mechanical Systems (MEMS) propose the use for civil structural health monitoring Higher data rate requirementsOur framework for this task augmented with video cameras

Structural Health Monitoring

Constantly monitor status of the structureDetect abnormal behaviorLocalize structural damageVarious SHM Systems

Visual Inspection by HumansTomography (Ultrasound, X-rays etc.)Vibration & Strain Monitoring...

Structural Health Monitoring

Essential characteristics Large number of sensorsSensor types

Vibration Tilt Strain

Data acquisition system for data recordingCentralized data interpretation

Structural Health Monitoring

Drawback of conventional wired systemExpensiveHuge mesh of cablesCentralized processing onlyLarger response timeLess Fault-Tolerant

Structural Health Monitoring

Advantages of using a wireless systemInexpensive Wireless data communicationDistributed processing possible

Improved response timeNear real-time performance

More fault-tolerant systemRedundancyModularity

Scalability

SHM of Bridges

Need of visual surveillanceTraffic monitoring Activity detection

Autonomous correlation betweenVideoOther sensor data (e.g. vibration, strain)

Intelligent sensing and actuation

SHM of BridgeAn example of wired SHM system

Drexel University / Commodore Barry Bridge

Proposed Framework

Wireless sensor nodesCentral station

Data collectionVideo cameras controller

Web-Server

Mote

Wireless Internet

Sensor Network

Mica motesMTS series sensor boardsSensors used

AccelerometerTemperature

Stationary video cameraPan/tilt/zoom featureControlled through central station

Network Architecture

Clusters based networkCluster head

Gateway node

Cluster member2-5 Sensing nodeBackup nodes (Gateway, Sensing)

Multi-hop to base station through cluster heads

Sensing node

Sensing node

Sensing node

Gateway node

Backup node

Backup node

Node Layout on Bridge

Sensing Node Backup Node Central Base Station

Data pathClusterGateway Node

Neighbor Discovery

Central station to gateway nodeGateway node to gateway node progressive listGateway node to sensing node progressive list

Sensing node

Sensing node

Gateway node

Sensing node

Sensing node

Gateway node

Sensing node

Sensing node

Gateway node

Vibration Sensing

Vibration data from Accelerometer2-axis of vibrationInduced vibration through impact hammerResponse from lab model of steel bridgeSensor with Mica2 mote

Vibration Sensing

Sampling frequency requirements100Hz-200HzHigher data rateSmaller network battery lifee.g. 6 nodes @ 150Hz x 2 axis~ 216 KB/min

Solution: Adaptive Sampling

Adaptive SamplingExploit silent zonesTransmit only useful dataModes of Sensing Nodes

Sleep modePassive sensing

Low sampling frequency ~80HzOnly two sampling nodes/cluster

Active sensingHigher sampling frequency ~ 150HzLimited time period (45 secs)Stored in Flash memoryTransferred through cluster head in allocated time slice

Adaptive Sampling

1 passive/rest sleepHigh vibration event detectedAll activeActive for limited timeRound-robin passive sensingHow to detect events?

Sensing Node

(Sleep)

Sensing Node

(Sleep)

Sensing Node

(Passive)

Gateway Node

Sensing Node

(Sleep)

Sensing Node

(Passive)

Sensing Node

(Sleep)

Gateway Node

Sensing Node

(Active)

Sensing Node

(Active)

Sensing Node

(Active)

Gateway Node

Event Detection

Vibration EventsActivity metric on 1-D vibration dataA good solution

Thresholded Mean Shift Vector

Mean Shift Vector

Always points towards the direction of maximum increase in densityOnline model: Can be applied in an iterative fashion

where,nx is the number of pointsyo is the last mean valuexi is ith sample

Interesting Events

SENSE_EVENTSuppression of unnecessary dataLower mean shift threshold TL

Change sensing mode (Passive/Active)CAMERA_EVENT

Critical event notification for visual inspectionHigher mean shift threshold TH

Trigger video camera request in cluster

Event Detection

SENSE_EVENTMean shift threshold TL exceededTrue positive

CAMERA_EVENT detected

Mean shift threshold TH exceededTrue positive

Camera Events

Gateway Nodes

Camera Event

Stationary wireless sensor nodes Controlled gateway node deploymentStationary pan/tilt/zoom video camera Central station attached to both video camera and WSNRegistered gateway nodes with approximate 1-D distance

Node Registration

Utilizing the gateway node-gateway node progressive listRegistering approx. gateway node positionsFOV wide enough to cover the cluster

Sensing node

Sensing node

Gateway node

Sensing node

Sensing node

Gateway node

Sensing node

Sensing node

Gateway node

Optimizations

Single time-stamp and current sampling frequency transmitted for the whole data packet Mean-shift vector test after 45secs of active samplingData packing for transmission saves ~38% of unused buffer

Test-bed Setup

Test-bed Setup

Demo Video

Overview of the test-bedVibration induced by impact hammerEvent detection by sensing nodes Camera motion

Conclusion

Wireless sensor network have a promising application in the area of SHMDistributed processing enhances the system effectivenessDomain specific knowledge for data compressionTested in controlled lab environment, effectiveness of the system remains to be seen in real-life situation