Distributed Pattern Recognition and Classification in

Introduction Fully Distributed Schemes Decentralised Schemes Current Work Conclusion Acknowledgements

Distributed Pattern Recognition andClassification in Wireless Sensor Networks

Alexander SeniorSupervisors: Y. Ahmet Sekercioglu Asad Khan

Department of Electrical and Computer Systems EngineeringFaculty of Engineering

Monash University

May 28, 2012

Alexander Senior ESCE Monash University

Distributed Pattern Recognition and Classification in Wireless Sensor Networks


Introduction

WSNs:small amount of memory‘weak’ processorsenergy poor – communication is THE biggest drain on battery

Want to focus on distributed schemes — computation andstorage are shared throughout the networkDistinction between the degrees of ‘distributed-ness’:

Fully distributed: absolutely no reliance on a base station (e.g.computer) or a root node that has authority over the entirenetworkDecentralised: requires a base station or network leader,though work is shared throughout the network

Focus on schemes that are (at least) adapted for WSNs, orhave had real-world evaluation




1 Introduction

2 Fully Distributed SchemesKohonen Self-Organising MapSupport Vector Machines

3 Decentralised SchemesGraph NeuronAdaptive Resonance TheoryOther decentralised schemes

4 Current Work

5 Conclusion

6 Acknowledgements




Kohonen Self-Organising Map

Kohonen Self-Organising Map (SOM)

Winner-takes all neural network — units/neurons compete towin the data

Each unit contains a prototype vector which is initiallyrandom, evolves as data is input into the map

Units win data that their prototype vectors are closest to

As units win data, all units update their prototype vectoraccording to:

current datadistance between their vector and winning vector — amount isaffected by neighbourhood functionlearning rate




Kohonen Self-Organising Map

Self-Organisation in Ad Hoc Networks: An Empirical StudyCatterall, van Laerhovena and Strohbach, 2003

Implemented SOM on Smart-It platform

One unit assigned to each nodeNodes broadcast their sensor readings to each other so thatwinning unit can be foundNodes can be added/removed in an ad-hoc fashion

Collected data on Smart-Its, and executed algorithm onsimilar hardware

Totally distributed scheme, no leader necessary

Method of choosing winner not discussed; broadcasts will notbe feasible in large networks; no evaluation of effectiveness




Support Vector Machines


Primarily used for classifying data points as either ‘positive’ or‘negative’

Linear classifiers operate on data directly to produce theirclassification

Non-linear classifiers employ the ‘kernel trick’ — data istransformed into higher-dimensional space, and the performtransformation in this space

Transformation can be found using optimisation techniqueswith training data





Support Vector Machine for Distributed Classification: aDynamic Consensus ApproachWang, Li and Zhou, 2009

Altered SVM algorithm for totally distributed evaluation

Nodes run their own SVM on their local data, and exchangedata locally to improve their estimate of the classification

Simulation verification with 36 nodes

Performance nearly as good as centralised method

Needs training

No mention of difficult of evaluating algorithm on nodes




Graph Neuron

Graph Neuron (GN)

Take a pattern and separate it into (position, value) pairs, e.g.’WSN’={(1,W ), (2,S), (3,N)}Every possible (position,value) pair is represented by a neuron

Neurons only activate if their pair is present in the pattern

Also have Stimulator and Interpreter (S&I)




Graph Neuron

GN continued

Active neurons broadcast their value toadjacent pairs — other active neurons usethese broadcasts to form their bias array:map from active neighbours (on left andright) to an index

Neurons determine if they haveencountered (sub-)patterns before byconsulting their bias array

S&I stores indices emitted by neurons

Memorising sub-patterns enables compactstorage




Graph Neuron

X(1)

O(2)

X(3)

X(1)

O(2)

O(3)

P1 P2

O(4)X(4)

X(2)

X(3)

X(4)

X(1)

O(2)

O(3)

O(4)

O(1)Port sequence:6 (bias RED)6 (bias BLUE)

2 (bias GREEN)

Port sequence:6,4 (bias RED)

2,8 (bias BLACK)

N1

N2

N3

N4

N5

N6

N7

N8

Port sequence:3 (bias RED)

Port sequence:1,3 (bias RED)1,7 (bias BLUE)

Port sequence:6,8 (bias BLUE)

2,8 (bias GREEN)

Port sequence:7 (bias BLUE)

3 (bias BLACK)7 (bias GREEN)

P1, P2

P1,P2

P1

P1

P2

P2

O(1)

X(2)

X(3)

P3

O(4)

Port sequence:2 (bias BLACK)

Port sequence:5,3 (bias BLACK)1,7 (bias GREEN)

X(1)

X(2)

O(3)

O(4)

P4




Graph Neuron

Hierarchical Graph Neuron

Problem with base GNalgorithm: crosstalk

Compare stored patterns‘abcdf’ and ‘fbcde’ with newpattern ‘abcde’: havecommon sub-patterns, falserecall

Use a hierarchy of arrays tosolve this problem:Hierarchical Graph Neuron(HGN)




Graph Neuron

HGN continued

Indices of lower arrays are fed asinput to higher arrays: upper levelbias arrays contain bias indices ofleft and right neurons and index ofnode directly below it — this solvescrosstalk problem

Hierarchy allows S&I to match withnoisy patterns: if a match cannotbe obtained from top-most level,can get consensus from lower levels

Have many more neurons




Graph Neuron

Cellular Microscopic Pattern Recogniser

Variant of GN

Neurons are arrangedlogically in a series ofconcentric circles (tracks)

Neurons function similarly asbefore, but also report toneurons in inner track

Inner neurons have moreauthority over network, aswith higher layers in HGN




Graph Neuron

Comparisons, advantages and disadvantages of GN

No complex computations needed — simple integer arithmetic

Storage needed proportional to unique sub-patternsencountered, not patterns

Simulations have shown good performance against otherrecognisers/classifiers such as SOM, SVM

Not fully distributed — need S&I to function

Will need to be adapted to cope with complex and changingtopologies of WSNs

No implementation in physical WSNs




Adaptive Resonance Theory

Adaptive Resonance Theory (ART)

Neural network like SOM, however:

have long and short term memoryadditional pattern classificationscan be learned by the systemwithout user intervention

Typical ART neural networkcomposed of input layer (L0),comparison layer (L1) and categorylayer (L2); also have a sensitivity(vigilance) threshold

Taken from Kulakov and Davcev, “IntelligentWireless Sensor Networks Using FuzzyARTNeural-Networks”, 2007.





ART continued

Category nodes contain prototypevectors — pattern ‘captured’ by nodesthat have the highest bottom-upactivation, and also have a highenough similarity

Prototype vector of activated categoryneuron updated, amount set bylearning rate

If no category neuron activates, inputforms a new category node

ART1 classifies binary data,FuzzyART classifies analog data

Taken from Kulakov and Davcev,“Intelligent Wireless Sensor NetworksUsing FuzzyART Neural-Networks”,2007.





Intelligent Wireless Sensor Networks Using FuzzyARTNeural-NetworksKulakov and Davcev, 2007

Used ‘clustered approach’:

cluster members all run a FuzzyART neural network, produceinteger category labelscluster leader runs an ART1 neural network to classify labels

Ran in small network with MicaZ motes

Sensitivity threshold is dynamically monitored so node’smemory is not overwhelmed

No evaluation of effectiveness of classification





Intruder Detection using a wireless sensor network with anintelligent mobile robot responseLi and Parker, 2008

Builds on work of Kulakov and Davcev, focus is on intruderdetection

Adds a Markov model after ART1 network in cluster leader todetermine if state transitions are abnormal

Scheme adapted to cope with missing data

Good improvement over previous scheme





Building Intrusion Detection with a Wireless SensorNetworkWalchli and Braun, 2010

Also uses FuzzyART — detection of abnormal activity inoffice environment

For cluster members, communication scheme is even simplerthan before: if a new category node has to be created, ‘1’ issent to cluster leader, otherwise ‘0’ implicitly assumed

Real-world implementation and testing with Embedded SensorBoards (MSP430 microcontroller)

Performed better than threshold technique, but no othercomparison made




Other decentralised schemes

Prototype modelling

Similar to neural network schemes in that data is classified viaprototype vectors

Prototype vectors formed by manipulating training data

Data is classified by determining which prototype vector it isclosest to




Other decentralised schemes

A System for Distributed Event Detection in WirelessSensor NetworksWittenburg et al., 2010

Focus is event detection - wireless alarm system forconstruction site

Real-life testing with Scatterweb nodes

Training data collected in network, then full readings sent tobase station

Prototype vector consider readings from several adjacentnodes — nodes broadcast to neighbours (GN)




Current Work

What if we had nodes with very limited abilities but in largeamounts and with large numbers of connections available?

Work is VERY preliminary




Conclusion

Distributed approaches:

SOM lacks validation and ability to scaleSVM is promising but requires testing in real-world networks

Decentralised approaches:

GN: light-weight, but needs implementation in real-worldnetworksART: most promising candidate yet, and have already hadimplementations; might have scaling issuesPrototype modelling: requires off-line training at base station,but have implementation




Acknowledgements

My thanks go to:

Supervisors Ahmet Sekercioglu and Asad Khan

STINT program

Sven Molin and Kim Ng

Fredrik Sandin and Blerim Emruli



Documents

Distributed Pattern Recognition and Classification in