Embed Size (px)
Citation preview
1
EnviroInfo Conference 2017
Disaster Management for Resilience
and Public Safety Workshop
Disaster Monitoring using UAV and Deep Learning
Andreas Kamilaris
13th September, 2017
Luxembourg
Problem
2
Monitoring and identification of disasters are crucial
for mitigating their effects on the environment and
on human population.
Motivation
3
Disaster monitoring can be facilitated by the use of
unmanned aerial vehicles (UAV), equipped with
camera sensors which can produce frequent aerial
photos of the areas of interest.
Motivation
4
Advantages of Drones:
• Small size
• Low cost of operation
• Exposure to dangerous environments
• High probability of mission success
• No risk of loss of aircrew resource
• High-resolution image sensing
• High operational flexibility
Motivation
5
Modern computer vision techniques:
• Artificial Neural Networks
• Scalable Vector Machines
• Multi-layer Perceptrons
• Random Forests
• Gaussian Mixture Models
• K-Nearest Neighbors
• Unsupervised feature learning
• Feature extraction techniques: Color, shape, texture
• Deep learning
Machine Learning-
based Approaches
Probabilistic
Modelling
Motivation
6
Advantages of Deep learning:
• Superior performance in terms of precision
• Perform classification and predictions particularly
well due to their structure.
• Flexible and adaptable
• No need for hand-engineered features
• Generalizes well
• Robust in low-resolution and -quality images.
Andreas Kamilaris and Francesc X. Prenafeta-Boldú, Deep Learning in Agriculture: A
Survey, Computers and Electronics in Agriculture Journal, 2017. [Under review]
Research Questions
7
Can drones and aerial image sensing be used for
real-time monitoring of physical areas and?
accurate identification of disasters?
Can deep learning be used in combination with
drones and aerial images for real-time disaster
monitoring/identification?
Deep Learning
8
Convolutional Neural Networks
Deep Learning
9
Convolutional Neural Networks
• Can be applied to any form of data, such as audio,
video, images, speech, and natural language.
• Various “successful” popular architectures: AlexNet,
VGG, GoogleNet, Inception-ResNet etc.
• Pre-trained weights
• Common datasets for pre-training CNN architectures
include ImageNet and PASCAL VOC.
• Many tools and platforms that allow researchers to
experiment with deep learning e.g. Keras, Theano.
General Idea
10Disaster!Nothing to
worry about!
State of the Art
11
No. Disaster Image source Accuracy
1.Fire (Kim, Lee, Park, Lee, &
Lee, 2016)Aerial photos
Human-like
judgement
2.Avalanche (Bejiga, Zeggada,
Nouffidj, & Melgani, 2017)Aerial photos 72-97%
3.Car accidents and fire (Kang
& Choo, 2016)CCTV cameras 96-99%
4. Landslides (Liu & Wu, 2016)Optical remote
sensing96%
5.
Landslides and flood (Amit,
Shiraishi, Inoshita, & Aoki,
2016)
Optical remote
sensing80-90%
Methodology
12
CNN Model: VGG architecture, pre-trained with the
ImageNet dataset of images.
Dataset: 544 aerial photos from Google images (min.
256x256 pixels), acquired using the query:
[Disaster]: earthquake, hurricanes, flood and fire.
[Landscape]: aerial views of cities, villages, forests and
rivers
[Disaster | Landscape] + "aerial view" + "drone"
Dataset
13
No. Image GroupNo. of
Images
Relevant Possible
Disaster
1. Buildings collapsed 101Earthquakes and
hurricanes
2. Flames or smoke 111 Fire
3. Flood 125
Earthquakes,
hurricanes and
tsunami
4. Forests and rivers 104 No Disaster
5. Cities and urban landscapes 103 No Disaster
Dataset: Disasters
14
Buildings collapsed
Flames or smoke
Flood
Dataset: Landscapes
15
Forests and rivers
Cities and urban landscapes
Setup
16
• 82% (444 images) of our dataset as training data
and 18% (100 images) as testing data.
• Random assignment of images in training/testing.
• Training procedure 20 minutes on a Linux
machine, testing 5 minutes for the 100 images.
• Learning rate: 0.001
• Used data augmentation techniques.
• 30 epochs
Results: Training Vs. Testing
17
83
84
85
86
87
88
89
90
91
92
82-18 70-30 75-25 85-15 90-10
Training Vs. Testing Percentage
Overa
ll P
recis
ion (
%)
Results: Training Vs. Precision
18
0
10
20
30
40
50
60
70
80
90
100
5 10 15 20 25 30 35
Ove
rall
Pre
cis
ion
(%
)
Number of Epochs
Results: Confusion Matrix
19
91% Precision
9% Error
Results: Analysis of Error
20
9% Error
Urban Vs. Buildings collapsed (4%) Urban Vs. Fire (2%)
Urban Vs. Flooding (1%)Flooding Vs. Buildings collapsed (2%)
Conclusion
21
Deep learning offers good precision and many benefits.
Can be successfully used in combination with UAV for
disaster monitoring/identification.
It has also some disadvantages:
• It takes (sometimes much) longer time to train.
• It requires the preparation and pre-labeling of a
dataset containing at least some hundreds of images.
Future Work
22
• Publish the dataset to the research community.
• Enhance the dataset with more images.
• Experiment with different architectures, platforms and
parameters.
• Increase overall precision to more than 95%.
• Perform a real-life case study with drones used for
monitoring some particular disaster e.g. indication of
fire.
Vision
23
Better disaster modelling,
especially when combining UAV
and deep learning with geo-
tagging of the events identified
and geospatial applications.
Facilitate the integration of relevant actors (i.e. action
forces/authorities, citizens/volunteers, other stakeholders)
in disaster management activities with regard to
communication, coordination and collaboration.
