Reactive Image-based Collision Avoidance System for - QUT ePrints

Queensland University of Technology

Reactive Image-based

Collision Avoidance

System for Unmanned

Aircraft Systems

Shane Degen B. Eng. (Hons) QUT

Australian Research Centre for Aerospace

Automation

Faculty of Built Environment & Engineering

This thesis is prepared as partial fulfilment of the

requirements for the Masters Degree.

May 2011

Reactive Image-based Collision Avoidance System for Unmanned Aircraft Systems

Shane Degen Page 2 of 129

This page is intentionally left blank.



I would like to dedicate the following thesis to Amanda, Lachlan, Elijah, Noah,

Gabriella and Moses.



Keywords

Collision Avoidance; Unmanned Aircaft Systems; Unmanned Aerial Vehicles;

Uninhabited Aerial Systems; UAS; UAV; Image-based Visual Servoing; Sense and

Avoid; See and Avoid; Sense and Act; Obstacle Avoidance; Collision Risk;

Guidance; Control; Gimballed Camera; Nonlinear Aircraft Control; Control and

Simulation; MATLAB; Monte Carlo Simulation; Equivalent Level of Safety; ELOS;

National Airspace System; Feature Based Manoeuvring; Position-based Avoidance;

Intruder Alert; Bearings-Only Tracking.



Abstract

Approximately 20 years have passed now since the NTSB issued its original

recommendation to expedite development, certification and production of low-cost

proximity warning and conflict detection systems for general aviation [1]. While

some systems are in place (TCAS [2]), ”see-and-avoid” remains the primary means

of separation between light aircrafts sharing the national airspace. The requirement

for a collision avoidance or sense-and-avoid capability onboard unmanned aircraft

has been identified by leading government, industry and regulatory bodies as one of

the most significant challenges facing the routine operation of unmanned aerial

systems (UAS) in the national airspace system (NAS) [3, 4].

In this thesis, we propose and develop a novel image-based collision avoidance

system to detect and avoid an upcoming conflict scenario (with an intruder) without

first estimating or filtering range. The proposed collision avoidance system (CAS)

uses relative bearing and angular-area subtended , estimated from an image, to

form a test statistic ASC . This test statistic is used in a thresholding technique to

decide if a conflict scenario is imminent. If deemed necessary, the system will

command the aircraft to perform a manoeuvre based on and constrained by the

CAS sensor field-of-view.

Through the use of a simulation environment where the UAS is mathematically

modelled and a flight controller developed, we show that using Monte Carlo

simulations a probability of a Mid Air Collision (MAC) MACRR or a Near Mid Air

Collision (NMAC) RiskRatio can be estimated. We also show the performance gain

this system has over a simplified version (bearings-only ). This performance gain

is demonstrated in the form of a standard operating characteristic curve.

Finally, it is shown that the proposed CAS performs at a level comparable to

current manned aviations equivalent level of safety (ELOS) expectations for Class E

airspace. In some cases, the CAS may be oversensitive in manoeuvring the owncraft



when not necessary, but this constitutes a more conservative and therefore safer,

flying procedures in most instances.



Table of Contents

KEYWORDS 4

ABSTRACT 5

TABLE OF CONTENTS 7

TABLE OF FIGURES 11

ACRONYMS 15

NOMENCLATURE 19

STATEMENT OF AUTHORSHIP 27

ACKNOWLEDGEMENTS 29

1 INTRODUCTION 31

1.1 Motivation 31

1.2 Collision Avoidance Problem 32

1.2.1 Definitions in the collision avoidance problem 32

1.2.2 Definitions 34

1.2.3 Types of sensors 35

1.3 Research Objectives 36

1.4 Significance 37

1.5 Research Contributions 38



1.6 Research Methodology 38

1.7 Publications 39

1.8 Content 40

2 BACKGROUND 41

2.1 Collision Avoidance Systems: Industry Developments 41

2.1.1 NASA ERAST 41

2.1.2 Auto ACAS 42

2.1.3 Northrop Grumman & AFRL 42

2.1.4 Discussion 44

2.2 Collision Avoidance Strategies 44

2.2.1 Optimal Strategies 44

2.2.2 Force Control Techniques 45

2.2.3 Geometric Approaches 47

2.2.4 Vision-based Obstacle Avoidance 48

2.3 Passive-only Collision Avoidance 49

2.3.1 Passive Ranging 51

2.4 Discussion 52

3 COLLISION AVOIDANCE 53

3.1 Collision Determination 53

3.1.1 Thresholding Technique for Collision Decision 54

3.1.2 Closest Point of Approach Distance 54

3.1.3 Time to Collision and Image Expansion 56

3.1.4 Collision Determination Algorithm 57

3.2 Avoidance Manoeuvre 59

3.2.1 Background 59

3.2.2 Relative-Bearing Based Manoeuvre 61



4 MODELLING & SIMULATION 63

4.1 UAS Model 63

4.1.1 Owncraft Coefficients 64

4.1.2 Atmospheric Model 65

4.1.3 Navigation Equations 66

4.2 UAS Controller 69

4.3 Camera Model 72

4.3.1 Configuration 72

4.3.2 Perspective Projection Model 73

4.4 Simulation Environment 77

4.4.1 The Vision System Emulator 77

4.4.2 The Conflict Scenario Emulator 77

4.4.3 The UAV Emulator 78

4.4.4 Simulator Adaptability 79

5 RESULTS AND ANALYSIS 81

5.1 Performance Analysis 81

5.1.1 Encounter Models 81

5.1.2 Performance Measures 82

5.2 Experiment Setup 88

5.2.1 Monte Carlo Simulations 88

5.2.2 Limitations and Assumptions 90

5.3 Results and Analysis 91

5.3.1 CAS Threshold Determination 91

5.3.2 Observations and Behavioural Patterns 92

5.3.3 Probabilistic Results 97

5.3.4 Performance Results 100

6 CONCLUSION 105



7 FUTURE RECOMMENDATIONS 107

8 APPENDICES 109

APPENDIX A 109

Data and aerodynamic coefficients of the Flamingo UAS [125] 109

Flamingo Limits 109

Inertial Data 110

Lift/Drag Data 110

Longitudinal Coefficients 110

Lateral Coefficients 111

Mach Coefficients 111

Control Coefficients 112

APPENDIX B 113

Flamingo Open-Loop Stability 113

Lateral Stability 113

Longitudinal Stability 116

APPENDIX C 119

Image Area Expansion 119

9 BIBLIOGRAPHY 122



Table of Figures

Figure 1 – Aviation layers of safety [24, 25] ....................................................... 33

Figure 2 – The sections of autonomous See-and-Avoid ....................................... 36

Figure 3 – Research developmental path ............................................................. 39

Figure 4 – Collision cone geometry [80] ............................................................. 47

Figure 5 – Collision avoidance system decision process ..................................... 53

Figure 6 – Geometry of a conflict scenario evolving over time .......................... 54

Figure 7 – Miss distance relationships ................................................................. 55

Figure 8 – Image plane characteristics ................................................................. 56

Figure 9 – Geometry of a conflict scenario .......................................................... 57

Figure 10 – Collision avoidance right-of-way sectors ......................................... 60

Figure 11 – Typical encounter scenarios ............................................................. 61

Figure 12 – Owncraft model used to define linear and angular variables ........... 63

Figure 13 – Aileron from heading and roll .......................................................... 69

Figure 14 – Rudder feed forward from sideslip (for coordinated turns) .............. 69

Figure 15 – Throttle for airspeed hold ................................................................. 70

Figure 16 – Elevator for altitude hold .................................................................. 70

Figure 17 – Two-camera perspective projection setup ........................................ 73

Figure 18 – Image of intruder as seen, without compensation, in the camera

frame (top) and with motion compensation (bottom) as calculated. The units are wrt

the focal length in millimetres. .................................................................................. 76

Figure 19 – IBCASE (simulator) architecture ..................................................... 78

Figure 20 – Example of a standard operating characteristics curve [111] ........... 83

Figure 21 – Possible outcomes for UAS with collision avoidance system [115] 84

Figure 22 – Random selection of intruder tracks encroaching owncraft ............. 90

Figure 23 – Distribution of min(CAS) for experiment 1 ....................................... 92



Figure 24 – Example of scenario where an owncraft reaches new heading and

immediately returns to the original heading. This is an example of a single

manoeuvre. ................................................................................................................. 94

Figure 25 – CAS behaviour for first 60s of Figure 24 track. The CAS test statistic

(red line) is between thresholds (±16) therefore a manoeuvre is made (3 secs). The

CAS is maintained at the last stable reading (blue dotted line) during the manoeuvre.

At the new heading, it is deemed safe to return to the original heading (20 secs),

where the stable CAS is held (blue dotted line) until on original heading. ................. 94

Figure 26 – Example of scenario where owncraft maintains new heading until θ

seconds before returning to original heading. This is an example of a maintained

manoeuvre. ................................................................................................................. 95

Figure 27 – CAS behaviour for first 60s of Figure 26 track. The CAS test statistic

(red line) is between thresholds (±16) therefore a manoeuvre is made (3 secs). The

CAS is maintained at the last stable reading (blue dotted line) during the manoeuvre.

At the new heading, it is still not safe to return to original heading (20 secs), so the

current heading is maintained for ϴ time until another CAS reading decides it is safe

to return to original heading (36 seconds).................................................................. 95

Figure 28 – Example of scenario where owncraft avoids and returns to original

heading, however CAS threshold is violated a second time. This is an example of a

repeated manoeuvre. .................................................................................................. 96

Figure 29 – CAS behaviour for first 70s of Figure 28 track. An avoidance

manoeuvre is made at 3 secs and then the CAS decision returns the owncraft to the

original heading (19 secs). When the owncraft has returned to the original heading a

second manoeuvre is performed (36 secs) and returns again (50 secs). .................... 96

Figure 30 – False Positive distributions before and after CAS is implemented .. 97

Figure 31 – A selection of Correct Avoidances made using implemented

algorithm. (a) top left – left intruder approach with maintained manoeuvre (b) top

right – right intruder approach with single manoeuvre (c) middle left – left intruder

approach with single manoeuvre (d) middle right – right intruder approach with

single manoeuvre (e) & (f) bottom – right intruder approach with repeated

manoeuvre. ................................................................................................................. 98



Figure 32 – Another selection of Correct Avoidances made using implemented

algorithm. (a) top left – left intruder approach with maintained manoeuvre (b) top

right – right intruder approach with single manoeuvre (c) & (d) middle – right

intruder approach with single manoeuvre (e) & (f) bottom – right intruder approach

with repeated manoeuvre. .......................................................................................... 99

Figure 33 – Failed avoidance detection or manoeuvres according to TABLE V

and Figure 21. (a) top right – Missed Detection (b) top left – Late Alert (c) bottom

left – Late Alert (d) bottom right – Late Alert on a repeated manoeuvre. .............. 100

Figure 34 – Standard Operating Characteristics (SOC) curve for CAS ............. 101

Figure 35 – SOC curve that displays original PCD and PFM ............................... 102

Figure 36 – Risk Ratio results for CAS ............................................................. 103






Acronyms

ACAS Airborne Collision Avoidance System (European term)

AFRL Air Force Research Laboratory (USA)

ARCAA Australian Research Centre for Aerospace Automation

ASTM American Society for Testing and Materials

ATC Air Traffic Control

BOT Bearings-Only Tracking

CA Correct Avoidance

CAS Collision Avoidance System

CASA Civil Aviation Safety Authority

CD Correct Detection

CNA Conflict with No Action

CPA Closest Point of Approach

CoG Centre of Gravity

DRA Defense Research Associates

EKF Extended Kalman Filter

ELOS Equivalent Level of Safety

ERAST Environmental Research Aircraft and Sensor Technology

EO Electro-optical

FA False Alarm

FAA Federal Aviation Administration (UAS)

FMV Försvarets Materielverk (Swedish Defence Materiel Admin.)



FOV Field of View

FM False Manoeuvre

HALE High Altitude/Long Endurance UAV

IBCASE Image Based Collision Avoidance Simulation Environment

ICAO International Civil Aviation Organization

IC Induced Conflict

IEEE Institute of Electrical and Electronics Engineers

KTAS Knots True Air Speed

LA Late Alert

LOS Line of Sight

MAC Mid Air Collision

MATLAB Matrix Laboratory (computer program)

MD Missed Detection

MILP Mixed Integer Linear Programming

MPC Model Predictive Control

NAS National Airspace System

NASA National Aerospace and Space Administration

NATO North Atlantic Treaty Organization

NED North-East-Down (coordinate frame)

NMAC Near Mid Air Collision

NMI Nautical Miles

NTSB National Transportation Safety Board (USA)

PaRCA Passive Ranging Collision Avoidance

PID Proportional-Integral-Derivative controller



PN Proportional Navigation

POI Point of Impact

POMDP Partially Observable Markov Decision Processes

PR Proper Rejection

QUT Queensland University of Technology

RR Risk Ratio

RTCA Radio Technical Commission for Aeronautics

SA Successful Alert

TCAS Traffic Collision Avoidance System

UA Unnecessary Alert

UAS Unmanned Aircraft System

UAV Unmanned Aerial Vehicle






Nomenclature

exp

I A Frontal cross-sectional area of intruder (m2)

b Owncraft wing span (m)

ASC Test statistic for collision avoidance

DC Drag coefficient total

LC Lift Coefficient total

lC Rolling moment coefficient

mC Pitching moment coefficient

nC Yawing moment coefficient

XC

X body-axis coefficient

YC Y body-axis coefficient

ZC Z body-axis coefficient

1 9c Moment equation coefficients listed in Equation (4.13)

c Owncraft mean wing chord (m)

elD

Derivative gain for altitude from elevator outer control loop

d Intruder size (a priori) (m)

MACE Expected number of MACS (MACs/hr)

NMACE Expected number of NMACS (NMACs/hr)

f Focal length of cameras (m)

fov Field of view for a single camera (˚)



g Gravity constant (m/s2)

h Altitude - Z earth-axis position of the owncraft, NED (m)

Ch Altitude commanded (m)

eh Altitude error (m)

engh Engine angular momentum about x-axis (N∙m∙s)

XI X body-axis moment of inertia (N∙m)

ZXI Z-X body-axis product of inertia (N∙m)

YI Y body-axis moment of inertia (N∙m)

ZI Z body-axis moment of inertia (N∙m)

elI Integral gain for altitude from elevator outer control loop

thI Integral gain on throttle from speed hold loop

kk Induced drag non-dimensional coefficient

k Time instance

L X body-axis aerodynamic moment component (N∙m)

apseL Lapse rate, of temperature with height (˚K/m)

M Y body-axis aerodynamic moment component (N∙m)

airM Molar mass of air (kg/mol)

m Owncraft mass (kg)

N Z body-axis aerodynamic moment component (N∙m)

n Camera number where [1,2]n

I n Integer number

P Air pressure (Pascals)



0P Air pressure at sea level (Pascals)

P Proportional gain for heading hold outer control loop

ailP Proportional gain for aileron from roll control loop

elP Proportional gain for altitude from elevator outer control loop

thP Proportional gain on throttle from speed hold loop

rudPff Proportional feed forward gain for coordinated turn of rudder from

sideslip

AlertP Probability of an alert being issued (hr-1)

ConP Probability of a entering conflict scenario (hr-1)

CDP Probability of a Correct Detection (hr-1)

CNAP Probability of a Conflict occurring if No Action (manoeuvre) is taken

(hr-1

)

FatalityP Probability of a fatality occurring (hr-1

)

FMP Probability of a False Manoeuvre

MACP Probability of a MAC occurring (hr-1

)

MACwithCASP Probability that a MAC occurs whilst a CAS is being used (hr-1

)

MACwoCASP Probability that a MAC occurs where no CAS is being used (hr-1

)

NMACP Probability of a NMAC occurring (hr-1

)

NMACwithCASP Probability that a NMAC occurs whilst a CAS is being used (hr-1

)

NMACwoCASP Probability that a NMAC occurs where no CAS is being used (hr

-1)

SAP Probability of a Satisfactory Alert (hr-1

)



UAP Probability of an Unnecessary Alert (hr-1

)

im

I

nP Intruder pixel location in the image of camera n where [1,2]n

I

cP Intruder pixel location in the joined camera plane

I

bP Intruder pixel location in the body-axis of the owncraft

I

NP Intruder pixel location in the earth-axis wrt owncraft

I

compP Motion compensated pixel location of intruder wrt owncraft

p X body-axis angular velocity component (rad/s)

q Y body-axis angular velocity component (rad/s)

q Dynamic Pressure (Pa)

R Range from owncraft to intruder (m)

gR Ideal gas constant (J/mol∙˚K)

kR Range from owncraft to intruder at time k (m)

RiskRatio Probability that a NMAC will occur with a CAS

ICRR Induced Conflict component of Risk Ratio

MACRR Probability that a MAC will occur with a CAS

unresolvedRR Unresolved risk component of Risk Ratio

r Z body-axis angular velocity component (rad/s)

S Owncraft wing area (m2)

kS Distance from owncraft to Point of Impact (POI) at time k (m)

I s Distance intruder travels in time I t (m)

T Engine Thrust (Newtons)



0T Temperature at sea level (˚Kelvin)

kT Perpendicular distance from intruder to POI at time k (m)

TCT Time to collision (s)

b

cT Cameras position in owncraft body axis (m)

I t Time it takes intruder to travel distance I s (s)

u X body-axis velocity of owncraft (m/s)

nu X image-axis pixel location of intruder in camera n

Vol Volume of airspace in encounter scenario (m3)

tV Velocity of the owncraft in the air (m/s)

C

tV Velocity commanded (m/s)

e

tV Velocity error (m/s)

v Y body-axis velocity of owncraft (m/s)

w Z body-axis velocity of owncraft (m/s)

X X body-axis aerodynamic force component (Newtons)

bx X body-axis position (m)

cx X camera-axis pixel location of the intruder

ex X earth-axis position of the owncraft, NED (m)

imnx X image-axis pixel location of intruder of camera n

compx X motion-compensated-axis pixel location of intruder

Nx X earth-axis position of intruder wrt owncraft (m)

Y Y body-axis aerodynamic force component (Newtons)



mY Closest point of approach distance of two aircraft (m)

lowerY Lower bound of random mY generated (m)

upperY Lower bound of random mY generated (m)

by Y body-axis position (m)

cy Y camera-axis pixel location of the intruder

ey Y earth-axis position of the owncraft, NED (m)

imny Y image-axis pixel location of intruder of camera n

compy Y motion-compensated-axis pixel location of intruder

Ny Y earth-axis position of intruder wrt owncraft (m)

Z Z body-axis aerodynamic force component (Newtons)

bz Z body-axis position (m)

Nz Z earth-axis position (altitude) of intruder wrt owncraft (m)

Angle of attack (rad)

Angle of sideslip (rad)

ail Aileron control surface deflection ( 1 1)a

el Elevator control surface deflection ( 1 1)e

rud Rudder control surface deflection ( 1 1)r

th Throttle deflection (0 1)th

X body-axis position of camera (m)

Y body-axis position of camera (m)

Roll - Euler angle of owncraft (rad)



C Roll command (˚)

e Roll error (rad)

Z body-axis position of camera (m)

Relative bearing of intruder wrt owncraft (˚)

k Relative bearing of intruder at time k (˚)

Relative bearing rate of intruder wrt owncraft (˚/k)

/ t Relative bearing rate of intruder wrt owncraft (˚/k)

Relative elevation of intruder wrt owncraft (˚)

spiral Spiral roll characteristic root

roll Roll characteristic root

Angle subtended by intruder in owncraft‟s image sensor (˚)

k Angle subtended by the intruder at time k (˚)

Angle subtended rate (image expansion, 1D) (˚/k)

t Angle subtended rate (image expansion, 1D) (˚/k)

I

e Intruders position in the earth-axes (m)

I

N Intruders position in the earth-axes wrt owncraft, NED (m)

O

e Owncraft‟s position in the earth-axes - ( , , )e ex y h (m)

e Owncraft‟s attitude – ( ) (rad)

Heading (Yaw) - Euler angle of owncraft (rad)

C

k Heading command at time k (˚)

e Heading error (rad)



max Maximum heading rate change (˚/s)

Air density (kg/m3)

Angle of heading alteration during manoeuvre (˚)

roll Roll time constant (s)

Time for a heading change of to take place (s)

Pitch - Euler angle of owncraft (rad)

Half the fov of the camera (˚)

b

c Rotation of the camera wrt body-axis (rad)

e

b Rotation of the body-axis wrt earth-axis aka e (rad)

DRn Dutch Roll undamped natural frequency (rad/s)

Pn Phugoid undamped natural frequency (rad/s)

SPn Short-period undamped natural frequency (rad/s)

x Rotation about the x-axis (rad)

y Rotation about the y-axis (rad)

z Rotation about the z-axis (rad)

Angular area subtended by the intruder (2D) (˚2)

t Angular area subtended rate (image area expansion, 2D) (˚2/k)

Angular area subtended rate (image area expansion, 2D) (˚2/k)

DR Dutch Roll damping ratio

P Phugoid damping ratio

SP Short Period damping ratio



Statement of Authorship

The work contained in this thesis has not been previously submitted to meet

requirements for an award at this or any other higher education institution. To the

best of my knowledge and belief, the thesis contains no material previously

published or written by another person except where due reference is made.

Signature: ______________________________________________________

Date: __________________________________________________________






Acknowledgements

This research was supported and funded by the Australian Research Council,

Australian Postgraduate Award, Faculty of Built Environment & Engineering,

Queensland University of Technology (QUT) Vice-Chancellor and Queensland

Government‟s Smart Skies Project.

I would like to express thanks to primary supervisor Dr Luis Mejias-Alvarez for

his guidance, also Dr Jason Ford for his support as an associate supervisor. Thanks

also to Prof. Rodney Walker for his motivation and patience.

I would also like to thank fellow researchers within the Australian Research

Centre for Aerospace Automation for all your assistance, patience and friendships.

I would like to express my thanks to my wife Amanda and our children Lachlan,

Elijah, Noah and Gabriella for your awesome support throughout this period, without

which, this very well could not have been possible.

Most of all I would like to thank the Lord Jesus Christ for the opportunity, ideas,

help, answers and inspiration for the following work.






1 Introduction

In this section, we introduce the motivation that drives this research; present the

problem definition; describe the overall objective and goals of this thesis; discuss its

significance; detail the novel contributions made; describe the adopted research

methodology; list the publications; and finally, detail the thesis format.

1.1 Motivation

Recently Australia celebrated a centenary of flight, which began with Harry

Houdini's pioneering flight at Diggers Rest, Victoria on March 16th

, 1910 [5]. In the

last 100 years, aviation has gone through considerable technological advancements.

Currently, the aerospace industry is increasing the trend towards automation,

replacing pilot functions with automated avionics. New systems are emerging that

use an increased level of automation; these are called Unmanned Aircraft Systems

(UAS). UAS origins can be traced back to 1914, when Elmer Sperry demonstrated

his gyro-stabilized Curtiss seaplane in a French airplane safety contest [6].

In the last decade or two, the ideas for pilotless plane operations have grown.

The applications and scenarios for UAS utilisation are expanding as industry is

becoming more aware of the functionality and capability of these autonomous

vehicles. Today UAS are the fastest growing sector in Aerospace [7]. Sales over the

next decade is projected to grow from $4.4 billion annually to $8.7 billion, with more

than $62 billion being spent [7].

For growth to continue on this scale, UAS operations need to expand beyond

controlled airspace and operate freely within the national airspace system (NAS) [8].

NAS integration requires that UAS are capable of performing at an equivalent level

of safety (ELOS) to that of manned aircraft [9-11]. A capability manned aircraft

have that UAS will require is see-and-avoid [8]. See-and-avoid technology is also

referred to as collision avoidance or sense-and-avoid.



Collision avoidance has been identified as one of the top priority technological

enablers of UAS into the NAS by various international organisations. United States

Office of the Secretary of Defence have recognised it as the second biggest matter

[3]. The Joint Air Power Competence Centre (NATO) has identified collision

avoidance as one of the top 26 needed capabilities facing UAS [12]. Also regulatory

bodies, Federal Aviation Administration (FAA) [8] and Eurocontrol [13] have

recognised the high priority of the collision avoidance issue. Various consortiums,

committees, studies and reports have been created to address collision avoidance [14,

15]. Standard specifications have even been drawn up by ASTM (American Society

for Testing and Materials) for a sense-and-avoid system design and its performance

requirements [16]. RTCA (Radio Technical Commission for Aeronautics) are also

expecting to have standards by 2011 [10].

1.2 Collision Avoidance Problem

1.2.1 Definitions in the collision avoidance problem

In the literature, there are a few distinctively different research areas, under the

name of collision avoidance [17-19]. In this thesis, we have used distinct definitions

to break this problem into the various categories. The first distinction made is

between UAS avoiding collisions with terrain/static obstacles as opposed to other air

traffic. In this thesis,

Obstacle avoidance is defined as avoiding collisions with terrain or

static obstacles.

Obstacle avoidance is not considered in this thesis, although a comparison is

addressed later in the literature review. Two main categories of collision avoidance

can be identified:



Cooperative collision avoidance, is where two aircraft that are in

communication with one another, negotiate a mitigation strategy.

This can happen distributively [20, 21] or with a central manager (separation

management) [22, 23]. On the other hand:

Non-cooperative collision avoidance is where the onus is solely

on each individual owncraft to find a way to avoid the conflict

scenario.

Non-cooperative collision avoidance is used as a safety backup in the event that

separation management fails or in case the aircraft are not in communication with

one another [24, 25]. The various levels of collision avoidance are shown in Figure

1. Levels 1-4 are considered separation management or cooperative collision

avoidance and level 5 is the non-cooperative collision avoidance [24, 26]. Non-

cooperative is the type of collision avoidance that governmental institutions and

regulatory bodies have identified as the major technological enabler for UAS to

operate freely in the NAS.

Figure 1 – Aviation layers of safety [24, 25]

Level 5 - See & Avoid

Level 4 - TCAS/CDTI –ACAS

Level 3 - Radar Separation Services

Level 2 - Strategic Sep. Services

Level 1 - Airspace Structures



1.2.2 Definitions

In this thesis, we make use of standard definitions from TCAS Minimum

Operations Performance Standards [27], but slightly redefined to accommodate the

context of our problem. The following definitions apply to our problem:

Near mid air collision (NMAC) occurs when two aircraft come

within 500 feet horizontally, which is 152.4 metres (also 100 feet

vertically but that component is ignored in this work) [27] .

Conflict scenario is defined for this research to be ‘an encounter

scenario between two aircraft whereby the aircraft come within

152.4m of each other laterally. This would result in a NMAC being

filed’.

Collision scenario is defined as ‘an encounter scenario whereby

two aircraft will collide with one another if an avoidance

manoeuvre is not made’.

Mid air collision (MAC) occurs when two aircraft collide with one

another.

In the context of this research, we define:

A mid air collision (MAC) as an ‘encounter scenario that would

lead to the two aircraft coming within 32m of one another’

This arises from realizing that a Boeing 747 (very large aircraft) has a wingspan

of 60m, we assume that our vehicle has a wingspan of 4m (which is a typical midsize



UAS), thus 32m1 seems a reasonable and even conservative figure. Typically, there

is no distance that defines two aircraft coming into contact with one another.

Because aircraft have volume and are not particles, this constant is arbitrarily

approximated. This is actually a reoccurring problem in CAS performance

evaluation [28].

1.2.3 Types of sensors

Of particular importance to non-cooperative collision avoidance is the

methodology used to acquire situational awareness of the environment in which the

owncraft is operating. There are generally two types of methods used to acquire

situational awareness; those involving either active or passive sensors.

Active sensors emit radiation and wait for a reflected signal in order to acquire

situational awareness [29]. This radiation is mostly radar for the UAS application,

but infrared, laser, ultrasonic can also be used. Active sensors are generally heavy,

larger, expensive, power-demanding and have large bandwidth requirements [29,

30]. As such, active sensors are often implemented on larger, more-expensive UAS

[31].

On the other hand, passive sensors acquire information from natural emissions

and reflections [29]. Passive sensors are normally electro-optical (EO), both infrared

and vision-based. In contrast to active sensors, passive sensors are typically much

cheaper, smaller, lighter and more power-efficient. They provide good bearing

information [26] and are easily interfaced with processors for research-friendly

computational analysis [32]. The main advantage is that passive sensors, in

particular electro-optical (EO) sensors, are much cheaper, generally lighter and less

power demanding than active sensors [29]. Thus, passive sensors are ideal for low-

cost UAS [29]. However, special attention should be placed in atmospheric effects

since it greatly affects the performance and quality of data [29]. Another limitation

of passive sensors is that no range information is directly observed [26].

1 Half wing span of each aircraft added.



1.3 Research Objectives

This research endeavours to use a passive sensor to address the non-cooperative

collision avoidance problem. Thus the principle research objective driving this thesis

is:

To investigate the best method to declare and avoid a conflict

scenario using image-based data.

Within this principle objective, the two major goals of this research are:

To identify and use the most relevant image-based features to determine

whether a conflict scenario will take place.

Upon determining that a threat is likely, use image-based features to

manoeuvre the owncraft to avoid colliding with an intruder.

Figure 2 – The sections of autonomous See-and-Avoid

Note that this research does not investigate the problem of detecting an intruder,

but only determining if a collision is likely given the intruders behaviour in the

image. The intruder detection is assumed to have taken place a priori and is outside

the scope of this research. This is illustrated in Figure 2. Intruder detection have

been investigated and addressed in other research [33].

Intruder Detection

Collision Determination

Avoidance Manoeuvre



1.4 Significance

As previously discussed, the collision avoidance problem is one of the major

hurdles to allowing UAS to operate freely in the NAS and thus ensure continued

UAS market growth. Accordingly, various major players within industry have

attempted to solve the collision avoidance problem (discussed in Section 2.1). The

most capable proposal put forth from industry was proposed by Chen [31]. It uses a

heavy and power-demanding sensor that costs approximately $200k. Such a system

would seem unreasonable for low-cost UAS.

A collision avoidance system using vision-only sensors would present a solution

for the low-cost UAS market and be a major technological enabler for the entire

UAS sector [29]. The collision avoidance algorithm presented in this research uses

vision-only sensors and can be implemented on general purpose hardware (costing<

$5k).

The significance this particular research has over other vision-only collision

avoidance algorithms, is that it:

Triggers an avoidance manoeuvre earlier than the range-estimate

dependant techniques such as the ones presented in [25, 30, 34]. This

research is able to react within three camera frames (typically 0.12

seconds) after intruder detection.

In addition, this research does not require range estimate in order to

decide to manoeuvre, unlike [25, 34, 35].

Use a less comprehensive set of scenarios to obtain a performance similar

to Kochenderfer et al. [36] which implements partially observable

Markov decision processes on bearings-only data for collision

determination. However, it should be compared in the appropriate context

given the considerable number of scenarios addressed by Kochenderfer et

al.



1.5 Research Contributions

The primary contribution of this thesis is in the design of a collision avoidance

system (CAS) that uses vision-only features to predict whether a collision is likely

and to decide how to manoeuvre. The algorithm is novel as it performs collision

avoidance without using any position-based information; it is performed using only

image-based information. Because of this feature, it is able to react almost

immediately; this is orders of magnitude faster than other systems [25, 30, 34] (see

Section 3.1).

As a derived contribution this research propose an avoidance manoeuvre

algorithm based on a relative-bearing control law (see section 3.2). This control

approach is consistent with see-and-avoid recommendations.

This research also contributes with an EO sensor model that combines two

standard EO sensors to achieve wide field of view (see Section 4.3). This sensor is

motion compensated accounting for the platform manoeuvres, with the advantage of

keeping the target in the field of view of the sensor during platform manoeuvring.

Finally, this thesis contributes with a comprehensive set of validations based on

Monte-Carlo simulations. This work provides using encounter models, performance

metrics that can be comparable with current aviation practices. Performance is shown

using standard operating curves.

1.6 Research Methodology

The research methodology followed in this thesis is illustrated in Figure 3. First,

a literature survey in collision avoidance has been performed, the findings of which

are presented in Section 2. Next, the problem is analysed and developed

geometrically, as shown in Section 3. Then a collision avoidance simulation

environment is developed using a model of the Flamingo UAS [37] and the collision

avoidance sensor model. To validate the proposed collision determination and

avoidance manoeuvre system, comprehensive Monte Carlo simulations with random



two-minute conflict scenarios are developed. The results are used to iteratively

develop and refine the original algorithm. The final algorithms‟ results are detailed

and analysed in Section 0.

Figure 3 – Research developmental path

1.7 Publications

There were two publications produced during this period of study. They are as

follows:

Conference Paper

S. C. Degen, L. Alvarez, J. J. Ford, and R. Walker, "Tensor field guidance for time-based waypoint arrival of UAVs by 4D trajectory generation," in Proceedings of the IEEE Aerospace Conference, Big Sky, Montana, 2009.

Submitted Journal Paper

S. C. Degen and L. M. Alvarez, "A reactive image-based collision avoidance system for Unmanned aircraft systems," IEEE Transactions on Aerospace and Electronic Systems, 2011 (submitted).

Literature Survey

Mathematical Development

Simulation & Analysis



1.8 Content

Thus far, we have discussed the motivation and rationale for the following

research. In addition, we have described the problem and its significance.

This thesis is structured as follows: Section 2 investigates the background of

collision avoidance in literature, both from industry and academia. Then we discuss

passive-only collision avoidance and positioning of this thesis within the overall

collision avoidance field.

Section 3 investigates the characteristics of an image that are pertinent to conflict

scenarios and then develops the IBCA technique that is used for detecting the

collision. It goes on to show the adopted method for manoeuvring the owncraft

around the intruder, once collision is detected.

Section 0 shows the mathematical modelling of the UAS and its controller. It

mathematically models the camera setup, which is the collision avoidance sensor,

and the controller that is used in the simulation. Finally, this section discusses the

simulation environment developed for the testing phase.

Section 0 discusses the performance metrics used to assess the safety of the

proposed CAS. Then it describes the setup of the experiment and its limitations.

Finally, it shows the results and discusses the implications.

The conclusions of this thesis are detailed in Section 6, with recommendations

for future work made in Section 7.



2 Background

This chapter surveys existing literature in the collision avoidance domain. Firstly,

the efforts of the aerospace industry are reviewed because they have been working on

non-cooperative collision avoidance for decades. Next, this section surveys the

academic literature examining at passive obstacle avoidance and non-passive

collision avoidance. Finally, we investigate the research specifically in the field of

passive-only collision avoidance.

2.1 Collision Avoidance Systems:

Industry Developments

2.1.1 NASA ERAST

In March 2003, National Aerospace and Space Administration (NASA)

Environmental Research Aircraft and Sensor Technology (ERAST) program flew

twenty-two conflict scenarios using a 35 GHz radar sensor [38]. The concept of

operation is for the collision detection system to provide situational awareness to a

human-in-the-loop who is responsible for performing the actual manoeuvre. This

program flew thirteen encounter scenarios with a single intruder aircraft and another

nine scenarios with two intruders approaching at the same time.

ERAST found that the pilots when unassisted would detect intruding aircraft at a

distance of approximately 1 1.5NMI (nautical miles) away. ERAST discovered

any speed greater than 300KTAS (knots true airspeed) and detection distance less

than 4 5NMI , is difficult for the pilot to comfortably avoid. This is attributed to

the human-in-the-loop, thus with an autonomous controller, the detection distance

would have provided plenty of time for a manoeuvre. This aligns with the findings

of Graham and Orr [39]. Overall, the collision detection system is deemed to



provide acceptable situational awareness by the pilots, except for a head on conflict

scenario where 600relv KTAS (using a FA-18). Where relv is the relative velocity

between the two approaching aircraft.

2.1.2 Auto ACAS

In 2002, Boeing, Lockheed Martin, Air Force Research Laboratories (AFRL),

Saab and FMV (Swedish Air Force) developed the Auto-ACAS (Airborne Collision

Avoidance System) for military aircraft that communicates along an established data

link [40]. It is able to negotiate conflict scenarios and has tested relative velocities

up to 860relv KTAS . This system predicts and transmits the owncraft‟s trajectory

into the future (5-10 secs). It compares the owncraft‟s trajectory predication against

all other aircraft transmitting their trajectory predictions. An avoidance manoeuvre

is carried out cooperatively should the risk exceed a threshold. Optimal control is

used to roll the aircraft away from one another, attempting to maintain a closest point

of approach (CPA) of 100m. Researchers found a violation of the CPA for one

conflict scenario where the CPA went as low as 80m [41].

2.1.3 Northrop Grumman & AFRL

In 2005, Northrop Grumman developed the passive ranging concept [25]. This

concept involves a manoeuvring owncraft acquiring a range estimate of the intruder

using a triangulation technique. In particular, for Northrop Grumman, this

manoeuvre involved a climb. Northrop Grummans‟ research uses passive sensors

and gets a range estimate to converge to 5-7% error (approximately) within 7

seconds. This convergence occurs at approximately 120ft of the 500ft, 20-second

climb. The climbing manoeuvre works well because the best motion for intruder

range estimation is one perpendicular to intruder line of sight (LOS) [42].

AFRL and Defense Research Associates (DRA) Detect And Avoid program used

an electro-optical (EO) sensor that had ~0.5 milli-radians (mRad) resolution [43]. In

simulation, this system detects the intruder at approximately 4NMI with near 100%

confidence and a 0.05% false detection rate. However, during flight testing there



were thousands of false tracks [44], of which, 68% were eliminated by tracking for

around 20 frames.

In 2005, AFRL and Northrop Grumman developed a see-and-avoid sensor suite

under the SeFAR program [45]. Later in 2007, Northrop Grumman and AFRL-DRA

collaborated on a series of flight trials with the sensor suite [46]. A Lear jet was used

as a surrogate High Altitude/Long Endurance (HALE) UAS for these trials. In

addition, the Northrop Grumman electro-optical sensor was combined with the traffic

collision avoidance system (TCAS). Twenty-seven different conflict scenario

geometries were flown with two (human piloted) intruder aircraft. Results from the

flight trial deemed the collision avoidance system (CAS) as successful, but admit

there is still much work to do. Currently, improvements are being made reducing the

range at which the intruder is detected and reducing the number of false tracks. It

was found that long-wave infrared cameras performed little, if no better than electro

optical (EO) sensors [46]. The PaRCA (Passive Ranging Collision Avoidance)

avoidance algorithm performed „well‟ (CPA distance exceeded 762m) except for the

head on case when the detection range was only 1.5NMI .

PaRCA is an evolutionary algorithm by Shakernia et al. [25] that pre-empts the

avoidance manoeuvre before executing it. It then calculates the range during the

execution of this avoidance manoeuvre, which is later incorporated in the controller.

Thus, the owncraft will not have to do a reversal if the passive ranging manoeuvre

increases the collision risk. PaRCA considers other constraints, like camera field of

view (FOV), owncraft dynamic limits, air traffic control (ATC) corridors etc.

In May 2008, the third evolution of the AFRL/Northrop Grumman programs flew

[31]. Radar, Airborne Dependant Surveillance Broadcast and TCAS were added to

the EO sensor and it became known as the Multiple Intruder Autonomous Avoidance

sensor. The ICV AI-130 radar was used for this series of flight trials. This system

flew many different geometries, with up to two intruders, and was deemed

successful. The pilot participants remarked that, “This is how a pilot would have

done it.” The fundamental problem with the AFRL approach for low-cost UAS is



the price, size, complexity, weight and power required for the sensor suite. The

evolved sensor suite now costs nearly $200k.

2.1.4 Discussion

The Auto-ACAS system [38, 39] has a dedicated communications channel,

making this a cooperative technology. The program showed almost no success but

helped others to understand the difficulty of the problem. The NASA ERAST

program [40, 41] involves radar, which is not a passive sensor. Even with this

expensive and very capable sensor, difficulties still arose in some scenarios.

However, the CAS was deemed a general success, just not in extreme situations.

Both of these programs are interesting to note because they work on the collision

avoidance program, but they are very different to the research of this thesis in that

they are either cooperative or non-passive.

The joint Northrop Grumman and AFRL program [25, 31, 42-46] started out

using a passive sensor but has very recently become a non-passive sensor due to the

difficulties involved in using a passive-only sensor. Therefore, this program is not

directly related to the work of this thesis.

2.2 Collision Avoidance Strategies

2.2.1 Optimal Strategies

Optimal control techniques for collision avoidance close the control loop and

make use of dynamic programming type approaches for avoiding obstacles [47].

Receding Horizon Control [48] (also known as Model Predictive Control (MPC)

[49]), which emerged from the field of chemical process control [50], has been

adapted for UAS nonlinear control. MPC closes the loop of open-loop optimal

control variables, at each time step, and incorporates the new environment variables,

which in the case of collision avoidance, is the intruder.



The first method considered here uses Mixed Integer Linear Programming

(MILP) with MPC strategies. This approach is first proposed by Pallottino et al. [51]

and also by Richards and How [47] to avoid vehicle collisions cooperatively. The

decentralized version of this MILP with MPC algorithm is still cooperative[47]. It

also requires a priori intruder position and uses cooperative techniques to acquire

this information; although, just as simply one could have used active sensors for non-

cooperative collision avoidance. Similarly, the non-linear MPC method introduced

by Shim and Kim [52] utilizes potential field methods (as discussed in Section 2.2.2)

and uses a priori position information about the intruder obtained from active

sensors, to safely navigate around obstacles.

MPC actually was first implemented using passive sensors by Frew [53]. Frew

derives the equations relevant to establishing the MPC and puts the target

information in a Fisher Information Matrix that models the probability of the

collision in an estimate covariance matrix. His approach uses live data to navigate

through unknown environments with static obstacles. For the collision avoidance

case, it uses a priori position and velocity information about the intruder. Thus, the

dynamic collision avoidance problem is unaddressed.

In the next version of this approach, Frew [48] includes adaptive control in the

MPC and builds upon this controller to form a global planner but still does not

address the dynamic obstacle problem. In later work, Frew [54] includes the

Unscented Transform on bearings-only information but on stationary targets

(obstacles).

2.2.2 Force Control Techniques

The notion of vector fields is similar to the idea of potential field methods, which

is the pioneering work of Khatib [55]. The use of potential functions has continued

to be one of the mainstream approaches to robotic task execution in the presence of

obstacles [56, 57]. A comprehensive summary of techniques that address the classic

geometric problem of constructing a collision-free path and traditional path-planning

algorithms is provided by Latombe [58]. Furthermore, progressive improvements



made to the general potential field methods over the last two decades [59-61]

continue today [57].

Potential field methods are force control techniques that navigate a robot by

sensing the environment and mapping it according to physical equations that make

that environment analogous to physical laws [62]. In the specific case of potential

field methods, the analogous physical law is the electrostatic charge equation, where

vehicle and/or obstacles and goal are treated as charges of opposite sign. Then the

path is mapped by calculating the Coulomb forces between every point of the

environment and the vehicle. The vehicle „falls‟ down the path of least resistance.

There are other force control techniques that come from mobile robot navigation

using physical equations for navigation; these are gaseous diffusion [63], Laplace‟s

equations [64] or mechanical stress fields [65]. These and many others have been

developed for the obstacle and collision avoidance problem over the decades.

Potential field methods more prevalent to UAS are the impedance force model

developed by Jang et al. [18]. Also, vector fields have been developed for UAS [66-

74] and used in commercial systems [75]. Vector fields have also been developed by

Degen et al. [76]. Sigurd and How [77] develop a method called total field collision

avoidance, for multiple UAS guidance and avoidance in a dense vehicle and obstacle

environment, once again using active sensors.

The free flight algorithm is another adaptation of potential field methods [20, 78].

It was developed by the RTCA in 1995 [79]. Aircraft repel one another according to

a „voltage potential function‟ in order to achieve a minimum closest point of

approach distance (CPA).

All these techniques require the relative position of the intruder to be known or

calculated. This can be done with vision-based sensors but implies extreme

complexity and is subject to calibration errors, otherwise it requires active sensors or

to be done cooperatively.



2.2.3 Geometric Approaches

2.2.3.1 Collision cones

The collision cone approach was first presented by Chakravarthy and Ghose [80].

Based on the geometry of Figure 4, a collision is avoided if the aircraft meets the

following conditions:

1) PR R

2) 0[ , ]rel f (outside of the cone)

3) 2

2

rel

rel

Figure 4 – Collision cone geometry [80]

The 3D version of this algorithm is presented by Watanabe et al. [81]. It uses

the active sensor (radar) planar implementation of Kumar and Ghose [82], but still

does not address the fundamental issue of dynamic obstacles without range

vo

v

i

-vi

-vrel

R

P

R

θ

θf θ0

ψrel

R

RP



information. Han and Bang [19] couple these with proportional navigation (PN)

strategies. This algorithm is also flown in the Carbone et al. [83] UAS.

2.2.3.2 Bearings-only tracking

Bearings-only tracking (BOT) is a technique that uses the relative bearing

information-only to filter the intruders range estimations. The range estimates allow

calculation of avoidance manoeuvres because positions of the relative parties are

known. The methods that are used for range estimation include: extended Kalman

filters (EKF) [84], posterior Cramer-Rao bounds [85] (that both Frew and How adopt

in almost all of the previously mentioned MPC work), particle filters [86] and some

for multi-target tracking [86, 87]. These have been adopted for modified polar

coordinates with better results [88] and the particle filter initialization issues are

addressed by Bréhard and Le Cadre [85]. It is demonstrated, that the particle filter

implementation yields the best results [84, 86, 89] for BOT filtering.

This area of BOT also covers the passive ranging concept that is discussed below

in Section 2.3.1.

2.2.4 Vision-based Obstacle Avoidance

The majority of the research in vision-based obstacle avoidance comes from the

optical flow field. Green and Oh [17] investigate obstacle avoidance (as defined in

Section 1.2.1.) using optical flow, by mounting one-pixel, 1-D, lightweight (4.8g)

optical flow sensors at ±45˚ on an indoor plane. The UAV is shown to avoid

collisions with a basketball net using rudder deflection only.

Another system, proposed by Fasano [26], couples an EO sensor, using pure

optical-flow based methods, with radar. This hybrid approach combines the higher

positive hit rate, range information and all-weather performance of radar with the

angular resolution of an EO sensor.

Recchia et al. [90] look at an EO only implementation, but show that this system

has many inherent limitations. This limitation comes from a requirement for a



stationary observer (for a moving target) or a stationary target (for a moving

observer).

Another vision-based technique for UAS is proposed by Watanabe et al. [81].

This algorithm uses optimal techniques to filter the position estimation of the

stationary obstacle. It then pre-empts the next waypoint to get a more efficient flight

path using minimum effort guidance.

Call et al. [91] investigate using an EO sensor to avoid stationary obstacles. A

corner-detecting image-processing algorithm is used to detect the obstacles. Next, a

reactive guidance algorithm, known as vector fields [71, 74], is used for the

avoidance manoeuvre. Griffiths et al. [72] use laser range finders (active sensors) to

avoid stationary objects that are located straight ahead, and use optical flow sensors

for navigating canyon corridors.

Shelnutt [92] develops a method for negotiating obstacles using optical flow,

similar to Griffiths method for navigating canyon corridors. It navigates between

two obstacles (on either side) by nullifying the LOS rate difference. This is

essentially the same as equalizing the optic flow on either side. It is similar to the

work presented in this thesis in that it uses the features of the image (in this case the

LOS rate) to navigate without first converting the image-based data into position-

based information.

2.3 Passive-only Collision Avoidance

Angelov et al. [93] propose a passive method to estimate the risk of a collision,

based on consecutive bearing measurements. Small changes in bearing indicate an

increased risk of collision (see Equation (3.1)). This paper fails to address the

decision aspect of this question i.e. what threshold of risk is deemed acceptable? It

also does not address the avoidance manoeuvre nor produce any performance results.

The methodology presented by Angelov et al. is closely related to that presented

in this thesis. However, the unpublished figures for avoidance performance make it

impossible to assess directly against the CAS presented here. Their performance



figures could be used to assess the collision determination/decision level of Section

5.3.3.

Kochenderfer et al. [36] use relative bearing and Time To Collision TCT

(extractable from angular-area subtended ) for hazard alerting. This is similar to

the work presented in this thesis. It is shown that a bearings-only ( only) CAS that

uses thresholding techniques, is rather ineffective. This type of CAS would have as

many false alerts as successful alerts (see Section 5.1.2). They implement partially

observable Markov decision processes (POMDP) to improve performance on a

bearings-only CAS. The POMDP system dynamically updates the underlying state

(impending collision) based on measurements (LOS rate) using Bayes‟ rule. An

observation model is obtained from simulation data. An impending collision belief is

updated from LOS measurements and compared against the model. The belief state

is thresholded, wherein a decision about manoeuvring is made.

The POMDP system (that implements Time To Collision TCT ) does obtain

notably better performance than the only system. The POMDP CAS employs a

comprehensive encounter model for simulation and testing, therefore, it cannot be

directly compared against the CAS of this thesis. However, the POMDP CAS is

compared to the only system to highlight performance gain. This thesis also

employs this strategy for displaying performance increase.

There are two reasons our system cannot be directly compared against the

POMDP. Firstly, it is not possible to replicate the POMDP work to compare against

directly without the observation model. Secondly, POMDP is implemented at the

collision determination level only (see Figure 2) and therefore does not manoeuvre.

This means that late alerts and induced conflicts (see Section 5.1.2) are not accounted

for.



2.3.1 Passive Ranging

An interesting geometric approach to passive-only collision avoidance involves a

technique henceforth known as passive ranging. This method manoeuvres the

owncraft and filters converging range estimates via a form of triangulation [46].

The idea of manoeuvring the owncraft for range estimation derives from the

bearing-only tracking (BOT) research field, Oshman and Davison [86] and

Logothetis et al. [87]. It first appeared in UAS obstacle avoidance in Frew and Rock

[94]. They investigated related issues like constraints on camera FOV and

measurement uncertainty. Other related research investigates different approaches

for optimizing the manoeuvre to increase awareness or reduce convergence time, e.g.

information theoretic approaches, Logothetis et al. [95].

The first time passive ranging appeared in UAS obstacle avoidance was in 2005,

Calise et al. [35]. They use optimal manoeuvring to obtain converging range

estimates on obstacles. In their paper, the camera is modelled similar to the CAS

sensor used in this thesis (see Section 4.1).

Passive ranging was first used for UAS collision avoidance by Shakernia and

Chen [25]. They achieved, in around 7 seconds, a range error convergence of 5-

10%, at a 120ft of a 500ft climb manoeuvre by exerting a 1.16 g manoeuvre for their

particular encounter scenarios. They noted that this climb manoeuvre is efficient

because a „perpendicular to line-of-sight‟ manoeuvre is required for quick

convergence and is close to optimal. They compared against a “dog-leg” lateral

manoeuvre that did not perform as well.

Voos [30] proposed a method for image-based (passive-only) collision avoidance

that does not require a manoeuvre. It filters range information with an EKF using

time to collision (see Section 3.1.3) information by monitoring intruder expansion

(pixels) in the image. This time-to-collision information takes around 3-4 seconds to

converge.



2.4 Discussion

The work done in this thesis is closest to the work done by Voos [30]. The

literature discussed in Section 2.3 filtered range information by either manoeuvring

the owncraft or by including image expansion and relative bearing rate. It could be

argued that a system that manoeuvres an owncraft is not passive, as it requires action

on the owncraft‟s behalf.

The work presented in this thesis does not manoeuvre the owncraft unless it

deems collision imminent. Shakernia et al. [25] and Frew [34] manoeuvre the

aircraft in order to get converging range estimates. Shakernia et al. [25], Frew [34]

and Voos [30] require time for these range estimates to converge. This convergence

time ranges from 4-15 seconds, which can be critical in a pending collision scenario.

The work in this thesis is similar to Voos [30] in that it uses image area

expansion and relative bearing rate . However, instead of filtering a range

estimate and then making a decision based on estimated intruder position, it makes a

decision directly from these image-based features (image area expansion and

relative bearing rate ). This is in a fashion similar to that of image-based visual

servoing research, Chaumette and Hutchinson [96, 97]. Thus, the following research

does not have to wait for a filter to converge on range and can act almost

immediately (within three camera frames ~ 0.12 seconds).



3 Collision Avoidance

This chapter presents the research that decides whether a collision is deemed

imminent using image based characteristics (Equation (3.15)). In addition, we show

the equations and algorithm that decide the avoidance manoeuvre based on image-

based characteristics.

3.1 Collision Determination

In this sub-section, we investigate the features in an image that directly affect a

conflict scenario, namely, intruder relative-bearing rate and image expansion.

Relative bearing rate (radians/sec) of the intruder is measured with respect to a

fixed body axis coordinate axes on the owncraft. Image expansion (pixels/second)

is the intruder‟s one-dimensional growth in the image. Image area expansion

(pixels2/second) is the intruder‟s two-dimensional growth in the image, which can

give greater resolution (as discussed in Section 3.1.3). We also develop the collision

determination algorithm whereby the owncraft decides whether to avoid the possible

upcoming collision. This addresses the first major research goal (see Section 1.3)

and represents a large portion of the novel contribution.

Measure

κ and μ

Is f(κ, μ) <

threshold?

Normal

Controller

Collision

Avoidance

System

What

Sector (κ )?

Manoeuvre

accordinglyYES

NO

Figure 5 – Collision avoidance system decision process



Figure 5 shows that the relative-bearing rate and image expansion are used as a

decision factor in whether the collision avoidance system (CAS) will manoeuvre.

The CAS makes a decision about the avoidance manoeuvre based on the sector of the

intruder (as discussed in Section 3.2.1).

3.1.1 Thresholding Technique for Collision Decision

To decide whether a collision is imminent, a thresholding technique is applied to

the test statistic for collision avoidance ASC later shown in Equation (3.15). This

system is depicted in Figure 5.

Figure 6 – Geometry of a conflict scenario evolving over time

3.1.2 Closest Point of Approach Distance

The first characteristic in an image that has relevance to the closest point of

approach (CPA) or miss distance, is the angular velocity of the centroid-image-of-

the-intruder across the FOV in the owncraft‟s compensated camera frame. This is

termed the relative-bearing rate of the intruder. When considering conflict scenario

geometry, both the Regan and Gray [98] and the Australian Transport Safety Bureau

[99] state that in terms of dynamic targets, a collision becomes imminent when the

κ

Rk+1

κ

Rk

Sk

Sk+1 μ

Tk+1

Tk

κk

κk+1



intruder stops moving in the inertial frame. This is shown in Figure 6. Where fov is

the camera field-of-view; a mid air collision (MAC) will occur if and only if

0, [ , ]fov fovt

(3.1)

However it is not just a MAC that needs to be avoided; a NMAC needs to be

reported to the authorities, so an ideal CAS would avoid colliding and not come

within 152.4m of the intruder (the defined NMAC zone). Aviation standards [16]

recommend for a CAS a minimum separation (CPA) of two aircraft always be more

than 152.4m away from each other.

Figure 7 – Miss distance relationships

Another characteristic that has direct impact on CPA distance is image expansion

. From Regan and Gray [98], the miss distance (CPA) is defined as:

/

/

I tn d

t

(3.2)

Where I n is an integer and d is from Figure 7. Also is defined as the angle

subtended by the intruder, thus t or is defined as the image expansion. It is

evident from Equation (3.2) that as / t approaches zero then so does the CPA

κk

Sk

Tk

d

μk



distance, making a NMAC more likely. Also, if t t then a NMAC is

likely to occur because the CPA distance would tend to zero. An important note is

that intruder size d is not known a priori for a conflict scenario.

3.1.3 Time to Collision and Image Expansion

Hoyle [100] presented a description appropriate for collision geometry. Time to

collision TCT , also known as time to pass (for the non-colliding case), is:

/

TC

RT

R t

. (3.3)

Where R is the range to the intruder. Using the small angle approximation

tan , Regan and Gray [98] showed that:

/

TCTt

(3.4)

Figure 8 – Image plane characteristics

The angle subtended by the intruder is a one-dimensional value. However, it

is expected that more accuracy (higher detail) could be attained from the two-

dimensional equivalent, termed angular-area subtended . Thus a greater resolution

R

d κ

κ

d

R



or estimation would be achieved for TCT using . This value for would be easily

measured in images as they are planar (two dimensional).

If an image pixel is considered to have approximately a square relationship (from

Figure 8). Then the angular-area subtended by a pixel is:

2 (3.5)

It is shown (in APPENDIX C) that the relationship between TCT and this

angular-area subtended is:

2

TCTt

(3.6)

Figure 9 – Geometry of a conflict scenario

3.1.4 Collision Determination Algorithm

Most of the collision avoidance approaches so far have used range kR in the

control law for ensuring that the miss distance is always larger than 152.4m or some

other threshold. In this research, we propose the use of passive-only sensors,

therefore knowledge of kR is not directly observed. Thus, we will determine a

dimensionless test statistic using conflict scenario image characteristics (time to

collision TCT and relative bearing rate k ) to ensure that 152.4kR m.

κ

S

κ

T

σS σR

R

T



From Figure 9 it is known that for R , the corresponding is:

tanT

S (3.7)

Using small angle approximation, which holds true for incremental time steps

T

S (3.8)

Inside this small angle approximation T is constant (as reflected in Figure 9), and

S R (3.9)

Thus,

1

R (3.10)

Taking the time derivative of (3.10)

2

1

t R

(3.11)

From (3.3) and utilizing that /R t is constant or approaching a constant for

conflict scenarios,

TCT R (3.12)

Equations (3.1) and (3.2) showed that risk of collision increases (CPA distance

decreases) as approaches zero. It is therefore intrinsic that risk of collision also

increases as TCT approaches zero, particularly if 0 . For a thresholding technique

in accordance with Figure 5, we could find a dimensionless test statistic ASC by

relating (3.11) and (3.12),

( )AS TCC f T (3.13)

2

2

1ASC R

R (3.14)

2

AS TCC T (3.15)

Where the inputs of (3.15) are directly determined from the image using (3.16)

and (3.17), i.e. and , respectively.



1k k k (3.16)

1k k k (3.17)

Where k is derived later in Equation (4.32).

3.2 Avoidance Manoeuvre

In this sub-section, we look at what the owncraft should do when it has decided

that a collision is imminent. Using image-based features, what is the best possible

avoidance manoeuvre? This addresses the second major research goal (see Section

1.3).

3.2.1 Background

From literature and aviation practice, the avoidance manoeuvre is typically

determined by the intruder‟s position, i.e. the relative bearing decides the

manoeuvre direction [30, 101]. This is driven by aviation regulations about which

aircraft has right-of-way for given scenarios, which are all position-based (ICAO

[102], FAA [103], CASA [104]). These aviation rules have exceptions to them,

depending on whether the intruder is unpowered or comparatively unresponsive.

Because the image-based collision avoidance system proposed in this research is

unable to discriminate the responsiveness of the intruder, the owncraft gives-way for

all encounter scenarios.

It is important to mention that aviation specifications [16] state that a CAS should

have a horizontal field of view of ±110˚ and a vertical field of view of ±15˚ as

reflected on Figure 10. Intruders in the rearward sector are considered overtaking

and thus they must give-way.

Authors have taken the above specifications and safety regulations and proposed

a method for avoidance based on sectors. Voos [30] and Sislak et al. [101] use this

sector based technique for their avoidance manoeuvre. Note from Figure 10 that

Sector 1 has no particular size. In addition, Figure 10 illustrates our particular CAS

sensors, which have a field of view (FOV) of 60˚. Although the image expansion of



Section 3.1.3 is measured from this camera, it is proposed to use a third gimballed

camera with a narrow FOV (e.g. 10˚ x 8˚) to get more accuracy in the TCT

information of Equation (3.6).

Voos [30] and Sislak et al. [101] separate the sectors according to:

Sector 1 – Oncoming Intruder – Each aircraft should alter course to the

right.

Sector 2 – Rearward Intruder – The intruder must give-way.

Sector 3 – Starboard Intruder (right-hand) – The owncraft must give-way.

Sector 4 – Portward Intruder (left-hand) – The intruder must give-way.

Figure 10 – Collision avoidance right-of-way sectors

These sectors represented here do not account for the previously mentioned

exceptions to the general give-way policy. In this thesis, the intruder detection

system does not identify the type of intruder, so all intruders will be treated as

comparatively unresponsive. Thus, regardless of the intruders sector or

responsiveness, the onus is on the owncraft to manoeuvre.

Sector 1

Sector 2

Sector 3

Sector 4

110˚

60˚



3.2.2 Relative-Bearing Based Manoeuvre

The method that is implemented for manoeuvring in this thesis aligns with

aviation rules and is similar to the previously discussed work of Voos [30] and Sislak

et al. [101]. The avoidance manoeuvre is made based upon the relative bearing

of the intruder. However, as Sector 1 of Figure 10 is half the FOV of our vision

sensor, it can introduce some problems to the developed method, increasing collision

risk unnecessarily. Thus, in this work Sector 1 is divided between Sector 3 or Sector

4 respectively. The owncraft will turn to the right ˚ (tau degrees) if the intruder is

on the right or it will turn to the left ˚ if the intruder is on the left, in accordance

with:

1 (C C

k k signum (3.18)

Figure 11 – Typical encounter scenarios

Where C

k is the commanded heading at time k .

For the encounter scenarios in Figure 11, Equation (3.18) would cause a right

hand turn because is positive (intruder is on the right). The results using Equation

(3.18) are shown and discussed in Section 5.3. TABLE I reflects the algorithm in

which this avoidance manoeuvre is implemented.

(a) (b) (c) (d)



TABLE I ALGORITHM FOR AVOIDANCE MANOEUVRE

Avoidance Manoeuvre Algorithm

max/ (16 seconds for Flamingo)

IF intruder not passed

IF ASC < threshold & Not in a turn (setturn==0)

setturn=1

Turn for time

ENDIF

IF CAS < (threshold + buffer) & In a turn (setturn==1)

Maintain turn for extra time

ENDIF

ENDIF



4 Modelling & Simulation

In this chapter, we present the mathematical development behind the UAS model,

control and the CAS sensor configuration. We also describe the architecture of the

simulation environment that is developed to validate the proposed CAS.

4.1 UAS Model

We model the UAS used in the image-based collision-avoidance simulation-

environment (IBCASE), to verify the collision determination and avoidance

manoeuvre developed in this thesis. We develop the equations of motion represented

by Equations (4.11) - (4.15).

Figure 12 – Owncraft model used to define linear and angular variables

Our dynamic model is based on a nonlinear 6-dof rigid-body dynamic model

[105]. We have used the aerodynamic coefficients for a Silvertone Flamingo UAS

(owncraft) [37], shown in Figure 12. Our simulation environment emulates this

model, given that this platform represents the experimental test-bed for future flight

xb-axis zb-axis

yb-axis

L, p

M, q

N, r

β

α Vt

Z, w X, u

Y, v



trials. However, it could easily be adapted for any UAS given coefficients and other

relevant data.

TABLE II NOMENCLATURE

Nomenclature

Quantity Axis Units

Velocity, angle-of-attack, sidelslip ,tV m/s, rad, rad

Aerodynamic force components (body) , ,X Y Z N

Aerodynamic moments (body-axes) , ,L M N N∙m

Translational velocities (body-axes) , ,u v w m/s

Angular velocity (body-axes) , ,p q r Rad/s

Euler Angles (roll, pitch, yaw) Rad

Position (earth-axes – NED) , ,e ex y h m

Aileron, elevator, rudder, throttle , , ,ail el rud th

Engine thrust, owncraft mass ,T m N, kg

Wing area, chord, span , ,S c b m2, m, m

Dynamic Pressure, gravity ,q g Pa, m/s2

4.1.1 Owncraft Coefficients

The particular aerodynamic coefficients and other relevant data are presented in

APPENDIX A. TABLE II shows the nomenclature used for the following sections.

The equations of motion use body-axes coefficients from Equation (4.2) instead of

wind-axes coefficients ( , , )D Y LC C C . These coefficients vary dynamically with

respect to state input ( , , , , , )tV p q r and control surface deflections‟

( , , , )ail el rud th :



0

0

2

2qL L L L

t

D D L

cC C C C q

V

C C kkC

(4.1)

sin cos

2

cos sin

r r

X L D

Y Y Y rud Y

t

Z L D

C C C

bC C C C r

V

C C C

(4.2)

0

2

2

2

r a p r

e q

r a p r

l l l rud l ail l l

t

m m m m el m m

t

n n n rud n ail n n

t

bC C C C C p C r

V

cC C C C C q C

V

bC C C C C p C r

V

(4.3)

4.1.2 Atmospheric Model

The dynamic pressure q (Pa) is obtained from pressure, P (Pa) at altitude h (m):

0

0

1

air

g apse

g M

R LaspeL h

P PT

(4.4)

TABLE III ATMOSPHERIC MODEL VARIABLES

Atmospheric Model Variables

Quantity Symbol Value Units

Gravity g 9.80665 m/s2

Air pressure @ 0m (STP) 0P 101 325 Pa

Temperature @ 0m (STP) 0T 288.16 ˚K

Idel gas constant gR 8.31447 J/(mol∙˚K)

Molar mass of air airM 0.0289644 kg/mol

Lapse rate apseL 0.0065 ˚K/m



Then density (kg/m3) at that altitude is:

0( )

air

g apse

P M

R T L h

(4.5)

Thus dynamic pressure is:

21

2tq V (4.6)

0

0 2

0

1

2 ( )

air

g apse

g M

R Laspe

air

t

g apse

L hM P

Tq V

R T L h

(4.7)

TABLE III shows the particular variables used to calculate Equation (4.7).

4.1.3 Navigation Equations

The owncraft navigation equations are defined in the flat-earth north-east-down

(NED) axes [105, 106]. These equations assume a stationary centre of gravity (CoG)

with constant mass and uniform gravitational field. They also ignore the rotational

forces of engine, i.e. 0engh .

sin

cos sin

cos cos

X

Y

Z

qSC Tu rv qw g

m

qSCv pw ru g

m

qSCw qu pv g

m

(4.8)

cos cos

sin

sin cos

t

t

t

u V

v V

w V

(4.9)

2 2 2

1

1

tan

sin

t

t

V u v w

w

u

v

V

(4.10)



2 2

2

2 1

t

t

t t

t

t

uu vv wwV

V

uw wu

u w

V v vV

vV

V

(4.11)

The time derivative of the translational states , ,tV in Equation (4.11) are

calculated from Equation (4.8) and (4.9) using previous time instance values from

Equation (4.10) (starting with a trimmed condition i.e. 0 0tV u and

0 0 0 0 00 , , , ,p q r v w also with 0 0 0 0 0 00 , , , , ,p q r u v w ).

The moment Equations (4.12) and kinematic Equations (4.14) are:

1 2 4 3 4

2 2

5 7 6 7

8 2 9 4 9 .

eng l n

eng m

eng l n

p c r c p c h q qSb c C c C

q c p c h r c p r qScc C

r c p c r c h q qSb c C c C

(4.12)

Where the coefficients used by Equation (4.12) are in Equation (4.13)

2

2

1

2

3

4

5

6

7

2

8

9

/

/

1/

X Z ZX

Y Z Z ZX

X Y Z ZX

Z

ZX

Z X Y

ZX Y

Y

X Y X ZX

X

I I I

c I I I I

c I I I I

c I

c I

c I I I

c I I

c I

c I I I I

c I

(4.13)



tan sin cos

cos sin

sin cos

cos

p q r

q r

q r

(4.14)

Finally, the owncraft navigation equations are:

( ) (

( ) (

.

e

e

e

x uC S v C S S S C w C S C S S

y uS C v S S S C C w S S C C S

h uS vC S wC C

(4.15)

Where ,C S are for cosine and sine respectively. The owncraft‟s position

( , , )O

e e ex y h , and attitude (e , in the earth-axes are calculated by

Euler integration involving Equations (4.11) - (4.15). Please refer to TABLE IV on

coordinate reference frames for the appropriate meaning of the suffix subscript for

the images, intruder position and owncraft position.

TABLE IV REFERENCE COORDINATE FRAMES

Reference Coordinate Frames

Symbol Coordinate frame

imn Image frame of nth camera (n=[1,2])

c Common camera frame (relative as if one camera)

b Body fixed coordinate frame

e Earth fixed coordinate frame, on earth (NED)

N Earth fixed coordinate frame on UAS

comp Attitude compensated coordinate frame



4.2 UAS Controller

The control approach for the UAS is divided into inner and outer loops. Where

the superscript ( e ) is the error signal and ( C ) is the commanded signal and xyzP ,

xyzI ,

xyzD & xyzPff are the PID gains on control surface xyz that are shown in TABLE

XIII of APPENDIX A. The inner loops that stabilise the owncraft are:

Aileron from roll (Figure 13). This controller determines how much to

deflect the aileron ail by applying a proportional gain ailP to the error

signal between the measured bank angle and the commanded bank

angle C .

( )e

ail ailP P (4.16)

Figure 13 – Aileron from heading and roll

Rudder from sideslip (Figure 14). This controller allows for coordinated

turns. It applies a deflection directly to the rudder rud that is determined

by applying a feed forward proportional gain rudPff to the calculated

angle of sideslip .

rud rudPff (4.17)

Figure 14 – Rudder feed forward from sideslip (for coordinated turns)

rudPff

rud Owncraft

Dynamics

,

P - +

e

C

ailP - +

ail

C e Owncraft

Dynamics

,



Throttle for speed hold (Figure 15). This controller determines throttle

deflection th by applying a Proportional-Integral gain ( , )th thP I to the

error signal between the commanded velocity C

tV and the measured

velocity tV .

0

k

ke e

th th t th t

n

P V I V

(4.18)

Figure 15 – Throttle for airspeed hold

The outer loops are for guidance and navigation purposes. These are:

Altitude hold using elevator (Figure 16). This controller determines an

elevator deflection el by applying a Proportional-Integral-Derivative

(PID) gain ( , , )el el elP I D to the error signal between the commanded

altitude Ch and the measured altitude h .

1

0

( )k

e e e e

el el el k el k k

n

P h I h D h h

(4.19)

Figure 16 – Elevator for altitude hold

elI + -

eh

h

Ch

+ el

Owncraft

Dynamics

h , tV

+ +

elP

elD

t

+ -

e

tV

tV

C

tV +

th Owncraft

Dynamics

tV , h

+

thP

thI



Heading hold using roll (Figure 13). This part of the controller

determines the commanded bank angle C by applying a proportional gain

P to the error signal between the actual heading and the commanded

heading C .

( )C CP (4.20)



4.3 Camera Model

In this sub-section, we model the perspective-projection camera (CAS sensor)

onboard the owncraft with the geometry of the conflict scenario modelled relative to

the CAS sensor.

4.3.1 Configuration

We model the sensor onboard the owncraft combining two camera projective

models in one common axis (as shown in Figure 17). When the intruder is observed

in one of the cameras (either of the two black axes), it is projected onto the common

image frame, which is assumed to be forward looking (in red). It is in this common

image frame that the control avoidance law is defined.

Aviation standards [16] have stated that a CAS should have a ±110˚ field-of-view

(FOV) in the horizontal and ±15˚ FOV in the vertical. For the purposes of this

research, two 60˚ FOV cameras in the horizontal have been modelled as reflected in

Figure 17. We have followed this approach considering the following:

Price and convenience – High quality sensors with these specifications

are obtained cheaply and conveniently. In addition, they can be

interfaced appropriately with powerful processors [32]; in the case where

computation-intensive image processing is required. They are also

lightweight and power efficient [29].

Lens Calibration – The image plane and lens distortion in this

configuration is minimal when compared with omni-directional sensors

[107].

FAA Accident Prevention Program Report – FAA report for manned

aircraft [108] recommend pilots regularly scan ±60˚ FOV horizontally in

order to prevent a mid air collision (MAC). In keeping with ELOS

expectations, a similar approach could be considered appropriate for an

automated system.



4.3.2 Perspective Projection Model

In Figure 17, if an intruder is identified in one of the cameras using detection

algorithms [33, 109], the intruder pixel location im , ,I

n n nP u v f (and in which

camera the intruder is detected) is extracted and the CAS notified, which ultimately

will determine if collision is imminent and possibly proceed with the avoidance

manoeuvre. In Figure 17, cameras are [1,2]n and f is the focal length of the

cameras. n0im is the origin of each individual image frame and 0c is the origin of the

common camera frame after the transformation of Equation (4.23). is the relative

bearing or azimuth to the intruder and is the elevation.

Figure 17 – Two-camera perspective projection setup

If I

e is the position of the intruder in the earth-axes (NED) and O

e is that of

the owncraft, then ( , , )I

N N N Nx y z is the intruder‟s position wrt the owncraft

CoG, i.e.

.I I O

N e e (4.21)

Although im

I

nP is obtained directly from the image, its relationship in the object

space is:

ψ

υ

-υ

xb

yb

zb

ze(D)

ye(E)

xe(N)

xc

yc

zc

yim1

yim2

xim1

xim2

zim1

zim2

f

f

f

(u,v) κ λ

IΠe(xe,ye,ze)

CoG bTc(x)

bTc(y)

R

0im1

0im2

0c



im

( )

( )( ) ( )

( )

I

n N N

I I I

n n N N NI I

N N I

N N

u xf f

P v yz z

f z

(4.22)

In this context, TABLE IV shows the subscript notation. Equation (4.23) rotates

the camera (rad) through the initial y-axis, where / 2fov . For the image

plane of the thn camera, the intruder position I

cP is now

im( )I I

c y nP P (4.23)

For the purposes of this thesis, the rotations through angle a wrt to axes ( , , )x y z

are:

1 0 0

( ) 0 cos sin

0 sin cos

cos 0 sin

( ) 0 1 0

sin 0 cos

cos sin 0

( sin cos 0

0 0 1

x

y

z

a a a

a a

a a

a

a a

a a

a a a

(4.24)

This sub-system subsequently transforms im

I

nP of the two rotated cameras to the

image plane of one forward-looking camera I

cP (red axis and image plane of Figure

17). Then all control law is developed on I

cP . The intruder‟s position in the body-

axes I

bP is then,

I b b I

b c c cP T P (4.25)

The cameras‟ focal point wrt the UAV CoG (translation) in the body-axes is b

cT .

If Tb

cT , then , and are typically < 2m, which is seemingly

insignificant compared to the distances the intruder are at and can be ignored [110].

Thus, Equation (4.25) simplifies to



I b I

b c cP P (4.26)

For the purposes of this simulation, the UAS configuration of Figure 17 rotates

2 2

b

c z x

(4.27)

Thus the intruder pixel location rotated through to the UAV CoG I

NP is

I e I

N b bP P (4.28)

The intruder‟s position needs to be monitored with the UAV motion compensated

for, such that relative bearing and azimuth are measured irrespective of the

UAV‟s behaviour. Because the NED earth-axes align with the UAV body-axes, the

only compensation considered necessary will be the Euler angles (attitude) of the

owncraft. In addition, it is not necessary to compensate for heading, but rather

heading changes, i.e. a north-always pointing CAS sensor is not necessary.

Therefore

0( ( )) ( ( )e

b z y x (4.29)

0 is the heading at the point where the intruder is first detected. Thus, the

motion compensated location of the intruder I

compP , wrt to a wings-level, original

heading is:

I c I

comp b eP P (4.30)

I c e b I

comp b b c cP P (4.31)

Then,

1

1

( )tan

( )tan

I

comp comp

I

comp comp

P x

f

P y

f

(4.32)

For now will be neglected and this thesis will concentrate on using . In

accordance with Equation (3.17) the relative bearing rate of the intruder is:

1k k k (4.33)



Figure 18 – Image of intruder as seen, without compensation, in the camera frame (top) and with

motion compensation (bottom) as calculated. The units are wrt the focal length in millimetres.

Figure 18 shows the evolving image of an intruder over time (40 seconds). The

intruder is first detected (green circle) on the right (with 041 ) and the aircraft

banks right and alters heading 20˚ right, considering that horizontal FOV is ±60˚. In

the uncompensated image on top, one can see the aircraft bank right and the intruder

pan right on the owncraft.

-6-4-20246

-2

-1

0

1

2

-6-4-20246

-2

-1

0

1

2

Image in motion compensated camera - I

compP

Image in camera - I

cP

(mm)

(mm)

(mm)

(mm)



4.4 Simulation Environment

In this sub-section, we describe the image-based collision-avoidance simulation

environment (IBCASE) that is developed using the models of Section 4.1-4.3 to

validate the image-based collision-avoidance control law of Section 3.

The architecture of the simulation environment is reflected in Figure 19. It is

designed to be generic and adaptable for any CAS sensor configuration or on any

UAS. IBCASE is implemented using MATLAB.

The system is divided into three main components.

The UAV emulator propagates the UAV throughout time using the

equations of Section 4.1.

The conflict scenario emulator generates the trajectory of the intruder.

The vision system emulator generates what the image would be onboard

the owncraft.

4.4.1 The Vision System Emulator

The vision sensor simulator is easily adapted for different CAS sensor

configurations via the CAS sensor configuration block. This vision sensor simulator

generates a motion compensated image I

compP using the known trajectory of the

intruder from the intruder trajectory generator and the attitude of the owncraft e

from the navigation equations block. It outputs to the collision avoider what the

intruder detection system (from Figure 2) would generate, in terms of a pixel location

for the intruder. The vision sensor simulator is developed in Section 4.3.2.

4.4.2 The Conflict Scenario Emulator

The intruder trajectory generator propagates the track of the intruder based on

various user inputs that are in the conflict scenario setup block. These inputs are

time of simulation, intruder speed, random start and stop positions etc. The conflict



scenario emulator used in the experimentation for validating this CAS is discussed in

Section 5.2.

VISION SYSTEM EMULATOR

UAV EMULATOR

UASCoefficients

NavigationEquations

Collision Avoider

UAS Controller

CONFLICT SCENARIO

EMULATOR

CAS SensorConfiguration

Vision Sensor Simulator

Conflict ScenarioSetup

Intruder TrajectoryGenerator

Figure 19 – IBCASE (simulator) architecture

4.4.3 The UAV Emulator

The UAV emulator uses the UAS coefficients of APPENDIX A in the navigation

equations block. The navigation equations block propagates the owncraft

throughout the conflict scenario using the equations of Section 4.1. It operates in

conjunction with the UAS controller block.

The UAS controller block has all the proportional-integral-derivative (PID)

controller gains that are tuned to give the owncraft an appropriate (realistic)

response. For the Flamingo UAS operating at 25Hz, these PID gains are in TABLE

XIII of APPENDIX A. The control law is prescribed in Section 4.2.

Inside the collision avoider is the novel contribution of this research (see Section

3). It determines whether to make an avoidance manoeuvre. If it chooses to



manoeuvre, then alternate desired heading and speed commands are issued to the

UAS controller.

The final responsibility for stability rests inside the UAS controller and is

therefore outside the scope of the collision avoidance simulator. However, the open

loop stability of the Flamingo is analysed and presented in APPENDIX B.

4.4.4 Simulator Adaptability

The simulator is designed for easy adaptation of different aircraft, types of

conflict scenarios and CAS sensor configurations. The yellow blocks can be

interchanged to vary the experiment:

The UAS coefficients block can be substituted with data for any UAS, the

data used in this simulation is based on the Flamingo UAS from

Silvertone [37]. The Flamingo data is found in APPENDIX A.

For each new UAS coefficients block a corresponding UAS controller has

to be defined that has the appropriate PID gains. The gains for the 25Hz

Flamingo model are in TABLE XIII of APPENDIX A.

Also adaptable is the CAS sensor configuration block; one can easily

redefine resolution, field-of-view, number of cameras, image sensor size

etc.

In addition, the conflict scenario setup block can be interchanged for

different experiments testing various types of conflict scenarios. The

conflict scenario setup block used in this thesis is described in Section

5.2.






5 Results and Analysis

In this chapter, we discuss performance metrics and present the experiment that

tests the proposed image-based CAS. We display the results, and then present

analysis and discussion.

5.1 Performance Analysis

In this sub-section, we discuss existing measures and encounter models for

assessing the performance of collision avoidance systems. There are three different

metrics that we use to characterise the performance of the developed CAS. The first

measure uses Standard Operating Characteristics (SOC) curves developed by Kuchar

[111], which are assessed over a range of ASC thresholds (the test statistic developed

in this thesis). The second method assesses Risk Ratio, which is a measure of UAS

performance in a conflict scenario (NMAC) with and without the CAS [112]. In the

third method we assess the CAS using Dalamagkidis et al. [10] ELOS expectations

for a collision scenario (MAC) at a nominal ASC threshold. The ASC test statistic

that is chosen for ELOS expectation performance measuring is the one that has the

lowest Risk Ratio (10). These three methods of displaying the CAS results

endeavour to benchmark the performance of the developed CAS against existing

systems.

5.1.1 Encounter Models

Before we start to discuss performance measures, it is important to understand

encounter models. Comprehensive encounter models gather radar and surveillance

data in a given NAS and analyse it to generate realistic encounter scenarios for CAS

testing in a simulation environment [113]. Models have evolved over the last few

decades and today incorporate non-cooperative encounter data [114] (known as

uncorrelated models). Perhaps the most comprehensive model is the Lincoln Labs



model [115]. It uses more than 420 000 encounters from 127 different radar sites in

the US, and learns the airspace encounter model using dynamic Bayesian networks.

This model can then generate random conflict scenarios that are statistically

representative of actual encounter scenarios of the NAS, with a known expected rate

for each encounter type. These scenarios are then used to validate CAS‟s in

simulation.

To undertake development of a comprehensive encounter model is beyond the

scope of this research, instead we use a Monte Carlo simulation with a comparatively

simplistic encounter scenario (detailed in Section 5.2). For the expected rate of

occurrence for a mid air collision MACE (MACs/flight hour) we use the Class E

airspace worst-case statistic from Weibel and Hansman [9].

5.1.2 Performance Measures

5.1.2.1 Standard Operating Characteristic Curves

There are two measures for characterising the performance of a CAS [111]. The

first is the success rate of the system, i.e. the probability that the UAV will avoid a

conflict scenario given that a conflict scenario is inevitable ( SAP and CDP from

below). The second is the false alarm rate i.e. the probability that the UAV will

attempt to avoid a collision when there is no conflict scenario ( UAP and FAP from

below). These probabilities change with various ASC thresholds. Adjusting the CAS

sensitivity to increase success rate will consequently increase the false alarm rate.

To capture this information, Kuchar [111] uses standard operating characteristic

(SOC) curves. These are an adaptation from signal detection theory [116] where

they are used to detect signals amongst background noise at various thresholds

(known as receiver operating characteristic curves). In these plots, Kuchar represents

the probability of correct detection CDP against the probability of a false alert FAP .

An example plot that displays the line-of-little-benefit is shown (dash-dot line) in



Figure 20. If a result occurs under the line-of-little-benefit then the false alert rate is

higher than the success rate, thus the system is of little benefit.

Figure 20 – Example of a standard operating characteristics curve [111]

Winder and Kuchar [117] break down these probabilities ( CDP and FAP ) to reveal

all the various possibilities in an encounter. TABLE V represents the various

possibilities for the outcomes illustrated in Figure 21.

TABLE V POSSIBLE OUTCOME CATEGORIES [115]

Category Abbreviation Alert

Necessary?

Alert

Issued?

Conflict

Occurred?

False Alert FA

Induced Conflict IC

Correct Avoidance CA

Late Alert LA

Missed Detection MD

Proper Rejection PR

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1Example Standard Operating Characteristic Curve

PFA

PC

D



Figure 21 – Possible outcomes for UAS with collision avoidance system [115]

Kochenderfer et al. [115] use SAP and UAP for their SOC curves. Although we

have displayed the results in this manner, we have also chosen to plot the SOC

curves using CDP and FMP . The various probabilities are defined:

1. Probability of Conflict ConP

Con

IC LA MDP

FA IC CA LA MD PR

(5.1)

2. Probability of Alert AlertP

Alert

FA IC CA LAP

FA IC CA LA MD PR

(5.2)

3. Probability of Satisfactory Alert SAP

SA

FA CAP

FA IC CA LA MD

(5.3)

4. Probability of Unnecessary Alert UAP

UA

FA ICP

FA IC CA LA MD

(5.4)

conflict region

NMAC

alert region FA

IC

CA

PR

RF

A

LA

MD



5. Probability of Conflict with No Action CNAP

CNA

CA LA MDP

FA IC CA LA MD PR

(5.5)

6. Probability of Correct Detection CDP

CD

CAP

CA LA MD

(5.6)

7. Probability of False Manoeuvre FMP

FM

FA ICP

FA IC PR

(5.7)

In Figure 21, the conflict region is defined by a cylinder of radius 152.4m, the

NMAC radius around the owncraft. The alert region is the region wherein it is

possible for the system to alert i.e the false detection region. The radius of the alert

region increases as the sensitivity of the CAS is increased (by increasing the ASC

threshold). In this research it was found that for the highest ASC threshold tested

(16), no manoeuvres were triggered when the aircraft did not have a CPA distance

greater than 1km (when tested 10 000 times). Thus the alert region has a radius of

1km and hence 1upperY km of Section 5.2.1.

5.1.2.2 Risk Ratio

Risk Ratio is a measure that has been traditionally used to assess performance of

a TCAS [112]. RiskRatio is the probability that a NMAC will occur with a CAS

against the probability it will occur without the CAS. Lincoln Labs [111, 115] and

Eurocontrol [112, 118] have published TCAS/ACAS RiskRatio results.

NMACwithCAS

NMACwoCAS

PRiskRatio

P (5.8)

Using TABLE V would give:

IC LA MD

IC FA CA LA MD PRRiskRatioCA LA MD

IC FA CA LA MD PR

(5.9)



IC LA MD

RiskRatioCA LA MD

(5.10)

RiskRatio is assessed over the alert region as shown in Figure 21. Risk Ratio

consists of two components [111], one that is due to the Induced Conflicts ICRR , and

another from the unresolved component unresolvedRR i.e. where avoidance failed.

IC unresolvedRiskRatio RR RR (5.11)

IC

ICRR

CA LA MD

(5.12)

unresolved

LA MDRR

CA LA MD

(5.13)

The current TCAS results (from a correlated model, that is using scenarios where

coordinated avoidance manoeuvres takes place) for RiskRatio from ICAO is 3.3%

[112] and within FAA is 5.5% [115]. For the uncorrelated model (uncooperative

scenarios, relevant to this research) the TCAS figures from ICAO are 22.9% [112]

(with ICRR at 13.7%) and 23% in America [119].

5.1.2.3 ELOS expectations

In Section 1.1, we discussed how UAS are expected to have an equivalent level

of safety (ELOS) to that of manned aircraft in order to enable/facilitate free

integration into the NAS [8-11, 25, 45, 120]. The National Transportation Safety

Board (NTSB) published figures for the probability of fatalities in piloted aircraft are

6 110FatalityP hr , but a more conservative figure like 7 110FatalityP hr should be

expected [28]. According to Dalamagkidis et al. [28] it is reasonable to assume that

for UAS, a mid air collision will result in a human fatality. Then from NTSB data

from 1983 to 2006 the probability of a MAC, 7 110MACP hr is proposed for UAS

[10, 28].

On the other hand, Eurocontrol use 83 10MACP [112]. For the NMAC case,

Eurocontrol use 73 10NMACP [112] and

71.7 10NMACP [118]. Kuchar and

Drumm [119] confirm these MAC rates. Weibel and Hansman [9] use a lower MACP



rate because it is based on class E airspace data. We will use the figures of Weibel

and Hansman [9] for our assessment because Class E airspace is similar to Class G

airspace, which is where this CAS is designed to operate.

Weibel and Hansman [9] use cooperative data in the Endoh aircraft collision gas

model [121] for encounter modeling, to define the expected number of collision

scenarios an owncraft is likely to encounter MACE ( /MACs hr ). An expected

collision occurs if the exposure volume overlaps with the UAV, the expected number

of collisions is equal to the ratio of total collision volume to the volume of airspace

[9]. Using the NTSB data, Weibel and Hansman [9] determine a figure for

54 10 /MACE collisions hr (at FL370). A worst-case conservative estimate of

410 /MACE collisions hr is proposed [10].

Dalamagkidis et al. [10] further develop Weibel and Hansman‟s [9] formula to

include that an owncraft that has a CAS may manoeuvre and avoid a collision,

known as the Risk Ratio that pertains to the MAC case MACRR .

MACwithCASMAC

MACwoCAS

PRR

P (5.14)

If the conflict region of Figure 21 is defined as a MAC (please see definition in

Section 1.2.2), then

MAC MAC MACP E RR (5.15)

Putting the proposed MACE back into Equation (5.15) will give:

310MACRR (5.16)

This estimate is based on Class E airspace and the same assumptions cannot be

made for Class G airspace because one is not able to monitor the traffic in Class G

[10].



5.2 Experiment Setup

In this section, we outline the system aim and the experiment, detailing the

objectives, scope and limitations of the simulation.

In this research we aim to prevent a near mid air collision (NMAC) laterally.

This means that the above controller is designed to make sure that the two

encountering aircraft stay more than 152.4m away from each other laterally, at all

times. Although we are aiming at preventing NMACs, we also assess the MAC

performance.

The objective of the proposed image-based CAS is to use an image-based sensor

to detect and avoid a conflict scenario, without inferring range. In order to validate

the performance of the proposed image-based collision avoidance system there are

two principle objectives that drive the two experiments. These are:

1) To see how successful the CAS is at avoiding a conflict scenario. This is

called the success rate experiment and from this experiment we will get SAP

and CDP .

2) To see how often the CAS performs an unnecessary avoidance manoeuvre,

and what is the outcome? This called the false alarm experiment and we will

obtain ConP , AlertP , CNAP , FMP and UAP .

5.2.1 Monte Carlo Simulations

The overall Monte Carlo simulation was run 50 000 times for both experiments

described above. These experiments make sure that at some random point mX , in

the owncraft‟s straight two-minute voyage, that an intruder will come within mY

metres of the owncraft. A small selection of 50 intruder tracks (green thin lines) is

shown in Figure 22. In Figure 22, the owncraft (thick blue line) is not attempting to

manoeuvre out of the way. The circles represent the beginning of the tracks.

This experiment operated within the following scope:



1. The point mX , of the two minute voyage is random and between 15 seconds

and 120 seconds. It has a normal distribution.

a. The bottom bound of 15 seconds is implemented because:

i. In the first 3 seconds nothing happened apart from owncraft

stabilisation and trimming;

ii. the next 12 seconds, comes from the fact the UAS are

expected to perform with an ELOS to manned aircraft [8, 25,

45, 120] and the literature states that a pilot takes about 12.5

seconds to detect and react [122]. Therefore, an unmanned

system would not be expected to avoid anything in less than

12 seconds.

b. The top bound of 120 seconds is a reasonable figure that is used; it

had no real significance, other than to affirm that few real life conflict

scenarios would take longer than two minutes from first detection

until passing, to play out. Some encounter models assess one minute

collision scenarios [113].

2. The value for mY had a lower bound lowerY and an upper bound upperY (metres).

a. For Experiment 1; 0lowerY m and 152.4upperY m .

b. For Experiment 2; 0lowerY m and 1upperY km .

3. The intruder had a random straight path. It had a normally random

distribution for the beginning and end-points of the intruder‟s track. The

following were applied to the intruder‟s path:

a. The intruder‟s maximum velocity is 250 KTAS (Airspace E, G rules).

b. The intruder began within the field-of-view of the CAS sensor (in our

case ±60˚ horizontal and ±23.4˚vertical forward-looking field-of-

view).

c. The intruder did not start within 1km of the owncraft. Effectively this

meant that the intruder is larger than 0.82m, because of the

implemented pixel resolution and our intruder detection system,

which is sub-pixel in nature [33].



Figure 22 – Random selection of intruder tracks encroaching owncraft

5.2.2 Limitations and Assumptions

The Monte Carlo simulations described above are only a preliminary experiment

to evaluate the performance of the proposed avoidance algorithm. Some of the

assumptions and limitations of these simulations are:

It does not consider the azimuth of the intruder and thus disregards altitudinal

aspects of manoeuvring in a three dimensional manner. Because of the

comparative responsiveness of the altitude controller and the smaller

restrictions on separation distance (100 feet or 30.5 m), it is expected that a

-2000

-1000

0

1000

2000

3000

4000

5000

6000

7000

8000

-5000 -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 5000

NO

RT

H (

m)

EAST (m)



3D version of this image-based collision avoidance system would have better

results, perhaps even guaranteeing the separation distances of a NMAC zone.

It only looks at detection of intruders with straight trajectories. It could be

expanded to include looking at intruder‟s with curved trajectories

It does not detect collisions whilst the owncraft is manoeuvring. That is, if

the owncraft goes into a manoeuvre, the part that is monitoring the test

statistic ASC , to see if it drops below the threshold, stops making decisions

until the owncraft has returned to level flight.

It does not actually monitor the complete ±110˚ horizontal FOV

recommended by aviation standards [16].

There are also the limitations of the actual sensor in terms of all weather

performance. Image-based sensors, whether vision or infrared, do not

perform well in cloudy conditions [29].

5.3 Results and Analysis

5.3.1 CAS Threshold Determination

A ASC threshold needs to be determined in order to assess a particular

configuration for ELOS expectations (as discussed in Section 3.1.1). In order to find

a reasonable value, we took the threshold that had the lowest RiskRatio . For the

CAS developed in this thesis, 10ASC . It was noticed that as the ASC threshold

increased (more than 10), the IC component increased as well.

Figure 23 shows the distribution for the minimum value of the ASC test statistic

( min( )ASC ) over 10 000 simulations that were all inside the perimeter of the NMAC

region.



5.3.2 Observations and Behavioural Patterns

When the algorithm of TABLE I is implemented using of 16 seconds, we

observe three behavioural patterns. The first, termed a single manoeuvre, is an

owncraft that performs an avoidance manoeuvre but returns to the original heading

immediately. The second behavioural pattern, called a maintained manoeuvre, turns

the owncraft onto the altered heading 0( ) and when it achieves this new heading,

continues to maintain it for seconds ( x number of times) before returning to the

original heading. The third type of behaviour observed, called a repeated

manoeuvre, sees the owncraft perform an avoidance manoeuvre and then

immediately return, however, it performs a subsequent manoeuvre because the ASC

threshold is again violated (risk of NMAC deemed high enough by CAS).

Figure 23 – Distribution of min(CAS) for experiment 1

The 120-second tracks of Figure 24, Figure 26 and Figure 28 show an owncraft‟s

original route (red dashed) that would have encountered an intruder (green dotted)

with an original closest point of approach (green square). The owncraft‟s avoidance

route is shown (blue solid) with the new closest point of approach (blue star). The

circles show the start of either aircraft‟s time track.

0 5 10 15 20 25 300

50

100

150

200

Histogram of CAS

No.

of

Occure

nces

min(CAS

)



The ASC behaviour plots of Figure 25, Figure 27 and Figure 29 displays the ASC

(solid red line) with the thresholds shown here to be at ±16 (straight black lines).

Because the ASC displays unstable behaviour during the avoidance manoeuvre, the

last accurate reading is maintained (blue dotted line) until the new heading

1( )c c

k k is attained.

Figure 24 depicts a single manoeuvre; it has the track of an owncraft that has

turned onto the new avoidance heading and upon achieving it, has deemed the

intruder as no longer a risk and immediately returned to the original heading. The

associated ASC plot is shown in Figure 25. From Figure 25, one can note the

avoidance manoeuvre is made at 3 seconds and initiates return almost immediately at

19 seconds, because it is then out of the threshold region.

Figure 26 illustrates a maintained manoeuvre; it is an example of an owncraft

that has detected an intruder, which has triggered an avoidance manoeuvre but then,

once on the new heading, the ASC is still under the threshold (at around 18 seconds),

so the new heading is maintained for (around 36 seconds). Figure 27 shows the

ASC behaviour of Figure 26 for the first 60 seconds. One can see the avoidance

manoeuvre is made at 3 seconds and the new heading is achieved at about 18

seconds. However, the ASC threshold is still violated, so it maintains the heading for

and initiates return at around 36 seconds, achieving original heading around 53

seconds.

Figure 28 shows the track for a repeated manoeuvre. This type is where an

owncraft has manoeuvred and then deemed that it is safe to return to the original

heading (because ASC is under a nominal threshold), however upon recovering the

original track, the intruder again violates the ASC threshold and the owncraft repeats

an avoidance manoeuvre. This can happen multiple times, although more than three

successive manoeuvres were very rare. Figure 29 illustrates the ASC behaviour. It

shows where the manoeuvre is triggered at 3 seconds and again at about 36 seconds.

The returns are triggered at around 19 seconds and at 50 seconds.



Figure 24 – Example of scenario where an owncraft reaches new heading and immediately returns to

the original heading. This is an example of a single manoeuvre.

Figure 25 – CAS behaviour for first 60s of Figure 24 track. The CAS test statistic (red line) is between

thresholds (±16) therefore a manoeuvre is made (3 secs). The CAS is maintained at the last stable reading

(blue dotted line) during the manoeuvre. At the new heading, it is deemed safe to return to the original

heading (20 secs), where the stable CAS is held (blue dotted line) until on original heading.

0

500

1000

1500

2000

2500

3000

-1000 -500 0 500 1000 1500 2000 2500 3000

NO

RT

H (

m)

EAST (m)

0 10 20 30 40 50 60-50

0

50

100

150

CAS

Time (s)

CA

S



Figure 26 – Example of scenario where owncraft maintains new heading until θ seconds before

returning to original heading. This is an example of a maintained manoeuvre.

Figure 27 – CAS behaviour for first 60s of Figure 26 track. The CAS test statistic (red line) is between

thresholds (±16) therefore a manoeuvre is made (3 secs). The CAS is maintained at the last stable reading

(blue dotted line) during the manoeuvre. At the new heading, it is still not safe to return to original

heading (20 secs), so the current heading is maintained for ϴ time until another CAS reading decides it is

safe to return to original heading (36 seconds).

0

500

1000

1500

2000

2500

3000

3500

-1000 -500 0 500 1000 1500

NO

RT

H (

m)

EAST (m)

0 10 20 30 40 50 60-50

0

50

100

150

CAS

Time (s)

CA

S



Figure 28 – Example of scenario where owncraft avoids and returns to original heading, however CAS

threshold is violated a second time. This is an example of a repeated manoeuvre.

Figure 29 – CAS behaviour for first 70s of Figure 28 track. An avoidance manoeuvre is made at 3 secs

and then the CAS decision returns the owncraft to the original heading (19 secs). When the owncraft has

returned to the original heading a second manoeuvre is performed (36 secs) and returns again (50 secs).

0

500

1000

1500

2000

2500

3000

3500

-1000 -500 0 500 1000 1500 2000

NO

RT

H (

m)

EAST (m)

0 10 20 30 40 50 60 70-100

-80

-60

-40

-20

0

20

40

60

80

100

CAS

Time (s)

CA

S



5.3.3 Probabilistic Results

Figure 30 shows the distribution of CPA‟s for 10 000 simulations, where an

avoidance manoeuvre is triggered (because of ASC threshold) but unnecessary

(because CPA distance is greater than the NMAC distance), from experiment two. It

shows the distribution for what would have been the CPA distance before the CAS is

implemented, and the new CPA after the CAS manoeuvred the owncraft. Therefore,

these correspond to the Induced Conflicts and False Alerts (Figure 21). Notice the

Induced Conflicts, because in the left diagram there are no CPA instances under

152.4m, but there are some shown in the right diagram (after the CAS manoeuvres).

Figure 30 – False Positive distributions before and after CAS is implemented

In this instance ( 16ASC ), the probability that the CAS would manoeuvre

falsely FMP is 51.21%. This is a measure of the oversensitivity and it is assessed

over the entire spectrum of ranges where the collision detector could trigger a

manoeuvre (for 16ASC this has 1upperY km and 0lowerY m ). Note that FMP

increases as the ASC threshold increases. However, of those that unnecessarily

manoeuvred, on average the CAS would increase the CPA distance between the two

aircraft 480.1m, and thus increase safety.

0 200 400 600 800 10000

10

20

30

40

50

60Avoidance Manoeuvres with CPA > 152.4m

min(CPA) - (m)

No.

of

Occure

nces

0 500 1000 1500 20000

10

20

30

40

50

60

70After Manoeuvre - New CPA

min(CPA) - (m)

No.

of

Occure

nces



Figure 31 – A selection of Correct Avoidances made using implemented algorithm. (a) top left – left

intruder approach with maintained manoeuvre (b) top right – right intruder approach with single

manoeuvre (c) middle left – left intruder approach with single manoeuvre (d) middle right – right intruder

approach with single manoeuvre (e) & (f) bottom – right intruder approach with repeated manoeuvre.

0

2000

4000

6000

8000

-6000 -4000 -2000 0 2000

NO

RT

H (

m)

EAST (m)

0

1000

2000

3000

-2000 -1000 0 1000

NO

RT

H (

m)

EAST (m)

0

1000

2000

3000

-1000 0 1000 2000

NO

RT

H (

m)

EAST (m)

0

500

1000

1500

2000

2500

3000

-1000 0 1000

NO

RT

H (

m)

EAST (m)

-1000

0

1000

2000

3000

4000

0 2000 4000

NO

RT

H (

m)

EAST (m)

0

1000

2000

3000

4000

5000

-2000 0 2000

NO

RT

H (

m)

EAST (m)

(a)

(e)

(c) (d)

(b)

(f)



Figure 32 – Another selection of Correct Avoidances made using implemented algorithm. (a) top left

– left intruder approach with maintained manoeuvre (b) top right – right intruder approach with single

manoeuvre (c) & (d) middle – right intruder approach with single manoeuvre (e) & (f) bottom – right

intruder approach with repeated manoeuvre.

0

1000

2000

3000

4000

5000

6000

-4000 -2000 0

NO

RT

H (

m)

EAST (m)

0

1000

2000

3000

4000

5000

-2000 0 2000

NO

RT

H (

m)

EAST (m)

0

500

1000

1500

2000

2500

3000

0 1000 2000

NO

RT

H (

m)

EAST (m)

0

500

1000

1500

2000

2500

3000

-1000 0 1000

NO

RT

H (

m)

EAST (m)

0

1000

2000

3000

4000

5000

6000

-2000 0 2000

NO

RT

H (

m)

EAST (m)

0

1000

2000

3000

4000

5000

6000

-2000 0 2000

NO

RT

H (

m)

EAST (m)

(a) (b)

(c) (d)

(e) (f)



Figure 31 shows a selection of cases where the owncraft successfully avoided the

NMAC. Figure 33 shows a selection of instances where the controller failed to avoid

the conflict. In the top left illustration of Figure 33 is a case of Miss Detection, this

had a CPA ( )mY of 93.5m, whereas the other three are Late Alerts.

Figure 33 – Failed avoidance detection or manoeuvres according to TABLE V and Figure 21.

(a) top right – Missed Detection (b) top left – Late Alert (c) bottom left – Late Alert (d) bottom right –

Late Alert on a repeated manoeuvre.

5.3.4 Performance Results

5.3.4.1 Standard Operating Characteristics

Figure 34 shows the SOC curves using SAP and UAP of Equations (5.3) and (5.4)

from [115]. Alternatively, results that implement CDP and FMP of Equations (5.6)

and (5.7), from the original [111] and others [36] are shown in Figure 35. Shown on

0

1000

2000

3000

0 500

NO

RT

H (

m)

EAST (m)

-2000

-1000

0

1000

2000

3000

-2000-1000 0 1000N

OR

TH

(m

)

EAST (m)

0

1000

2000

3000

-3000 -2000 -1000 0 1000

NO

RT

H (

m)

EAST (m)

0

1000

2000

3000

-500 0 500

NO

RT

H (

m)

EAST (m)

(a) (b)

(d) (c)



these plots is the line-of-little-benefit (red dashed line). When the system is

operating below this line, more false alarms than satisfactory alerts are triggered.

Figure 34 – Standard Operating Characteristics (SOC) curve for CAS

As a basis for comparison, we have also shown a system that operates solely on

relative bearing rate ( ) – green dotted line. Also, seen in Figure 34, is that Induced

Conflicts increase as UAP increases, because SAP tends away from unity. This is not

reflected on the /CD FMP P plots in Figure 35. This is the main reason for using a

SOC curve with SAP and UAP .

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1Standard Operating Characteristic Curve

PUA

PS

A

Ideal

CAS

CAS

Relative Bearing

No benefit

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.8

0.85

0.9

0.95


PUA

PS

A



Figure 35 – SOC curve that displays original PCD and PFM

5.3.4.2 Risk Ratio

Displayed in Figure 36 are the results for RiskRatio (in blue solid lines) at

various thresholds (on the x-axis) for both ASC (top) and from the relative bearing

experiment (bottom). The Induced Conflict component of the Risk Ratio ICRR

from Equation (5.12) is shown as the red dash-dot line. From the top diagram of

Figure 36 one can see that 10 has the lowest RiskRatio . This is why it is chosen as

the threshold for ELOS analysis below in Section 5.3.4.3. From Figure 36, one can

see that at the chosen ASC threshold of 10, a 1.266%RiskRatio with

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8


PFM

PC

D

Ideal

CAS

CAS

Relative Bearing

No benefit

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.8

0.85

0.9

0.95


PFM

PC

D



0.596%ICRR is produced. Historically 10% of NMACs lead to MACs [123],

which can be seen in the MACP and NMACP Eurocontrol use [112].

Figure 36 – Risk Ratio results for CAS

When compared with the TCAS results for uncooperative scenarios (23%) [112],

these results are an order of magnitude better, however there is no real comparison

between TCAS and this CAS, as TCAS is tested using manoeuvring intruders in a

comprehensive encounter model (as discussed in Section 5.1.1); whereas for our

CAS, a simplified encounter model is used. Also noteworthy is that TCAS is

designed in particular for the cooperative domain, whereas this CAS is designed

specifically for the uncooperative scenarios, therefore comparison is not relevant.

0 2 4 6 8 10 12 14 160

0.1

0.2

0.3

0.4

0.5

Risk Ratio for CAS

Ris

kR

atio

CAS

test statistic

RiskRatio

RRIC

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5Risk Ratio for Relative Bearing

Ris

kR

atio

Test statistic (x 10-4)



Even though 10ASC has a high FMP rate of 27.35% (oversensitivity),

manoeuvring one out of four times unnecessarily is deemed reasonable to get the

high level of safety (low RiskRatio ).

5.3.4.3 ELOS Expectations

For computing ELOS expectations requirements, we determined the total Risk

Ratio for a MAC is 31.27 10MACRR . This comes from the assumption that 10%

of NMACs lead to MACs [112, 123]. The conservative figures released by

Dalamagkidis et al. [28, 124] for MACRR state that a CAS would need to meet the

manned aviation ELOS, which was said to be around 31 10MACRR , from

Equation (5.16), for Class E airspace. (Remembering that there are no figures for

Class G airspace). Thus, one can see that the figures of the collision avoidance

system of this research are in the same order of magnitude. It is therefore reasonable

to claim that this collision avoidance system has results that are comparable to

current ELOS expectations for operations in Class E airspace.



6 Conclusion

As previously discussed in Section 1.1, the collision avoidance problem is one of

the major hurdles to allowing UAS to operate freely in the NAS and thus ensure

continued UAS market growth [8]. Various major players within industry have made

a reasonable attempt at solving the collision avoidance problem [38-46]. However,

the system industry proposes uses a sensor that costs around $200k, uses a lot of

power and is heavy [31]. A system of this magnitude is currently unreasonable for

low-cost UAS.

A collision avoidance system using vision-only sensors would present a solution

for the low-cost UAS market and be a major technological enabler for the entire

UAS sector [29]. This thesis has presented a well-defined methodology for a

collision avoidance algorithm that uses vision-only data to negotiate a conflict

scenario without calculating range. The fact that no range estimates are made means

that action is able to take place almost immediately (within ~0.12 seconds) which is

orders of magnitude faster than its rival systems [25, 30, 34] and thus improves the

overall safety of the collision avoidance system (CAS).

In this thesis, we investigated the intruder‟s characteristics in an image that

directly affect the miss distance in a conflict scenario: namely intruder image

velocity ; and time to collision TCT , which is derived from intruder‟s angular-area

subtended and its rate (known as image area expansion). These image-based

characteristics are implemented in a CAS that uses a test statistic in a thresholding

approach to detect a conflict, ASC . This algorithm also uses these image-based

characteristics to manoeuvre the owncraft to avoid the collision.

The objective of the developed CAS is to be able to avoid a mid-air collision

(MAC) and a near mid air collision (NMAC). That means for successful collision

avoidance the two aircraft need to exceed a closest point of approach (CPA) distance

of 152.4 metres (500ft) laterally.



We mathematically modelled the UAS platform (Flamingo) that the proposed

CAS is tested on, along with the UAS controller and the CAS vision sensor that is

motion-compensated, using feedback from the owncraft inertial instruments. We

outlined the architecture of the simulation environment used to test the proposed

CAS.

Finally, a simplified encounter model is implemented in a Monte-Carlo

simulation that is used to simulate NMACs, defined to have a CPA distance of less

than 152.4m. These simulations are run 50 000 times at various test statistic

thresholds. The developed CAS is gauged against a system that uses only intruder

image velocity in a similar thresholding approach. The results are displayed in a

standard operating characteristic (SOC) curve for both CAS‟s over a range of test

statistics. These SOC curves display correct detection performance against false

alarm rate. Established is that the CAS of this thesis which utilises TCT performs

much better than a system that uses only.

Also shown is that for Class E airspace the published probability of a MAC

ELOS expectation is 310MACRR . The figure that this CAS produced in simulation

experiments is 31.27 10MACRR . This is in the same order of magnitude. Thus, it

is reasonable to say that this CAS has results that are comparable with current ELOS

expectations for Class E airspace.



7 Future Recommendations

This research aimed to prove the feasibility of using vision-only data in a heavily

regulated environment, such as aviation, for collision avoidance. We have shown

that the results are favourable and comparable to the ELOS requirements such a

system needs.

From here, it is recommended to expand the research to include collision

determination in the second image dimension (of elevation ) and thus include

manoeuvring in the third dimension (altitude). Because the NMAC defined zone for

the vertical plane is only 30.5m and the responsiveness of an owncraft in the vertical

plane is often quicker, it is reasonable to expect even better results.

Next one would include manoeuvring intruders (on constant curves); however,

more encounter modelling also needs to be completed in this area. Alternately

running this algorithm on one of the noted comprehensive models would go towards

showing realisable results. After this, multiple intruder collision avoidance would be

the next logical step.

As well as having the more comprehensive encounter model, it would be good to

get flight test results, as other unforeseeable problems may need addressing.






8 Appendices

APPENDIX A

Data and aerodynamic coefficients of the Flamingo UAS [125]

TABLE VI FLAMINGO DATA

Flamingo Values

Variable Symbol Value Units

Mass m 20 kg

Mean chord c 0.29 m

Surface area S 1.15 m2

Wingspan b 4.0 m

Centre of gravity CoG 0.25 c

Airfoil 2415 NACA

Flamingo Limits

TABLE VII FLAMINGO LIMITS

Flamingo Limits

Variable Symbol Value Units

Maximum thrust maxT 24.5 N

Thrust slew limit maxT 5 N/k

Max angle of attack max 16 ˚

Stall speed stallV 13 m/s

Typical operation speed TV 27 m/s

Maximum speed maxV 40 m/s

Climb rate maxh 92 m/min

Max stable roll angle max 20 ˚

Max heading rate max 3.75 ˚/s



Inertial Data

TABLE VIII INERTIAL DATA

Inertial Values (N∙m)

Axis Value

XI 5.0

YI 6.28

YI 9.18

ZXI 0

Lift/Drag Data

TABLE IX LIFT/DRAG DATA

Lift/Drag Values

Coefficient Value

0LC 0.04

LC 6.0

qLC 7.729

0DC 0.02

kk 0.0039

Longitudinal Coefficients

TABLE X LONGITUDINAL COEFFICIENTS

Longitudinal Values

Coefficient Value

0mC -0.055

mC

-0.85

emC

-1.571

qmC -41.3

mC

-10.7



Lateral Coefficients

TABLE XI LATERAL COEFFICIENTS

Lateral Values

Coefficient Value

YC -0.308

rYC

0.2

pYC 0.0

rYC 0.588

lC

-0.089

rlC

0.015

alC

0.177

plC -0.6

rlC (CL/3.5)-0.063

nC 0.038

nC 0.0

rnC

-0.055

anC

-0.0354*CL

pnC -0.032

rnC -1.157

Mach Coefficients

TABLE XII MACH COEFFICIENTS

Mach Values

Coefficient Value

umC 0

uLC 0

uDC 0



Control Coefficients

TABLE XIII CONTROL COEFFICIENTS

Control Values

Loop Symbol Value

Heading hold using roll P -1

Aileron from roll ailP -0.2

Altitude hold using elevator

elP -0.01

elI -0.00005

elD -1

Rudder from sideslip (coord. turns) rudPff -10

Throttle for speed hold thP 3

thI 0.008



APPENDIX B

Flamingo Open-Loop Stability

This open loop stability analysis is developed using Nelson [124] with q as

446.4886 Pa, Temperature at 15˚C and altititude of sea level. Also, take LC as 0.4 for

typical operations.

Lateral Stability

Lateral-directional Derivatives

l

X

qSbCL

I

(8.1)

2

2

rn

r

Z t

qSb CN

I V (8.2)

2

2

rl

r

X t

qSb CL

I V (8.3)

n

Z

qSbCN

I

(8.4)

2

2

pl

p

X t

qSb CL

I V (8.5)

YqSC

Ym

(8.6)

2

rY

r

t

qSbCY

mV (8.7)



Spiral approximation

2

2

2

0.089 1.157 0.05129 0.038446.4886 1.15 4

2 9.18 27 0.089

18.81

r r

r r

spiral

l n l n

Z t l

L N L N

L

C C C CqSb

I V C

(8.8)

Roll approximation

2

2

2

2

2

446.4886 1.15 4 0.6

2 5 27

18.26

p

p

l

roll p

X t

l

X t

qSb CL

I V

qSb C

I V

(8.9)

Dutch Roll approximation

2

2

2

2

446.4886 4.6 446.4886 4.6 1.157 0.308 0.038 0.588 40 0.038

2 20 9.18 27

2053.848 2053.848 0.35636 0.02231 1.52

267688.8

2.30

DR

r r

r r t

n

t

n Y n Y n

Z t

Y N N Y V N

V

qSb qSb C C C C mC

mI V

(8.10)



2

2

1

2

1

2 2

1 446.489 1.15 0.308 4 1.157

2 2.37 27 20 2 9.18

4.11

DR

r

DR

t r

DR

n t

y n

n t Z

Y V N

V

C b CqS

V m I

(8.11)

Lateral Flying Qualities

Spiral

This represents awesome roll characteristics (Level 1) according to table 5.5 of

Nelson. spiral is the characteristic root due to spiral mode.

18.81spiral (8.12)

Roll

This represents awesome roll characteristics (Level 1) according to table 5.5 of

Nelson. Where roll is the roll time constant and roll is the characteristic root due to

roll.

18.26

1

0.0547

roll

roll

roll

roll

(8.13)

Dutch Roll

The Dutch Roll characteristics represent very good or Level 1 according to

Nelson table 5.6. Where DRn is the undamped natural frequency and DR is the

damping ratio due to Dutch Roll.



2.37

4.11

DRn

DR

(8.14)

Longitudinal Stability

Longitudinal Derivatives

0( 2 )

uL L

u

t

C C qSZ

mV

(8.15)

0( 2 )

uD D

u

t

C C qSX

mV

(8.16)

0( )L DC C qS

Zm

(8.17)

2qq m

t Y

c qScM C

V I (8.18)

m

Y

qScM C

I (8.19)

2

m

t Y

c qScM C

V I (8.20)

Phugoid mode

0

2

2

( 2 )

446.4886 1.15 9.80665 2 0.04

20 27

0.1662

P

u

un

t

L L

t

Z g

V

qSg C C

mV

(8.21)



0

2

( 2 )

2

446.4886 1.15 2 0.02

2 0.1662 20 27

0.1144

P

u

P

uP

n

D D

n t

X

qS C C

mV

(8.22)

Short Period mode

0

2

2

( )

2

446.4886 1.15 0.29 446.4886 1.15 0.29 6.02 41.30.85

6.28 2 20 27

7.09

SP

q

q

n

t

L D m

m

Y t

Z MM

V

qSc C C CqScC

I mV

(8.23)

0

0

2

2

2

2 2

0.29 41.3 10.7446.4886 1.15 6 0.02

2 7.0892 27 20 2 6.28

0.22

SP

q

SP

q

SP

n

m mL D

n t Y

ZM M

u

c C CC CqS

V m I

(8.24)

Longitudinal Flying Qualities

Phugoid

These phugoidal characteristics are considered level 1 or good (according to

Nelson table 4.10). Where Pn is the undamped natural frequency and P is the

damping ratio due to phugoid.



0.1662

0.1144

Pn

P

(8.25)

Short Period

These short period characteristics are considered level 2 or acceptable (according

to Nelson table 4.10). Where SPn is the undamped natural frequency and SP is the

damping ratio in the Short Period.

7.09

0.22

SPn

SP

(8.26)



APPENDIX C

Image Area Expansion

An image pixel is considered to have approximately a square relationship (From

Figure 8). Thus, the angular area subtended by the image is:

2 (8.27)

From Figure 8 we can let

2A d (8.28)

Now if we assume (for small angles) that

d

R (8.29)

Then

A

R (8.30)

So,

2

A

R (8.31)

Thus,

2

A

t t R

(8.32)

From quotient rule:

2

u vv u

u t t

t v v

(8.33)

Substituting (8.32) into (8.33) gives:



22

4

A RR A

t t

t R

(8.34)

However,

0A

t

(8.35)

So Equation (8.34) becomes,

2

4.

A R

t R t

(8.36)

Now substitute in Equation (8.31),

2

( . ).

R R

t R t

(8.37)

The product rule states,

.v u

u v u vt t t

(8.38)

So

( . )

2 .R R R

Rt t

(8.39)

And it is known that,

R

Vt

(8.40)

Thus putting Equation (8.39) and (8.40) back into Equation (8.37) becomes,

2

.2 .RVt R

(8.41)

But we know

R

VT

(8.42)

Where T is TCT . Therefore putting Equation (8.42) back into Equation (8.41)



2

2

2.

TC

R

t R T

(8.43)

So finally,

2

TCTt

(8.44)



9 Bibliography

[1] J. Baldwin, et al., "General aviation collision avoidance-challenges of full

implementation," in Digital Avionics Systems Conference, 1994. 13th DASC.,

AIAA/IEEE, 1994, pp. 504-509.

[2] K. M. Caldwell, "TCAS II testing conflicts and resolutions," in 18th

Proceedings of the ICAS Congress, Beijing, China, 1992, pp. 9-17.

[3] US-OSoD, Unmanned systems roadmap 2009-2034, Fourth ed. Washington,

District of Columbia: Office of the Secretary of Defense, 2009.

[4] M. T. DeGarmo, "Issues concerning integration of unmanned aerial vehicles

in civil airspace," MITRE, Center for Advanced Aviation System

Development, McLean, Virginia, 2004.

[5] B. C. Meyer, "Notes on flying and dying," Psychoanalytic Quarterly, vol. 52,

pp. 327-352, 07 1983.

[6] L. R. Newcombe, Unmanned aviation: a brief history of unmanned aerial

vehicles: AIAA, 2004.

[7] Teal, World unmanned aerial vehicle systems - market profile and forecast,

2010.

[8] DoD. (2009, 16/02/2009). Due regard technology for unmanned aerial

systems.

[9] R. E. Weibel and R. J. Hansman, "Safety considerations for operation of

unmanned aerial vehicles in the national airspace system," MIT International

Center for Air Transportation, Cambridge, Massachusetts2005.

[10] K. Dalamagkidis, et al., "On unmanned aircraft systems issues, challenges

and operational restrictions preventing integration into the national airspace

system," Progress in Aerospace Sciences, vol. 44, pp. 503-519, 2008.

[11] A. D. Zeitlin, "Technology milestones–detect, sense & avoid for unmanned

aircraft systems," in Proceedings of the AIAA Infotech@Aerospace

Conference and Exhibit, Rohnert Park, California, 2007.

[12] JAPCC, "The Joint Air Power Competence Centre (JAPCC) flight plan for

unmanned aircraft systems (UAS) in NATO," Joint Air Power Competence

Centre15/03/2007 2007.

[13] CARE, "Integration of UAVs into future air traffic management," Co-

operative Actions of R&D in Eurocontrol2001.

[14] S. Attila, "Technology demonstration study on sense and avoid technologies

for long endurance unmanned aerial vehicles," European Defense Agency,

Brussels, Belgium2007.

[15] Air4All, "Air4All workshop 1," ed, 2008.

[16] ASTM Standards, F2411-07 - Standard specification for design and

performance of an airborne sense-and-avoid system, 2007.

[17] W. E. Green and P. Y. Oh, "Optic-flow-based collision avoidance," IEEE

Robotics & Automation Magazine, vol. 15, pp. 96-103, 2008.



[18] E. S. Jang, et al., "Collision avoidance of a mobile robot for moving obstacles

based on impedance force control algorithm," in Proceedings of the

IEEE/RSJ International Conference on Intelligent Robots and Systems IROS,

Edmonton, Canada, 2005, pp. 382-387.

[19] S. C. Han and H. Bang, "Proportional navigation-based optimal collision

avoidance for UAVs," in Proceedings of the 2nd International Conference on

Autonomous Robots and Agents, Palmerston North, New Zealand, 2004.

[20] J. M. Hoekstra, et al., "Free flight in a crowded airspace," Progress in

Astronautics and Aeronautics, pp. 533-545, 2001.

[21] A. Bicchi and L. Pallottino, "On optimal cooperative conflict resolution for

air traffic management systems," IEEE Transactions on Intelligent

Transportation Systems, vol. 1, pp. 221-231, 2000.

[22] B. Wetherby, et al., Full AERA services operational description. McLean,

Virginia: The MITRE Corporation, 1993.

[23] H. Erzberger, "The automated airspace concept," presented at the Proceedings

of the 4th USA/Europe Air Traffic Management R&D Seminar, Santa Fe,

New Mexico, 2001.

[24] A. R. Lacher, et al., "Unmanned aircraft collision avoidance – technology

assessment and evaluation methods," in Proceedings of The 7th Air Traffic

Management Research & Development Seminar Barcelona, Spain, 2007.

[25] O. Shakernia, et al., "Passive ranging for UAV sense and avoid applications,"

in Proceedings of the AIAA Infotech@Aerospace Conference and Exhibit,

Arlington, Virginia, 2005, pp. 1-10.

[26] G. Fasano, "Multisensor based fully autonomous non-cooperative collision

avoidance system for UAVs," PhD Thesis PhD Thesis, Aerospace Systems

Research Group, University of Naples, Naples, Itlay, 2008.

[27] SKYbrary. (2010, 6/4/2010). Near mid air collision (NMAC). Available:

http://www.skybrary.aero/index.php/NMAC

[28] K. Dalamagkidis, et al., On integrating unmanned aircraft systems into the

national airspace system: issues, challenges, operational restrictions,

certification, and recommendations: Springer Verlag, 2009.

[29] B. C. Karhoff, et al., "Eyes in the domestic sky: an assessment of sense and

avoid technology for the army's "Warrior" unmanned aerial vehicle," in IEEE

Systems and Information Engineering Design Symposium, 2006, pp. 36-42.

[30] H. Voos, "UAV "see and avoid" with nonlinear filtering and non-cooperative

avoidance," in Proceedings of the 13th IASTED International Conference

Robotics and Applications, Wurzburg, Germany, 2007.

[31] W.-Z. Chen, "Sense and avoid (SAA) technologies for unmanned aircraft

(UA)," National Cheng Kung University2008.

[32] nVidia. (2009, 17/02/2009). GeForce GTX 280. Available:

http://www.nvidia.com/object/product_geforce_gtx_280_us.html

[33] J. Lai and J. J. Ford, "Relative Entropy Rate based Multiple Hidden Markov

Model Approximation," IEEE Transactions on Signal Processing,

2009(accepted to appear).

http://www.skybrary.aero/index.php/NMAC

http://www.nvidia.com/object/product_geforce_gtx_280_us.html



[34] E. W. Frew, "Observer trajectory generation for target-motion estimation

using monocular vision," PhD Thesis PhD Thesis, Department of Aeronautics

and Astronautics, Stanford University, San Francisco, California, 2003.

[35] A. J. Calise, et al., "Estimation and guidance strategies for vision based target

tracking," in Proceedings of the American Control Conference ACC,

Portland, Oregon, 2005, pp. 5079-5084 vol. 7.

[36] M. Kochenderfer, et al., "Hazard alerting using line-of-sight rate," in

Proceedings of the AIAA Guidance, Navigation and Control Conference and

Exhibit, Honolulu, Hawaii, 2008, pp. 2003-2011.

[37] B. Young. (2009, 10 December 2009). Silvertone Flamingo UAV. Available:

http://www.silvertoneuav.com/flamingo.php

[38] R. C. Wolfe, "NASA ERAST non-cooperative DSA flight test," 2003.

[39] W. Graham and R. H. Orr, "Separation of air traffic by visual means: an

estimate of the effectiveness of the see-and-avoid doctrine," Proceedings of

the IEEE, vol. 58, pp. 337-361, 1970.

[40] Y. Ikeda, et al., "Automatic air collision avoidance system," in Proceedings

of the 41st SICE Annual Conference, Osaka, Japan, 2002, pp. 630-635.

[41] Y. Ikeda and J. Kay, "An optimal control problem for automatic air collision

avoidance," in Proceedings of the 42nd IEEE Conference on Decision and

Control CDC, Maui, Hawaii, 2003, pp. 2222-2227.

[42] L. Matthies, et al., "Kalman filter-based algorithms for estimating depth from

image sequences," International Journal of Computer Vision, vol. 3, pp. 209-

238, 1989.

[43] J. Utt, et al., "Test and Integration of a Detect and Avoid System," in

Proceedings of the AIAA 3rd "Unmanned Unlimited" Technical Conference,

Workshop and Exhibit, Chicago Illinois, 2004.

[44] J. Utt, et al., "Development of a sense and avoid system," in Proceedings of

the AIAA Infotech@Aerospace Conference and Exhibit, Arlington, Virginia,

2005.

[45] K. R. Suwal, et al., "SeFAR integration test bed for see and avoid

technologies," in Proceedings of the AIAA Infotech@Aerospace Conference

and Exhibit, Arlington, Virginia, 2005, pp. 1-7.

[46] O. Shakernia, et al., "Sense and avoid (SAA) flight test and lessons learned,"

in Proceedings of the AIAA Infotech@Aerospace Conference and Exhibit,

Rohnert Park, California, 2007.

[47] A. Richards and J. P. How, "Aircraft trajectory planning with collision

avoidance using mixed integer linear programming (MILP)," in Proceedings

of the American Control Conference ACC, Anchorage, Alaska, 2002, pp.

1936-1941.

[48] E. W. Frew, et al., "Adaptive receding horizon control for vision-based

navigation of small unmanned aircraft," in Proceedings of the American

Control Conference ACC, Minneapolis, Minnesota, 2006.

[49] A. Richards and J. How, "A decentralized algorithm for robust constrained

model predictive control," in Proceedings of the American Control

Conference ACC, Boston, Massachusetts, 2004, pp. 4261-4266 vol.5.

http://www.silvertoneuav.com/flamingo.php



[50] J. B. Froisy, "Model predictive control - Building a bridge between theory

and practice," Computers and Chemical Engineering, vol. 30, pp. 1426-1435,

2006.

[51] L. Pallottino, et al., "Conflict resolution problems for air traffic management

systems solved with mixed integer programming," IEEE Transactions on

Intelligent Transportation Systems, vol. 3, pp. 3-11, 2002.

[52] D. H. Shim, et al., "Decentralized nonlinear model predictive control of

multiple flying robots," in Proceedings of the 42nd IEEE Conference on

Decision and Control CDC, Maui, Hawaii, 2003, pp. 3621-3626.

[53] E. W. Frew, "Receding horizon control using random search for UAV

navigation with passive, non-cooperative sensing," in Proceedings of the

AIAA Guidance, Navigation and Control Conference and Exhibit, San

Francisco, California, 2005, pp. 1-13.

[54] E. W. Frew, "Approximating information content for active sensing tasks

using the unscented transform," in Proceedings of the IEEE/RSJ International

Conference on Intelligent Robots and Systems IROS, Nice, France, 2008, pp.

2559-2564.

[55] O. Khatib, "Real-time obstacle avoidance for manipulators and mobile

robots," The International Journal of Robotics Research, vol. 5, pp. 90-98,

March 1 1986.

[56] P. Corke, "Mobile robot navigation as a planar visual servoing problem," in

Robotics Research. vol. 6, ed: Springer, 2003, pp. 361-372.

[57] J. Ren, et al., "Modified Newton's method applied to potential field-based

navigation for nonholonomic robots in dynamic environments," Robotica,

vol. 26, pp. 117-127, 2007.

[58] J. C. Latombe, Robot motion planning: Kluwer Academic Publishers, 1991.

[59] I. Ulrich and J. Borenstein, "VFH+: reliable obstacle avoidance for fast

mobile robots," in Proceedings of the IEEE International Conference on

Robotics and Automation ICRA, Leuven, Belgium, 1998, pp. 1572-1577.

[60] I. Ulrich and J. Borenstein, "VFH*: local obstacle avoidance with look-ahead

verification," in Proceedings of the IEEE International Conference on

Robotics and Automation ICRA, San Francisco, California, 2000, pp. 2505-

251.

[61] J. Borenstein and Y. Koren, "The vector field histogram-fast obstacle

avoidance for mobile robots," IEEE Transactions on Robotics and

Automation, vol. 7, pp. 278-288, 1991.

[62] Y. Koren and J. Borenstein, "Potential field methods and their inherent

limitations for mobile robot navigation," in Proceedings of the IEEE

International Conference on Robotics and Automation ICRA, Sacramento.

California, 1991, pp. 1398-1404.

[63] G. K. Schmidt and K. Azarm, "Mobile robot navigation in a dynamic world

using an unsteady diffusion equation strategy," in Proceedings of the

lEEE/RSJ International Conference on Intelligent Robots and Systems IROS,

Raleigh, North Carolina, 1992, pp. 642-647.



[64] C. I. Connolly, et al., "Path planning using Laplace's equation," in

Proceedings of the IEEE International Conference on Robotics and

Automation ICRA, Cincinnati, Ohio, 1990, pp. 2102-2106.

[65] A. A. Masoud, et al., "Robot navigation using a pressure generated

mechanical stress field: the biharmonic potential approach," in Proceedings

of the IEEE International Conference on Robotics and Automation ICRA, San

Diego, California, 1994, pp. 124-129

[66] D. A. Lawrence, "Lyapunov vector fields for UAV flock coordination," in

Proceedings of the AIAA 2nd "Unmanned Unlimited" Technical Conference,

Workshop and Exhibit, San Diego, California, 2003.

[67] E. W. Frew, "Cooperative standoff tracking of uncertain moving targets using

active robot networks," in Proceedings of the IEEE International Conference

on Robotics and Automation ICRA, Rome, Italy, 2007, pp. 3277-3282.

[68] E. W. Frew and D. Lawrence, "Cooperative stand-off tracking of moving

targets by a team of autonomous aircraft," in Proceedings of the AIAA

Guidance, Navigation and Control Conference and Exhibit, San Francisco,

California, 2005.

[69] S. R. Griffiths, "Vector field approach for curved path following for

miniature aerial vehicles," in Proceedings of the AIAA Guidance, Navigation

and Control Conference and Exhibit, Keystone, Colorado, 2006, pp. 63-64.

[70] E. W. Frew, et al., "Lyapunov guidance vector fields for unmanned aircraft

applications," in Proceedings of the American Control Conference ACC, New

York City, 2007, pp. 371-376.

[71] E. W. Frew, et al., "Coordinated standoff tracking of moving targets using

Lyapunov guidance vector fields," AIAA Journal of Guidance, Control, and

Dynamics, vol. 31, p. 290, 2008.

[72] S. Griffiths, et al., "Maximizing miniature aerial vehicles," IEEE Robotics &

Automation Magazine, vol. 13, pp. 34-43, 2006.

[73] S. Griffiths, et al., "Obstacle and terrain avoidance for miniature aerial

vehicles," IEEE Robotics & Automation Magazine, vol. 13, pp. 34-43, 2006.

[74] D. R. Nelson, et al., "Vector field path following for miniature air vehicles,"

IEEE Transactions on Robotics, vol. 23, pp. 519-529, 2007.

[75] Procerus. (2010, 12/05/2010). Kestral autpilot. Available:

http://www.procerusuav.com/productsKestrelAutopilot.php

[76] S. C. Degen, et al., "Tensor field guidance for time-based waypoint arrival of

UAVs by 4D trajectory generation," in Proceedings of the IEEE Aerospace

Conference, Big Sky, Montana, 2009.

[77] K. Sigurd and J. How, "UAV trajectory design using total field collision

avoidance," in Proceedings of the AIAA Guidance, Navigation and Control

Conference and Exhibit, Austin, Texas, 2003.

[78] M. Massink and N. De Francesco, "Modelling free flight with collision

avoidance," in Proceedings of the IEEE International Conference on

Engineering of Complex Computer Systems, Skovde , Sweden, 2001, pp. 270-

279.

[79] RTCA, "Final report of the RTCA task force 3: free flight implementation,"

RTCA, Washington, District of Columbia1995.

http://www.procerusuav.com/productsKestrelAutopilot.php



[80] A. Chakravarthy and D. Ghose, "Obstacle avoidance in a dynamic

environment: a collision cone approach," IEEE Transactions on Systems,

Man and Cybernetics, Part A, vol. 28, pp. 562-574, 1998.

[81] Y. Watanabe, et al., "Vision-based obstacle avoidance for UAVs," in

Proceedings of the AIAA Guidance, Navigation and Control Conference and

Exhibit, Hilton Head, South Carolina, 2007, pp. 20-23.

[82] B. Ajith Kumar and D. Ghose, "Radar-assisted collision avoidance/guidance

strategy for planar flight," IEEE Transactions on Aerospace and Electronic

Systems, vol. 37, pp. 77-90, 2001.

[83] C. Carbone, et al., "A novel 3D geometric algorithm for aircraft autonomous

collision avoidance," in Proceedings of the 45th IEEE Conference on

Decision and Control CDC, San Diego, California, 2006, pp. 1580-1585.

[84] S. Arulampalam, et al., "Bearings-only tracking of manoeuvring targets using

particle filters," Journal on Applied Signal Processing vol. 15, pp. 2351–

2365, 2004.

[85] T. Bréhard and J. P. Le Cadre, "Initialization of particle filter and posterior

Cramér-Rao bound for bearings-only tracking in modified polar coordinate

system," IEEE Transactions on Aerospace and Electronic Systems, 2004.

[86] Y. Yu and Q. Cheng, "Particle filters for maneuvering target tracking

problem," Signal Processing, vol. 86, pp. 195-203, 2006.

[87] G. Lei, et al., "Posterior Cramer-Rao lower bounds for multitarget bearings-

only tracking," Journal of Systems Engineering and Electronics, vol. 19, pp.

1127-1132, 2008.

[88] V. Aidala and S. Hammel, "Utilization of modified polar coordinates for

bearings-only tracking," IEEE Transactions on Automatic Control, vol. 28,

pp. 283-294, 1983.

[89] S. Arulampalam and B. Ristic, "Comparison of the particle filter with range-

parameterized and modified polar EKFs for angle-only tracking," in

Proceedings of the Signal and Data Processing of Small Targets 2000,

Orlando, Florida, 2000, pp. 288-299.

[90] G. Recchia, et al., "An optical flow based electro-optical see-and-avoid

system for UAVs," in Proceedings of the IEEE Aerospace Conference, Big

Sky, Montana, 2007, pp. 1-9.

[91] B. Call, et al., "Obstacle avoidance for unmanned air vehicles using image

feature tracking," in Proceedings of the AIAA Guidance, Navigation and

Control Conference and Exhibit, Keystone, Colorado, 2006.

[92] P. J. Shelnutt, "Collision avoidance for UAVs using optic flow measurement

with line of sight rate equalization and looming," MSc Thesis, Air Force

Institute of Technology, Air University, Wright-Patterson Air Force Base,

Ohio, 2008.

[93] P. Angelov, et al., "A passive approach to autonomous collision detection and

avoidance," in Proceedings of the Tenth International Conference on

Computer Modeling and Simulation, Cambridge, United Kingdom, 2008, pp.

64-69.

[94] E. W. Frew and S. M. Rock, "Trajectory generation for constant velocity

target motion estimation using monocular vision," in Proceedings of the



IEEE International Conference on Robotics and Automation ICRA Taipei,

Taiwan, 2003, pp. 3479-3484 vol.3.

[95] A. Logothetis, et al., "An information theoretic approach to observer path

design for bearings-only tracking," in Proceedings of the 36th IEEE

Conference on Decision and Control, CDC, San Diego, California, 1997, pp.

3132-3137 vol.4.

[96] F. Chaumette and S. Hutchinson, "Visual servo control. I. Basic approaches

[Tutorial]," IEEE Robotics & Automation Magazine, vol. 13, pp. 82-90, 2006.

[97] F. Chaumette and S. Hutchinson, "Visual servo control. II. Advanced

approaches [Tutorial]," IEEE Robotics & Automation Magazine, vol. 14, pp.

109-118, 2007.

[98] D. Regan and R. Gray, "Visually guided collision avoidance and collision

achievement," Trends in Cognitive Sciences, vol. 4, pp. 99-107, 2000.

[99] ATSB, "Limitations of the see and avoid principle," Canberra, Australia2004.

[100] F. Hoyle, The black cloud: Penguin, 1957.

[101] D. Sislak, et al., "Negotiation-based approach to UAVs," in IEEE Workshop

on Distributed Intelligent Systems: Collective Intelligence and Its

Applications, DIS, Prague, Czech Republic, 2006, pp. 279-284.

[102] Chapter 3 - Section 3.2: Right of way, 2009.

[103] Subpart B - Section 91.113: Right-of-way rules; Except water operations,

2004.

[104] Division 1 - Regulation 162: Rules for prevention of collision, 1988.

[105] F. R. Garza and E. A. Morelli, "A collection of nonlinear aircraft simulations

in matlab," National Aeronautics and Space Administration, NASA/TM-2003-

212145, 2003.

[106] B. L. Stevens and F. L. Lewis, Aircraft control and simulation. Georgia Tech

Research Institute/University of Texas: Wiley-Interscience, 1992, p 9.

[107] N. Cowan, et al., "Vision-based follow-the-leader," in Proceedings of the

IEEE/RSJ International Conference on Intelligent Robots and Systems IROS,

Las Vegas, Nevada, 2003, pp. 1796-1801 vol.2.

[108] FAA, "P-8740-51 - How to avoid a mid air collision," 1987.

[109] L. Mejias, et al., "Towards implementation of vision-based UAS sense and

avoid systems," in International Council of the Aeronautical Sciences ICAS,

Nice, France, 2010.

[110] O. Bourquardez and F. Chaumette, "Visual servoing of an airplane for

alignment with respect to a runway," in Proceedings of the IEEE

International Conference on Robotics and Automation ICRA, Rome, Italy,

2007, pp. 1330-1335.

[111] J. Kuchar, "Methodology for alerting-system performance evaluation," AIAA

Journal of Guidance, Control, and Dynamics, vol. 19, pp. 438-444, 1996.

[112] Eurocontrol, "Final report on studies on the safety of ACAS II in Europe,"

ACAS/ACASA/02-014, 2002.

[113] M. J. Kochenderfer, et al., "A comprehensive aircraft encounter model of the

National Airspace System," Lincoln Laboratory Journal, vol. 17, pp. 41-53,

2008.



[114] M. Kochenderfer, et al., "Uncorrelated encounter model of the National

Airspace System version 1.0," Massachusetts Inst Of Tech Lexington,

Lincoln Lab,2008.

[115] M. J. Kochenderfer, et al., "Model-based optimization of airborne collision

avoidance logic," Massachusetts Inst Of Tech Lexington, Lincoln Lab, 2010.

[116] M. Barkat, Signal detection and estimation: Artech House Publishers, 2005.

[117] L. Winder and J. Kuchar, "Evaluation of collision avoidance maneuvers for

parallel approach," AIAA Journal of Guidance, Control, and Dynamics, vol.

22, pp. 801-807, 1999.

[118] Eurocontrol, "Final report on the safety of ACAS II in the European RVSM

environment," ASARP/WP9/72/D, 2006.

[119] J. Kuchar and A. Drumm, "The Traffic Alert and Collision Avoidance

System," Lincoln Laboratory Journal, vol. 16, pp. 277-296, 2007.

[120] T. Hutchings, et al., "Architecting UAV sense & avoid systems," in

Proceedings of the IET Conference on Autonomous Systems, London, United

Kingdom, 2007, pp. 1-8.

[121] S. Endoh, "Aircraft collision models," MSc Thesis, Department of

Aeronautics and Astronautics, Massachusetts Institute of Technology,

Cambridge, Massachusetts, 1982.

[122] Number 90-48C, 1983.

[123] M. Kochenderfer, et al., "Airspace encounter models for estimating collision

risk," AIAA Journal of Guidance, Control, and Dynamics, vol. 33, 2010.

[124] R. C. Nelson, Flight stability and automatic control, 2nd ed. University of

Notre Dame: McGraw Hill, 1998.

[125] G. Bonin, "Flamingo model data," Riverwood, NSW, 2009.

Documents

Reactive Image-based Collision Avoidance System for - QUT ePrints