EN4 Dynamics and Vibrations Design Project Designing a position controller for a quadcopter

Synopsis In this project you will design and test a control system (written in MATLAB) that will fly a quadcopter along a prescribed path. 1. Organization

• This project must be done in groups of up to 4 students. • The project `deliverables’; are (i) A Matlab code that will control the quadcopter; and (ii) A short

(2-3 page) report (one report per group) describing your design calculations, and (iii) a short oral presentation describing your design procedures, and demonstrating the performance of your design; and (iv) an evaluation of the performance of your team-mates on the project.

• You will demonstrate your design to faculty on Saturday May 6th. As part of the testing, you should give a short (10 min) presentation describing your design procedure. Your written report is due at the presentation.

• Warning: We only have 4 sets of equipment to test designs. 2. Overview of design requirements The goal of this project is to design and implement a control system that will fly a quadcopter along a prescribed path. You will be provided with the following equipment

1. A Microsoft Kinect 2, which will measure the position of the quadcopter; 2. A Crazyflie2 micro-quadcopter; 3. A Crazyradio PA USB radio that will send control signals to the quadcopter; 4. A battery charger and spare batteries for the quadcopter 5. Python software that will (i) manage the radio interfaces the quadcopter; and (ii) run your

MATLAB scripts that fly the quadcopter; 6. A Matlab code that will do the image processing to track the quadcopter. 7. A template MATLAB code that you can extend to implement your controller

Before you start: Download and save the template Matlab controller code from the course website. You can also download the following functions – you can modify these if you like. Each kinect sensor has its own unique optics so we have had to pre-install a customized version on each of the computers in B&H096 – if you want to use your own modified versions please contact Prof Bower for assistance.

• getKinectCoords_sample.m (reads the Kinect and calculates coordinates from the images) • preview_kinect_image_sample.m (an interactive MATLAB GUI that helps set up the image

processing) Some additional software is needed to manage communications between the quadcopter, the Kinect, and your control scripts. This only works on recent Windows operating systems and is tricky and time-

consuming to install, so we have provided a dedicated computer with each quadcopter. You will need to copy your codes onto these computers to test your designs. To complete the design, you will need to: (i) Complete some preliminary hand-calculations and MATLAB simulations to understand how the

control system works, and to calculate values of important parameters in the control system. (ii) Write a MATLAB code that will calculate roll/pitch/yaw/thrust settings that will fly the

quadcopter to where it should be. (iii) Test and fine-tune your control system on an actual quadcopter (iv) Write a code to make your quadcopter do something interesting – you can find a few suggestions

in Section 7. Please be gentle with the quadcopters! – if they break they are hard to repair or replace quickly.

The steps in the project are described in more detail in the sections to follow. 3. Design calculations The basic design problem you must solve is (i) measure the position of the quadcopter; and (ii) use the measured position to calculate the roll, pitch, thrust and yaw settings to be sent to the quadcopter. You will do this using a so-called ‘PID feedback-control system’ (The PID stands for Proportional, Integral, Derivative). To implement your design, you should:

(i) Do some quick hand-calculations to understand how the PID controller works; (ii) Write a more sophisticated MATLAB code that will solve the equations of motion for the

quadcopter and an idealized version of the control system (iii) Use your MATLAB code to estimate values for the parameters needed in the actual control

system. (iv) Write a matlab script (there is a template online) to implement the feedback controller.

3.1 Hand calculations The following simple calculations will help you understand how a feedback controller works. The figure shows an idealization of a quadcopter. To keep things simple, we will focus only on controlling its height, and assume that its roll, pitch and yaw are stabilized separately. Assume that:

• The quadcopter has mass m. • At time t=0 the quadcopter is on the ground, at z=0. • We wish to hold the quadcopter at a constant altitude *z . • We can measure the actual height z of the quadcopter; • We control the motors by sending a signal (a number between 0 and 60000) τ to the quadcopter.

A computer on-board the quad will then adjust the motor speeds to produce a total vertical thrust T βτ= , where β is a constant (which we have measured for you).

You will now derive the equations of motion for the quadcopter and its controller, and determine how it will behave.

Target altitude

zz*

m

(a) Write down F=ma for the quadcopter and hence show that its position is related to τ by 2

2d zm mgdt

βτ= −

(b) For a basic control system we could choose to set *( )PK z zτ = − where PK is a constant (proportional control). For this choice, use the solutions to vibration problems to write down the solution for ( )z t in terms of *z , m, g, PK and β . Does this controller work?

(c) For a better system, we can determine the velocity /dz dt of the quadcopter, and set *( )P D

dzK z z Kdt

τ = − − (proportional-derivative control). For this choice, write down the modified

equation of motion for y, and hence use your understanding of vibrations to (a) Give a formula for the value of DK that will make the quadcopter go to its final position as quickly as possible; and (b) For this choice, derive a formula for ( )z t in terms of *z , m, g, PK and β . .

(d) Notice that the quadcopter never gets to the right height with the controller in (c). We can do a bit

better if we measure the mass, and use *0 ( )P D

dzK z z Kdt

τ τ= + − − , where 0 /mgτ β= is the thrust

setting that will just support the weight of the quadcopter. What is the final position with this choice?

Summary: This idea is called ‘feedback control’ – the basic idea is to apply a control setting (the number τ ) that is determined by the difference *e z z= − between where you would like the quadcopter to be

*( )z , and where it actually is (z). For P-D control, we choose P DdeK e Kdt

τ = + . We can use the same

idea to control x,y position and the yaw of the quadcopter too: but instead of sending a thrust setting, we will send signals that control the roll, pitch and yaw rate. If you are wondering ‘What about the I in PID’ stay tuned – we will add this later. Complications: As is usual in engineering the basic operating principles in this design are simple but several complications arise when we try to implement it. The two big ones here are (a) The motor thrust does not just depend on τ - because of the airflow around the quad, the thrust depends on how the quadcopter is moving; and (b) It takes us a finite amount of time to read the Kinect images, process them to calculate the quadcopter position, and calculate the necessary value of τ . Your code will be able to run at about 20 frames per second. For this reason, we need to select controller settings that will make the proportional controller oscillate at about 0.5 cycles per second or slower. The main purpose of your design calculations is to estimate the required controller settings – with this initial estimate, you can fine-tune the design by hand. Optional additional calculations: If you like, you can set up a similar model that will help design the controller for horizontal motion. In this case, you will send pitch and roll settings (in degrees) to the quadcopter.

3.2 Writing a MATLAB quadcopter simulator You can do much more sophisticated design calculations by using MATLAB to solve the equations of motion for the quadcopter instead of doing them by hand. In this section, you will write a MATLAB code to simulate the motion of a feedback-controlled quadcopter in 3D, and will use the code to find values for the P-D control parameters ,PZ DZK K and ,PXY DXYK K controlling vertical and horizontal motion. For this step, we will consider the full 3D ( , , )x y z motion of the quadcopter. Assume that:

• The position and velocity of the quadcopter is specified in a fixed {i,j,k} basis, with i along the line of sight of the kinect, and k vertical (see the figure).

• We specify the desired position of the quadcopter as * * *( , , )x y z (these could vary with time) • A sensor on the quadcopter will measure the yaw angle ψ of the quadcopter. The yaw is the

rotation of the quadcopter about the Ne axis (with right hand convention) shown in the figure. We use the convention that yaw is zero when Le is parallel to i.

• The controller will send signals to the quadcopter that specify the thrust (a number between 0 and 60000) τ ; the roll and pitch angles ,θ φ (which represent rotations of the quadcopter about the ,L Te e axes, in degrees); and the yaw rate λ (in degrees per second) .

• These settings cause the motors to apply the thrust at an angle. The resulting vector components of thrust are

sin sin cos sin coscos sin sin sin cos

cos cos

x

y

z

TTT

θ ψ θ φ ψθ φ ψ θ ψ βτ

θ φ

+ = −

• The quadcopter will respond to the yaw signal λ by rotating about the Ne axis at a rate ddtψ λ=

With this pre-amble, work through the following calculations: (a) Use F=ma and the usual manipulations to show that the equations of motion for position ( , , )x y z ,

velocity , ,x y zv v v and yaw ψ are

(sin sin cos sin cos ) /(cos sin sin sin cos ) /

(cos cos ) /

x

y

z

x

y

z

x vy vz v

d v mdt

v mv m g

θ ψ θ φ ψ βτθ φ ψ θ ψ βτ

θ φ βτψ λ

= + −

−

(b) Write a MATLAB script that will use ode45 to calculate ( , , )x y z , velocity , ,x y zv v v and yaw ψ with constant values for , , ,τ φ θ λ . Use the following parameters:

• m=35 grams • 51.17 10 Nβ −= × (the units are Newtons) • Initial position and velocity ( 0, 0, 0)x y z= = = , 0, 0, 0x y zv v v= = =

Check your code by predicting the position and velocity of the quadcopter for a few (constant) values of , , ,τ φ θ λ and make sure the code gives the answers you expect.

(c) Next, add the control system to the calculation. As before, the basic idea is to make the controller

settings τ ; the roll and pitch angles ,θ φ (in radians); and the yaw rate λ respond to any deviation between the actual and desired positions of the quadcopter. Here is one way to do this1:

*0

* *

*

( )cos sin

cos sin

( ) ( )

( )

PZ DZ z

y x

x y

x PXY DXY x y PXY DXY y

P

K z z K vC C

C C

C K x x K v C K y y K v

K ψ

τ τθ ψ ψ

φ ψ ψ

λ ψ ψ

= + − −

= − +

= + +

= − − = − −

= −

Modify your MATLAB code to use these formulas to calculate τ ; ,θ φ inside the ‘EOM’ function.

1 The trig terms in the formulas for yaw and pitch need a little explanation. If the yaw is zero, then pitching the quadcopter will make it move parallel to the x direction, and rolling it will make the quadcopter move parallel to the y direction. But if the quadcopter yaws, this is no longer the case. The trig terms do the necessary basis change for you.

3.3 Adding Integral Control The P-D controller you implemented in the preceding section works quite well, but if the quadcopter is not perfectly balanced (i.e. it does not stay in one place when it is flying with zero roll and pitch angle), or if the thrust is not calibrated perfectly, the quadcopter tends to fly some distance away from the desired point, instead of exactly where it should be. The ‘I’ term in the P-I-D controller is designed to fix this problem. It works like this.

• The controller keeps track of the integrated error between the desired and actual position, by calculating

* * *

0 0 0( ) ( ) ( )

t t t

x y ze x x dt e y y dt e z z dt= − = − = −∫ ∫ ∫

• The feedback control signal is modified to include a term proportional to ( , , )x y ze e e as follows. *

0

* *

*

( )cos sin

cos sin

( ) ( )

( )

PZ DZ z IZ z

y x

x y

x PXY DXY x IXY x y PXY DXY y IXY y

P

K z z K v K eC C

C C

C K x x K v K e C K y y K v K e

K ψ

τ τθ ψ ψ

φ ψ ψ

λ ψ ψ

= + − − +

= − +

= + +

= − − + = − − +

= −

To understand why this works, notice that if the quadcopter is flying any distance away from the desired position the integrated error will continue increasing. A term proportional to the error is fed back to the controller. This means that the thrust and roll/pitch settings will be adjusted until the error no longer increases: if the error stops increasing, the quadcopter must be exactly at the desired position. You can test the effects of the ‘integral’ control by adding it to your MATLAB design code. When you write your quadcopter control code (section 6) you will need to do the integrals to calculate the error, but in the MATLAB design code it is more convenient to have the MATLAB ODE solver do the integrals. To do this, you can modify the equations in Section 3.2(a) to include the error variables to the unknowns. The modified system of equations becomes:

*

*

*

(sin sin cos sin cos ) /(sin sin cos sin cos ) /

(cos cos ) /

x

y

z

x

y

z

x

y

z

vxvyvz

mvmvd

m gvdt

x xee y ye z z

θ ψ θ φ ψ βτθ ψ θ φ ψ β

θ φ βτλψ

+ + = − − − −

At time t=0 we know 0x y ze e e= = = . Add the additional three lines to your matlab code to compute the error, and add the extra ‘I’ term to the feedback control equations. Then follow the steps in Section 3.4 to test your code, and to calculate values for the PID coefficients in your controller.

3.4 Tuning the controller parameters Finally, run the following calculations to obtain initial estimates for the controller parameters. In each case run the simulation for about 20-30 seconds, and plot the position and yaw as functions of time. Please follow these instructions carefully – incorrect values for the controller gains cause crashes that often break the tracking ball off the quadcopter. This is not a big problem as they are designed to be replaceable, but replacing them does take some time….

(a) Set the initial position, velocity, yaw and error to zero. Set the desired coordinates to * * * *0 1x y zψ= = = = . Set the controller parameters 0PXY DZ DXY IXY IZK K K K K= = = = = .

Then adjust the value of PZK to give a period of vertical oscillation of about 4 seconds. The value of PZK will probably be between 1000 and 10000

(b) Next, keep the initial and desired positions in (a) and set 0PXY DXY IXY IZK K K K= = = = . With the value of PZK in part (a) adjust DZK to give a slightly under-damped response (you should see a small overshoot of the desired position). The value of DZK will be similar to the value of PZK .

(c) Repeat (b) with the new values of PZK and DZK , but now gradually increase IZK until you just start to see a second oscillation in the height.

(d) Next, you need to tune the horizontal controller. Set the initial position, velocity and error in the quadcopter to zero. Set the desired coordinates to * * * *0 0.1z y xψ= = = = . Set

0DXY IXYK K= = and adjust the value of PXYK to give a period of horizontal oscillation (in the x direction) of about 3 seconds. The value of PXYK will probably be less than 1.

(e) Next, adjust DXYK to give a slightly under-damped response (you should aim to produce a small overshoot of the desired position, but no observable subsequent oscillations).

(f) Finally, increase IXYK until you just start to see a second oscillation start to appear in the x position.

(g) The last parameter to set is the yaw constant PK ψ . To set this * * * *0 1x y z ψ= = = = . Then adjust PK ψ so that the yaw reaches 90% of the desired value in about 2.5 seconds.

You now have values for the parameters in the feedback controller – you will use these values when you write your Matlab script to control the quadcopter (Section 5). 4. Thrust calibration To function correctly, the control system that you designed in the preceding sections requires a value for the constant 0τ that specifies thrust required to just support the quadcopter’s weight. (To see what happens when this term is not included in the control system, see the last part of this video). This turns out to be a little more complicated than it seems, because β is a function of the battery voltage: as the battery drains, the motor produces progressively less thrust. You can deal with this by continuously monitoring the battery voltage and using a formula to calculate

0τ . We have found that the following function works well:

0 101600 15300Vτ = − where V is the battery voltage.

5. Implementing the controller You will control your quadcopter by writing a MATLAB script that calculates the roll, pitch, thrust and yaw-rate settings to be sent to the quadcopter. The Python client that manages communications with the quadcopter will repeatedly call your function to obtain values for the control settings. The sample rate is typically of order 15-20 per second. A code template has been provided for you. The input and output arguments of this function should not be changed (they have to be compatible with the Python code that manages communications with the quadcopter). The template should be fairly straightforward to follow, but there is one feature of the code that needs a little explanation. Before take-off, the Python client will call your code to make sure that the Kinect has been able to detect the quadcopter and to obtain its initial position. During this phase the motor will not be turned on. The INITIALIZING variable will be set to 1 during this phase. You can test the value of this variable, and when it is set to 1, open any output files you need, and so on. Your MATLAB script should accomplish the following tasks: (i) Obtain the position and velocity of the quadcopter from the Kinect images. A MATLAB code

(described in the Appendix) has been provided for this purpose, but you can modify it if you wish. The code is capable of tracking several objects, as long as they have RGB color signatures that can be distinguished.

(ii) Calculate the integrated error measures

* * *

0 0 0( ) ( ) ( )

t t t

x y ze x x dt e y y dt e z z dt= − = − = −∫ ∫ ∫

You can use a simple Euler time-stepping scheme to continuously update the error, as follows: Let ( )xe t denote the error in x at time t. The error at a short time t t+ ∆ later is

*( ) ( ) ( )x xe t t e t t x x+ ∆ = + ∆ − You will need to store the values of ( , , )x y ze e e as persistent variables in your Matlab script (see the code template for instructions).

(iii) Calculate 0τ using the formula in Section 4 or the results of your measurements. (iv) Calculate the required controller settings from the formulas

*0

* *

*

( )cos sin

cos sin

( ) ( )

( )

PZ DZ z IZ z

y x

x y

x PXY DXY x Ix x y PXY DXY y Iy y

P

K z z K v K eC C

C C

C K x x K v K e C K y y K v K e

K ψ

τ τθ ψ ψ

φ ψ ψ

λ ψ ψ

= + − − +

= − +

= + +

= − − + = − − +

= −

The formulas above assume that the angles ,θ φ and the angular velocity λ are in radians. Note that the yaw sensor on the quadcopter gives ψ in degrees. You must also convert ,θ φ andλ into degrees when you return their values (as output variables from your function) to the flight controller.

(v) Print data recording the performance of the flight to a file. We suggest printing at least the desired position and the actual position of the quadcopter to a csv file. You can then read the file in MATLAB and plot the data after the flight, so you can see how accurately your controller is working. You can print a lot of other data as well, e.g. the velocity; the controller settings; the battery voltage; and (if you chose to log them) the accelerometer and gyro signals.

Of course, the desired positions * * *( , , )x y z can be functions of time. You will need to write code to make the quadcopter fly along some path. Don’t try anything too fancy for your first test – try

(1) Set the desired yaw * 0ψ = (2) With * *( , )x y held fixed, raise the quadcopter at constant speed for 5 sec until it reaches a height

0.4m above its initial position. (3) Next, with x,z held fixed move *x 0.4m towards the Kinect at constant speed for 5 s. (4) Hold position steady for 10 sec (5) With * *( , )x y Lower the quad to 0.1m below its initial height over 5 sec (6) At the end of the flight activate the ‘crash land’ sequence by returning FAIL=1. This will cause

the controller to power down the motor for a reasonably soft landing. Once you get this to work, you can try more complicated paths. It’s best not to try to fly too close to the ground – ‘ground effect’ makes the quadcopter very difficult to control, and the ground can also interfere with the Kinect’s depth measurement system. 6. Testing your controller: Once you have all your codes written, you can try a test flight. This is straightforward:

1) Make sure the takeoff stand is about 1.25m away from the Kinect 2) Log into the computer provided with your Brown user name and password (the same one you use

for the computers in the instructional computer lab). 3) Copy the MATLAB code controller.m to the desktop (if you wish, you can store it in some other

directory, but then you will need to browse to select the directory every time you open the flight manager).

4) Open the flight manager from the PC desktop. This will (eventually) open up a GUI. 5) If you stored your matlab codes somewhere other than the desktop, use the buttons provided to

browse to the folder. 6) Place the quadcopter on the takeoff stand, with its front facing away from the Kinect (the battery

connection is at the back of the quadcopter). 7) While the quadcopter is level, press the small switch at its front to turn it on. The LEDs should

flash, and the motors spin briefly if the power-up was successful. If not, try turning it on a second time.

8) By default, only the battery and yaw signals are logged from the quadcopter. If you wish, you can log data from the accelerometers and gyros on the quadcopter as well. This will slow down the controller communications slightly. Note that if you click the checkboxes the radio link will disconnect. A new radio link needs to be established to update the logger.

9) You can enter parameter values to be passed to your controller script if you need them (they get passed to your controller.m script and your code has to do something with them. If you enter nothing, that’s fine….).

10) You can preview the Kinect image if you wish (it is a good idea to do this before your first flight, just to make sure nothing red other than the ball appears in the field of view and that a depth image). This will open some matlab windows – click them to bring them to the front. The flight manager GUI will not respond until you close the Matlab windows (use the ‘close’ button at the bottom of the window to do this otherwise the connection to the python script will break).

11) Press the ‘scan’ button to search for quadcopters. If they are found they will be listed in the drop-down menu. Each quadcopter is set to communicate on a different radio channel – make sure you select the one you want to fly and not one being used by another group!. Then press ‘connect’ to initialize the radio connection. If the connection is successful the link quality

bar will display, a message appears in the Status bar, and (shortly afterwards) the battery voltage gets displayed. If the connection doesn’t work try again – also try restarting the quadcopter.

12) Press the ‘Start Flight’ button to begin a flight. It is helpful to have someone ready to catch the quadcopter for your first flight in case it goes crazy in case it goes crazy. If something goes wrong you can press the red button to abort the flight – this will activate a ‘crash land’ sequence that ramps down the motors and hopefully brings the quadcopter to a reasonably soft landing. Once you have a bug-free controller code and the correct controller gains the quadcopter should behave well, as long as it doesn’t lose radio contact.

Troubleshooting:

• If you press the ‘Start Flight’ button and the motors don’t spin up within a few seconds it is because of radio interference. In this case press the ‘Abort Flight’ button, disconnect the radio, and re-connect. You can try turning off and restarting the quadcopter as well. The problem usually resolves itself in a few minutes, but we haven’t found a way to stop this happening.

• Radio interference can also cause the controller to lose contact with the quadcopter during flight, in which case it might plummet to the ground, or fly off in some strange direction. Just try the flight again (assuming the quadcopter survives)

• The battery chargers are flakey. Move the battery connectors around in the sockets until the lights come on. The light turns off again when the battery is fully charged.

• Consult a TA or faculty member for help repairing damage… To fine-tune your controller, it is helpful to write a MATLAB script that will read the .csv file recorded during the flight and plot graphs showing the actual and desired positions of the quadcopter (and any other data you might find interesting). You can then try adjusting the controller settings to see if you can improve performance. You will find that the quadcopter is stable only for a rather narrow range of values of the feedback-controller constants – with the wrong values the quadcopter will oscillate around the desired position, sometimes quite violently. You will find that the optimal controller settings are functions of the battery voltage and will change as the battery discharges – you can either compromise and use a setting that works for the full range of battery voltages, or try to find a function that will compensate for the voltage drop. Once you are happy with your controller you can work on getting the quadcopter to do a more interesting flight. Here are a few suggestions:

(1) Use the Kinect to detect a (stationary) ball of a different color (green or blue) and make the quadcopter hover over it.

(2) Have the quadcopter follow a path specified by several balls (if you use the scripts provided, the balls would have to have distinct colors. You could modify the script that processes the Kinect image to distinguish between multiple balls of the same color, but this would be quite challenging)

(3) Program the quadcopter to follow a moving colored ball – you can attach a colored ball to the end of a stick and move it around by hand.

(4) Try to code some version of ‘Quadcopter tennis’ in which two people can direct the path of the quadcopter with different colored balls.

8. Report Guidelines Your report should

(1) Give the values you selected for your feedback control parameters (2) Describe how you calculated the feedback-control parameters, and show some graphs that predict

the expected behavior of the feedback-controller. (3) If you measured the motor thrust, report the results of this experiment. (4) Describe any measurements you made to test the actual performance of the control system (5) Describe anything interesting you might have programmed your quadcopter to do.

8. Grading Rubric Your project will be graded as follows:

• Matlab design calculations 10 points • Coding and tuning a feedback controller 5 points • Report writing and presentation 10 points • Oral presentation 10 points • Performance during testing 5 points • Peer Evaluation: 5 points

(45 Points)

APPENDIX: Image Processing MATLAB codes to read the Kinect images and calculate coordinates have been provided for you. This section provides a short overview of how this is done. The MATLAB codes are available on the course website and you can modify them if you like. The Kinect behaves much like a camera, but in addition to recording a digital color image, it also records a ‘depth’ image, which measures the distance of objects in the field of view from the sensor (it does this by measuring the time it takes for a laser beam to leave the kinect, reflect off a surface, and return to the Kinect sensor – see this article for some more detail). When you fly your quadcopter, you will be able to preview the images captured by the Kinect, and may need to tweak the default settings to make sure the images are captured correctly. A brief outline of the tracking algorithm follows. When you press the ‘preview kinect image’ button in the flight manager, a MATLAB window will open that displays the color image (the quadcopter GUI will be unresponsive until you close the matlab window). The figure below shows a representative image.

Some brief explanations:

• The red border in the color image marks the edge of the region where the quadcopter is permitted to fly. If the quadcopter crosses the red border an ‘out of frame’ error will be returned and the controller will activate an automatic ‘crash land’ sequence that powers down the motors.

• The software will try to detect the red ball attached to your quadcopter. It does this by searching for red pixels in the image (the algorithm will get confused if anything else red appears in the image).

• If enough red pixels are detected to estimate the position of the ball, a rectangle surrounding the quadcopter (or whatever red object is found) will be displayed in the color image. To speed up the tracking algorithm, only the image inside the green rectangle is searched for red pixels during flight. The code keeps track of the velocity of the quadcopter and tries to move the search rectangle to the expected position of the quadcopter each time the Kinect is sampled.

You can identify (and modify) which regions of the color image are detected by the code to be part of the red ball by pressing the ‘filtered color image’ button below the color image. Pixels that are red are shaded black in this image, any other color is white. If the coordinates can be extracted from the images, they are also displayed. A representative example is shown in the figure below (in the image shown, the window has been configured to tack a blue ball – the color of the rectangle surrounding the filtered pixels indicate the color filtering)

Note that

• Red pixels are identified by their ‘RGB’ values (red, green blue). The Matlab code searches for pixels that have RGB values that lie within a specified range.

• You can change the value and range for the RGB values by using the buttons on the bottom right hand corner of the filtered color image. Their values will be provided to your MATLAB control code while you fly your quadcopter.

• You can use the ‘objects’ menu to add additional colors to be tracked. If you press the ‘add’ button a static image will be displayed, and you can drag a small rectangle to sample the colors of the pixels you would like to track. After selecting the rectangle, double-click its center. A window will pop up displaying the average RGB values of the sampled pixels, and (once you acknowledge the message box) pixels filtered using the new RGB values will be displayed. You can adjust the filter mean values and filter windows until you see a clean image.

The color image is not used directly to calculate coordinates. Instead, it is used to determine the approximate region inside the ‘depth image’ that contains the quadcopter. You can view the ‘depth image’ by pressing the ‘depth image’ button at the bottom of the color or filtered color images.

• A white border on the outside edge of the depth image shows the edges of the region where the quadcopter is permitted to fly. If the quadcopter crosses the border an ‘out of frame’ error will be returned and the controller will activate an automatic ‘crash land’ sequence that powers down the motors.

• Dark blue pixels (with a value of 0) are areas where the laser-beam did not reflect properly and the Kinect could not measure a depth value.

• If one or more tracked objects are detected in the color image, the approximate region containing the object is outlined by a white rectangle in the depth image.

• The Matlab script determines the quadcopter co-ordinates by finding the closest point to the Kinect inside the rectangular region, and then calculating the centroid of all the pixels in the rectangular sub-region with depth values within 140mm of the closest point. This yields three numbers: (1) The distance of the centroid from the kinect, in mm; (2,3) The pixel coordinates of the centroid in the depth image. These have to be converted into physical (x,y,z) coordinates. The code provided uses a basic pinhole camera approximation to do this.