Radiometric calibration of a UAV thermal camera

Julia Kelly (j.r.kelly.919096@swansea.ac.uk)1, Lars Eklundh2 and Natascha Kljun12

1 Department of Geography, Wallace Building, Swansea University, Singleton Park,

Swansea, SA2 8PP, UK 2 Department of Physical Geography and Ecosystem Science, Lund University, Sölvegatan

12, S-223 62 Lund, Sweden

Acknowledgements

This work was funded by a Short Term Scientific Mission grant from COST Action

OPTIMISE ES1309

1. Introduction

Thermal cameras specifically designed for use on Unmanned Aerial Vehicles (UAVs) have

become more widely available in the last few years, but producing accurate, high-quality

temperature data with these cameras is still non-trivial (Berni, et al., 2009; Stark, et al., 2014;

Ribeiro-Gomes, et al., 2017). Thermal cameras designed for use on UAVs use uncooled

microbolometers which are lightweight and consume very little power compared to

conventional cooled thermal cameras. However, uncooled microbolometers do not maintain a

constant temperature and thus are very sensitive to changes in camera and external atmospheric

temperature (Budzier, et al., 2015). These issues are especially problematic when conducting

UAV surveys as the camera is subjected to changing atmospheric temperature, solar radiation

and wind conditions. Uncooled microbolometer cameras can also exhibit significant sensor

noise, due to the ‘vignetting’ effect, whereby the wide-angle of the lens causes distortion in

temperature measurements across the sensor, and non-uniformity noise as a result of differing

responses of individual pixels across the sensor (Meier et al., 2011; Goodall et al., 2016).

Further challenges arise when using non-radiometric (but more affordable) thermal camera

models. They provide only raw data in digital numbers (DN) and calibration must be performed

by the user to produce surface temperature data (Gómez-Candón, et al., 2016).

The aim of this Short Term Scientific Mission (STSM) was to perform a radiometric

calibration, and evaluate the performance, of a FLIR Vue Pro 640 UAV thermal infrared

camera. The FLIR Vue Pro is a non-radiometric thermal camera with an uncooled vanadium

oxide (VOx) microbolometer. Laboratory experiments were conducted to meet the following

four objectives:

a. Assess the stabilization time of the camera

b. Generate blackbody calibration curves for the camera

c. Assess the effect of wind and heating on the camera

d. Assess sensor noise

The knowledge generated from these experiments was used to design a set of recommended

best practices for conducting UAV flights. A workflow for processing thermal camera images

from UAV flights in Agisoft Photoscan Pro was also produced based on several UAV flights

conducted over a mire ecosystem at Skogaryd Research Station (Sweden).

2

2. Methods

A blackbody radiator (emissivity: 0.992) was placed 32cm away from the camera so that the

entire device was visible in the camera field of view. A PT100 resistance thermometer recorded

the temperature of the blackbody radiator every second. The camera was mounted on a tripod,

with a copper-constantin thermocouple attached to the top surface of the camera case recording

its temperature every second. A Kestrel 4500 pocket weather monitor recorded air temperature,

relative humidity and wind speed every 30 seconds. The experimental setup is illustrated in

Figure 1.

2.1 Assessing camera stabilization time

The purpose of the experiment was to assess the length of time required after switch on for the

camera to produce stable DN values. The camera was switched on and recorded images of the

blackbody radiator every 30 seconds for 2 hours. The blackbody radiator maintained a constant

temperature at 18.4°C (σ=0.2°C). To assess the change in camera raw signal over time, a

circular area representing the centre of the blackbody was selected on all the images recorded.

The same circle was used for all images and covered 33% of the total number of pixels in an

image. The mean DN value of all pixels within this circle was calculated for each image and

plotted over time. The results of the experiment showed the camera required a one hour

stabilization time and this was implemented before all further experiments.

2.2 Blackbody calibration curve generation

The FLIR Vue Pro 640 is not radiometric therefore it was essential to generate calibration

curves to model the relationship between camera raw signal and observed temperature. The

blackbody radiator was cooled in an incubation chamber overnight (Conviron CMP4030) at

4°C (the minimum temperature of the chamber). The minimum temperature reached by the

blackbody was 6.5°C. After stabilization of the camera, the blackbody was placed 32cm way

Kestrel 4500

recording air

temperature

and humidity

Blackbody

radiator

Copper-constantan

thermocouple recording

FLIR Vue Pro case

temperature

FLIR Vue Pro 640

Blackbody radiator

power supply

PT100 resistance

thermometer recording

blackbody temperature

Figure 1. Photo of experimental setup and equipment for radiometric calibration of

FLIR Vue Pro.

3

from the camera. The temperature of the blackbody was increased in 5°C steps (6.5°C, 10°C,

15°C, 20°C, 25°C, 30°C, 35°C and 40°C). The camera recorded 20 images of the blackbody at

each temperature interval. During the experiment, room temperature was stable at 20.6°C

(σ=0.3°C).

The same experiment was repeated to investigate how camera temperature affects measured

DN values. The camera and blackbody were placed inside the incubation chamber which

maintained an atmospheric temperature at 10.8°C (σ=0.1°C). The blackbody was cooled

overnight in the chamber at 10°C and reached a minimum temperature of 10.3°C. The

temperature of the blackbody was then increased in 5°C temperature steps (10.3°C, 15°C,

20°C, 25°C). The maximum temperature attainable by the blackbody in the chamber was 25°C

at the maximum voltage supplied by its power source. Image processing for both experiments

followed the same method as described in section 2.1. The standard deviation of the DN values

was also calculated for each temperature step.

2.3 Assessing the effect of wind and heating on the camera

An experiment was performed to test the influence of wind and external heating on camera

performance, mimicking the windy conditions and direct sunlight the camera is exposed to

during UAV flight. The camera recorded images of the blackbody every 30 seconds. A fan was

placed 50cm away from the blackbody and camera to blow wind over the camera. The Kestrel

4500 pocket weather monitor was positioned opposite the fan to record air temperature,

humidity and wind speed. After the camera had stabilized, the fan was set to its lowest speed

setting (2.0m/s, σ=0.2m/s wind speed) for 30 minutes. The fan was then switched off and the

camera stabilized for 1 hour. Next the fan was switched to its highest speed setting (3.3m/s,

σ=0.2m/s wind speed) for 30 minutes. The camera was then stabilized for 1 hour. The fan speed

used is comparable to the speed at which the UAV is flown (<5m/s). A control period of 30

minutes ensued when the camera continued recording images of the blackbody without any

wind or heating effect. Finally, a 500W halogen lamp was placed several centimeters above

the camera to heat it. The camera was heated until it reached 45°C (5°C below the maximum

operating temperature of the camera). The lamp was then turned off to avoid overheating the

camera, and the camera continued recording images of the blackbody for 30 minutes. Image

processing followed the same method as described in section 2.1.

2.4 Assessing sensor noise

An experiment was performed to test for noise among pixels on the sensor array. The camera

was moved closer to the blackbody radiator (8cm apart) so that the blackbody radiator entirely

filled the field of view of the camera. In this way, each pixel on the camera sensor should have

recorded the same temperature. The blackbody temperature was stable at 19.1°C (σ=0.1°C).

After stabilization, the frame rate of the camera was increased to record one image every second

for 30 minutes (1800 images captured). Operating at a higher frame rate mimicked the

conditions during UAV flight when the camera is recording images very frequently.

To determine the spatial noise affecting the sensor, two statistical analyses were performed. To

test for the vignetting effect, the mean DN recorded by each pixel across all images was

calculated. The mean value of each pixel (‘pixel mean’) was compared to the mean value of all

pixels across all images (‘image mean’). The sensor layout was plotted, with the colour of each

pixel representing the relationship between the pixel mean and image mean. The final plot thus

shows the mean deviation of each pixel from the image mean (variation across spatial area of

the sensor).

4

The method outlined in Aubrecht et al (2016) was used to assess the severity of non-uniformity

noise patterns across the sensor. The mean image DN was subtracted from each image. The

deviation of each pixel from the image mean across the 1800 recorded images was then

correlated to the deviation of the centre pixel from the image mean across all images. The

correlation coefficients where then plotted for each pixel in their respective locations across

the sensor to show whether neighbouring pixels exhibit similar behaviour (as would be

expected if striped patterns were present on the sensor).

3 Results

3.1 Assessing camera stabilization time

The camera experienced significant fluctuations in raw signal output during the first 15 minutes

of operation and required approximately 1 hour to fully stabilize (Figure 4). The experiment

was repeated and produced similar results.

3.2 Blackbody calibration curve generation

Calibration curves were generated at room temperature (camera temperature = 38°C) and in a

cold incubation chamber (camera temperature = 20°C; Figure 5). The camera raw signal has a

linear response to observed temperature. The camera uses a narrow range of DN values (FLIR

Vue Pro records 14-bit images that range between 0-16383 DN). Furthermore, the average

image signal varies little between frames recorded at each temperature step (σ <4 DN for all

temperature steps recorded) suggesting the camera is sensitive to <1°C temperature changes.

However, the object temperature recorded by the camera is clearly affected by the temperature

of the camera itself, with lower camera temperature associated with lower DN for a given object

temperature.

Figure 2. FLIR Vue Pro raw signal (digital numbers) over 2 hours after

camera switch on.

5

3.3 Effect of wind and heating on the camera

There were large fluctuations in camera DN when subjected to wind or heating treatment

(Figure 6). Using the calibration equations derived in Figure 5, the fluctuations in DN values

of the camera during the wind and heating experiment are equivalent to up to ~18°C change in

observed temperature. It was therefore difficult to determine a clear relationship between

camera temperature and DN that could be used to correct for the influence of wind and heating

on the camera. In this experiment, camera temperature is negatively related to DN, whereas the

calibration curve experiment (Figure 5) showed the opposite response.

3.4 Assessing sensor noise

To test for the effects of vignetting, the mean value of individual pixels was compared to the

image mean for all images recorded during the experiment (Figure 8). As expected, pixels

towards the edge of the sensor recorded lower than average values, while those towards the

centre recorded higher than average values. Although this effect could be attributed to uneven

heat distribution over the surface of the blackbody, a similar effect is noticeable on images

captured during UAV flight (Figure 9). The same analysis of vignetting effect was conducted

for 10 images captured during a UAV flight over a mire ecosystem at the Skogaryd Research

Station fieldsite. The lack of large, distinctive features in the mire vegetation enables it to act

as a mostly uniform surface (Figure 9a). Pixels in the lower corners of the images are darker

than those towards the top (Figure 9b).

Based on the calibration equations in Figure 5, it is estimated that a ~1°C temperature change

is observed by the camera as a change of ~25 DN (Figure 5). The results displayed in Figure

Figure 3. Blackbody calibration curves of FLIR Vue Pro 640. Calibration

curves were performed at room temperature (camera temperature = 38°C)

and in a cold incubation chamber (camera temperature = 20°C). Error bars

are standard deviation of raw camera signal but are very small (>4 DN).

6

8, show that the spatial noise across the camera sensor caused by the vignetting effect could

cause variability which is >25 DN. The results imply that the camera is accurate to 2°C.

Non-uniformity noise, which appears as a pattern of vertical or horizontal stripes is also present

on the FLIR Vue Pro sensor (Figure 10). The majority of the correlation coefficients are close

to 0, implying these noise patterns have low impact on overall image quality.

Figure 4. Effect of fan wind and lamp heating on FLIR Vue Pro 640 DN. Blue shaded areas indicate fan was switched on.

Red shaded area indicates lamp was switched on. Grey shaded area is control period with no wind or heat. No shading

(white areas) are camera stabilization periods with no wind or heat. Camera signal was stabilized for 1 hour following

each fan treatment.

Figure 8. Vignetting effect on the

FLIR Vue Pro. Individual pixel

means are compared to the image

mean. Grey pixel means fall within 1σ

of the image mean, whilst black and

white pixel means are < or > 1σ from

the image mean, respectively. Image

mean = 9167 DN, σ=8.26 DN.

7

3.5 Thermal image processing in Agisoft Photoscan Pro

Several orthomosaics were generated from test flights conducted before and after the STSM

over a mire ecosystem. The main challenge for orthomosaic construction was the low contrast

and low image quality (due to slow camera integration time causing blurry images).

Availability of flight data (telemetry data including pitch, yaw and roll) or ground-based

Figure 10. Correlation coefficients between each pixel and the centre pixel.

A B

Figure 9. Test for the vignetting effect using 10 images from a UAV flight over a mire ecosystem at the Skogaryd Research Station fieldsite.

A) Example image of the mire captured by the FLIR Vue Pro. B) Individual pixel means are compared to the image mean. Grey pixel

means fall within 1σ of the image mean, whilst black and white pixel means are < or > 1σ from the image mean, respectively.

8

geometric control points was essential for producing orthomosaics. Suggested parameters for

processing thermal imagery in Agisoft Photoscan Pro and a workflow can be found at the

following link: https://juliageographer.wordpress.com/2017/05/03/agisoft-photoscan-

workflow-for-flir-vue-pro/

4 Discussion and recommendations for camera use and UAV flight planning

The experimental results indicate that the FLIR Vue Pro 640 thermal infrared camera is

sensitive to observed temperature changes >2°C, accounting for variability across the camera

sensor. However, the accuracy of the measurements may be much lower due to the strong

dependence of camera DN on atmospheric conditions (Figures 5 and 6). Since the camera uses

an uncooled microbolometer, the temperature of the sensor will respond to changes in ambient

temperature. Fluctuations in sensor temperature have a significant impact on measurement

accuracy for two reasons. Firstly, each pixel on the array emits infrared radiation according to

the sensor temperature and receives infrared radiation emitted by the camera housing according

to its temperature. Secondly, the sensitivity of individual pixels and the relationship between

pixel voltage and signal output is dependent on the temperature of the pixel. Each pixel in the

array has a large field of view and therefore ‘sees’ a large portion of the camera interior

(Budzier and Gerlach, 2015). As a result, small changes in camera temperature can have a large

impact on the measurement. According to the manufacturer, since the FLIR Vue Pro is an

uncalibrated camera, neither of these effects are accounted for.

Theoretically, it should be possible to correct measured DN for camera temperature by

analysing the relationship between the two variables. A positive relationship between camera

temperature and measured DN would be expected since measured DN includes a large

proportion of the thermal radiation emitted by the camera interior. However, the results of the

wind and heating experiment show that there is no consistent relationship between camera

temperature and DN. It is unclear why the DN values show both large negative and positive

deviations from the control period value when the camera is either cooled or heated. Calls to

the manufacturer provided no further information on the grounds that a detailed explanation of

internal camera functioning and manufacturing would reveal proprietary information.

Given the strong temperature-dependency of uncooled, uncalibrated thermal cameras, their

ability to record stable measurements of surface thermal radiation while flying on a UAV will

be limited. It is therefore strongly recommended to purchase a radiometrically calibrated

camera which will include a mechanism or algorithm to account for the temperature-

dependency of the sensor. Nevertheless, the accuracy of these cameras is still fairly low (± 5°C

for the FLIR Vue Pro R) and the sensitivity and offset of individual sensor pixels which varies

according to sensor temperature may not be corrected for (Ribeiro-Gomes, et al., 2017).

Therefore the following best practices are recommended to minimize the effects of changing

atmospheric conditions during UAV flight on the camera:

• Turn on camera at least 15mins before take-off to allow camera startup sequence and a

stable operating temperature to be reached (Berni, et al., 2009; Gómez-Candón, et al.,

2016)

• Add extra flight lines before the start of measurements to allow camera to acclimatize

to ambient temperature, wind and solar radiation conditions while flying

• Shelter camera when on UAV, either by placing the camera in the body of a fixed wing

UAV or by building a small shelter when the camera is flying on a quadcopter

9

• Fly slowly to reduce the cooling effect of wind on the camera and to allow collection

of high quality, non-blurry images

• Temperature calibration in the field is essential for uncalibrated cameras in order to

convert DN to surface temperature but also provides validation data for radiometrically

calibrated cameras.

Field calibration can be achieved by measuring ground targets with an IR radiometer, a second

IR camera or thermocouples (Berni, et al., 2009; Jensen, et al., 2013, Gómez-Candón, et al.,

2016). Ideally, ground target temperature measurements should be made during a UAV flight

or as soon as possible before or after the flight. Temperature calibration or validation should

be performed separately for each UAV flight since atmospheric conditions and camera

temperature will likely change between each flight. By measuring ground targets with a wide

range of temperatures, a simple empirical line calibration can be performed. Calculating a

direct relationship between DN and ground target surface temperature omits the need to

measure and account for the proportion of measured DN that comes from radiation emitted by

the camera interior. It also accounts for changes in atmospheric conditions (air temperature,

humidity, solar radiation) which affect the relationship between at-sensor measured radiance

and surface temperature (Berni, et al., 2009; Hammerle, et al., 2017). Ground targets should be

large enough to cover several thermal camera pixels so that they can be easily located on the

images and have a high emissivity.

The analysis of sensor noise revealed a significant vignetting effect and minimal ‘stripy’ noise

patterns across the sensor. The stripes are caused by non-uniformity noise patterns whereby

individual pixels behave differently as a result of slight differences during the manufacturing

process and degradation of the sensor over time. Most thermal cameras including the FLIR

Vue Pro perform automatic non-uniformity or flat-field corrections as the non-uniformity noise

can be relatively large compared to the desired infrared radiation signal (Budzier, et al., 2015).

During the flat-field correction the shutter is closed, presenting an isothermal surface to the

sensor and thus the offset coefficients of each pixel can be adjusted. In radiometrically

calibrated cameras, the NUC can also be used to account for internal camera temperature

fluctuations with the assumption that the sensor temperature is the same temperature as the rest

of the camera interior. The correction appears to be working well in the FLIR Vue Pro as only

minimal non-uniformity noise patterns are visible across the sensor and these are generally not

visible on individual images taken by the camera. Although the vignetting effect remains

significant, it is usually compensated for when images are stitched together in photogrammetry

software.

Orthomosaic creation from thermal images is generally more challenging than when using

optical data (Ribeiro-Gomes, et al., 2017). Thermal imagery can have low contrast when

measuring areas with small variations in surface temperature since they are often designed to

measure very large temperature ranges (for example, -40°C to 550°C for the FLIR Vue Pro R).

As a result, a lower number of tie points are identified in UAV image stitching software causing

some images to be omitted from the orthomosaic, high uncertainty in the georeferencing

process and introduction of stitching artefacts into the final orthomosaic. Furthermore, the

slow integration time of the camera often causes blurry images to be captured during flight. To

minimize these issues, several recommendations for flight campaign design are outlined below:

Fly slowly to reduce occurrence of blurry images and extend flight lines beyond

measurement area so that images taken during UAV turns can be omitted from the

orthomosaic

10

Ensure a high front- and sidelap between images (ideally >80% in both directions) so

that there is plenty of redundancy and the lowest quality images can be omitted from

the orthomosaic

Use a gimbal or other solution to minimize vibration of the camera

Conduct several flights over the same area at different angles to minimize the effects of

bi-directional reflectance and increase the image density to increase the quality of the

final orthomosaic

References

Aubrecht, D. M.; Helliker, B. R.; Goulden, M. L.; Roberts, D. A.; Still, C. J. & Richardson, A.

D. Continuous, long-term, high-frequency thermal imaging of vegetation: Uncertainties and

recommended best practices. Agricultural and Forest Meteorology, 2016, 228–229, 315 - 326

Berni, J. A. J; Zarco-Tejada, P. J.; Suárez, L. & Fereres, E. Thermal and Narrowband

Multispectral Remote Sensing for Vegetation Monitoring from an Unmanned Aerial Vehicle.

IEEE Transactions on Geoscience and Remote Sensing, 2009, 47, 722-738.

Budzier, H. & Gerlach, G. Calibration of uncooled thermal infrared cameras. Journal of

Sensors and Sensor Systems, 2015, 4,187-197.

Gómez-Candón, D.; Virlet, N.; Labbé, S.; Jolivot, A. & Regnard, J-L. Field phenotyping

of water stress at tree scale by UAV-sensed imagery: new insights for thermal acquisition

and calibration. Precision Agriculture, 2016, 17, 786-800.

Goodall, T. R.; Bovik, A. C. & Paulter, N. G. Tasking on Natural Statistics of Infrared Images.

IEEE Transactions on Image Processing, 2016, 25, 65-79

Hammerle, A.; Meier, F.; Heinl, M.; Egger, A. & Leitinger, G. Implications of atmospheric

conditions for analysis of temperature variability derived from landscape-scale thermography.

International Journal of Biometeorology, 2017, 61, 575-588.

Jensen, A. M.; McKee, M., & Chen, Y. Calibrating thermal imagery from an unmanned aerial

system – AggieAir. IEEE International Geoscience and Remote Sensing Symposium - IGARSS,

Melbourne, VIC, 2013, 542-545.

Meier, F.; Scherer, D.; Richters, J. & Christen, A. Atmospheric correction of thermal-infrared

imagery of the 3-D urban environment acquired in oblique viewing geometry. Atmospheric

Measurement Techniques, 2011, 4, 909-922

Ribeiro-Gomes, K.; Hernández-López, D.; Ortega, J. F.; Ballesteros, R.; Poblete, T. &

Moreno, M., A. Uncooled Thermal Camera Calibration and Optimization of the

Photogrammetry Process for UAV Applications in Agriculture. Sensors, 2017, 17, 2173.

Stark, B.; Smith, B. & Chen, Y. Survey of Thermal Infrared Remote Sensing for Unmanned

Aerial Systems. International Conference on Unmanned Aircraft Systems (ICUAS), Orlando,

FL, 2014, 1294-1299