76
AUTOMATIC THERMAL INSPECTION OF ALUMINIUM REDUCTION CELL June 2012 Guðjón Hugberg Björnsson Master of Science in Electrical Engineering

AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

  • Upload
    lamhanh

  • View
    212

  • Download
    0

Embed Size (px)

Citation preview

Page 1: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

AUTOMATIC THERMALINSPECTION OF ALUMINIUM

REDUCTION CELL

June 2012Guðjón Hugberg Björnsson

Master of Science in Electrical Engineering

Page 2: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering
Page 3: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

AUTOMATIC THERMAL INSPECTION OFALUMINIUM REDUCTION CELL

Guðjón Hugberg BjörnssonMaster of ScienceElectrical EngineeringJune 2012School of Science and EngineeringReykjavík University

M.Sc. RESEARCH THESISISSN 1670-8539

Page 4: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering
Page 5: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Automatic thermal inspection of aluminium reduction cell

by

Guðjón Hugberg Björnsson

Research thesis 60 ECTS submitted to the School of Science and Engineeringat Reykjavík University in partial fulfillment of

the requirements for the degree ofMaster of Science in Electrical Engineering

June 2012

Research Thesis Committee:

Jón Guðnason, SupervisorDR, Reykjavik University

Agni Ásgeirsson, Co-SupervisorMSc, Alcan á Íslandi hf.

Joseph Timothy Foley, ExaminerDR, Reykjavik University

Page 6: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

CopyrightGuðjón Hugberg Björnsson

June 2012

Page 7: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Date

Jón Guðnason, SupervisorDR, Reykjavik University

Agni Ásgeirsson, Co-SupervisorMSc, Alcan á Íslandi hf.

Joseph Timothy Foley, ExaminerDR, Reykjavik University

The undersigned hereby certify that they recommend to the School of Sci-ence and Engineering at Reykjavík University for acceptance this researchthesis entitled Automatic thermal inspection of aluminium reduction cellsubmitted by Guðjón Hugberg Björnsson in partial fulfillment of the re-quirements for the degree of Master of Science in Electrical Engineer-ing.

Page 8: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Date

Guðjón Hugberg BjörnssonMaster of Science

The undersigned hereby grants permission to the Reykjavík University Li-brary to reproduce single copies of this research thesis entitled Automaticthermal inspection of aluminium reduction cell and to lend or sell suchcopies for private, scholarly or scientific research purposes only.

The author reserves all other publication and other rights in association withthe copyright in the research thesis, and except as herein before provided, nei-ther the research thesis nor any substantial portion thereof may be printed orotherwise reproduced in any material form whatsoever without the author’sprior written permission.

Page 9: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Automatic thermal inspection of aluminium reduction cell

Guðjón Hugberg Björnsson

June 2012

Abstract

Modern aluminium production facilities all over the world experience cellleakages on a regular basis. This is one of the most dangerous parts of thealuminium production process. No aluminium manufacturer has managed toeliminate this problem entirely; this is a persistent problem without a solu-tion. The objective of this project is to develop a system that can log tempera-ture measurements of collector bars in an aluminium reduction cell and storethem in a database. The system would replace manual thermal inspection.Thermal images of the cell collector bars were captured with a FLIR E40thermal camera. The thermal images were analyzed with modern computervision techniques: linear boundaries are used to split the images into classes.The thermal images are relatively homogenous and can be split into cate-gories with 99.95% specificity. The thermal reading from the images havethe same accuracy as the thermal camera: ±2°C or ±2%. The applicationof this system is more frequent thermal inspection with data logging for longterm process control. This should result in a safer industrial environment andmore cost efficient pot maintenance.

Page 10: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Sjálfvirkt hitaeftirlit með álframleiðslukerum

Guðjón Hugberg Björnsson

júní 2012

Útdráttur

Kerlekar eru þekkt vandamál við álframleiðslu um allan heim. Engum ál-framleiðanda hefur enn tekist að koma algerlega í veg fyrir kerleka, en þettaer einn hættulegasti þáttur framleiðslunnar. Markmið þessa verkefnis er aðhanna kerfi sem mælir hitastig bakskauta rafgreiningarkera í Ísal á sjálfvikanhátt og vistar niðurstöður í gagnagrunn. Þetta eftirlit kemur til með að leysaaf hólmi það eftirlit sem nú er framkvæmt af starfsmönnum Ísal. Myndir afbakskautum rafgreiningarkeranna eru teknar með FLIR E40 hitamyndavél.Myndirnar eru síðan greindar með myndgreiningarforriti þar sem hitastigbakskautanna er reiknað. Hitamyndirnar eru frekar einsleitar og því var unntað skipta þeim í flokka með 99.95% nákvæmni. Tilraunir sýndu fram áað aðferðir sem notaðar eru í verkefninu við greiningu hitastigs bæta ekkiá ónákvæmni hitamyndavélarinnar. Nákvæmni reyndist vera sú sama ognákvæmni hitamyndavélarinnar eða ±2°C eða ±2% af mældu gildi. Meðverkefninu er sýnt fram á að sjálfvirkt hitaeftirlit mun nýtast vel við þessarmælingar. Sjálfvirkt hitaeftirlit hefur í för með sér mun tíðari mælingar semgeta leitt af sér meira öryggi fyrir starfsmenn auk hagræðingar í viðhaldi raf-greiningarkera.

Page 11: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

v

Acknowledgements

I want to thank Dr. Jón Guðnason for his help and guidance through the course of thisassignment. Símon Elvar Vilhjálmsson, for his inspiration and advices. Stefán FreyrStefánsson, for his insight and help with computer vision. Elín Adda Steinarsdóttir,Reynir Hólm Gunnarsson and Hrafn Leó Guðjónsson for their help with reviewing thethesis.

Ísal for providing access to thermal camera and permission to take live thermal measure-ments. Furthermore, Agni Ásgeirsson at Ísal, for his motivation, insight and knowledgeof the aluminium production process and providing data for the project. Pétur Hreinssonat Ísal, for his advice and providing data for the project.

Page 12: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

vi

Page 13: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

vii

Contents

List of Figures ix

List of Tables xi

1 Introduction 1

2 Background 32.1 Computer vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.1.1 Computer vision programming language . . . . . . . . . . . . . . 42.2 Thermography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2.1 Emissivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2.2 Infrared Thermal sensors . . . . . . . . . . . . . . . . . . . . . . 72.2.3 Field of view and camera resolution . . . . . . . . . . . . . . . . 82.2.4 Atmospheric attenuation . . . . . . . . . . . . . . . . . . . . . . 9

2.3 Thermal properties of an aluminium reduction cell . . . . . . . . . . . . 102.4 System setup and data collection . . . . . . . . . . . . . . . . . . . . . . 12

2.4.1 Thermal camera for data collection . . . . . . . . . . . . . . . . 132.4.2 Camera setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.4.3 System dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . 142.4.4 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 Image analysis 193.1 Image features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.2 Feature extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.2.1 Crop image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.2.2 Contour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.2.3 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.3 Feature selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.3.1 Feature selection methods . . . . . . . . . . . . . . . . . . . . . 24

Page 14: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

viii

3.3.2 Decision boundary test method . . . . . . . . . . . . . . . . . . . 273.4 Image analysis results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.4.1 Feature selection uncombined methods . . . . . . . . . . . . . . 283.4.2 Feature selection combined methods . . . . . . . . . . . . . . . . 323.4.3 Image analysis for unseen test data . . . . . . . . . . . . . . . . . 37

3.5 Image analysis summary . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4 Thermal inspection 394.1 Temperature measurement . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.1.1 Cell temperature . . . . . . . . . . . . . . . . . . . . . . . . . . 394.1.2 Maximum temperature measurement . . . . . . . . . . . . . . . 404.1.3 Temperature measurement analysis method . . . . . . . . . . . . 40

4.2 Notifications and data storage . . . . . . . . . . . . . . . . . . . . . . . . 434.3 Thermal inspection result . . . . . . . . . . . . . . . . . . . . . . . . . . 44

4.3.1 Accuracy of temperature measurement . . . . . . . . . . . . . . 444.3.2 Repeatability of temperature measurement . . . . . . . . . . . . 454.3.3 Angle of attack . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4.4 Thermal inspection summary . . . . . . . . . . . . . . . . . . . . . . . . 47

5 Summary and Discussion 49

Bibliography 51

A Cell leakages in Ísal 53

B Thermal inspection 55

Page 15: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

ix

List of Figures

2.1 Blue bottle cap. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.2 Infrared image of a fuse box. . . . . . . . . . . . . . . . . . . . . . . . . 52.3 Steel pot at different temperatures . . . . . . . . . . . . . . . . . . . . . 62.4 Two types of thermal sensors. . . . . . . . . . . . . . . . . . . . . . . . . 82.5 IFOV versus distance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.6 Atmospheric attenuation. . . . . . . . . . . . . . . . . . . . . . . . . . . 102.7 Reduction cell with prebaked anodes. . . . . . . . . . . . . . . . . . . . 112.8 Thermal energy losses of a reduction cell. . . . . . . . . . . . . . . . . . 122.9 Flir E series thermal camera. . . . . . . . . . . . . . . . . . . . . . . . . 132.10 Carrier setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.11 Set of collector bars. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.12 Degrees of freedom. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.13 Sequence of images when the carrier is moving forward. . . . . . . . . . 162.14 Sequence of images how the carrier can move sideways in operation. . . . 162.15 Basement in Ísal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.1 Smelter setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.2 Types of images. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.3 Image properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.4 Image moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.5 Center point location boundary . . . . . . . . . . . . . . . . . . . . . . . 263.6 Major principal axis orientation . . . . . . . . . . . . . . . . . . . . . . . 273.7 Center point distribution for methods X1 and X2. . . . . . . . . . . . . . 283.8 False acceptance ratio versus false rejection for methods X1 and X2. . . . 293.9 False acceptance ratio and false rejection ratio versus center bias for meth-

ods X1 and X2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.10 Object orientation versus image number for methods X3 and X4. . . . . . 303.11 False acceptance ratio versus false rejection ratio for methods X3 and X4. 31

Page 16: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

x

3.12 False acceptance ratio and false rejection ratio versus object orientationfor methods X3 and X4. . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.13 Object orientation for combined methods X1 and X3,X4. . . . . . . . . . 333.14 False acceptance ratio versus false rejection ratio for combined methods

X1 and X3,X4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.15 False acceptance ratio and false rejection ratio versus object orientation

for combined methods X1 and X3,X4. . . . . . . . . . . . . . . . . . . . 343.16 Orientation versus image number for combined methods X2 and X3,X4. . 353.17 False acceptance ratio versus false rejection ratio for combined methods

X2 and X3,X4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.18 False acceptance ratio and false rejection ratio versus the orientation for

combined methods X2 and X3,X4. . . . . . . . . . . . . . . . . . . . . . 363.19 Center position of objects in test data from pot room 1 and pot room 3. . . 37

4.1 Cell temperature measurement. . . . . . . . . . . . . . . . . . . . . . . . 404.2 Thermal accuracy measurement setup. . . . . . . . . . . . . . . . . . . . 414.3 Angle of attack temperature measurement. . . . . . . . . . . . . . . . . . 434.4 Temperature versus image width. . . . . . . . . . . . . . . . . . . . . . . 464.5 Temperature versus image height and image width. . . . . . . . . . . . . 46

Page 17: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

xi

List of Tables

3.1 False acceptance and false rejection ratio for all methods . . . . . . . . . 323.2 False acceptance and false rejection ratio for all methods . . . . . . . . . 36

4.1 Temperature measurements in oven . . . . . . . . . . . . . . . . . . . . 444.2 Repeatability temperature measurements . . . . . . . . . . . . . . . . . . 45

A.1 Cell leakages in Ísal from 2000-2012 . . . . . . . . . . . . . . . . . . . . 53

B.1 Mean and standard deviation for each collector bar for each run . . . . . . 55

Page 18: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

xii

Page 19: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

1

Chapter 1

Introduction

Reykjavik University in cooperation with Ísal (Rio Tinto Alcan Iceland) have been work-ing on an automatic thermal inspection system for use in Ísals aluminium smelter inStraumsvík Iceland. The project started as an undergraduate course in Reykjavik Uni-versity’s engineering department called Hönnun X. The students designed and built thefirst prototype of the carrier robot and control system.

The aluminium is produced in aluminium reduction cells. Over time, these cells start tobreak down and the insulation fails. When this happens, molten aluminium and electrolytecomes in contact with the outer shell of the cell. These molten metals are highly corrosiveand rapidly perforate the outer shell allowing the metals to freely leak out.

Cell leakage is a frequent malfunction in aluminium smelters all over the world. In theyears from 2000 to 2012 about 80% of all cells in Ísal ended their service life with somekind of cell leakage (see Table A.1). Every failing cell is a great danger to workers. Itcan also damage equipment and facilities. Finally some pollution always escapes to theenvironment. No aluminium smelter has managed to entirely eliminate cell leakages: thisis a persistent problem without an existing solution.

In normal operation, a great deal of the cell’s heat escapes through the side walls andcollector bars. When the insulation material starts to break down, the thermal resistanceof the material decreases and more heat can escape though the collector bars and sidewalls causing them to increase in temperature.

Today, a thermal inspection is performed regularly on the cell’s collector bars. This in-spection is done manually without any data logging. Manual data logging of the collectorbar temperatures is very demanding, if not impossible, because the total number of col-lector bars in the smelter is 19200. This periodic inspection has decreased the number of

Page 20: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

2 Automatic thermal inspection of aluminium reduction cell

cell tap outs per year, but it has not eliminated the problem. This inspection and othermeasurements in Ísal detect about 20% of all failing cells and are entirely based on theoperators assessment of situations each time.

Objective of the project

The objective of this project is to develop a system that can log temperature measurementsof the collector bars in aluminium reduction cells and store them in a database. Thetemperature measurements will be analyzed from thermal images coming from a movingFlir E40 thermal camera. The system is intended to be a part of an autonomous inspectionrobot that carries the thermal camera between cells. The system will be analyzed in termsof accuracy, resolution and repeatability.

Thesis Structure

The project is split into two elements. The first element focuses on the image analysis.The image has to be analyzed in terms of feature extraction and feature selection to beable to measure temperature of the cell. The field of computer vision offers many differentmethods to implement this analysis. This part will mainly discuss the methods that wereused in this project and how they were developed.

The second element considers the thermal inspection automation. When images havebeen analyzed the temperature information can be extracted. The main focus in this partis the quality of the thermal inspection in terms of temperature measurement accuracy,resolution and repeatability.

Page 21: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

3

Chapter 2

Background

The project relies on many different tools and techniques from different fields, such ascomputer vision, thermography and the physical properties of aluminium reduction cells.This chapter is intended to give the reader an insight into the projects background.

2.1 Computer vision

Throughout the years machines and robotic devices have relied on all kinds of sensorsto gather data about the surrounding environment in order to be able to take intelligentdecisions. For the last decade or so, vision systems have become more and more common.This is an byproduct of the development of personal computers. Computers are alwaysgetting smaller, faster, cheaper, and more energy efficient and thus are always gettingmore available to be used in a robot.

Computer vision is far from being simple to implement on robots. When a human sees aball, it knows from years of experience that this is a ball. We see the colour of the ball,under a huge range of lighting conditions. When a computer sees a ball it is just an arrayof numbers generated from light that fell on the camera sensor array. Each pixel has norelation to the pixels surrounding it, so even if there is a straight line on the image, thecomputer does not notice it without appropriate image processing algorithm.

Figure 2.1 shows three images of a bottle cap. Figure 2.1a shows the original image ofa blue bottle cap. Figure 2.1b shows the image of the cap in low resolution in grayscale.Figure 2.1c shows the pixel values for the low resolution image of the cap. It is easyfor the human eye to see that this is a blue cap of a bottle in Figure 2.1a. On the lowresolution image(Figure 2.1b) we still see that this is some circular object. The computer

Page 22: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

4 Automatic thermal inspection of aluminium reduction cell

(a) (b) (c)

Figure 2.1: (a) A blue bottle cap. (b) Grayscale low resolution version of Figure (a). (c)Pixel value array of Figure (b). The computer sees the image as an array of numbers asshown on Figure (c).

experiences the world as an array of numbers as on Figure 2.1c. The purpose of imageprocessing algorithms is to extract features and recognize objects in the measured arrayof numbers which is the image. Figure 2.1a has the resolution of 1073x1134 pixels in3 layers so the computer has examine nearly four million values for this single image.The image processing software provides tools to make the computer able to interpret theimage.

2.1.1 Computer vision programming language

Computer vision can be practically implemented in nearly any programming languagein any programming environment. There are many readily available computer visionlibraries with precompiled optimized functions to accelerate image processing.

In this project, OpenCV was the function library used with the Python programming lan-guage. OpenCV in Python is a good combination, as OpenCV provides a vast range offunctions that are optimized for realtime applications and Python provides an easy pro-gramming language that does not need compilation. This is a good combination allowingfast development without sacrificing system performance. All data analysis for featureselection was performed in Matlab.

2.2 Thermography

There are three ways for thermal energy to be transferred to or from an object: conduction,convection and radiation. The flow of energy is from a warmer object to the colder object.

Page 23: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 5

Conduction and convection are familiar to most people. Heat conduction occurs whena pot is on a turned-on stove. The thermal energy in the stove heats up the pot withconduction on the bottom surface of the pot. When a mass of air is flowing through aradiator cooling a car’s engine is forced heat convection. The classic example of radiationis warm sunlight on a dark surface.

All objects with temperature over 0 K radiate thermal energy. The energy increases withtemperature by power of four(Equation 2.1). Most thermal energy is radiated as electro-magnetic waves at infrared wavelengths between 0.74 µm up to 100 µm. If an objectradiates thermal energy at higher frequency it gets visible, glowing red hot. Incandescentlight bulbs use this physical property to make visible light by heating a wire up to 3000 Kor 2726°C [1].

With the use of a thermal camera, the invisible part of the electromagnetic spectrum iscaptured and portrayed as a visible image. Instead of showing colours, this image showsthe temperature of the scene. This feature can be extremely useful as it makes noncontactthermal inspection possible in a dangerous environment. Figure 2.2 shows an infraredimage of a fuse box under a load. The image has an iron colour pallet to interpreter thetemperatures to make analysis easier for the operator. The scale on the right is showingtemperature in °C. From this image, it is clear that one circuit is under much heavier loadthan the others. Figure 2.2b shows the fuse. This kind of analysis is difficult with tradi-tional thermometers, the circuit has to be taken out of operation and many temperaturemeasurements have to be performed to get thermal readings from all objects.

(a) (b)

Figure 2.2: Infrared image of a fuse box. One circuit is under much heavier lode than theothers. Hotter wires indicate more current.

Page 24: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

6 Automatic thermal inspection of aluminium reduction cell

2.2.1 Emissivity

All objects emit thermal energy, but they can also absorb and reflect energy. A materialthat absorbs all of the energy that falls on it is called a perfect black body. The perfectblack body is also a perfect emitter [1].

To estimate the temperature of an object, the objects ability to emit energy has to beknown. This ability is described with emissivity, a number from 0-1. The emissivity isdifferent between materials and the surface finish of the material. Polished steel has lowemissivity and highly rusted steel has emissivity close to unity.

(a) (b) (c)

Figure 2.3: The pot on Figure (a) contains cold water. The pot on Figure (c) contains hotwater. Figure (b) shows an image of the pot. A piece of black electrical tape attached tothe pot has thermal emissivity close to unity and shows the right temperature of the pot.Polished steel has poor thermal emissivity as shown on the figure.

Figure 2.3 shows how emissivity is important for correct thermal analysis. The steel poton the left is filled with cold water and the one on the right is filled with hot water. Steelis a good conductor and by touching the pot the user can feel the temperature of thecontent. While steel is a good conductor, polished steel is a poor emitter. A piece ofplastic tape has emissivity of about 0.95 and therefore is a good emitter displaying theaccurate temperature of the pot [2].

Infrared thermal meters measure thermal energy H emitted from the object. To calculatethe objects temperature this formula is used solved for T ,

H = Aeσ(T 4 − T 4s ) (2.1)

where T is the temperature of the object in K, H is the thermal energy measured by thecamera, A is the surface area of the object or the area of the camera pixel at the dis-tance of the object, e is the emissivity, σ is the Stefan Boltzman constant σ = 5.670400 ·10−8W/m·K4 and Ts is the ambient temperature or the reference temperature of the cam-era in K [1].

Page 25: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 7

When the only available thermal information is relative to a fixed emissivity constant e itis relatively easy to calculate the temperature of an object with other emissivity constantenew by setting H = Hnew. The surface area A and σ are constants. By solving for Tnewthe formula is,

Tnew = 4

√eT 4

enew− eT 4

s

enew+ T 4

s .

When the emissivity of a material is unknown it can be measured by heating the object,preferably with air, to a known temperature T and the emissivity can be calculated withthe same formula solved for enew [1],

enew =e(T 4

ir − T 4s )

T 4 − T 4s

where e is the emissivity setting of the thermal camera, Tir is the thermal reading fromthe camera and T is the actual temperature of the object.

2.2.2 Infrared Thermal sensors

Sensors in thermal cameras can be split into two groups, thermal and photon detectors.Thermal sensors work at two different wavebands of the electromagnetic spectrum, midwave 3 − 5 µm and long wave 7.5 − 13.5 µm. Thermal sensors are not practical at wavelengths from 5 − 7.5 µm, this will be better explained in Section 2.2.4.

Thermal detectors respond directly to the thermal energy that falls on the sensor. The sen-sor is created from metal or semiconductor material. Thermal energy changes physicalproperties in the sensor which influence capacity, resistance, voltage, or current depend-ing on the system being used. This change in physical properties is then measured andinterpreted into temperature. The most common version of thermal sensors is the microbolometer which works in the long wave band.

Photon detectors react directly to the photons, where the photon helps free electrons inthe sensor to reach a higher energy state, changing the physical properties of the sensor,capacity, resistance, voltage or current. The physical properties are very sensitive to theenergy state in the sensor, so these sensors need cooling to minimize influences. By cool-ing these sensors down to 200K-70K, depending on the sensor material, these sensorsbecome very sensitive and can react to small changes in thermal energy. Photon sensor

Page 26: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

8 Automatic thermal inspection of aluminium reduction cell

material works on a limited gap in the spectrum making them limited for usage in gen-eral purpose thermometers. Photon detectors are widely used in advanced defense andresearch systems.

Most commercial thermal cameras use a sensor from the thermal family of sensor or themicrobolometer. These sensors have a broad thermal spectrum, they are small, they areenergy efficient and have a low production cost. The disadvantage of this type of sensorsis that they have a slower response time (12 ms) than the photon sensors (1 µs) [2].

Figure 2.4 shows two thermal camera cores. The size and weight difference is quite big.The uncooled microbolometer core on Figure 2.4a weighs 28.5g with lens and the cooledcore on Figure 2.4a weighs around 500g. Both these cores are one of the lightest in theircategory of sensors.

(a) FLIR Quark uncooled mi-croboolometer sensor

(b) FLIR Photon HRC cooledsensor

Figure 2.4: Two types of thermal sensors http://www.flir.com/cvs/cores/.

2.2.3 Field of view and camera resolution

To be able to detect the object of interest, the thermal sensor needs sufficient resolutionto detect the object in one or more pixels. The camera lens and distance between thesensor and the object determine the field of view at the plane of interest. The energy thatfalls on one pixel is averaged, if the resolution is too low it is possible to miss valuableinformation. For example if half of the pixel is covered by radiation equivalent to 23°Cand the other half by 70°C , the pixel would read an intermediate value and neither tem-perature would be detected. This commonly happens when measuring the temperature ofa wire.

For this application, the distance between the sensor and the object is 2.5 m and theobject of interest is 5 x 15 cm making the 5 cm the smaller constant hence the dominantconstant. To guarantee that the object is visible in one or more pixels, each pixel needs tocover 2.5 cm or less making instantaneous field of view (IFOV) 25x25mm. The standard

Page 27: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 9

lens on most cameras has field of view (FOV) 25° by 19° or 4/3. The minimum resolutioncan be calculated,

R =2 · tan F

2· L

I(2.2)

where R is the resolution, F is in lens FOV in degrees, either the horizontal field of view(HFOV) or the vertical field of view (VFOV), L is the distance from the camera to theobject in meters and I is the instantaneous field of view in meters. For this applicationthe minimum resolution would be 44x33 pixels. Figure 2.5 shows how the pixels IFOVchanges with increased distance from a camera with 160x120 pixel resolution [2].

0 500 1000 1500 2000 2500 3000 3500 40000

2

4

6

8

10

12

IFO

V [

mm

]

Distance [mm]

Figure 2.5: IFOV versus distance.

2.2.4 Atmospheric attenuation

When thermal radiation travels through air it tends to attenuate because of absorption andscattering. Different vapors and gases in the atmosphere absorb energy at different wave-lengths. Figure 2.6 shows atmospheric attenuation for the electromagnetic spectrum from0 to 14 µm. The absorbing molecules can be seen in the table below the chart. The atmo-spheric attenuation is the highest from 5.5 to 7.5 µm where water molecules absorb theenergy. As mentioned in Section 2.2.2 the infrared sensors work at two different bands,long and mid wave. These bands are called atmospheric windows because infrared energyis transmitted through the atmosphere without much attenuation at those bands.

Page 28: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

10 Automatic thermal inspection of aluminium reduction cell

Figure 2.6: The top part of the figure shows transmission versus wavelength or the atmo-spheric attenuation. The bottom part of the figure shows the absorbing molecules at eachwavelength [3].

2.3 Thermal properties of an aluminium reduction cell

Aluminium is the most abundant metal in the earth’s crust. Aluminium is very chemi-cally unstable and is not found in its pure form. Every pure aluminium element on earthhas been chemically produced by man. Aluminium is produced by the Hall-Héroult elec-trolysis process where: aluminium oxide is dissolved in molten cryolite bath. Figure 2.7shows typical reduction cell with prebaked carbon anodes. This is the most common setupin modern aluminium smelters.

The cell consists of carbon lined pot with thermal insulation and thick steel shell workingas the cathode. There are raw material feeders, covers and movable carbon anodes onthe top. The bath is made of liquid alumina at the bottom and molten electrolyte ontop. The top and sides are framed by frozen layer of the electrolyte protecting the liningfrom corrosion. Alumina (Al2O3) is dissolved in the molten electrolyte where the mainingredient is cryolite (Na3AlF6). The oxygen from the alumina is discharged to theanodes where the carbon reacts with the oxygen forming carbon dioxide (CO2) and purealuminium falls to the carbon lining at the bottom. The over all chemical reaction isexplained with formula 2.3

2Al2O3 + 3C = 4Al + 3CO2 [4]. (2.3)

Page 29: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 11

������������ ��

���� ���� �����������������������������������������������

����������

������ ������ ������ ������

��� ����������

���������� ����������

���������

�����������

�����������

�����

����������� �������

Figure 2.7: Reduction cell with prebaked anodes.

The electrolysis process runs at about 977°C and all heat comes from the process itself.The heat comes in the from of electrical energy from the supply and thermal energy fromthe carbon. The voltage drop over each cell is about 4 to 6 V of which 2.21 V are neededfor electrolysis and the rest goes into heat.

Energy balance is an important property of a stable aluminium reduction cells. Enoughthermal energy has to be stored in the cell to help the process but, too much will deterioratethe cell prematurely.

Much of the heat escapes through the top of the cell or between 40 to 60%. Figure2.8 shows the thermal energy loss distribution under normal operation. The side wallshave considerable energy loss in percentage because the surface area is much bigger thanmost other areas of the cell. The surface temperature on these areas varies with ambienttemperature, humidity and air convection.

With time the insulation, lining and cathode block deteriorate and thermal energy lossincreases through sidewalls, bottom and collector bars. Insulation material increases inthermal conductivity and the cathode blocks swell and break down. When the cathodeblock breaks down, the electrolyte and aluminium have clear access to the collector bars.The distance between the molten metal and the bars decreases. The increase in conductivethermal energy losses can be explained with Equation 2.4,

H = kATH − TC

L(2.4)

Page 30: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

12 Automatic thermal inspection of aluminium reduction cell

������������ ��

���� ���� ������������������������������

����������

������ ������

��� ����������

����������

��

!�������

"#$

%$

����&���

�� ��

%$

����

Figure 2.8: Thermal energy losses of a reduction cell.

where k is the thermal conductivity, A is the surface area, L is the distance betweenthermal zones and T is the temperature of those zones.

With greater heat transfer from the electrolyte and aluminium to the outer layers, the tem-perature of those layers will increase. With respect to size, the collector bars transfer moreenergy per area than the sidewalls and bottom so they will be warmer than the sides andbottom. In addition, the collector bars are fed into the center of the cell and therefore aremore likely to come in contact with the processed metals and thus transfer more energy tothe colder areas outside the cell. When the overall energy transfer increases, the collectorbars will rise in temperature but if one or more bar come in direct contact to the processedmetals those bars will rise much more than others and the problem can be detected. Withrise in temperature the radiated energy transferred from the collector bars will rise to thepower of four. The thermal energy is calculated with Equation 2.1[4][1].

2.4 System setup and data collection

In this section the thermal camera and system properties will be introduced. The specksof the thermal camera are also described and how the data is queried for analysis.

Page 31: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 13

2.4.1 Thermal camera for data collection

The thermal camera is the key component for the success of this project. The camera isthe carrier’s eye of the world and has to fulfill its needs. To collect data for this project Ísalprovided one of their inspection cameras: a Flir E40 thermal camera intended for indus-trial inspection. This camera has all the features needed to successfully collect data in Ísaland to do emissivity tests. The E40 model has 160x120 pixel uncooled microbolometersensor. The sensing range is from -20°C to +650°C. Emissivity can be set from 0.1-1.Figure 2.9 shows an image of the camera.

Figure 2.9: Flir E series thermal camera www.flir.com.

The thermal data is scaled up and sent to the computer as 320 x 240 pixel three layer 8 bitarray (this is done for the status information of the image, battery information etc.). Thetemperature setting of the camera is interpreted by 8 bits (values from 0-255). The cameracan send out images with various colour palettes to interpret the temperature as coloursfor a human operator. For computer vision the image is sent as a gray scale image. Allthree layers have the same value so only one has to be processed. The camera sensitivityis <0.07°C and the measurement accuracy is ±2°C or ±2% of reading [5].

The sensing range of the camera can be tuned manually to increase the temperature res-olution. The camera is set at +150°C to +350°C making the resolution 0.78°C/bit. Tocalculate the actual temperature of the image the value has to be mapped with the for-mula,

T = Tl +Pv · (Th − Tl)

256(2.5)

where T is the actual temperature, Tl is the lower temperature set point of the camera(+150°C), Th is the higher temperature set point (+350°C), Pv is the pixel value from thecamera and 256 is the resolution.

In operation, the camera sensor builds up an error with different values for each pixelin the sensor. To eliminate this error, the camera has a built in equalizer which slips in

Page 32: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

14 Automatic thermal inspection of aluminium reduction cell

front of the sensor as needed so the sensor can calibrate this error out. This equalizer is ablack metal plate with even temperature. This process is automatic and when the camerais calibrating, the image stream stops with out any warning.

2.4.2 Camera setup

The thermal camera is placed in a camera box on top of a carrier facing up with a viewof the collector bars. An Ultrasonic distance sensor is on the right side of the carriermeasuring the distance between the carrier and the right side wall. Temperature sensoris placed on the carrier for ambient temperature measurements. Figure 2.10 shows thesetup.

Carrier

ACell Collector bar

~2,5

m

25°

Wall

FLIR E40

A B19°

Collector bars

Direction of motion

Temperature SensorUltrasonic sensor

FLIR E40

A BA B

Figure 2.10: The camera is located on top of the carrier. The Ultrasonic distance sensoris located on the side of the carrier and a temperature sensor is placed on the carrier.

The thermal camera view under the cell is little more than one set of collector bars. Tocover all the collector bars the carrier has to drive on both sides of the cell. Figure 2.11shows a typical view from the rover with thermal and visual image.

2.4.3 System dynamics

Under normal surveillance the carrier drives on a concrete floor along a wall on the rightside. The carrier is controlled by an operator or automatically by measuring the distanceto the wall and its approach angle. None of those properties are ideal, the floor and wallhave cracks and inequalities and the operator or control system is not perfect. The carrieris able to move freely or by control in five ways, surge, sway, roll, pitch and yaw Figure

Page 33: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 15

(a) (b)

Figure 2.11: (a) a thermal image of a set of collector bars. (b) a visual image of a set ofcollector bars. The thermal camera sees the rusted portion of the collector bars noted withcircles.

2.12 (five degrees of freedom). This movement has to be taken into account and can notinfluence the image processing algorithm. The sixth degree of freedom for a free body isheave or the up and down movement. The inequalities of the floor can influence the pitchbut are neglected otherwise.

RollPitch

SurgeSway

Yaw

Figure 2.12: The figure shows the carriers degrees of freedom.

The camera is placed in a vertical position in the carrier with the bottom of the imagefacing in direction of motion as shown on Figure 2.10. Figure 2.13 shows a typical imagesequence when the carrier is surging (moving forward) and Figure 2.14 shows images ofhow the carrier sways (side movements) in operation.

Page 34: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

16 Automatic thermal inspection of aluminium reduction cell

Figure 2.13: Sequence of images when the carrier is moving forward.

Figure 2.14: Sequence of images how the carrier can sway in operation.

2.4.4 Data collection

Before the image analysis functions can be created, training and test data have to becollected from site in Ísal. This data is collected by capturing the image stream fromthe thermal camera while the carrier is driven underneath the aluminium reduction cells.Before the data can be captured the thermal camera has to be configured. Emissivity isset at e = 0.95 and the sensor gain is set at 0-650°C. The colour palette is set to grayscale.

After some testing in Ísal the thermal scale of the camera was set to 150-350°C. Thecoldest cells have lower temperatures than 200°C and the thermal limit of the cell is320°C. This thermal scale should capture all collector bars and the thermal resolution ismaximized.

The training data is collected in the north west corner of pot room 3. The carrier is driveneast alongside the middle section of the pot room for 30 cells. The camera has a viewof half the cell at a time, so to capture the cell in full the carrier is driven west along thesouth wall on the way back. Figure 2.15 shows the route of the carrier.

To verify that the data from those 30 cells is representative for all the pot rooms, test datawas collected in pot room 1, and at the high current zone in pot room 3. The test data

Page 35: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 17

Col

lect

or b

ars

Col

lect

or b

ars

West along south wall

East along middle section

Figure 2.15: Route of the carrier for data collection in Ísal’s basement.

from pot room 1 has the thermal limit set at 100-258°C and the test data from pot room 3has the thermal limit set at 200-358°C.

After the data has been captured it is hand labeled as accepted or rejected. Images thatare labeled as accepted have a full view of the entire set of collector bars. The file nameof the accepted images are logged in a text file so the image analysis test method can seeif the image should be accepted or rejected.

Page 36: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

18

Page 37: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

19

Chapter 3

Image analysis

In this project the purpose of the image analysis processing step is to detect images thatcontain two collector bars and label them accordingly. When an image has been labeledit is analyzed by methods which are described in Chapter 4.

Images are streamed from the camera at a rate of up to 30 images a second. The majorityof the images contain no interesting information and should to be rejected by the imageanalysis. To be able to accept or reject an image with acceptable error rate, it is necessaryto use constants and image properties that stay unchanged between images and are truefor all reduction cells. For example, when looking for a red ball you use the constant thatthe thing you are looking for is red and has the shape of a ball. This decision would bebased on two decision boundaries. The first boundary checks for red areas on the imageand the second checks the shape of the object.

The criteria for detecting images are described in this chapter. First, the image features aredescribed, then the methods for image feature extraction and the methods to find featuresthat distinguish images that contain interesting information.

3.1 Image features

In image analysis the main challenge is to find constants that can be used to decidewhether to reject an image or accept it. Often these constants are based on the geom-etry of the environment or geometry of the object of interest. In this project there areseveral constants that can be evaluated.

The aluminium smelter in Straumsvik has 3 pot rooms, all with the same cells and sameset up. There are 160 cells in each pot room, divided in to two rows of 80 cells. Each row

Page 38: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

20 Automatic thermal inspection of aluminium reduction cell

has 8 banks of 10 cells and each cell has 40 collector bars, 20 on each side. All the cellshave the same architecture so all dimensions are constants. Figure 3.1 shows the smeltersetup.

POT ROOM 1

3158

POT ROOM 3

3159

POT ROOM 2

POT ROOM 3

2 rows of 80 cells, 8 banks in each row

3156 3154 3152 3150 3148 3146 3142 31403144 3138

31573159

3160

3155 3153 3151 3149 3147 3145 3143 3141 3139 3137

10 cells in a bank

20 collector bars on each side, total 40 on each cell

2 collector bars in a set

Figure 3.1: Ísal has three pot rooms shown in the top part of the diagram. Each pot roomhas 2 rows of 80 cells, divided into 8 banks of 10 cells. The middle part of the diagramshows two parallel cell banks. The bottom part of the diagram shows one cell containing40 collector bars, 20 on each side.

The collector bar location and dimension can be used to categorize the images and thecell setup can be used to numerate the cells. Since the cells are divided into banks of 10cells the gap between banks can be used to correct for errors in numeration.

It is important to identify if the images can be split into different image classes to sim-plify the image analysis. Different decision boundaries can then apply to different imageclasses. With closer look at the images from Ísal, they can be split into image classesdepending on special features in the image.

Figure 3.2 demonstrates sample images for each subgroups in this project. Figure 3.2ashows a typical image of a set of collector bars. The image illustrates only one big objectbut the more frequent situation is that other parts of the cell reflect radiation coming fromthe collector bars making reflections in the image, as shown in Figure 3.2b. Relatively

Page 39: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 21

(a) Image that containsone big object of two col-lector bars. Pass.

(b) Image that containstwo objects, one big oftwo collector bars andone reflective object tothe side or the top of thecollector bars. Pass.

(c) Image that containsone set of collector barsseen as two separate ob-jects. Pass.

(d) Image that containssegment of a set of collec-tor bars. Fail.

(e) Image that containstwo segments of a set ofcollector bars. Fail.

(f) Image that containscell corner. Fail.

(g) Image that containscell corner and a reflec-tion or a cell sidewall.Fail.

(h) Image that contains areflection or a cell side-wall.

Figure 3.2: Types of images.

cold collector bars radiate thermal energy at low intensity and are seen as two objectsshown in Figure 3.2c.

The camera is constantly capturing images while the carrier is moving (see Section 2.4.3).Many of those images show nothing at all, while others show corners, reflections and partof collector bars and therefore need to be rejected. Figures 3.2d to 3.2h all show samplesof these images that need to be rejected.

3.2 Feature extraction

When an image is captured it cannot be analyzed immediately. First it needs some prepa-ration so the image analysis algorithms can work on it. The preparation consist of threefunctions. First the image has to be converted from RGB to gray scale to minimize datathat needs to be processed. RGB image contains three layers, one for each colour. Thedata from the thermal camera is the same for all layers and thus converting the imagefrom RGB to gray scale is relatively easy and the data is minimized by one third. Next theimage is cropped, so all unnecessary information that is attached to the image from thecamera doesn’t interfere with the analysis. After the image has been cropped and changed

Page 40: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

22 Automatic thermal inspection of aluminium reduction cell

to gray scale, image features can be extracted by finding the objects on each image andthe properties of each object. This feature extraction is done with contour and momentfunctions.

3.2.1 Crop image

When the image comes from the thermal camera it contains information about batterystatus, connectivity thermal scale and so on. The image processing algorithm does notknow what is a thermal reading or a logo coming from the camera. Figure 3.3a showsa raw image as it comes from the camera with battery info, USB status and thermal ref-erence bar. This information is useful to the operator but the computer vision algorithmreads those white objects as temperature readings. The crop function defines the RegionOf Interest (ROI) in the image so the rest is cropped out as seen in Figure 3.3b.

3.2.2 Contour

The temperature of the cell varies in pixel values from 0-255, so the pixel value does nothelp the image process accepting or rejecting images. The image processing algorithmcan rely on the shape and position of the objects in the image. A helpful tool to link ad-jacent white pixels is the contour function. The contour function processes the image andcounts the number of objects by looking at adjacent pixels using the Suzuki85 algorithm.Figure 3.3c shows Figure 3.3b after it has been processed by the contour function.

(a) Raw image from camera (b) Image after being cropped (c) Image with contours

Figure 3.3: Image properties

The contour function saves information about each contour in an array of layers. Eachlayer can then be processed by another function to find more properties of the image, likesize and orientation. This is done with the moment function[6].

Page 41: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 23

3.2.3 Moments

The contour function finds the objects but it has no information about the size, orientationor the center point of that object. To find this information, the output from the contourfunction is processed by the moment function. The moment function returns central andspatial moments up to third order. With central and spatial moments the size, orientationand center location can be calculated among many other properties of the image [6].

In general the moment function can be defined by this formula,

mp,q =

∞∫−∞

∞∫−∞

xpxqf(x, y) dx dy (3.1)

where x, y refers to pixel location and f(x, y) refers to pixel values. Function 3.1 assumesthat f(x, y) is continuous but images are discrete so the formula can be written as,

mp,q =w∑

x=0

h∑y=0

xpyqf(x, y) (3.2)

where w represents the width and h the height of the image. For the first moment m00,x0y0 = 1 so one will be added to m for all none zero pixels defining the size of the object.Moment m1,0 adds all locations of none zero x pixels and m0,1 adds the location of nonezero y pixels. By dividing m1,0 and m0,1 with m0,0 the average location of x and y isfound, defining the centroid of the object.

(x, y) =(m10

m00

,m01

m00

)(3.3)

The images shape and orientation is found with the second order central moments. Thefirst order moments are normalized by the area but the central moments are normalizedby location. The central moments can be described as,

µp,q =w∑

x=0

h∑y=0

(x− x)p(y − y)qf(x, y) (3.4)

Each object has major and minor principal axes running through the center point. Themajor and minor principal axes can be calculated with this formula,

tan Θ =1

2

(µ02 − µ20

µ11

)± 1

2µ11

√µ202 − 2µ02µ20 + µ2

20 + 4µ211 (3.5)

Page 42: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

24 Automatic thermal inspection of aluminium reduction cell

where µ11, µ02 and µ20 are calculated with formula 3.4. The angle is then,

θ = arctan

(1

2

(µ02 − µ20

µ11

)± 1

2µ11

√µ202 − 2µ02µ20 + µ2

20 + 4µ211

). (3.6)

Figure 3.4a shows a circular object. The major and minor principal axis have equal lengthso the orientation is undefined. Figure 3.4b has a major principal axis running along they axis and thus the object is vertical. Figure 3.4c shows object with major principal axisrunning along x axis and is thus horizontal.

These properties are easy to observe for the human eye but this is useful for image analyticalgorithm.

(a) Circular object (b) Major principal axis along theY axis

(c) Major principal axis along theX axis

Figure 3.4: Image moments

The moment function has many other features such as shape eccentricity, radius of gyra-tion and moment invariants. These features are not used in this project so they will not bedescribed in details [6].

3.3 Feature selection

The objective of the feature selection stage is to find the most suitable subset of features todetect the set of collector bars. The method employed here was that of trial and error. Thedecision process is then based on a combination of the selected features using a decisiontree [7].

3.3.1 Feature selection methods

Computer vision algorithms that processes images where the object or the camera is mov-ing, need to know if the object of interest is in the frame or not. The algorithm has to find

Page 43: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 25

out if the whole object is in the frame or if the camera only sees part of it. In this situationcorrect constants are crucial so the algorithm can make decisions whether to reject animage or accept it.

When the image database is analyzed it is clear that there are many features that couldbe applicable to separate the images. Many features were tested, but are not useful to theprocess and therefore will not be described in detail.

Features that turned out to be effective are:

X1 Location of average center point

X2 Location of center point of largest object

X3 Major principal axis orientation

X4 Mean major principal axis orientation if objects are more than one

Center point location

By looking at the sample images that show both collector bars, it is clear that the averagecenter of all objects on the image is distributed around the image center. With the useof the contour and moment function, the center point of the object can be calculated. Ifthe object’s center is distributed around the image center this property should make twoboundaries parallel to the image top and bottom edge.

There are two ways to perform this evaluation. Method X1, locating the average centerpoint for the objects on the image and method X2 by checking the location of the largestobject on the image. The average center location is calculated with this formula,

∆y =y1 · a1 + y2 · a2

a1 + a2

where ∆y is the average center point of the object, yn is the center point of the object andits area is denoted as an. The center point of both objects is multiplied with the size of thearea of each object and divided by the total area of both objects. This gives the averagecenter point. If one object is much larger than the other, the center is closer to the centerof the larger object thus giving it higher weight. The largest object in the image is alwaysnumber 1, so the larger object location is simply,

∆y = y1.

Page 44: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

26 Automatic thermal inspection of aluminium reduction cell

Figure 3.5a shows where a set of collector bars is in between the boundary and the imageis accepted. Figure 3.5b and 3.5c show where the average location is in between theboundary and the image is accepted. This is a false acceptance which another boundaryhas to reject. Figure 3.5d shows where the set of collector bars is not in range of the imageand the image is rejected.

(a) Entire set is withinboundaries

(b) Average center withinboundaries and center oflargest object outside ofboundaries

(c) Corner within bound-aries

(d) Center outside ofboundaries

Figure 3.5: Center point location boundary

Principal axis

When the image sample set is observed with regard to orientation of the object it showsthat the images that contains a complete set of collector bars tend to have major principalaxis oriented towards 90° , while corners and reflections are oriented towards 0°. Thisboundary checks if the orientation can separate the images using the major principal axis.Since the minor and major principal axis are perpendicular they should show the sameresult only with 90° difference.

There are two methods to verify the separation of the objects. Method X3 observes theaverage orientation of objects in the image and method X4 observes the orientation of thelargest object. The average orientation is calculated,

∆θ =θ1 · a1 + θ2 · a2

a1 + a2

where ∆θ is the average angle and θ1 and θ2 are the major principal axis of the largestobjects with area a1 and the second object with area a2. θ is calculated with formula 3.6.The major principal axis orientation of the larger object is simply θ1. Figure 3.6 showstwo images where the principal axis method is implemented.

Page 45: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 27

(a) Compleat set of collector bars with ma-jor principal axis pointing in 89°

(b) Corner image where major principalaxis is pointing in 13°

Figure 3.6: Major principal axis orientation

3.3.2 Decision boundary test method

In Section 3.3 four methods are described to identify data separation to make decisionboundary that can accept or reject images with an acceptable error rate. Each method istested on a set of training data captured in real environment in Ísal. The training dataconsists of 3887 images that show 30 cells in north vest corner of potroom three. Thecells bare even numbers from 3102 to 3160.

The training data is hand labeled as accepted or rejected. Images that are labeled asaccepted have a clear view of two collector bars. The feature extraction function collectsdata from each image and saves it to a table. The table is then imported to Matlab for dataanalysis. Each method is tested with 100 different boundary settings and false acceptanceratio and false rejection ratio are measured for each setting.

Each method is then evaluated from its false acceptance ratio and false rejection ratio. Ifthe method helps categorizing images it is evaluated in combination with other methodsuntil acceptable ratio is gained.

When the decision making algorithm has been formed it is tested with unseen data sets.These sets have different camera settings emulating colder climate and hotter cells. Thishelps estimating the performance of the algorithm.

Page 46: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

28 Automatic thermal inspection of aluminium reduction cell

3.4 Image analysis results

3.4.1 Feature selection uncombined methods

When all four methods that were described in Section 3.3 have been tested, the results areanalyzed one at a time with respect to false acceptance ratio and false rejection ratio.

Location of center point

There are two methods that analyzes center point location. Figure 3.7a shows the distri-bution for method X1, where the average center is calculated for all objects. Figure 3.7bshows the distribution for method X2, the center of the largest object. The y axis showsthe center point relative to the image height and x axis is the image number for all imagesin the set.

0 500 1000 1500 2000 2500 3000 35000

10

20

30

40

50

60

70

80

90

100

Cen

ter

po

siti

on

[%

]

Image number

Center of manually accepted imagesCenter of manually rejected images

(a)

0 500 1000 1500 2000 2500 3000 35000

10

20

30

40

50

60

70

80

90

100

Cen

ter

po

siti

on

[%

]

Image number

Center of manually accepted imagesCenter of manually rejected images

(b)

Figure 3.7: Distribution of objects center points versus sample images, (a) for method X1and (b) for method X2.

The two vertical gaps in the data around image number 1300 and image number 2700, areimages that are taken between banks of cells. It is clear from Figure 3.7 that there are twoboundaries parallel to the top and bottom edge of the image. The location of the boundaryis tested in 100 steps from the top and bottom edge until they meet in the middle, coveringthe whole height of the image. The false acceptance and false rejection ratio is measuredfor each step. To find the best location, the false acceptance ratio is plotted versus falserejection ratio seen in Figure 3.8.

The best value for the boundary is where the false rejection ratio starts to increase fasterthan the false acceptance ratio decreases. At this point the false acceptance ratio is min-

Page 47: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 29

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Fal

se a

ccep

tan

ce r

atio

[%

]

False rejection ratio [%]

(a)

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Fal

se a

ccep

tan

ce r

atio

[%

]

False rejection ratio [%]

(b)

Figure 3.8: False acceptance ratio versus false rejection ratio, (a) for methods X1 and (b)for method X2.

imized without rejecting many images. It can be better to accept images that should berejected than to reject images that may contain valuable information. As seen in Fig-ure 3.8a the best value for method X1 is located where false acceptance ratio is 34.83%and false rejection ratio is 1.76%. As seen in Figure 3.8b the best value for method X2 islocated where false acceptance ratio is 11.96% and false rejection ratio is 0%.

To locate the boundary the center bias has to be found. The center bias is proportion ofthe image height. If the center bias is 10%, the boundary is located 10% of the imageheight from the top and bottom edge. To find the actual location of the boundary, thefalse rejection ratio and false acceptance ratio are plotted versus the center bias seen onFigure 3.9.

0 5 10 15 20 25 30 35 40 45 500

10

20

30

40

50

60

70

80

90

100

Rat

io [

%]

Center bias [%]

False acceptanceFalse rejection

(a)

0 5 10 15 20 25 30 35 40 45 500

10

20

30

40

50

60

70

80

90

100

Rat

io [

%]

Center bias [%]

False acceptanceFalse rejection

(b) t

Figure 3.9: False acceptance ratio and false rejection ratio versus center bias, (a) formethods X1 and (b) for method X2.

Page 48: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

30 Automatic thermal inspection of aluminium reduction cell

The best location for the boundary for method X1 is where false acceptance ratio is34.83% and false rejection is 1.76%. This gives center bias of 29% of the image height.The best location for the boundary for method X2 is where false acceptance ratio is11.96% and false rejection is 0% that gives center bias of 29%. The actual bias in termsof pixels would be,

yb = yh · b

where yb is the location of the boundary from either edge, yh is the total height of theimage in pixels and b is the Center bias value found from Figure 3.9.

Principal axis

There are two methods that analyze the object orientation. Figure 3.10a shows the averageorientation of all objects using method X3. Figure 3.10b shows the orientation of thelargest object with method X4. The y axis shows the proportional orientation of theobject and x axis shows the image number for all images in the set.

0 500 1000 1500 2000 2500 3000 35000

10

20

30

40

50

60

70

80

90

100

Ori

enta

tio

n [

%]

Image number

Orientation of manually accepted imagesOrientation of manually rejected images

(a)

0 500 1000 1500 2000 2500 3000 35000

10

20

30

40

50

60

70

80

90

100

Ori

enta

tio

n [

%]

Image number

Orientation of manually accepted imagesOrientation of manually rejected images

(b)

Figure 3.10: Object orientation versus image number, (a) for methods X3 and (b) formethods X4.

The data separation on Figure 3.10 is not complete but there are signs of clustering on bothimages. The clustering is much better for Figure 3.10b than for Figure 3.10a. To locatethe separation a boundary is located in 100 steps over the whole range of orientation andthe false acceptance and false rejection ratio measured for each step. The false acceptanceratio versus the false rejection ratio can be seen on Figure 3.11.

Page 49: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 31

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Fal

se a

ccep

tan

ce r

atio

[%

]

False rejection ratio [%]

(a)

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Fal

se a

ccep

tan

ce r

atio

[%

]

False rejection ratio [%]

(b)

Figure 3.11: False acceptance ratio versus false rejection ratio, (a) for methods X3 and(b) for method X4.

The best location for the boundary for method X3 is where false acceptance is 65.27% andfalse rejection is 0.08%. For method X4 the best location is where the false acceptanceratio is 45.26% and the false rejection ratio is 7.21%. To find the value for the boundarythe false acceptance ratio and false rejection ratio are plotted versus the object orientationseen on Figure 3.12.

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Rat

io [

%]

Average object orientation [%]

False acceptanceFalse rejection

(a)

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Rat

io [

%]

Object orientation [%]

False acceptanceFalse rejection

(b)

Figure 3.12: False acceptance ratio and false rejection ratio versus object orientation, (a)for methods X3 and (b) for method X4.

The optimum value for a linear boundary for method X3 is where object orientation is42% and for method X4 at 92%.

Page 50: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

32 Automatic thermal inspection of aluminium reduction cell

Summary for uncombined feature selection

Table 3.1 shows the false acceptance and false rejection ratio for each method. It can beseen that method X1 gets the lowest false acceptance and false rejection ratio. Low falseacceptance and false rejection ratio indicate high data separation. All the methods haverelatively high false acceptance ratio and are therefore not suitable to separate the data onits own.

Table 3.1: False acceptance and false rejection ratio for all methods

Method False acceptance ratio [% ] False rejection ratio [%]

X1 34.83 1.76X2 11.96 0.00X3 65.27 0.08X4 45.26 7.21

3.4.2 Feature selection combined methods

The methods are used in combination with other methods to get better results. The besttwo methods from Section 3.4.1 are chosen to accept and reject images individually, sothat the methods can be compared. The accepted images are then passed on to methodX3 and X4. The same method is used as in Section 3.4.1 to find the best value for theboundary.

The two best methods from Table 3.1 are methods X1 and X2. The best performance formethods X1 and X2 in Section 3.4.1 was found to be when the boundary was located 29%of the image height from the image top and bottom edge.

Average center point X1 and principal axis methods

Figure 3.13 shows orientation of all images that passed the boundary of method X1.Method X1 has 1.76% false rejection ratio. The falsely rejected images are denoted withgreen ’+’ on the figure.

Page 51: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 33

0 500 1000 1500 2000 2500 3000 35000

10

20

30

40

50

60

70

80

90

100

Ori

enta

tio

n [

%]

Image number

Oreintation of manually accepted imagesOrientation of manually rejected imagesOrientation of falsely rejected images

(a)

0 500 1000 1500 2000 2500 3000 35000

10

20

30

40

50

60

70

80

90

100

Ori

enta

tio

n [

%]

Image number

Orientation of manually accepted imagesOrientation of manually rejected imagesOrientation of falsely rejected images

(b)

Figure 3.13: Object orientation for combined methods, (a) for methods X1 and X3 and(b) for methods X1 and X4.

When Figure 3.13 is compared to Figure 3.10, where images didn’t have to pass theboundary of method X1 the data shows a little better data separation. To see this clearerthe false acceptance and false rejection ratio is plotted in one graph seen on Figure 3.14.

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Fal

se a

ccep

tan

ce r

atio

[%

]

False rejection ratio [%]

(a)

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Fal

se a

ccep

tan

ce r

atio

[%

]

False rejection ratio [%]

(b)

Figure 3.14: False acceptance ratio versus false rejection ratio for combined methods, (a)for methods X1 and X3 and (b) for methods X1 and X4.

The gain in performance from Figure 3.14 compared to Figure 3.11 is large. The bestperformance for method X3 alone was 65.27% false acceptance rate and for method X4this was 45.26%. With the addition of method X1 the false acceptance ratio is reducedto 15.48%(Figure 3.14a) for method X3 and 10%(Figure 3.14b) for method X4. To seethe location of the boundary the false acceptance and false rejection ratio versus objectorientation is plotted in one graph seen on Figure 3.15.

Page 52: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

34 Automatic thermal inspection of aluminium reduction cell

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Rat

io [

%]

Average object orientation [%]

False acceptanceFalse rejection

(a)

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Rat

io [

%]

Object orientation [%]

False acceptanceFalse rejection

(b)

Figure 3.15: False acceptance ratio and false rejection ratio versus object orientation forcombined methods, (a) for methods X1 and X3 and (b) for methods X1 and X4.

Figure 3.15 shows the false acceptance and false rejection ratio versus object orientation.The main difference between those two methods is the distribution of object orienta-tion. For method X3 the average object orientation is distributed from 40-100% while formethod X4 the object orientation of the largest object is distributed from 85-100%. It isthere for easier to separate the data with method X1 in combination with method X4 thanin combination with method X3.

Center point of largest object X2 and principal axis methods

Figure 3.16 shows orientation of all images that passed the boundary of method X2.Method X2 has no false rejection and thus there are no green ’+’ on the figure.

Page 53: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 35

0 500 1000 1500 2000 2500 3000 35000

10

20

30

40

50

60

70

80

90

100

Ori

enta

tio

n [

%]

Image number

Orientation of manually accepted imagesOrientation of manually rejected images

(a)

0 500 1000 1500 2000 2500 3000 35000

10

20

30

40

50

60

70

80

90

100

Ori

enta

tio

n [

%]

Image number

Orientation of manually accepted imagesOrientation of manually rejected images

(b)

Figure 3.16: Orientation versus image number for combined methods, (a) for methods X2and X3 and (b) for methods X2 and X4.

Figure 3.16 show much better data separation than Figure 3.10, where the images didn’thave to pass boundary X2. It is clear from looking at Figure 3.16 that it is possible tomake a linear boundary that can separate the data almost completely. Figure 3.17 showsthe false acceptance versus false rejection ratio.

10−1

100

101

102

10−1

100

101

102

Fal

se a

ccep

tan

ce r

atio

[%

]

False rejection ratio [%]

(a)

10−1

100

101

102

10−1

100

101

102

Fal

se a

ccep

tan

ce r

atio

[%

]

False rejection ratio [%]

(b)

Figure 3.17: False acceptance ratio versus false rejection ratio for combined methods, (a)for methods X2 and X3 and (b) for methods X2 and X4.

The gain in performance of this method is very good. Figure 3.17a shows a little falseacceptance or 0.19%. Figure 3.17b shows close to no false acceptance or 0.05%.

Page 54: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

36 Automatic thermal inspection of aluminium reduction cell

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Rat

io [

%]

Average object orientation [%]

False acceptanceFalse rejection

(a)

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Rat

io [

%]

Object orientation [%]

False acceptanceFalse rejection

(b)

Figure 3.18: False acceptance ratio and false rejection ratio versus the orientation forcombined methods, (a) for methods X2 and X3 and (b) for methods X2 and X4.

Figure 3.18 shows the false acceptance and false rejection ratio versus object orientation.Figure 3.18 and Figure 3.15 show nearly the same false rejection curve but since methodX2 has much better performance the data gets nearly separated. The location for theboundary with method X2 and X4 in combination has really good safety margins causethe algorithm is not as sensitive for yaw movement of the carrier. For method X2 and X3in combination the safety margins are not as big and thus are not as good.

Summary for combined feature selection

Table 3.2 shows a summary of the results for combined methods.

Table 3.2: False acceptance and false rejection ratio for all methods

Method X1 X2False accept.[%] False reject.[%] False accep.[%] False reject.[%]

X3 15.48 1.82 0.19 0.02X4 10.00 1.98 0.05 0.00

By combining methods X3 and X4 to methods X1 and X2 the false acceptance ratio getsmuch better. Each method improves the data separation and thus the false acceptanceratio. By combining methods X2 and X4 the false acceptance gets down to 0.05% withno false rejection. The data separation shown on Figure 3.18b has a wide pass where thefalse acceptance and false rejection ratio is close to zero and thus the method is consideredrobust. When methods X2 and X3 are combined the data separation is on a much narrower

Page 55: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 37

scale on the orientation axis shown on Figure 3.18a so the methods are not considered tobe robust since small changes in the data would cause larger change in the false acceptanceor false rejection ratio.

3.4.3 Image analysis for unseen test data

The image analysis has been able to get good separation on the training data set. To testhow robust the image analysis is, the combined methods X2 and X4, are tested with testdata sets from Ísal. These data sets are captured in pot room 1 and 3 with different thermalscale limitation for the camera (see Section 2.4.4). Figure 3.19 shows the center positionof objects in the test data.

0 10 20 30 40 50 60 700

10

20

30

40

50

60

70

80

90

100

Cen

ter

po

siti

on

[%

]

Image number

Center of manually accepted imagesCenter of manually rejected imagesCenter of wrongfully rejected images

(a)

0 10 20 30 40 50 60 700

10

20

30

40

50

60

70

80

90

100

Cen

ter

po

siti

on

[%

]

Image number

Center of manually accepted imagesCenter of manually rejected imagesCenter of wrongfully rejected images

(b)

Figure 3.19: Center position of objects in test data from (a) pot room 1 and (b) pot room3.

When the image analysis is tested with the test data from pot room 1, the results show thatall images that should be accepted are accepted and all images that should be rejected arerejected. When the image analysis is tested with the test data from pot room 3, the resultsare the same. No image is wrongfully rejected or accepted.

3.5 Image analysis summary

The focus of the image analysis is to categorize images captured by the thermal camera.The task is split into two subgroups. First all the features of the image are extracted withthe contour and moment function. When features have been extracted they are evaluated

Page 56: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

38 Automatic thermal inspection of aluminium reduction cell

and tested against linear boundaries X2 and X4. If the image passes the boundary it issent on to the thermal inspection algorithm.

After the image analysis functions have been tested, the feature extraction function onlyhas to collect a fraction of the data the system is capable of collecting. The image analy-sis methods only relay on information about the largest object, the centroid and the majorprincipal axis orientation. Although numerous features and combinations of features weretested for this part of the project, the final result is uncomplicated and gives a false accep-tance ratio of 0.05% with the training data and 0% with untrained data.

Page 57: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

39

Chapter 4

Thermal inspection

When images have been separated as accepted or rejected images they can be checked forcell temperature, the data can be stored and notifications can be sent to the operator. Inthis chapter all the temperature measurements will be described and analyzed in terms ofaccuracy and repeatability.

4.1 Temperature measurement

Thermal inspection takes place after the images have been diagnosed as containing col-lector bars (accepted) or not (rejected). All images are checked for maximum temperatureand all accepted images are checked for cell temperature.

4.1.1 Cell temperature

When an image has been analyzed and accepted it is checked for collector bar tempera-ture. There are two collector bars on each image. To check each collector bar the imageis split into two regions of interest about the centroid of the largest object. Each region isthen checked for the highest pixel value: the warmest pixel. Figure 4.1 shows a tempera-ture measurement for a set of collector bars.

When more than one image is accepted in a row the data is collected and saved as onedata point. This feature is also used to mark the set of cathodes and cells. First group ofaccepted images gets the number 1 and the cell is marked as 1. Next set of images getsthe number 2 and the cell is marked 1 and so on until 10 sets of images have been ana-

Page 58: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

40 Automatic thermal inspection of aluminium reduction cell

Figure 4.1: Cell temperature measurement. The object is split into two regions by it’scentroid. The highest pixel value from each side is captured and stored as the collectorbar temperature.

lyzed then the cell number is increased by one. This is all according to the cell geometrydescribed in Section 3.1.

4.1.2 Maximum temperature measurement

There is always a possibility that the image analysis algorithm rejects an image of a severeover-temperature situation. The algorithm might not generalize to such a case and rejectan image which is critical to capture.

To prevent this from happening there is a safety feature that checks all incoming imagesfor maximum temperature. If the image has temperatures over a certain threshold the op-erator is notified about the situation and location regardless of further image analysis. Theoperator can then respond and check if there is a possible cell leakage in progress.

4.1.3 Temperature measurement analysis method

The quality of the temperature measurement is assessed using three qualities, discussedin this section. They are: accuracy, repeatability and angle of attack.

Accuracy

The accuracy of the temperature measurement can be evaluated by measuring the actualtemperature of a collector bar and comparing the result against the thermal camera tem-

Page 59: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 41

perature. The temperature has to be measured of a collector bar material heated in an ovenbecause a collector bar in use can not be touched due to electric shock danger. The ac-curacy of the measurement is subject to the thermal camera temperature setting, thermalcamera temperature reading accuracy and the collector bar emissivity.

To test the temperature accuracy a piece of rusted collector bar material was heated up inan oven. The camera was set as it would be in operation, emissivity set at e = 0.95 andtemperature scale set at 0-650°C. For comparison the material temperature is measuredwith Omega OM-DAQPRO-5300 thermometer and the Flir E40 thermal camera. Themeasurement is performed at three temperature settings. For each setting the temperatureis measured 3 times with the thermometer and the thermal camera. Figure 4.2 shows themeasurement setup.

(a) Thermal image of oven (b) Setup

Figure 4.2: Figure (a) shows a thermal image of the piece in the oven. Figure (b) showsthe thermal camera in front of the oven and the Omega thermometer to the left of it.

Repeatability

The images are processed at the rate of up to 30 images pr. second. At this rate severalimages are processed for each set of collector bars. Each image has different view of thecollector bar in both x and y axis. By determining the repeatability of the temperaturemeasurement the reproducibility of the measurement method can be confirmed.

To evaluate the repeatability the following conditions have to be met,

• The same measurement procedure has to be used each time.

• The same observer has to conduct each measurement.

• The same measurement instrument has to be used each time, under the same condi-tions.

Page 60: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

42 Automatic thermal inspection of aluminium reduction cell

• The measurement has to be performed at the same location each time.

• The measurement has to be repeated over a short period of time.

The same reduction cell is measured 5 times, over as short period of time as possible.The camera is always placed in the same position on the carrier with the same settingsand the carrier is driven the same path. The carrier path can change a little betweenrounds so the temperature measurement can have error due to different angle of attack.For this measurement it is assumed that the cell has constant temperature over the wholeexperiment [8].

The mean is calculated for each collector bar in each run and the standard deviation ofthe mean for all the runs. This is done for each collector bar in the cell and then themean of the standard deviation is calculated giving the expected error for the temperaturemeasurements between runs.

Angle of attack

When the carrier is surging the camera captures several images of each set of collectorbars. When the carrier drives past the cell the camera gets radiation from the collectorbars at a different angles each image, giving different angles of attack.

To measure the temperature difference between images at different angles the camera isplaced under a cell with different view at it. The area where the collector bars are inthe view is roughly divided into 6 columns. In each column 4 images are captured withdifferent view at the set of collector bars. The camera is moved, not tilted so the change inposition is on y axis. Figure 4.3a shows an image for the first image for column 1. Figure4.3b shows the fourth image for column 2. Where the camera has been moved on the yaxis.

Page 61: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 43

(a) (b)

Figure 4.3: The figure is split into 6 columns. Four images are captured in each columnwith different view of the collector bars. Figure (a) shows the first image in first columnand Figure (b) shows the last image in the second column.

4.2 Notifications and data storage

The system is intended for thermal monitoring of aluminium reduction cells collectorbars. In normal operation the system collects thermal data about each collector bar. Whenthe temperature of the images has been analyzed, a monitoring algorithm checks if thedetected temperature values are abnormal or not. If the temperature is above a specificset-point the operator is informed via SMS and /or email. For now the email function hasbeen implemented. The notification set point is at 320°C.

When cell temperature has been analyzed the data is stored in a CSV file. There is oneentry stored for each set of collector bars and one image. The images are stored for lateranalysis and development purposes, but should not necessary be in the final implementa-tion. Each entry contains the following variables:

• Date [day.month.year]

• Time of day [HH:mm:ss]

• Cell number [[Pot room nr. 1-3][Cell nr.1-160]

• Collector bar number[1-40]

• Mean temperature of collector bar A [°C]

• Standard deviation of temperatures for collector bar A [°C]

• Ambient temperature [°C]

Page 62: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

44 Automatic thermal inspection of aluminium reduction cell

• Number of analyzed images [n]

• Carrier distance from right side wall [mm]

• Name of corresponding image in database [day_month_year/cell_bar.png]

4.3 Thermal inspection result

4.3.1 Accuracy of temperature measurement

Table 4.1 shows the result from the temperature measurements. The measurements wereperformed for three different temperatures, 100°C, 150°C and 200°C. Three measure-ments were taken for each temperature setting for comparison.

Table 4.1: Temperature measurements with Omega OM-DAQPRO-5300 temperature me-ter and Flir E40 thermal camera. Emissivity = 0.95, Thermal scale = 0-650°C automatic

Measurement # Omega [°C] Flir E40 [°C] ERROR [°C] ERROR[%]

1 107.2 108 0.8 0.742 107.5 108 0.5 0.463 108.2 108 0.2 0.194 163.1 161 2.1 1.305 163.5 162 1.5 0.936 162 162 0 0.007 201.9 201 0.9 0.458 202.4 202 0.4 0.209 204 203 1 0.49

Mean 0.8 0.53

Table 4.1 shows that the expected temperature measurement error is 0.53%. This error isless than the expected error given from manufacturer and thus isn’t the dominant error.The data sheet for the Flir E40 thermal camera states that the thermal accuracy of thecamera is ±2°C or ± 2% of reading. This measurement confirms that the emissivity ofthe collector bar material is e = 0.95, else the contact measurement and the infraredmeasurement would be inconsistent.

Page 63: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 45

4.3.2 Repeatability of temperature measurement

Table 4.2 show the measurement results.

Table 4.2: Repeatability temperature measurements

Bar nr. Round 1 [°C] Round 2 [°C] Round 3 [°C] Round 4 [°C] Round 5 [°C] Std. of mean [°C]

1A 268.40 268.50 268.22 266.00 268.40 1.072B 268.50 267.63 269.56 269.29 269.10 0.773A 274.30 274.13 273.22 276.57 272.10 1.654B 270.40 270.00 268.44 269.43 268.80 0.815A 270.30 270.63 270.11 270.43 265.30 2.276B 273.40 272.75 272.78 275.14 270.20 1.777A 276.50 276.75 276.33 277.86 272.00 2.258B 279.40 280.13 279.00 282.71 277.50 1.929A 273.30 274.13 274.00 275.43 271.60 1.40

10B 273.70 269.88 271.67 272.57 271.20 1.4411A 270.40 268.75 270.33 270.43 270.20 0.7212B 271.00 269.50 270.44 270.43 269.70 0.6113A 267.80 267.13 267.22 267.86 265.20 1.0814B 275.20 274.88 274.78 277.29 273.30 1.4315A 271.20 270.38 272.67 271.43 269.40 1.2216B 276.60 273.38 275.22 277.14 274.80 1.5017A 272.40 274.50 274.11 273.86 272.60 0.9418B 274.90 277.25 276.89 276.57 275.60 0.9719A 279.60 278.56 278.50 277.89 276.90 0.9920B 272.30 272.78 272.50 272.56 271.70 0.41

Mean of std 1.30

The temperature difference between rounds in table 4.2 is very low compared to the pos-sible errors produced by the resolution of the camera, slight temperature change of thecell and movement on x axis (swaying) of the carrier. This table gives clear indicationthat the measurement is reproducible. Table B.1 in appendix B shows standard deviationfor each collector bar in each run.

4.3.3 Angle of attack

When the carrier surges the view of the collector bars changes with image height. Thischange in angle of attack has no influences on the thermal measurements. When thecarrier sways or changes its driving path the view of the collector bars changes withimage width. This change in angle of attack has considerable influences on the thermal

Page 64: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

46 Automatic thermal inspection of aluminium reduction cell

measurement. Figure 4.4 shows all the measurements. The figure shows changes intemperature measurements with image width. The temperature drops linearly from left toright with the slope of 0.23°C per percentage of image width. The figure doesn’t show theimage height but as the data isn’t scattered on the y axis there is no temperature changewith image height.

0 10 20 30 40 50 60 70 80 90 1000

50

100

150

200

250

300

350

Image width[%]

Tem

pera

ture

[°C

]

Poly line t=−x0.2285+225.3Measured values

Figure 4.4: Temperature versus image width.

Figure 4.5 shows the temperature for each measurement versus the image height and im-age width. This figure confirms that there is no considerable error with image height.

Figure 4.5: Temperature versus image height and image width.

Page 65: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

Guðjón Hugberg Björnsson 47

4.4 Thermal inspection summary

The thermal inspection of the system collects the thermal data coming from the camerawhich is the output of the system. The thermal inspection has two parallel stages, allincoming images are analyzed for maximum temperature and all qualified images fromthe image analysis are checked for collector bar temperature. When images have beenanalyzed for temperature the output is stored and notification sent to the operator if tem-peratures are abnormal.

The accuracy of the temperature measurements is crucial for the functionality of the sys-tem. The temperature measurement is evaluated with respect to accuracy, repeatabilityand location of the object in view or the angle of attack.

The accuracy of the temperature measurements from the thermal camera compared to thethermometer that relies on conduction is 0.53%. The camera view of the object can in-fluence the measurements if the carrier is unstable. The influences can be compensatedcause this change is linear. The repeatability of the measurements is good. The repeata-bility test shows an expected error of 1.30°C between measurements while the resolutionof the thermal camera is 0.78°C.

The overall thermal performance of the system is fairly accurate. According to the re-peatability an accuracy measurements, the overall system accuracy is better than the ac-curacy of the thermal camera according to the manufacturer.

The thermal measurement methods of this project don’t add to the cameras inaccuracy,and the measurement accuracy of the camera is also subject to thermal changes in the sen-sor environment. The overall thermal accuracy for the system is the same as the thermalaccuracy of the camera or ±2°C or ±2% of reading.

Page 66: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

48

Page 67: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

49

Chapter 5

Summary and Discussion

The idea behind this project was to see if the temperature measurements of collector barsin Ísal could be automatized. The premise is that a robot can drive with a thermal cameraaround the smelter, collect real time heat measurement from all collector bars and alert thepot room manager of potential danger. The benefit of such system would involve frequentheat measurements and data logging for long term trend monitoring. The overall result issafer environment and more cost efficient pot maintenance.

The project’s objectives were split in two independent parts, image analysis and thermalinspection. The image analysis accepts and rejects images. It performance is measuredwith false acceptance and false rejection. The performance of the function was found tobe 0.05% false acceptance ratio and 0% false rejection ratio. In other words the functioncan separate the images correctly 99.95% of the time.

The thermal inspection checks accepted images for temperature. The main interest is therepeatability and accuracy of the function. The repeatability measurement showed an ex-pected error for the temperature measurement to be ±1.3°C when the average temperatureof the sample was 274°C. This makes the expected error of ±0.5%. The accuracy mea-surements show near complete correlation between thermal camera measurements andthermometer measurements or expected error of ±0.53%. These results show strong indi-cation that the measurement methods don’t add to the measurement errors of the system,so the overall measurement accuracy is the same as for the thermal camera or ±2°C or±2% of reading.

The results of this research show that implementing an automatic temperature inspec-tion system is feasible. The results also support that it would be reliable and reasonablyaccurate.

Page 68: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

50 Automatic thermal inspection of aluminium reduction cell

Future work

The carrier robot is one of the fundamental parts for this project. Development of therobot ongoing in Reykjavik University, preliminary results are encouraging. Next step inmaking this inspection automatized is to test its operation over a long period of time. Themain focus of this project has been on the smelter setup in Ísal. Future designs are goingto be more versatile and more suitable for other smelter designs.

Page 69: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

51

Bibliography

[1] H. Young and R. Freedman, University Physics. Pearson Education, 2008.

[2] FLIR, “The ultimate infrared handbook for rd professionals.” FLIR AB.

[3] U. Sharma, “Infra red detectors,” 2004.

[4] K. Grjotheim and H. Kvande, Introduction to aluminium electrolysis. Aluminium-Verlag, 1993.

[5] FLIR, “E-series infrared camera.” FLIR Systems Pty Ltd.

[6] L. O’Gorman, M. Sammon, and M. Seul, Practical Algorithms for Image Analysis.Cambridge, 2009.

[7] C. Bishop, Pattern Recognition and Machine Learning. Springer, 2007.

[8] B.Taylor and C.Kuyatt, “Guidelines for evaluating and expressing the uncertainty ofnist measurement results,” tech. rep., National Institution of Standards and Technol-ogy, 1994.

Page 70: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

52

Page 71: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

53

Appendix A

Cell leakages in Ísal

Table A.1: Cell leakages in Ísal from 2000-2012

Year Leakage Stopped Total Cell relining Leakage ratio

2012 (until May 24.) 48 10 58 83%2011 116 42 158 73%2010 75 21 96 78%2009 69 24 93 74%2008 112 41 153 73%2007 167 81 248 67%2006 109 42 151 72%2005 89 16 105 85%2004 87 12 99 88%2003 122 12 134 91%2002 100 9 109 92%2001 117 15 132 89%2000 89 10 99 90%

Total 1300 335 1635 80%

Page 72: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

54

Page 73: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

55

Appendix B

Thermal inspection

Table B.1: Mean and standard deviation for each collector bar for each run

Bar nr. Round 1 Round 2 Round 3 Round 4 Round 5 Round 1-5

Mean Std Mean Std Mean Std Mean Std Mean Std Mean Std1A 268.40 2.99 268.50 1.77 268.22 1.56 266.00 3.00 268.40 2.50 267.90 1.072B 268.50 2.99 267.63 2.45 269.56 2.60 269.29 2.29 269.10 2.02 268.81 0.773A 274.30 4.79 274.13 3.91 273.22 4.35 276.57 1.51 272.10 5.51 274.06 1.654B 270.40 0.84 270.00 2.45 268.44 2.19 269.43 1.81 268.80 1.87 269.41 0.815A 270.30 2.75 270.63 2.26 270.11 2.93 270.43 2.99 265.30 2.45 269.35 2.276B 273.40 2.80 272.75 3.54 272.78 2.99 275.14 1.95 270.20 2.86 272.85 1.777A 276.50 4.77 276.75 2.82 276.33 4.24 277.86 1.95 272.00 3.71 275.89 2.258B 279.40 5.19 280.13 2.70 279.00 4.82 282.71 1.98 277.50 4.48 279.75 1.929A 273.30 3.62 274.13 2.03 274.00 3.74 275.43 2.37 271.60 3.20 273.69 1.40

10B 273.70 3.89 269.88 2.36 271.67 3.74 272.57 0.53 271.20 3.36 271.80 1.4411A 270.40 4.30 268.75 3.06 270.33 3.64 270.43 2.70 270.20 3.01 270.02 0.7212B 271.00 5.75 269.50 1.20 270.44 3.81 270.43 1.90 269.70 3.62 270.21 0.6113A 267.80 4.02 267.13 3.00 267.22 4.38 267.86 1.07 265.20 1.93 267.04 1.0814B 275.20 4.34 274.88 3.14 274.78 3.96 277.29 3.35 273.30 3.02 275.09 1.4315A 271.20 2.94 270.38 2.45 272.67 3.54 271.43 1.62 269.40 4.06 271.01 1.2216B 276.60 4.14 273.38 2.83 275.22 4.12 277.14 2.73 274.80 2.97 275.43 1.5017A 272.40 3.86 274.50 2.27 274.11 3.95 273.86 2.91 272.60 3.75 273.49 0.9418B 274.90 4.23 277.25 2.12 276.89 4.46 276.57 3.15 275.60 3.78 276.24 0.9719A 279.60 3.06 278.56 2.24 278.50 1.65 277.89 2.42 276.90 4.65 278.29 0.9920B 272.30 1.89 272.78 0.67 272.50 1.51 272.56 1.81 271.70 1.57 272.37 0.41

Mean of std 1.30

Page 74: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

56

Page 75: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering
Page 76: AUTOMATIC THERMAL INSPECTION OF … thermal inspection of aluminium reduction cell by Guðjón Hugberg Björnsson Research thesis 60 ECTS submitted to the School of Science and Engineering

School of Science and EngineeringReykjavík UniversityMenntavegur 1101 Reykjavík, IcelandTel. +354 599 6200Fax +354 599 6201www.reykjavikuniversity.isISSN 1670-8539