www.ainia.es 1

Hyperspectral imagingRicardo Diaz

Hyperspectral imagingRicardo Diaz

TRAINING SCHOOL IN MONELLS (08/09/14-09/09/14) AND GIRONA (10/09/14)

“NON-DESTRUCTIVE ON-LINE TECHNOLOGIES TO DETERMINE QUALITY OF MEAT AND MEAT PRODUCTS: FUNCTIONING PRINCIPLE AND CHEMOMETRICS”

www.ainia.es 2

1. Theoretical aspects- Introduction: concepts- Operating principle- Hypercube: spectra or images?- Multispectral or Hyperspectral?- Data analysis- Chemical image construction- Algorithm parallelization- Applications

2. Practical application- Definition of the problem to be solved- Instrumentation- Hypercube acquisition- Signal correction with references- Points selection- Pre-treatment- Model calibration- Chemical image generation: resulting images

1. Theoretical aspects- Introduction: concepts- Operating principle- Hypercube: spectra or images?- Multispectral or Hyperspectral?- Data analysis- Chemical image construction- Algorithm parallelization- Applications

2. Practical application- Definition of the problem to be solved- Instrumentation- Hypercube acquisition- Signal correction with references- Points selection- Pre-treatment- Model calibration- Chemical image generation: resulting images

www.ainia.es 3

1. Theoretical aspects1. Theoretical aspects

www.ainia.es 4

1.1. Introduction: concepts

The basis of this technique is based in the interaction between infrared light and matter, where the light is absorbed in different wavelengths of the light.

The basis of this technique is based in the interaction between infrared light and matter, where the light is absorbed in different wavelengths of the light.

www.ainia.es 5

Spectrum: characteristic fingerprint of matter depending on its composition in the NIR region

spectrum

www.ainia.es 6

www.ainia.es 7

Light diffraction

Light breakdown to measure absorption in different wavelengths

www.ainia.es 8

Image analysis

Application of algorithms to a 2D image to obtain information related with physical properties: length, size, colour…

www.ainia.es 9

NIR spectroscopy:

Analysis of the NIR spectrum to measure composition through the absorption of light in different wavelengths

Non destructive technique to obtain chemical information from one point of thesample

www.ainia.es 10

Machine vision

Acquisition of images using image sensor to obtain information related with physical properties

Non destructive technique to obtain physical information from the whole sample

www.ainia.es 11

Hyperspectral vision

Generation of artificial images analysing the NIR spectrum of each pixel of the sample

Non destructive technique to obtain chemical and physical information from each point of the whole sample

www.ainia.es 12

The optics focus a line where the light of each point is diffracted by a spectrographic optic and is projected on the NIR matrix sensor.

1.2. Operating principle

s

λ

www.ainia.es 13

CAMERA

SPECTROGRAPHIC OPTIC

FOCUSING OPTIC

www.ainia.es 14

Parts of a hyperspectral system:

HYPERSPECTRAL CAMERA

PC+GPU

PLC

ENCODER

CONVEYOR BELT

IR LIGTH

www.ainia.es 15

HYPERSPECTRAL CAMERA

NIR LIGHT SYSTEM

REJECTION SYSTEM

PC + GPU

CONVEYOR BELT

www.ainia.es 16

Optical configurations:

REFLECTANCE INTERACTANCE TRANSMITANCE

www.ainia.es 17

spectrum of xj

hypercube

λ

Abs

image in λis

t

λi

xj

λ

1.3. Hypercube: spectra or images?

www.ainia.es 18

1.4. Multispectral or Hyperspectral approach?

Multispectral approach: selection of several wavelengths

Hyperspectral approach: uses all the wavelengths of the spectrum

λ1 λ2

λ1

λ2

www.ainia.es 19

1.5. Data analysis

1. Machine vision approach: application of image algorithms to the images in the selected wavelengths.

λ1 λ2

λ1 λ2

www.ainia.es 20

2. Spectroscopic approach: application of multivariate analysis to estimate composition or classification of each pixel/point of the sample.

λ1 λn

www.ainia.es 21

Characteristics of multivariate analysis:

- More than one input variable (absorbance in each wavelength) andmore than on output variable.

- Reduction of variables (hundreds of absorbance measurements in different wavelength) in a reduced set of data (scores) with thevariance of the dataset (hypercube).

- Concentration Maps Generation (e.g. fat determination in meat):

+ PLS (Projection to Latent Structures)

- Classification of each point of the sample (e.g. classification depending on quality):

+ PCA+AD (Principal Component Analysis + Discriminant Analysis)

+ PLSDA (Projection to Latent Structures DiscriminantAnalysis)

www.ainia.es 22

1.6. Chemical image construction

Assigning a different color to a each pixel depending on its class or composition, we obtain an artificial image (“chemical image”) representing composition map of the sample.

Background

Loin

Fat

Bone

www.ainia.es 23

1.7. Algorithm parallelization

High computation capability is needed:

■ Pre-processing algorithm of each spectrum

■ Scores obtaining algorithm

■ Prediction/classification algorithm

Additional processors are needed:

■ FPGA (Field Programmable Gate Array)

■ DSP (digital signal processor)

■ GPU (Graphics Processing Unit)

www.ainia.es 24

Example of time consuming of a hyperspectral image of 320 x 256.using an Intel Pentium i5 2500K 3.3 GHz with 8 Gb RAM under 64 bits OS without (blue) and with (red) GPU (GeForce 560 Ti (384 cores). Algorithm is PCA with CE pre-processing using 6 scores in the model. Time is seconds.

www.ainia.es 25

Hyperspectral inspection system in real time in ainia pilot plantHyperspectral inspection system in real time in ainia pilot plant

www.ainia.es 26

1.8. Applications

Concentration of the AP in pharmaceutics©ainia

www.ainia.es 27

Detection of hazelnut shells©ainia

www.ainia.es 28

Applications in meat

• Quality Control: measurement of chemical and physical properties with a non-destructive method analysing the 100% of the batch. E.g. tenderness, pH, water activity, fat, moisture…

• Classification of samples based on the quality: RFN (reddish, firm y non exudative), PSE (pale, soft y exudative) y DFD (dark, firm y dry)

• Composition control of the whole production: moisture, protein and fat

• Foreign bodies detection independently of its density

• Defect detection in carcasses: tumours, strokes, faecal residues

www.ainia.es 29

Foreign bodies detection in meat productsDiaz R et al. 2011. Proc. Eurosensors XXV ©ainia

Detection and identification of contaminants PET, HDPE, LDPE film, metal, insect, bone and fat in meat products such as pork loin with detection sensitivity from 1 mm.

www.ainia.es 30

Prediction of drip-loss, pH, and color for pork using ahyperspectral imaging technique

J. Qiao et al. Meat Science 76 (2007) 1–8

Correlation coefficient for predicting the water activity of 0.77, pH of 0.55 and 0.86 color.

J. Qiao et al. Meat Science 76 (2007) 1–8

www.ainia.es 31

Determination of the intramuscular fat % in meat by NIR hyperspectral vision system

© ainia

www.ainia.es 32

In a batch of 28 samples with 400 faecal residues was achieved by detecting the 95% hyperspectral vision system 400 to 1000 nm.

K.C. Lawrence et al., J. Near Infrared Spectrosc. 14, 223–230 (2006)

Contaminant detection on poultry carcasses

www.ainia.es 33

Prediction of beef tenderness using hyperspectral vision

Classification error of 96.4% among 111 samples of beef (longissimus dorsi) in tender, intermediate and hard.

G. K. Naganathana et al. Computers and electronics in agriculture 64 (2008) 225–233

www.ainia.es 34

Poultry skin tumor detection with hyperspectral system

Through a selection of 8 spectral bands in the VIS / NIR could be detected 32 of 40 skin tumors.

S. Nakariyakul, D.P. Casasent / Journal of Food Engineering 94 (2009) 358–365

www.ainia.es 35

Prediction of drip-loss, pH, and color for pork using ahyperspectral imaging technique

Classification of pork loin steaks in RFN (reddish, firm and non-exudative), PSE (pale, soft and exudative) and DFD (dark, firm and dry)

D. Barbin et al. / Meat Science 90 (2012) 259–268

RFN PSE DFD

www.ainia.es 36

NIR hyperspectral imaging for rapid and reagentlessdetermination of Enterobacteriaceae on chicken fillets

PLSR model based on 3 wavelengths (930, 1121 and 1345 nm) to estimate the Enterobacteriaceae presence with a root mean squared errors (RMSEs) > 0.47 log10CFU g-1.Y.-Z. Feng et al. / Food Chemistry 138 (2013) 1829–1836

www.ainia.es 37

www.ainia.es 38

https://www.youtube.com/watch?v=mTmsWQP8Mpw

www.ainia.es 39

2. Practical application2. Practical application

www.ainia.es 40

2.1. - Definition of the problem to be solved

Map generation composition of pork chops withhyperspectral vision in the near-infrared:

- Sample preparation- Instrumentation- Hypercube acquisition- Signal correction with references- Pre-treatments- Model calibration- Validation- Chemical image generation

www.ainia.es 41

Hyperspectral camera:

Camera XEVA-CL (1100 y 2500 nm) with 320x256 px

Spectrograph Imspector N25E

Optics 22mm

Conveyor belt with servomotor

75 W NIR halogen lamps

PC Intel i5, 8 Gb RAM with GPU

Processing Ainia’s software for acquisition, data processing and implementation in real time.

2.2. - Instrumentation

www.ainia.es 42

Instrumentation tuning

- Turn on the lamps

- Turn on the SWIR camera

- Focusing the optics

- Speed regulation

- Adjustment of lighting conditions

- Set the integration time

www.ainia.es 43

2.3. - Hypercube acquisition

Acquisition of the video sequences (hypercubes):

- Black reference

- White reference

- Sample sequences

www.ainia.es 44

2.4. - Signal correction with references

Signal correction avoid external influences caused by light changes

Black reference: internal noise from the sensor

White reference: variations of light in time

Rλ - BλWλ- Bλ

R Reflectance in λwavelengthB Black reference in λwavelengthW White reference in λwavelength

www.ainia.es 45

2.4. – Points selection

Selection of the point to extract the spectra of each class or sample to build the data matrix for calibration

www.ainia.es 46

2.5. - Pre-treatments

Spectrum pre-treatments: a method to improve the quality of the spectrum and to solve some problems during the acquisition process.

The most commonly used methods are:

■ Autoscale: center and scale

■ Mean center

■ Signal Normal Variate (SNV)

■ Multiplicative Scatter Correction (MSC)

■ Savitzky-Golay derivatives

www.ainia.es 47

2.6. - Model calibration

Chemometrics analysis:

PLSDA (Projection to Latent Structures Discriminant Analysis)

Choose cross validation “random subsets”

www.ainia.es 48

2.7. - Chemical image generation: resulting images

www.ainia.es 49

Thanks for your attention

Ricardo Diaz rdiaz@ainia.es