2016-10-10

1

Color standardization for the immunohistochemically stained

tissue section images Anna Korzyńska, Jakub Żak,

Łukasz Roszkowiak, Dorota Pijanowska, Tomasz Markiewicz

2016 IEEE International Conferences on

Imaging Systems & Techniques

Presentation scheduler

• Introduction to the problem of WSI color standardization • Methods:

– The glass slide for the color standardization

– The procedure of the color standardization

– The method of color comparison

• Results of color changes during color standardization

• Conclusions

2 2016 IEEE International Conference on Imaging Systems and Techniques (IST)

2016-10-10

2

Introduction to the problem of color standardization (I)

Number or publications about digital imaging in pathology has grown almost exponentially during the period of the last decade owing to improvements in slide scanners and development of methodology of computer image processing for whole slide digital images (WSI). But the problem of color consistency and color dissimilarity for images produced by different digitalization tools called also batch effect is still visible in any collaborative repository of histopathological and immunohistopathological images, where data is collected from many hospitals and clinics. It is difficult to determine whether the colors of a digital slide resemble colors of a microscope slide. As software for its analysis is frequently dependent from color and texture, the color standardization procedure is needed. So, there is a need to propose a procedure, that makes colors of two images created by various slides digitalization tools as similar as possible.

3 2016 IEEE International Conference on Imaging Systems and Techniques (IST)

3DHistech

Apero

Introduction to the problem of color standardization (II)

• The color variation in images captured by various tools for the same microscope slides digitalization, such as scanners or microscopes as a result of different optics, light sources, cameras (sensors and mosaics) and software for image restoration.

• These methods of color management in histological slides digitalization tools were published by: – Yagi 2011 then by Murakami in 2012 are based on pixels’ color statistics which

also vary and depend on the tissue and pathology presented in an image. – Revie and co-workers in 2014 [8] presents calibrating slides that are produced

based on the biopolymers stained with typical dyes used in the pathology labs. But they are still not available commercially.

– Bautista and co-workers in 2015 [11] converts colors in a digital slide according to information captured from color calibration slide, adjusted to slides stained with Hematoxylin and Eosine (H&E).

4 2016 IEEE International Conference on Imaging Systems and Techniques (IST)

2016-10-10

3

The glass slide

for the color standardization

in digital images

from immunohistochemically stained

tissue sections

The proposed microscope slide is constructed using a typical glass slide on which color samples of polycarbonate plastic and deep dyed polyester film with known light transmittance (Substrate: PET – Polyethylene Terephathalate from ROSCO LTD ) are fixed. It is assembled using following color phantoms:

red, green, blue, and 50% gray as primary colors, and also additional colors, which are

generally observed in DAB&H tissue slides: brown, amber, yellow.

5 2016 IEEE International Conference on Imaging Systems and Techniques (IST)

The procedure of the color standardization

• The first step is digitalization of the color correction microscope slide (1) using particular digitalization tool and the result image is used to prepare matrix of coefficients for color correction (2 & 3)

• The second step to correct colors in WSI digitalized by the tool for which correction matrix had been calculated (4 & 5).

6 2016 IEEE International Conference on Imaging Systems and Techniques (IST)

2016-10-10

4

Correction matrix:

1.5821 -0.6471 0.0800

-0.0229 0.9877 0.1932

0.1679 -0.5475 1.5589

7 2016 IEEE International Conference on Imaging Systems and Techniques (IST)

The method of color comparison • Two types of evaluation techniques are used:

– the distances in CIELAB

– mean square error in RGB between colors of corresponding pixel

• Since images from each scanning tool have different size in pixels and different optical

distortion embedded in images, any comparison should be performed after matching and transforming procedure which adjustment images to be similar in pixel size and in co-localization of corresponding objects.

• The MATLAB software for matching and calculating transformation and matching one image to

the other with bicubic interpolation has been used to adjust images from different scanners.

8 2016 IEEE International Conference on Imaging Systems and Techniques (IST)

2016-10-10

5

Materials The method has been validated using

several glass slides prepared in: • Pathology Department of

Military Institute of Medicine (MIM, Warsaw) and

• Pathology Department of Hospital de Tortosa Verge de la Cinta, Tortosa

digitalized using following digitalization tools: – Microscope Olympus + ImagePro

Premier – Microscope Nikon – CoolScope 1 by Nikon – Pannoramic DESK from 3D

Histech – Aperio AT2 from Leica – ZEISS Axio Scan.Z1

The visual compression have been done using self calibrating monitors EIZO ColorEdge CS230-BK + I1 Display PRO rent. 9 2016 IEEE International Conference on Imaging Systems and Techniques (IST)

Fragments of two images digitalized by microscope Olympus (top) and CoolScope (middle). The microscope slide fragment before color standardization (left) and after (right) are shown in the first two rows. Graphs in the bottom row demonstrate a-b chromaticity space with colors presented in image digitalized by microscope in yellow and digitalized by CoolScope in blue.

2016 IEEE International Conference on Imaging Systems and Techniques (IST) 10

Results of color changes during color standardization (I)

2016-10-10

6

Results of color changes during color standardization (II)

The quantitative comparison have been done using experimental data and artificial synthetic images with and without noise on the level typical for slide scanners. The mean distance between pair of experimental images before and after standardization is decreased by the process of standardization in statistically significant way.

Image pairs of

micros-cope slides

Comparison

Scanner

Normalized

distance in

CIELAB

(mean /

standard

deviation)

Mean square

error calculated

in RGB

( mean /

standard

deviation)

Before After Before After

Refernce

vs

Artificial – without

noise

CoolScope

S1 0.0016* 0.0004* 11.2 9.1

Pannoramic

DESK 0.0036* 0.0014* 11.3 9.7

Refernce

vs

Artificial – with

noise

CoolScope

S1 0.0015* 0.0008* 10.9 10.8

Pannoramic

DESK 0.0036* 0.0022* 11.3 10.6

Set of fragments of

experimental

samples

CoolScope

S1 6,4±0.76 9,4±0.7 12.4±1.3& 8.7±1.7&

Pannoramic

DESK

•*perceptually similar

•&statistically significant for p<0.001

using T-students test

11 2016 IEEE International Conference on Imaging Systems and Techniques (IST)

Conclusions • This method is applied in the MIAP (PIKAOM) platform developed in MIM by the

consortium of three institutions: MIM, Nalecz Institute of Biocybernetics and Biomedical Engineering, and Warsaw University of Technology under national applied project. To pathologists and reaseachers from registered institution the MIAP offers tools for quantitative analysis of digital images of IHC stained tissue samples with 3,3’- Diaminobenzidine (DAB) and Haematokxilin (H), some of which are based on color and texture features of objects. As the samples are acquired by different digitalization tools, the users are obligated to add the digitalized version of the calibration slide to use the proposed software.

• The proposed 8 colors of samples are selected to correct DAB&H stained images. With an addition of learning dynamic neural network, the software for correction matrix calculation and digital standardization is independent from colors of samples and their position on the glass. So, it can be easily adapted to other spots’ location shape and colors.

12 2016 IEEE International Conference on Imaging Systems and Techniques (IST)

2016-10-10

7

Acknowledgment This study was supported by the National

Centre for Research and Development, Poland

(grant PBS2/A9/21/2013).

We all thank Anna Wesołowska-Szcześniak, Marek

Dawgul, PhD, and Konrad Dudzinski, PhD, for help in calibration glass assembling and cleansing

processes.

Thanks for your attention! Nalecz Institute of Biocybernetics

and Biomedical Engineering Polish Academy of Sciences

Warsaw, Poland 13 2016 IEEE International Conference on Imaging Systems and Techniques (IST)