The Results of Automated Image Analysis Workshop at the

10th European Congress on Telepathology and 4th International Congress on Virtual Microscopy

Arvydas Laurinavičius

Pathology Visions 2010

VILNIUS UNIVERSITY NATIONAL CENTRE

OF PATHOLOGY

Background and Disclaimer

• Pathologist (Renal)• Director, National Center of Pathology, LT• Professor, Vilnius University• EU COST Telepathology Network• EU LLL EUROPALS• MB, IHTSDO (SNOMED CT)• International Member, CAP• User of Aperio and TissueGnostics tools• No competing interests

2010 Vilnius Lithuania2012 Venice Italy

 Telepathology - Programhttp://www.telepathology2010.com/31Screen clipping taken: 2010-09-27; 11:14

  

http://www.telepathology2010.com

The Goal

To provide an overview of automated image analysis tools in terms of their robustness and workflow efficiency in a structured and comparable fashion

Outline

• Why – A Pathologist’s Vision of the Digital

• How – Workshop Design and Results

• What – Ways to Go

>19th 20th 21st century

Evolution of Pathology

DIGITALMOLECULAR

IMUNOHISTOCHEMISTRYMICROSCOPYMACROSCOPY

Spectrum of Technologies

Pathology Lab …

transforms biological information into medical

Biospecimen

Pathology Diagnosis

Patient

Clinical Decision

Spectrum of Technologies

Adding Digital Path-WayTissue collected

Tissue sampled

Tissue processed

Slides produced

Pathologist ”reads” slides

Pathologist interprets images Computer

scans slides

Computer analyzes images

Digital patholo

gy

Competition? Ignorance? Synergy?

MacroscopyMicroscopy

Immunohistochemistry

Questions asked:

• Does this work?• Why is Digital better than Conventional?• Tool or Toy? Long way to go…

More specifically:• Shall I scan everything?• Should scanners be certified for diagnostic use?• Is it legal to make a diagnosis on virtual slides?• Can I work faster on digital images?• Are quantification results reliable?

Innovation versus Routine

Psychology

Technology

Involvement needs awareness

Treat the Tools and Humans equally

Pathologist #1 Pathologist #1

Pathologist #2

perfect

Pathologist #2moderate

Tool #1 Tool #1

Tool #2

perfect

Tool #2perfect

inte

r-ob

serv

er

intra-observer

mod

erate

???

Are different tools in agreement? Are they better than we?

Partitioning the Observer

Tissue collected Tissue sampled Tissue 

processed

Slides produced

Pathologist ”reads” slides

Pathologist interprets images

Computer scans slides

Computer analyzes images

1st European Scanner Contest

Automated Image Analysis Workshop

“2 in 1”

“2 in 1”

“Software” “Hardware”

Outline

• Why – A Pathologist’s Vision of the Digital

• How – Workshop Design and Results

• What – Ways to Go Next

Workshop Design

• Keep simple, explore feasibility of a Contest • Estrogen Receptor and HER2 IHC

– Whole slide and TMAs from the Spanish QA Program

• HER2 FISH– Whole slides from the Ntl Ctr Pathol

• Available for scanning >1 month (at the 1st ESC)• Participants presented their workflow and results

at the Workshop• Presentations posted at http://www.telepathology2010.com

Participants

Company Speaker IHC FISH

Aperio Kate Lillard √

Leica/SlidePath Sean Costello √

BioImagene Vikram Mohan √ √

MetaSystems Christian Schunck √*

3DHistech  Csaba Hankó √ √

* Used for analysis the FISH slides provided

Workshop Results

Concordance testing of the results was out

of scope, however, some output data

provided by the Participants were

analyzed

Estrogen Receptor, % Pos Nuclei3 outlier cases by A, variable ROI selection?

Estrogen Receptor, Total Nuclei CountedB and C, different size of ROI?

Participant B and C Correlation 0.898 p<.0001

Estrogen Receptor, % Pos NucleiStrong correlation; nonlinearity possible?

Nonlinear regression p<.0001

B

Estrogen Receptor, % Pos NucleiNonlinearity: C outputs higher values (frequent 100%) than B

Estrogen Receptor, % Pos NucleiB tends to output lower values than A and C

Not significant

HER2 IHC Score Agreement between B and C

 C  0 1+ 2+ 3+ n/a TotalB0 4 0 0 1 0 5

1+ 4 4 1 0 0 9

2+ 0 0 1 0 0 1

3+ 0 2 2 16 0 20

n/a 1 0 0 0 4 5

Total 9 6 4 17 4 40

Simple Kappa 0.61Weighted Kappa 0.69 Note: different cutoff used by B and C for 3+ (10 vs 30%)

Lessons learned

• Plan thoroughly, involve Participants

• Improve scanning logistics, especially FISH

• Provide gold standard slides, preferably TMAs

• Define sampling

– whole slide, manual annotation, automated ROI

detection

• Harmonize output formats

Outline

• Why – A Pathologist’s Vision of the Digital

• How – Workshop Design and Results

• What – Ways to Go

Ways to Go

• Do nothing

• Do the same

• Do inter-observer (inter-Tool) variability study

• Develop an ongoing QA program

• Disseminate the results

Disseminate

D-PathLympics

Digital Pathology League

Scanner Contest

Image Analysis Contest