Adarza Technologies Bacteria Classification Assistant ENSC 440 Presentation April 19 th, 2005 Members: Edward Goh Linda Wu Josna Rao Geeda Ip

Adarza TechnologiesAdarza TechnologiesBacteria Classification Bacteria Classification AssistantAssistantENSC 440 Presentation

April 19th, 2005

Members:Edward Goh Linda WuJosna RaoGeeda Ip

© Adarza Technologies 2005

Agenda for TodayAgenda for Today• Introduction

• System Overview

• Data Acquisition

• Image Processing

• Data Filtering

• User Interface

• Testing Procedure

• Cost Analysis

• Future Features

• Conclusion

• Acknowledgements

• Q&A

• Demonstration


Company ProfileCompany Profile

• Adarza Technologies– Edward Goh (Chief Executive Officer)– Linda Wu (Chief Operations Officer and Chief

Financial Officer)– Josna Rao (Chief Technology Officer)– Geeda Ip (Chief Information Officer)


IntroductionIntroduction

• The Bacteria Classification Assistant– Helps user identify bacteria– Assists with training new technicians

• Technologically Relevant– microbiology field– Applications – ID microscopic objects


System OverviewSystem Overview

• .NET framework with Open Computer Vision Library

• Microsoft Access - Database

• Windows Forms - GUI

• Accepts JPEG images from hardware

• Returns subset of database

• Classification Criteria:– Colour– Size– Boundary Shape– Colony Shape


BCA Block DiagramBCA Block Diagram


Data AcquisitionData Acquisition

• Digital Camera Microscope

• Camera - Canon Powershot G5 with Scopetronix MaxView Plus accessories for the microscope

• Microscope - Olympus VMT fitted with 1X, 2X objectices


Image Processing AgendaImage Processing Agenda

• Preprocessing Modules– Edge– Edge sequence– Centre Finding

• CC Processing Modules– Size– Colour– Boundary Shape– Colony Shape


Pre-processingPre-processing

• Features that are not specifically classification criteria

• Edge detection

• Edge sequencing

• Centre detection


Edge ProcessingEdge Processing

• Returns image of colony in white, on black background

• Method:– Split image into red, green, and blue planes– Remove noise– Find edges in each plane by calculating

gradient and setting threshold– Adjust threshold until contour closes– Floodfill colony white and background black


Original Image

Blue Edge Green Edge Red Edge

Final Edge


Edge ExamplesEdge ExamplesStreptococcus sanguis


Edge ExamplesEdge ExamplesChryseomonas luteola


Edge ExamplesEdge ExamplesCandida albicans


Edge ExamplesEdge ExamplesNeisseria gonorrhoeae

• Transparent Edge

• No distinct colour difference


Edge SequencingEdge Sequencing

• Detects contours

• Sorts and orders the points from contours

• Returns an OpenCV sequence object

• Used to determine boundary shape and colony shape


Centre Finding ModuleCentre Finding Module

• Centroid equations

• Returns cvPoint

• Relies on edge image

• Centre used for boundary and colony shape


Centre SamplesCentre Samples


Size ModuleSize Module

• Goal: 4 Categories– Tiny, Small, Medium, Large

• Comparison based on 2D area

• Uses library function to count non-zero pixels

• Requires monochrome version of sample and calibration images

• Ratio of calibration and sample area taken


Size SamplesSize Samples


Colour ModuleColour Module

• Goal: 7 Categories– Non-pigmented: White, Gray, White-Yellow-Gray, Red– Pigmented: Blue, Green, Yellow, Purple

• Returns two values: primary and secondary– Primary: most likely– Secondary: for borderline cases

• Method: analyze image in Hue-Saturation-Brightness channels– Search for local maxima

• Some examples


Hue-Saturation-BrightnessHue-Saturation-Brightness

• Hue: 0° ~ 360°

• Saturation: 0% ~ 100%

• Brightness: 0% ~ 100%


Colour Module MethodColour Module Method

• Image separated into 3 planes

Hue Plane Saturation Plane Value Plane


Colour Module MethodColour Module Method

• Each plane analyzed with mask

Hue Plane Saturation Plane Value Plane

100 184239



• Pigmented colonies:


Colour ExamplesColour Examples

• Yellow Colony



• Non-pigmented Colonies

Red - Transparent

White-Yellow-Gray

GrayWhite


Colour ExampleColour Example

• White Colony



• Test Cases1. Strongly pigmented colonies

– Ex: Chryseomonas luteola

2. Very white colonies– Ex: Stomatococcus mucilaginosis

3. Borderline cases– Relies on secondary colour for identification– Relax contraints


Colony Shape ModuleColony Shape Module

• Goal: Classify colony as Circular or Irregular

• Method:– Histogram of Radial Lengths

• Circular:– Single large peak

• Irregular:– Multiple peaks of lower height



• Histogram Analysis:– Locate max peak in histogram– Evaluate average height of surrounding columns– Use threshold to determine if height great enough to

be circular

Length of radial vector

Number of edge points



Irregular

Circular



• Test Cases:1. Colonies of different sizes:

– Small number of edge points: Passed(correct classifier returned)

– Large number of edge points: Passed

2. Different edge cases:– Washout edges: Passed (always irregular)


Boundary Shape ModuleBoundary Shape Module

• Goal: 4 Categories– Smooth, Undulating, or Finger-like

• Method of edge angles– Histogram of Edge Angles– Difference between tangent angle

and normal angle

• Smooth– 0°

• Undulating– Some at 0°

• Fingerlike– Widely varying from 0° to 90°



• Histogram Analysis:– Evaluate average height of bins around 0°– Evaluate max peak height and location– Evaluate average height of bins around 90°– Combination of 4 major thresholds

Edge Angle0° 90 °

Number of edge points

10%



Undulating

Smooth

Finger-like



Smooth

Undulating

Finger-like



• Reliability:– Borderline cases:

• Certain colonies have characteristics of more than one shape type

• Edge may also be unreliable, perhaps due to blurry colony

– Solution:• Peaks closer to thresholds in number and angle

are given two shape classifiers• Both classifiers used to filter results from database



• Test Cases:1. Colonies of different sizes: Passed2. Different edge cases:

– Washout edges: Passed (always finger-like)– Blurry edges: Two shape classifiers returned in

85% of tests

3. Colonies of different shapes:– Circular: Passed– Irregular: Passed– Highly irregular (length more than twice width):

– Leads to some unreliability, often classified as finger-like


DatabaseDatabase

• Introduction

• Technology – Microsoft Access 2000– ADO.NET

• Considerations in design:– Normalization– Size– Portability– Adaptability


Database Structure OverviewDatabase Structure Overview

PhotoTable

PK PHOTOINDEX

PHOTOFILEPATH CALIBRATIONINDEX

SizeTable

PK SIZEINDEX

SIZENAME

BoundaryShapeTable

PK BOUNDARYSHAPEINDEX

BOUNDARYSHAPENAME1 m

LightingCondTable

PK LIGHTCONDINDEX

LIGHTCONDDESC

1

m

ColourTable

PK COLOURINDEX

COLOURNAME ISPIGMENTATION

BacteriaPictureTable

PK BPINDEX

NAME PHOTOINDEX ISREFERENCEDATA BCAVERSION BOUNDARYSHAPEINDEX COLONYSHAPEINDEX COLOURINDEX SIZEINDEX ISGRAMPOSITIVE AGARMEDIUM SMELL TEXTURE CONTAMINATION INCUBTEMP INCUBTIME INCUBATMOSCOND IDCODE COMMENTS UPPERLEFT LOWERRIGHT INSIDEUPPERLEFT INSIDELOWERRIGHT

m

m

m

1

1

1

ColonyShapeTable

PK COLONYSHAPEINDEX

COLONYSHAPENAME

m 1

CalibrationTable

PK CALIBRATIONINDEX

CALIBFILEPATH LIGHTCONDINDEX

m

1


Bacteria Picture TableBacteria Picture TableBacteriaPictureTable

PK BPINDEX

NAME PHOTOINDEX ISREFERENCEDATA BCAVERSION BOUNDARYSHAPEINDEX COLONYSHAPEINDEX COLOURINDEX SIZEINDEX ISGRAMPOSITIVE AGARMEDIUM SMELL TEXTURE CONTAMINATION INCUBTEMP INCUBTIME INCUBATMOSCOND IDCODE COMMENTS UPPERLEFT LOWERRIGHT INSIDEUPPERLEFT INSIDELOWERRIGHT

• Where the main data for each record is stored.

• Can store reference or sample entries

• BCAVERSION field


Boundary Shape TableBoundary Shape Table

BoundaryShapeTable

PK BOUNDARYSHAPEINDEX

BOUNDARYSHAPENAME

• Structure:

• Sample Data:

BOUNDARYSHAPEINDEX BOUNDARYSHAPENAME1 Smooth2 Undulating3 Finger-like


Colony Shape TableColony Shape Table

• Structure:

• Sample Data:

ColonyShapeTable

PK COLONYSHAPEINDEX

COLONYSHAPENAME

COLONYSHAPEINDEX COLONYSHAPENAME1 Circular2 Irregular


Colour TableColour Table• Structure:

• Sample Data:

ColourTable

PK COLOURINDEX

COLOURNAME ISPIGMENTATION

COLOURINDEX COLOURNAME ISPIGMENTATION1 White N2 Grey N3 Red Y4 Blue Y5 Green Y6 Yellow Y7 White-Yellow-Green N8 Purple Y


Size TableSize Table

• Structure:

• Sample Data:

SizeTable

PK SIZEINDEX

SIZENAME

SIZEINDEX SIZENAME1 Tiny2 Small3 Medium4 Large


Photo and Calibration TablesPhoto and Calibration TablesPhotoTable

PK PHOTOINDEX

PHOTOFILEPATH CALIBRATIONINDEX

LightingCondTable

PK LIGHTCONDINDEX

LIGHTCONDDESC

1

m

CalibrationTable

PK CALIBRATIONINDEX

CALIBFILEPATH LIGHTCONDINDEX

m

1

• Photo Table

• Calibration Table

• Lighting Condition Table– Describes the lighting condition under which

the batch of pictures were taken.

– Each entry in the Photo Table points to a bacteria picture file on disk.

– Each entry in the Calibration Table points to a calibration picture file on disk.

– Each entry also points to an entry in the Calibration Table.

– Pictures from the same batch would point to same Calibration entry.

– Each entry also points to an entry in the Lighting Condition Table.


How Files are Stored on DiskHow Files are Stored on Disk

• Top Level – Where the BCA

program is located.

BCABCA.exe

DatabaseBCA.mdb

DataPicture1.jpg

Picture2.jpg

Etc.

• Database Level– Where the Access database

(BCA.mdb) is located.

• Data Level– Where all the pictures are located.


Goals of the GUIGoals of the GUI

• Collect all necessary input from user.

• Guide user through sample image analytical process.

• Present information from the database in a user-friendly manner.

• Allow user to easily search, view, and modify database.


Startup ScreenStartup Screen

• Choice 1: Analyze sample image

• Choice 2: View/Modify database


Browsing Through DatabaseBrowsing Through Database

Modified information appears in Modified information appears in blueblue

Warning before navigating awayWarning before navigating away


Warning before deletionWarning before deletion


After DeletionAfter Deletion


Searching and Setting FiltersSearching and Setting Filters

• Values between criteria are AND-ed

• Values within criteria are OR-ed


Before Filters are SetBefore Filters are Set

• Check marks indicate which filters to set.

• Check box items are in black.

• Once we click “Set Filters”…


After Filters are SetAfter Filters are Set

• Check box items appear in blue.

• Number of filtered records are reflected in the title bar:

• Can undo filters or add/remove additional filters.


After Filters are SetAfter Filters are Set

• Selecting/deselecting additional filters will toggle the appropriate filter buttons.


Analyze Sample Walk-ThroughAnalyze Sample Walk-Through

• Get the calibration image file.

• Get a point inside the calibration image.

• Get the sample image file.

• Get a crop of a single bacterial colony.

• Get a rectangle within the colony.

• Next…




Add to DatabaseAdd to Database

• Automatically fill in calculated values.

• Notifies user if required information is missing.


Testing ProcedureTesting Procedure

• Individual Module testing– Setting threshold levels– 100% accuracy on non-border cases– At least 90% accuracy on border cases

• Consider both primary and secondary choices

• Integrated testing– Multiple testing of reference set– Searching for consistency– Compromise between computer and human

perception


Cost AnalysisCost AnalysisItem Estimated Market Price (CDN) Our Estimated Cost Our Real Cost

Research expenses (photocopying, printing, etc.) $100.00 $100.00 $10.25

Textbooks on C++ and Visual Studio $127.98

Developmental tools and software

· MS Visual Studio .NET Academic Edition 2003 $135.00 $0.00 $0.00

· Matlab Student Version with image acquisition, image processing, and fuzzy logic toolboxes. $177.00 $0.00 $0.00

· Paint Shop Pro 9.0 $144.99 $0.00 $0.00

· CVS Advanced $12.00

Sony Cyber-shot 3.2 DSC-P72 mega-pixel digital camera $300.00 $0.00 $0.00

Microscope by C&A Scientific $135.00 $0.00 $0.00

Lab Petri Dishes 100 x 15mm (20/Pk) $13.50 $0.00 $0.00

Travel expense for consultation meetings $15.00 $15.00 $7.75

Website $15.00 $15.00 $18.11

Consultation Lunch $24.04

Contingencies $20.00 $20.00 $0.00

Total: $1,055.49 $150.00 $200.13


Future FeaturesFuture Features

• Image Processing Features– Active contours– Additional classification criteria

• User Interface Features– Advanced search capabilities– Remove extra input from user– Results ordered by probability


ConclusionConclusion

• Proof of concept complete

• Further development is possible


AcknowledgementsAcknowledgements

• Dr. Mirza Faisel Beg, Assistant Professor of the School of Engineering Science

• Mr. David Chan, Senior Lab Technician @ BCCDC

• Dr. Swee Han Goh, Associate Professor @ BCCDC


Questions?Questions?


DEMODEMO

Documents

Adarza Technologies Bacteria Classification Assistant ENSC 440 Presentation April 19 th, 2005 Members: Edward Goh Linda Wu Josna Rao Geeda Ip