Image Processing and Computer Vision with MATLAB and … · 37 Point Cloud Processing in Matlab As of R2015a Computer Vision System Toolbox (R2014b/15a) Computational Geometry in

1© 2015 The MathWorks, Inc.

Image Processing and Computer Vision

with MATLAB and SIMULINK

By Dr Jasmina Lazić

Application Engineering, MathWorks

2

Computer

Intelligence

Robotic Vision

Non-linear SP

Multi-variable SP

Cognitive

Vision

Statistics

Geometry

Optimization

Biological Vision

Optics

Smart

Cameras

Computer

Vision

Machine

Vision

Image

Processing

Physics

Imaging

Neurobiology

Mathematics

Machine

Learning

Control

RoboticsArtificial

IntelligenceSignal Processing

Physics

Imaging

Neurobiology

Mathematics

Machine

Learning

Control

RoboticsArtificial

IntelligenceSignal Processing

Computer Vision

System Toolbox™

Neural Network Toolbox™

Simulink™

Signal Processing

Toolbox™

SimScape™

Image Processing

Toolbox™SimBiology™

Embedded

Coder™

Optimization

Toolbox™

MATLAB™

Statistics Toolbox™

SimMechanics™

Control System

Toolbox™DSP System

Toolbox™

Image Acquisition

Toolbox™

The Computer Vision Ecosystem

3

Applications: Image and Video Processing

Medical imaging

Surveillance

Robotics

Automotive safety

Consumer electronics

Geospatial computing

Machine vision

and more…

4

Agenda

• Introduction • Feature Detect, Extract & Match

• Image Acquisition • Object Detection and Tracking

• Explore-Enhance-Segment-Analyze • 3D Vision

• Image Registration • Embedded Applications

5

http://www.mathworks.co.uk/products/image-video-processing


6

Agenda





• Feature Detect, Extract & Match




• Introduction





• Introduction





• Introduction


• Object Detection and Tracking



• Introduction


• Image Acquisition Demo: MATLAB live video with Microsoft Kinect

Demo: Image Acquisition App• Object Detection and Tracking



7

Image Acquisition

Acquire images and video directly into MATLAB and Simulink

Configure device properties

8

Agenda

• Introduction


• Image Acquisition Demo: Live video in MATLAB

Demo: Image Acquisition App

Demo: Live video in SIMULINK




• Introduction





• Introduction


• Image Acquisition




• Introduction






• Introduction




• 3D Vision


• Introduction




• Explore-Enhance-Segment-Analyze Demo: Tissue segmentation

Demo: Image analysis Apps

Demo: Digital display reader

• 3D Vision


9

Standard Segmentation Workflow

10


Explore image

11


Explore image

Enhance image

12


Explore image

Enhance image

Segment image

13


Explore image

Enhance image

Segment image

Analyze image

14

Demos: Explore-Enhance-Segment-Analyze

15

Image Processing Apps

NEW!NEW!

Image Viewer

Color Thresholder

Region Analyzer

16

Optical Character Recognition

Support for English, Japanese + others

Simple interface:

>> results = ocr(I)

17

Segmentation – other techniques

Watershed and Active Contours (NEW)

18

Agenda

• Introduction




• Explore-Enhance-Segment-Analyze Demo: Tissue segmentation

Demo: Image analysis Apps

Demo: Digital display reader

• 3D Vision


• Introduction




• 3D Vision


• Introduction




• Explore-Enhance-Segment-Analyze

• 3D Vision


• Introduction





• 3D Vision


• Introduction





• 3D Vision

• Embedded Applications

• Introduction





• 3D Vision

• Image Registration

Demo: Video Mosaicking (Simulink) • Embedded Applications

19

Image Registration

2D and 3D

Align multi-model images

20

Image Registration

NEW: Non-rigid image

registration and warping

21

Demo: Video Mosaicking

22

Agenda

• Introduction





• 3D Vision


Demo: Video Mosaicking (Simulink) • Embedded Applications

• Introduction





• 3D Vision


• Introduction





• 3D Vision



• Introduction





• 3D Vision



• Introduction




• 3D Vision




Geometry Estimation




• 3D Vision



23

Feature Detection, Extraction and Matching

Corner Template SURF MSEREdge

24

Point Tracker

Simple interface:

points = detectXFeatures(I);

tracker = vision.PointTracker;

initialize(tracker, points.Location, frame);

while ~finished

frame = step(videoFileReader);

[points, validity] = step(tracker, frame);

end

25

Geometry Estimation

RANSAC – Random Sampling and Consensus

Gets geometry from a minimal sample set and checks it against all matches

26

Agenda


Geometry Estimation




• 3D Vision



• Introduction




• 3D Vision







• 3D Vision







• 3D Vision






• 3D Vision





Demo: Pedestrian Tracking


• 3D Vision



27

Object Detection and Tracking

28

Face Detection, People Detection and Foreground Detection

29

Training Image Labeler App

30

Demo: Tracking multiple objects with a Kalman Filter

31

Demo: Tracking multiple objects with a Kalman Filter

Kalman Filter interface:

– Configure:

– Predict:

– Update:

predictedCentroid = predict(kalmanFilter);

kalmanFilter = configureKalmanFilter( ...

'ConstantVelocity', centroid, ...

[200, 50], [100, 25], 100);

c2 = correct(kalmanFilter, centroid);

32

Bag of Words for Category Classification

1. Create Bag of Features

2. Encode Input Images

3. Repeat for all classes

4. Use machine learning algorithm

to discriminate between classes

33

Agenda



Demo: Multi-object Tracking


• 3D Vision






• 3D Vision






• 3D Vision






• 3D Vision










• Explore-Enhance-Segment-Analyze • 3D Vision Demo: Camera Calibrator App

Demo: 3D Reconstruction

Demo: Depth Mapping



34

3D Vision

-4-3-2-10123

-8

-6

-4

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

x (meters)z (meters)

y (

mete

rs)

Pyramid with dynamic programming and sub-pixel accuracy

0

5

10

15

35

NEW Apps: Camera Calibrator and Stereo Camera Calibrator

Simplified workflow estimating

camera intrinsic and extrinsic

parameters

Removes the effects of lens

distortion from an image

Automatically detects

checkerboard patterns

37

Point Cloud Processing in Matlab

As of R2015a

Computer Vision System Toolbox (R2014b/15a)

Computational Geometry in base Matlab

Shipping Example: 3-D Point Cloud Registration and Stitching

pointCloud Object for storing a 3-D point cloud

pcdenoise Remove noise from a 3-D point cloud

pcdownsample Downsample a 3-D point cloud

pcmerge Merge two 3-D point clouds

pcregrigd Register two point clouds with ICP algorithm

pctransform Rigid transform a 3-D point cloud

showPointCloud Plot 3-D point cloud

Pcread Read a 3-D point cloud from PLY file

Pcwrite Write a 3-D point cloud to PLY file

alignColorToDepth Align Kinect color image to depth image

depthToPointCloud Convert Kinect depth image to a 3-D point cloud

http://www.mathworks.com/examples/matlab-computer-vision/4318-3-d-point-cloud-registration-and-stitching

39

Agenda



• Explore-Enhance-Segment-Analyze • 3D Vision Demo: Camera Calibrator App

Demo: 3D Reconstruction

Demo: Depth Mapping

























• Image Registration • Embedded Applications Demo: Ball Tracker

Demo: Quad-copter Model

40

Embedded Applications

More than 60 Image Processing

functions support code generation

Most Computer Vision functionality

supports code generation

Many features generate platform-

independent code

bwdist

bwlookup

bwmorph

bwpack

bwselect

bwtraceboundary

bwunpack

conndef

edge

fitgeotrans

fspecial

getrangefromclass

histeq

im2double

im2int16

im2single

im2uint16

im2uint8

imadjust

imbothat

imclearborder

imclose

imcomplement

imdilate

imerode

imextendedmax

imextendedmin

imfill

imfilter

imhist

imhmax

imhmin

imlincomb

imopen

imquantize

imreconstruct

imregionalmax

imregionalmin

imtophat

imwarp

iptcheckmap

iptcheckconn

label2rgb

mean2

medfilt2

multithresh

ordfilt2

padarray

rgb2ycbcr

strel

stretchlim

ycbcr2rgb

imref2d

imref3d

affine2d

projective2d

41

Demo: Ball Tracker

42

Agenda









43

Explore-Enhance-Segment-Analyze

3D Vision

Apps

Feature Detection

Object Detection and Tracking

Image Registration

Image

AcquisitionImage

Viewer

Color

Thresholder

Region

Analyzer

Camera

Calibrator



46© 2015 The MathWorks, Inc.

Documents

Image Processing and Computer Vision with MATLAB and … · 37 Point Cloud Processing in Matlab As of R2015a Computer Vision System Toolbox (R2014b/15a) Computational Geometry in