Upload
others
View
69
Download
7
Embed Size (px)
Citation preview
1© 2014 The MathWorks, Inc.
Image Processing and Computer Vision
with MATLAB and SIMULINK
By Joss Knight
Senior Developer, GPU and Parallel Algorithms
2
Computer
Intelligence
Robotic Vision
Non-linear SP
Multi-variable SP
Cognitive
Vision
Statistics
Geometry
Optimization
Biological Vision
Optics
Smart
Cameras
Computer
Vision
Machine
Vision
Image
Processing
Physics
Imaging
Neurobiology
Mathematics
Machine
Learning
Control
RoboticsArtificial
IntelligenceSignal Processing
Physics
Imaging
Neurobiology
Mathematics
Machine
Learning
Control
RoboticsArtificial
IntelligenceSignal Processing
Computer Vision
System Toolbox™
Neural Network Toolbox™
Simulink™
Signal Processing
Toolbox™
SimScape™
Image Processing
Toolbox™SimBiology™
Embedded
Coder™
Optimization
Toolbox™
MATLAB™
Statistics Toolbox™
SimMechanics™
Control System
Toolbox™DSP System
Toolbox™
Image Acquisition
Toolbox™
The Computer Vision Ecosystem
3
Applications: Image and Video Processing
Medical imaging
Surveillance
Robotics
Automotive safety
Consumer electronics
Geospatial computing
Machine vision
and more…
4
Agenda
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
5
http://www.mathworks.co.uk/products/image-video-processing
6
Agenda
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
• Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition Demo: Live video in MATLAB
Demo: Image Acquisition App
Demo: Live video in SIMULINK
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
7
Image Acquisition
Acquire images and video directly into MATLAB and Simulink
Configure device properties
8
Agenda
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition Demo: Live video in MATLAB
Demo: Image Acquisition App
Demo: Live video in SIMULINK
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze Demo: Tissue segmentation
Demo: Image analysis Apps
Demo: Digital display reader
• 3D Vision
• Image Registration • Embedded Applications
9
Standard Segmentation Workflow
10
Standard Segmentation Workflow
Explore image
11
Standard Segmentation Workflow
Explore image
Enhance image
12
Standard Segmentation Workflow
Explore image
Enhance image
Segment image
13
Standard Segmentation Workflow
Explore image
Enhance image
Segment image
Analyze image
14
Demos: Explore-Enhance-Segment-Analyze
15
Image Processing Apps
NEW!NEW!
Image Viewer
Color Thresholder
Region Analyzer
16
Optical Character Recognition
Support for English, Japanese + others
Simple interface:
>> results = ocr(I)
17
Segmentation – other techniques
Watershed and Active Contours (NEW)
18
Agenda
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze Demo: Tissue segmentation
Demo: Image analysis Apps
Demo: Digital display reader
• 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
Demo: Video Mosaicking (Simulink) • Embedded Applications
19
Image Registration
2D and 3D
Align multi-model images
20
Image Registration
NEW: Non-rigid image
registration and warping
21
Demo: Video Mosaicking
22
Agenda
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
Demo: Video Mosaicking (Simulink) • Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
Geometry Estimation
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
23
Feature Detection, Extraction and Matching
Corner Template SURF MSEREdge
24
Point Tracker
Simple interface:
points = detectXFeatures(I);
tracker = vision.PointTracker;
initialize(tracker, points.Location, frame);
while ~finished
frame = step(videoFileReader);
[points, validity] = step(tracker, frame);
end
25
Geometry Estimation
RANSAC – Random Sampling and Consensus
Gets geometry from a minimal sample set and checks it against all matches
26
Agenda
• Introduction • Feature Detect, Extract & Match
Geometry Estimation
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
Demo: Multi-object Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
27
Object Detection and Tracking
28
Face Detection, People Detection and Foreground Detection
29
Training Image Labeler App
30
Demo: Tracking multiple objects with a Kalman Filter
31
Demo: Tracking multiple objects with a Kalman Filter
Kalman Filter interface:
– Configure:
– Predict:
– Update:
predictedCentroid = predict(kalmanFilter);
kalmanFilter = configureKalmanFilter( ...
'ConstantVelocity', centroid, ...
[200, 50], [100, 25], 100);
c2 = correct(kalmanFilter, centroid);
32
Agenda
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
Demo: Multi-object Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision Demo: Camera Calibrator App
Demo: 3D Reconstruction
Demo: Depth Mapping
• Image Registration
• Embedded Applications
33
3D Vision
-4-3-2-10123
-8
-6
-4
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
x (meters)z (meters)
y (
mete
rs)
Pyramid with dynamic programming and sub-pixel accuracy
0
5
10
15
34
NEW Apps: Camera Calibrator and Stereo Camera Calibrator
Simplified workflow estimating
camera intrinsic and extrinsic
parameters
Removes the effects of lens
distortion from an image
Automatically detects
checkerboard patterns
35
Demo: 3D Reconstruction of Point Cloud
Detect calibration grid to define
world coordinates
Detect and match features on
object
Project into scene using triangulate
36
Demo: Disparity Maps and Distance detection
37
Agenda
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision Demo: Camera Calibrator App
Demo: 3D Reconstruction
Demo: Depth Mapping
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration
• Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications Demo: Ball Tracker
Demo: Quad-copter Model
38
Embedded Applications
56 Image Processing functions
support code generation
Most Computer Vision functionality
supports code generation
Many features generate platform-
independent code
bwdist
bwlookup
bwmorph
bwpack
bwselect
bwtraceboundary
bwunpack
conndef
edge
fitgeotrans
fspecial
getrangefromclass
histeq
im2double
im2int16
im2single
im2uint16
im2uint8
imadjust
imbothat
imclearborder
imclose
imcomplement
imdilate
imerode
imextendedmax
imextendedmin
imfill
imfilter
imhist
imhmax
imhmin
imlincomb
imopen
imquantize
imreconstruct
imregionalmax
imregionalmin
imtophat
imwarp
iptcheckmap
iptcheckconn
label2rgb
mean2
medfilt2
multithresh
ordfilt2
padarray
rgb2ycbcr
strel
stretchlim
ycbcr2rgb
imref2d
imref3d
affine2d
projective2d
39
Demos: Ball Tracker and Quad-copter model
40
Agenda
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
• Introduction • Feature Detect, Extract & Match
• Image Acquisition • Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration • Embedded Applications
41
Explore-Enhance-Segment-Analyze
3D Vision
Apps
Feature Detection
Object Detection and Tracking
Image Registration
Image
AcquisitionImage
Viewer
Color
Thresholder
Region
Analyzer
Camera
Calibrator
http://www.mathworks.co.uk/products/image-video-processing
43© 2014 The MathWorks, Inc.