Introduction to Linear Image Processing 1 Introduction to...

1 Introduction to Linear Image Processing

IPAM - UCLA July 22, 2013

Introduction to Linear Image Processing

Iasonas Kokkinos

Center for Visual Computing

Ecole Centrale Paris / INRIA Saclay

Computer Vision

Image to Symbols

Computer Graphics

Symbols to Image

Imaging

Physics to Image

Image Processing

Image to Image

Image Sciences in a nutshell

Images as functions

Continuous d=1: Gray

d=3: Color

Discrete

Image Denoising

Key assumption: clean image is smooth

Moving Average in 2D

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

Slide Source: S. Seitz

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10 20

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10 20 30

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10 20 30 30

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10 20 30 30 30 20 10

0 20 40 60 60 60 40 20

0 30 60 90 90 90 60 30

0 30 50 80 80 90 60 30

0 20 30 50 50 60 40 20

10 20 30 30 30 30 20 10

10 10 10 0 0 0 0 0

Denoising: input

Denoising: first application of averaring filter

Denoising: tenth application of denoising filter

Denoising: application of larger box filter

Weighted averaging

σ = 2 σ = 5

Weighting kernel

σ = 10

Standard deviation, σ: determines spatial support

Gaussian function:

Moving average

Gaussian blur

Image Processing

image image filter

Linearity

Translation Invariance

Linear, Translation-Invariant (LTI) system

Linear Image Processing

image image filter

From time-invariance: useful bases.

image image filter

Linear algebra reminder

Orthonormal basis:

Expansion coefficients:

Basis: N linearly independent vectors

Expansion on basis:

Expansion:

Canonical basis

Canonical basis for 2D signals

Kronecker delta

Canonical basis for signals: expansion

Unit sample function

Signal expansion:

Identify terms:

Rewrite:

Sifting property:

Canonical basis for signals and LTI filters

Any signal:

impulse response

Translation-invariance

Output of any LSI filter for any input:

convolution of input with filter’s impulse response

Convolution sum

sample

By linearity:

Convolution – discrete and continuous

2D convolution sum:

2D convolution integral:

image image filter

Associative Property:

Associative property & efficiency

Separability of Gaussian:

Slow Fast

Associative property & accuracy

Derivative of Gaussian:

approximate exact

Associative property & multi-scale processing

Semi-group property of Gaussian:

Denoising: first application of averaging kernel

Denoising: 10th application of denoising kernel

Distributive property & efficiency

Distributive property:

Steerable fliter:

W. Freeman and E. Adelson, ‘The design and use of steerable filters’, PAMI, 1991

Linear algebra reminder: eigenvectors

Eigenvectors:

Full-rank, real and symmetric: eigenbasis

Eigenvectors and eigenfunctions

Eigenvector:

Eigenfunction:

Input:

Output:

Eigenfunctions for LTI filters

LTI filter:

Let’s guess:

It works:

Frequency response:

Expansion on harmonic basis

From orthonormality:

Inner product for complex functions:

Discrete-time:

Continuous-time:

Change of basis

Canonical expansion:

Eigenbasis expansion:

Rotation matrix from eigenbasis:

Fourier transform: change of basis Rotation from canonical basis to eigenfunction basis

Fourier Analysis

Fourier synthesis equation

Continuous-time:

Discrete-time:

Convolution theorem of Fourier transform

Input expansion:

Output:

Expansions:

image image filter

Convolution theorem

Fourier Analysis Fourier Synthesis

Convolution theorem and efficiency

Fast Fourier Transform Fast Fourier Transform

Gaussian blur Time

Frequency

Moving average Time

Frequency

Modulation property and Gabor filters

Modulation property:

Gaussian:

Time Frequency

Modulation property and Gabor filters

Modulation property:

Gaussian:

Gabor:

Time Frequency

• Consider many combinations of and

Frequency responce

isocurves

Increasing

2D Gabor filterbank

2D Gabor filterbank and texture analysis

Thursday’s lecture: Pyramids, Scale-Invariant Blobs/Ridges, SIFT,

HOG, Log-polar features, Harmonic analysis on surfaces…

• Linear Time-Invariant filters

• Convolution

• Fourier Transform

• (Derivative-of) Gaussian filters

• Steerable filters

• Gabor filters

Summary

Introduction to Linear Image Processing 1 Introduction to...

Documents

Introduction to introduction to introduction to … Optimization

Compositional Convolutional Neural Networks: A Deep ...ayuille/JHUcourses/...models and deep networks into compositional convo-lutional neural networks, a uniﬁed deep model with

An A* perspective on deterministic optimization for ...ayuille/courses/Stat238-Winter12/astar_pr00J.pdf · To Appear in Pattern Recognition Letters 1 Introduction A promising approach

Image Parsing - Department of Computer Science › ~ayuille › JHUcourses › ProbabilisticModelsOfVisu… · Image Parsing Alan Yuille (UCLA/JHU). Segmentation, Detection, and Recognition

An Introduction for An Introduction for Introduction for ......An Introduction for An Introduction for Introduction for TeachersTeachersTeachers . exChange, Blackburn Cathedral’s

The Concave-Convex Procedure (CCCP).ayuille/pubs/ucla/A177_ayuille_NC2003.pdf · 2016. 8. 5. · The Concave-Convex Procedure (CCCP) A. L. Yuille and Anand Rangarajan Smith-Kettlewell

1. Introduction 1. Introduction 1. Introduction 1 ... · Introduction 1. Introduction 1. Introduction 1. Introduction 6061B0242401. The Phone Screen From the Phone screen, you can

INTRODUCTIONS (LEADS) Academic introduction Creative introduction Action introduction Dialogue introduction Overarching societal statements Personal introduction

Cue Coupling - Department of Computer Scienceayuille/courses/Stat271-Fall15/BasicCueCoupling.pdfCue Coupling This section describes models for coupling di erent visual cues. The ideas

Introduction Introduction (ctnd.)

The Concave-Convex Procedure (CCCP)ayuille/pubs/optimize_papers/cccp_nc03J.pdfThe Concave-Convex procedure (CCCP) is a way to construct discrete time it-erative dynamical systems which

Introduction to Retinaayuille/courses/Stat271-Fall15/RetinaIntroduction.pdf · Purpose of the Retina • The anatomy and electrophysiology of the retina has been studied extensively

Course: Vision as Bayesian Inference. Lecture 1 - …ayuille/courses/Stat238-Winter10/chp1.pdf · Course: Vision as Bayesian Inference. Lecture 1 ... it has become possible to determine

Deep Networks - Department of Computer Scienceayuille/courses/Stat271-Fall15/Deep Networks Vitta… · Taigman et al. DeepFace: Closing the Gap to Human-Level Performance in Face

PatchAttack: A Black-box Texture-based Attack CCVL, Johns ...ayuille/JHUcourses/VisionAsBayesianInference202… · PatchAttack: A Black-box Texture-based Attack with Reinforcement

1 Recursive Compositional Models for Vision: description ...ayuille/Pubs10_12/JMIVyuilleB.pdfAlthough our work closely relates to this literature our emphasis is more on discriminative

Lambertian model of reflectance I: shape from shading and ...ayuille/courses/Stat238-Winter14/... · •Lambertian SFS produces an eikonal equation 𝛻 6= 6+ 6= 1 𝐸 6 −1 •Right

Relative luminance and binocular disparity preferences are ...ayuille/courses/Stat271-Fall13/PNAS-2012-Samond… · Relative luminance and binocular disparity preferences are correlated

· Contents INTRODUCTION Introduction

Introducción / Introduction / Introduction