If you can't read please download the document
Upload
lucretia-armando
View
44
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Learning sparse representations to restore, classify, and sense images and videos. Guillermo Sapiro University of Minnesota. Supported by NSF, NGA, NIH, ONR, DARPA, ARO, McKnight Foundation. Ramirez. Martin Duarte. Lecumberry. Rodriguez. Overview. Introduction - PowerPoint PPT Presentation
Citation preview
Learning sparse representations to restore, classify, and sense images and videosGuillermo Sapiro
University of MinnesotaSupported by NSF, NGA, NIH, ONR, DARPA, ARO, McKnight Foundation
Learning Sparsity*Martin DuarteRodriguezRamirez
Lecumberry
Learning Sparsity
Learning Sparsity* OverviewIntroductionDenoising, Demosaicing, InpaintingMairal, Elad, Sapiro, IEEE-TIP, January 2008
Learn multiscale dictionaries Mairal, Elad, Sapiro, SIAM-MMS, April 2008
Sparsity + Self-similarityMairal, Bach, Ponce, Sapiro, Zisserman, pre-print.
Incoherent dictionaries and universal codingRamirez, Lecumberry, Sapiro, June 2009, pre-print
Learning to classifyMairal, Bach, Ponce, Sapiro, Zisserman, CVPR 2008, NIPS 2008Rodriguez and Sapiro, pre-print, 2008.
Learning to sense sparse signalsDuarte and Sapiro, pre-print, May 2008, IEEE-TIP to appear
Learning Sparsity
Learning Sparsity*Introduction I: Sparse and Redundant Representations
Webster Dictionary: Of few and scattered elements
Learning Sparsity
Learning Sparsity* Restoration by Energy Minimization y : Given measurements x : Unknown to be recoveredRestoration/representation algorithms are often related to the minimization of an energy function of the formBayesian type of approach
What is the prior? What is the image model?
Learning Sparsity
Learning Sparsity* The Sparseland Model for Images M
Learning Sparsity
Learning Sparsity* What Should the Dictionary D Be?
Learning Sparsity
Learning Sparsity*Introduction II: Dictionary Learning
Learning Sparsity
Learning Sparsity* Measure of Quality for DField & Olshausen (96)Engan et. al. (99)Lewicki & Sejnowski (00)Cotter et. al. (03)Gribonval et. al. (04)Aharon, Elad, & Bruckstein (04) Aharon, Elad, & Bruckstein (05)Ng et al. (07)Mairal, Sapiro, Elad (08)
Learning Sparsity
Learning Sparsity* The KSVD Algorithm General Aharon, Elad, & Bruckstein (`04)
Learning Sparsity
Learning Sparsity*Show me the pictures
Learning Sparsity
Learning Sparsity* Change the Metric in the OMP
Learning Sparsity
Learning Sparsity* Non-uniform noise
Learning Sparsity
Learning Sparsity* Example: Non-uniform noise
Learning Sparsity
Learning Sparsity* Example: Inpainting
Learning Sparsity
Learning Sparsity* Example: Demoisaic
Learning Sparsity
Learning Sparsity* Example: Inpainting
Learning Sparsity
Learning Sparsity*Not enough fun yet?: Multiscale Dictionaries
Learning Sparsity
Learning Sparsity*Learned multiscale dictionary
Learning Sparsity
Learning Sparsity*
Learning Sparsity
Learning Sparsity*Color multiscale dictionaries
Learning Sparsity
Learning Sparsity*Example
Learning Sparsity
Learning Sparsity*Video inpainting
Learning Sparsity
Extending the Models Learning Sparsity*
Learning Sparsity
Universal Coding and Incoherent Dictionaries
ConsistentImproved generalization propertiesImproved active set computationImproved coding speedImproved reconstructionSee poster by Ramirez and Lecumberry
Learning Sparsity*
Learning Sparsity
Sparsity + Self-similarity=Group SparsityCombine the two of the most successful models for images
Mairal, Bach. Ponce, Sapiro, Zisserman, pre-print, 2009 Learning Sparsity*
Learning Sparsity
Learning to Classify
Learning Sparsity*Global Dictionary
Learning Sparsity
Learning Sparsity*Barbara
Learning Sparsity
Learning Sparsity*Boat
Learning Sparsity
Learning Sparsity*Digits
Learning Sparsity
Which dictionary? How to learn them?Multiple reconstructive dictionary? (Payre)
Single reconstructive dictionary? (Ng et al, LeCunn et al.)
Dictionaries for classification!
See also Winn et al., Holub et al., Lasserre et al., Hinton et al. for joint discriminative/generative probabilistic approaches
Learning Sparsity*
Learning Sparsity
Learning Sparsity*Learning multiple reconstructive and discriminative dictionariesWith J. Mairal, F. Bach, J. Ponce, and A. Zisserman, CVPR 08, NIPS 08
Learning Sparsity
Learning Sparsity*Texture classification
Learning Sparsity
Semi-supervised detection learningMIT -- Learning Sparsity*
Learning Sparsity
Learning Sparsity*Learning a Single Discriminative and Reconstructive Dictionary
Exploit the representation coefficients for classificationInclude this in the optimizationClass supervised simultaneous OMPWith F. Rodriguez
Learning Sparsity
Learning Sparsity*Digits images: Robust to noise and occlusions
Learning Sparsity
Learning Sparsity*Supervised Dictionary LearningWith J. Mairal, F. Bach, J. Ponce, and A. Zisserman, NIPS 08
Learning Sparsity
Learning to Sense Sparse Images
MotivationCompressed sensing (Candes &Tao, Donoho, et al.)SparsityRandom samplingUniversalityStabilityShall the sensing be adapted to the data type?Yes! (Elad, Peyre, Weiss et al., Applebaum et al, this talk).Shall the sensing and dictionary be learned simultaneously?
Learning Sparsity*
Learning Sparsity
Some formulas.
Learning Sparsity*+ RIP (Identity Gramm Matrix)
Learning Sparsity
Design the dictionary and sensing togetherLearning Sparsity*
Learning Sparsity
Just Believe the Pictures Learning Sparsity*
Learning Sparsity
Just Believe the PicturesLearning Sparsity*
Learning Sparsity
Just Believe the PicturesLearning Sparsity*
Learning Sparsity
Learning Sparsity*ConclusionsState-of-the-art denoising results for still (shared with Dabov et al.) and videoGeneralVectorial and multiscale learned dictionariesDictionaries with internal structureDictionary learning for classificationSee also Szlam and Sapiro, ICML 2009See also Carin et al, ICML 2009Dictionary learning for sensing
A lot of work still to be done!
Learning Sparsity
Please do not use the wrong dictionaries12 M pixel image7 million patchesLARS+online learning: ~8 minutes
Mairal, Bach, Ponce, Sapiro, ICML 2009
Learning Sparsity*
Learning Sparsity
Learning Sparsity*
Learning Sparsity