Online Dictionary Learning for Sparse

CodingInternational Conference on Machine Learning, 2009

Julien Mairal, Francis Bach, Jean Ponce and Guillermo Sapiro

Overview

• Introduction• Problem Statement• Online Dictionary Learning• Experimental Validation• Conclusion

Introduction

• The linear decomposition of a signal using a few atoms of a learned dictionary has recently led to state-of-the-art results for image processing tasks.• While learning the dictionary has proven to be critical to achieve results,

effectively solving the corresponding optimization problem is a significant computational challenge. (There may include millions of training sets.)

Introduction

Overview

• Introduction• Problem Statement• Online Dictionary Learning• Experimental Validation• Conclusion

Problem Statement

• Classical dictionary learning techniques consider a finite training set of signals and optimize the empirical cost function.

• Note that should be small if D is “good” at represent the signal x.

Problem Statement

• To prevent D from being arbitrarily large (which would lead to arbitrarily small values of α), it is common to constrain columns to have an L-2 norm less than or equal to one.

• The problem of minimizing can be rewritten as a joint optimization problem with respect to dictionary D and coefficient α,

Problem Statement

• A nature approach to solving this problem is to alternate between the two variables, minimizing over one while keeping the other one fixed.• In the case of dictionary learning, classical projected first-order

stochastic gradient descent consists of a sequence of updates of D:

• The dictionary learning method authors present falls into the class of online algorithms based on stochastic approximations, processing one sample at a time, but exploits the specific structure of the problem to efficient solve it.

Overview

• Introduction• Problem Statement• Online Dictionary Learning• Experimental Validation• Conclusion

Online Dictionary Learning-Algorithm Outline

Online Dictionary Learning – Sparse Coding• The sparse coding problem of Eq. (2) with fixed dictionary is an L1-

regularized linear least-squares problem.• The columns of learned dictionaries are in general highly correlated,

so authors use LARS-Lasso algorithm (Osborne et al., 2000; Efron et al., 2004) to provide whole regularization path (i.e. for all possible values of λ).

Online Dictionary Learning – Dictionary Update• Proposed algorithm for updating the dictionary uses block-coordinate

descent with warm restarts, and one of its main advantages is that it is parameter-free and does not require any learning rate tuning.• Since the vectors are sparse, the coefficients of matrix A are in

general concentrated on the diagonal.• Since algorithm uses the value of as warm restart for computing , a

single iteration has empirically been found to be enough.

Overview

• Introduction• Problem Statement• Online Dictionary Learning• Experimental Validation• Conclusion

Experimental Validation

Overview

• Introduction• Problem Statement• Online Dictionary Learning• Experimental Validation• Conclusion

Conclusion

• Authors have introduced in this paper a new stochastic online algorithm for learning dictionaries adapted to sparse coding tasks.• Preliminary experiments demonstrate that it is significantly faster than

batch alternatives on large datasets that may contain millions of training example.

An Efficient Frame-Content Based Intra Frame Rate Control

for HEVCIEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO 7, JULY 2015

Miaohui Wang, King Ngi Ngan, and Hongliang Li

Overview

• Introduction• Proposed Rate Control Method• Simulation Results• Conclusion

Introduction

• In this letter, authors propose a new gradient based R-λ model for the HEVC intra frame rate control, where the gradient is used to measure the frame-content complexity.• In addition, a novel bit allocation method is developed for CTU rate

control.

Overview

• Introduction• Proposed Rate Control Method• Simulation Results• Conclusion

Modeling the Relationship Between Rate-Gradient and λ for the HEVC Frame Coding• Due to that different frames have different encoding complexities, the

frame-content complexity measure is incorporated into the proposed method for HEVC intra frame coding.

Bit Allocation – GOP Level Bit Allocation

Original – GOP Level Proposed – GOP Level

Original – Frame Level Proposed – Frame Level

Original – CU Level Proposed – CU Level

Model Parameter Update• Original • Proposed

Overview

• Introduction• Proposed Rate Control Method• Simulation Results• Conclusion

Simulation Configuration

1. HM 10.0 : the original HM 10.0 without rate control2. JCT-VC K0103: the original HM 10.0 with the default rate control3. JCT-VC M0257: the original HM 10.0 with the default intra frame rate

control4. Proposed method

Simulation Results

Simulation Results

Overview

• Introduction• Proposed Rate Control Method• Simulation Results• Conclusion

Conclusion

• In this letter, a frame-content based rate control method is proposed for the HEVC intra frame coding.• The frame-content complexity is measured by its gradient, which has

been incorporated into an improved R-λ model.• A new bit allocation scheme with content complexity is developed at

the CTU level.