Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Blind Source Separation using Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

May, 2015

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Outline

Background

NMF

Cost function

PSA

FASST

Objectives and source model

Optimization problem: decomposition

Implementation

Evaluation

Try it!

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Outline

Background

NMF

Cost function

PSA

FASST

Objectives and source model

Optimization problem: decomposition

Implementation

Evaluation

Try it!

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Non-negative Matrix Factorization (Lee and Seung, 2001)

V ≈ V = WH

I V and V are spectrograms ∈ <≥0,M×N

I W are the bases where ∈ <≥0,M×K

I H is the activation matrix where H ∈ <≥0,K×N

I Positive → Meaningful

Example:

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Considering β-divergence as cost function

Definition:

dβ(x |y) =

1

β (β−1)

(xβ + (β − 1) yβ − β x yβ−1

)β ∈ R\{0, 1}

x (log x − log y) + (y − x) β = 1

xy− log x

y− 1 β = 0

Observations:

I Note that for different β some common cost functions are defined:

I β = 2: Euclidean distance

I β = 1: generalised Kullback-Leibler divergence

I β = 0: Itakura-Saito divergence

Gradient Descend and Multiplicative Updates:

I b = a− γ∇F (a)

I for γ small enough, then F (a) ≥ F (b)

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Prior Subspace Analysis: composing W

W is formed by two parts:

I Fixed Trained Patterns (Wtar ) where Wt ∈ <≥0,M×Ktar

I Adaptive Patterns (Wbgnd ) where Wt ∈ <≥0,M×Kbgnd

(Note that Ktar + Kbgnd = K)

Example:

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Outline

Background

NMF

Cost function

PSA

FASST

Objectives and source model

Optimization problem: decomposition

Implementation

Evaluation

Try it!

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Objectives:

I Flexible Audio Source Separation Toolbox (FASST) (Ozerov et al. 2012)

is a general purpose framework able to model SS problems.

FASST: source model (I). Based on an excitation-filter model:

V ≈ V = WH =∑j

Vj ; Vj = Vexj � Vft

j

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

FASST: source model (II). Based on an excitation-filter model:

Vj = Vexj � Vft

j = Eexj Pex

j � Eftj Pft

j =

= (Wexj Uex

j Gexj Hex

j )� (Wftj Uft

j Gftj Hft

j )

Vexj is further decomposed as:

I Spectral patterns (Eexj ) are modulated by time activations (Pex

j ).

Spectral patterns (Eexj ) are linear combinations of:

I Narrow band spectral patterns Wexj with weights Uex

j .

Time activations Pexj are a combination of:

I time-localized patterns Hexj with weights Gex

j .

The filter spectral power Vftj is similarly expressed.

EXTREMELLY VERSATILE! Fixed and learnt basis.

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

FASST: source model (III). Based on an excitation-filter model:

Vj = Vexj � Vft

j = Eexj Pex

j � Eftj Pft

j =

= (Wexj Uex

j Gexj Hex

j )� (Wftj Uft

j Gftj Hft

j )

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Optimization problem: decomposition

FASST estimation of the sources is achieved in two steps:

I Step 1. Given initial parameter values θinit , the model parameters θ are

estimated from the mixture V using an iterative Generalized Expectation

Maximization (GEM) algorithm; where each iteration is achieved by

means of Multiplicative Update (MU) rules inspired from the NMF

methodology.

I Step 2. Given the mixture V and the estimated model parameters θ,

source estimates Vj are computed using Wiener filtering.

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Decomposition block diagram

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Outline

Background

NMF

Cost function

PSA

FASST

Objectives and source model

Optimization problem: decomposition

Implementation

Evaluation

Try it!

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Implementation details (I)

I FASST is set for separating the bass, drums, melody and the remaining

instruments from a stereo music recording.

I Is available online written in Matlab.

Relies on two main concepts:

I Having prior knowledge about the sources to label them automatically

(PSA).

I Joint modelling of all sources instead of a cascading approach (separating

the drums, then the melody, etc.).

The sound scene is modelled with 12 sources (Vj ):

I 4 sources representing the bass.

I 4 sources drums.

I 4 sources modelling other instruments.

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Implementation details (II)

The bass and drums sources are constrained to Vj = Wexj Gex

j with:

I Gexj being adaptive

I Wexj being fixed and pre-trained (with the same framework) from isolated

bass and drum samples from the RWC music database.

The melody and to the remaining instruments are constrained to

Vj = Wexj Uex

j Gexj where:

I Uexj and Gex

j are adaptive.

I Wexj is a fixed dictionary representing the narrow band spectral patterns

modelling the harmonic pitch.

Considering:

I ERB spectrogram (providing higher low-frequency resolution than the

STFT desired for modelling drums and bass).

I With random initialization to all the adaptive parameters.

The melody is reconstructed by choosing the most energetic source of the 4

sources modelling other instruments.

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Outline

Background

NMF

Cost function

PSA

FASST

Objectives and source model

Optimization problem: decomposition

Implementation

Evaluation

Try it!

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

BSS Evaluation (I)

I Emmanuel Vincent, Remi Gribonval, Cedric Fevotte. Performance

measurement in blind audio source separation. IEEE Transactions on

Audio, Speech and Language Processing, Institute of Electrical and

Electronics Engineers (IEEE), 2006.

I Objective measures accepted for the community.

Formalization of the problem:

starget(n) = starget(n) + einterf (n) + enoise(n) + eartif (n)

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

BSS Evaluation (II)

Formalization of the objective measures:

I Source to Distortion Ratio (SDR):

SDR = 10log10‖starget(n)‖2

‖einterf (n) + enoise(n) + eartif (n)‖2

I Source to Interference Ratio (SIR):

SIR = 10log10‖starget(n)‖2

‖einterf (n)‖2

I Sources to Noise Ratio (SNR):

SNR = 10log10‖starget(n) + einterf (n)‖2

‖enoise(n)‖2

I Sources to Artifacts Ratio (SAR):

SAR = 10log10‖starget(n) + einterf (n) + enoise(n)‖2

‖eartif (n)‖2

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Outline

Background

NMF

Cost function

PSA

FASST

Objectives and source model

Optimization problem: decomposition

Implementation

Evaluation

Try it!

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Download and use FASST

I http://bass-db.gforge.inria.fr/fasst/

I Download Version 1 (Matlab)

I Uncompress

I Run:

EXAMPLE prof rec sep drums bass melody FULL(’Shannon Hurley Sunrise prof prod mix.wav’, ’example data’,

’example data’)

I And... HAVE FUN! :)

Blind Source

Separation using

Non-negative Matrix

Factorization: FASST

Jordi Pons i Puig

Background

NMF

Cost function

PSA

FASST

Objectives and source

model

Optimization problem:

decomposition

Implementation

Evaluation

Try it!

Thanks

Thanks for your attention!