Balazs KÖVESI, Dominique MASSALOUX

Present document contains informations proprietary to France Telecom. Accepting this document means for its recipient he or she recognizes the confidential nature of its content and his or her engagement not to reproduce it, not to transmit it to a third party, not to reveal its content and not to use it for commercial purposes without previous FTR&D written consent.

Method of Packet Errors Cancellation Suitable for any Speech and Sound Compression Scheme

STQ Workshop, Sophia-Antipolis, February 11th 2003

Balazs KÖVESI, Dominique MASSALOUX

France Telecom R&D

Diffusion of this document is subject to France Telecom authorization D2

Introduction

s ContextQApplication like VoIP or audio streaming

–Possible high packet loss rate (up to 10 %)

s Proposition of a frame error concealment (FEC) methodQCopes with high packet loss rateQRelies on CELP synthesis schemeQIndependent from the codec typeQSpeech oriented but also suitable for musicQIncludes adaptive gain controlQAvoids "robot" voiceQEnsures the decoder memory updateQSmoothing after an erased period

France Telecom R&D


Plan

s Basic principle of the new FEC method

s Implementation in a MDCT codec

s Generalization to other codec types

s Conclusion

France Telecom R&D


Basic principle of the new FEC

Decoded signaldecoder

Valid data

Indication of erased data

Storage of decoded samples

Synthesis of missing samples

Decoder update

reconstructed signal

Smoothing with

decoded signal

France Telecom R&D


s The MDCT transformQAnalysis with 50 % overlap

Q Synthesis with overlap-add

Implementation in a MDCT codec

memory for the next frame overlap-add

-windowing-TT FF transform

-FF TT transform-windowing-overlap-add

Analysis window n-1

20 ms

Analysis window n

t

Synthesis window n-1

decoded frame n

20 mst

Synthesis window n

France Telecom R&D


s Effect of frame erasure

Qthe loose of x bitstream frames affects x+1 output frames Qthese frames have to be synthesized in the decoder


Disturbed: frames n-1 & n 20 ms

Erased: frame n

t

France Telecom R&D


s Memorizing part

QAfter decoding a valid frame

–The 40 ms output memory is updated–The energy of the frame is calculated–The energy memory buffer is updated ( 5 s )


Valid dataDecoded signaldecoder

Erased data indication



Decoder update

synthesized signal

Smoothing with

decoded signal

France Telecom R&D







Decoder update

synthesized signal

Smoothing with

decoded signal

Synthe-sized signal

LTP parameters (B(z))

LPC parameters (A(z))

LPCanalysis

calc. past excitation

signal

LTP filtering1/B(z)

LPC synthesi

s1/A(z)

Adaptive gain control

Memory of past decoded signal

LTP analysis & V/UV detection

France Telecom R&D







Decoder update

synthesized signal

Smoothing with

decoded signal

Synthe-sized signal



LPCanalysi

s



signal

LTP filtering1/B(z)

LPC synthesi

s1/A(z)



QLPC filter modelizes the spectral envelope

QCoefficients not transmitted –LPC analysis order can be higher than in a usual CELP (32 @

16kHZ) better performance on music

past decoded signalLPC coefficientsfilter A(z)

classical method

20 ms

France Telecom R&D







Decoder update

synthesized signal

Smoothing with

decoded signal

Synthe-sized signal



LPCanalysis

LTP analysis & V/UV

detection


signal

LTP filtering1/B(z)

LPC synthesi

s1/A(z)



QPrecise pitch estimation is crucial for the good performanceQOnly integer pitch (P) values are examined [50 Hz, 600 Hz]QNormalized correlations on the last 2P samplesQPitch criteria: maximum correlation +

multiple & fractional verificationsQV/UV criteria: selected correlation value +

energy value 5 s energy memory energy evolution of the last two pitch

periods

past decoded signal

p1

p2

p3

p4

correlationcalculations

France Telecom R&D







Decoder update

synthesized signal

Smoothing with

decoded signal

Synthe-sized signal



LPCanalysis


calc. past

excitation signal

LTP filtering1/B(z)

LPC synthesi

s1/A(z)



QLPC analysis filtering

past decoded signalA(z)

past excitation signal

France Telecom R&D







Decoder update

synthesized signal

Smoothing with

decoded signal

Synthe-sized signal



LPCanalysis



signal

LTP filterin

g1/B(z)

LPC synthesi

s1/A(z)



QExcitation signal generation for the LPC synthesis filteringQvoiced excitation:

–2 components–harmonic, lower frequency bands

LTP filter combined with a low pass filter–less harmonic, higher frequency bands

LTP filter combined with a high pass filter + randomly evolving pitch

France Telecom R&D


QExcitation signal generation for the LPC synthesis filteringQvoiced excitation:

–2 components–harmonic, lower frequency bands

LTP filter combined with a low pass filter–less harmonic, higher frequency bands

LTP filter combined with a high pass filter + randomly evolving pitch






Decoder update

synthesized signal

Smoothing with

decoded signal

Synthe-sized signal



LPCanalysis



signal

LTP filterin

g1/B(z)

LPC synthesi

s1/A(z)



Qunvoiced excitation–non harmonic, lower frequency bands–“randomized” LTP filtering + low pass filtering

+ sudden energy variations are suppressed

France Telecom R&D







Decoder update

synthesized signal

Smoothing with

decoded signal

Synthe-sized signal



LPCanalysis



signal

LTP filtering1/B(z)

LPC synthes

is 1/A(z)



QLPC synthesis filtering

excitation signal synthesized signal

A(z)1

France Telecom R&D







Decoder update

synthesized signal

Smoothing with

decoded signal

Synthe-sized signal



LPCanalysis



signal

LTP filtering1/B(z)

LPC synthesi

s1/A(z)



QImportant in case of long erased periods (> 20 ms)Q2 adaptation laws:

–stationary

–non-stationary

–The adaptations also depend on the pitch value Qdecision available from the LTP analysis

background noise level

40 ms t

t

France Telecom R&D


s Recoverable information


Synthesized frames n-1, n, n+1 20 ms

Erased: frame n-1 & n

t

decoded frame n+2

France Telecom R&D


s Recoverable informationQfor the first erased frame


MDCT transform on the first 2 synthesized frames

t

Partly recovered frame n-1IMDCT transform

France Telecom R&D


s Decoder memory update


t





Decoder update

synthesized signal

Smoothing with

decoded signal

MDCT transform on the last 2 synthesized frames

IMDCT memory to update

IMDCT transform(FT + windowing)

updated IMDCT memory

France Telecom R&D


s Recoverable informationQfor the last erased frame


t

Partly recovered frame n+1IMDCT transform

France Telecom R&D


s Smoothing part Qwithout smoothing


t





Decoder update

synthesized signal

Smoothing with

decoded signal

Synthesized frames n-1, n, n+1 decoded frame n+2

discontinuity

synthesized domain

error-free domain

France Telecom R&D


s Smoothing part QA codec independent solution:


t





Decoder update

synthesized signal

Smoothing with

decoded signal

Synthesized frames n-1, n, n+1 decoded frame n+2

synthesized domain

error-free domain

Extra synthesized samples

crossfading 1

0

France Telecom R&D


s Smoothing part Qwith MDCT smoothing


t

smooth transition at frame n+1overlap-addlike crossfading





Decoder update

synthesized signal

Smoothing with

decoded signal

synthesized domain

error-free domain

France Telecom R&D


s can be adapted to any coding schemeQwas successfully implemented in

–temporal codecs (G.711, G.721, G722)–in a CELP codec (G.723.1) –in a hierarchical codec composed of a CELP and a transform codec

QMemorizing and synthesis part are codec independentQDecoder memory update

–very important for recursive codecs (CELP)–general solution: coding – decoding on the synthesized frames

–too complex for CELP–less complex solution: backtracking

QSmoothing –a general solution: crossfading–more efficient smoothing can be found for some coding schemes

(ex.: MDCT)–the decoder memory update ensure the smoothing in CELP codecs

Generalization to other codec types





Decoder update

synthesized signal

Smoothing with decoded

signal

France Telecom R&D


s A general FEC method for any coding schemeQoptimal for speech voice, good performances on music Qavoids too synthetic sound for voiced framesQkeeps the nature of the unvoiced frames Qenhanced energy managementQcareful update of the decoder memoryQsmoothing after an erased period

s Informal subjective tests have shown its good behavior

s Successfully implemented in group communication applications

s Perspectives:Qspeech / music decision + enhanced music modeQ…

Conclusion

Documents

Balazs KÖVESI, Dominique MASSALOUX