Fund. of Digital Communications Chapter 5: Demodulation ...GRAZ UNIVERSITY OF TECHNOLOGY Signal Processing and Speech Communications Lab Optimum Detection (cont’d) Maximum Likelihood

GRAZ UNIVERSITY OF TECHNOLOGY

Signal Processing and Speech Communications Lab

Fund. of Digital CommunicationsChapter 5: Demodulation, Detection

Klaus Witrisal

[email protected]

Signal Processing and Speech Communication Laboratory

www.spsc.tugraz.at

Graz University of Technology

January 23, 2014

Fund. of Digital CommunicationsChapter 5: Demodulation, Detection – p. 1/46



Outline� 5-1 Correlation-Type Demodulator and Matched Filter� 5-2 Optimal Detection� 5-3 Error Probability� 5-4 Equalization (and Channel Distortions)

� Reference: (Figures taken from these books.)� J. G. Proakis and M. Salehi, “Communication

System Engineering,” 2nd Ed., Prentice Hall, 2002� J. G. Proakis and M. Salehi, “Grundlagen der

Kommunikationstechnik,” 2. Aufl., Pearson, 2004 (inGerman – Achtung! tw. fehlerhafte Übersetzung)




Correlation-type demodulator� Channel: Additive white Gaussian noise (AWGN) is

addedr(t) = sm(t) + n(t)

� transmitted signal {sm(t)},m = 1, 2, ...M is representedby N basis functions {ψk(t)}, k = 1, 2, ...N

� received signal r(t) is projected onto these basisfunctions {ψk(t)}∫ T

0

r(t)ψk(t)dt =

∫ T

0

[sm(t) + n(t)]ψk(t) dt

rk = smk + nk, k = 1, 2, ..., N

� Output vector in signal space: r = sm + n




Correlation demod. (cont’d)

[Proakis 2002]




Correlation demod. (cont’d)� Received signal

r(t) =

N∑k=1

smkψk(t) +

N∑k=1

nkψk(t) + n′(t)

=

N∑k=1

rkψk(t) + n′(t)

� Correlator outputs r = [r1, r2, ...rN ]T are sufficientstatistik for the decision� i.e.: there is no additional info in n′(t)� n′(t) is part of n(t) that is not representable by

{ψk(t)}� Interpretation of r: noise cloud in signal space




Correlation demod. (cont’d)

[Proakis 2002]




Correlation demod. (cont’d)� Characterization of the noise component

� Mean values

E{nk} = ... =

∫ T

0

E{n(t)}ψk(t) dt = 0

� Correlations (= covariances, since means = 0)

E{nknm} = ... =N0

2δ[m− k]

� i.e.: N noise components {nk} are zero-mean,uncorrelated, Gaussian random variables

� their variances σ2n = N0/2 (because ||ψk(t)||2 = 1)




Correlation demod. (cont’d)� Statistics of the correlator outputs {rk}

� Mean valuesE{rk} = E{smk + nk} = smk

� Variances (identical)σ2r = σ2n = N0/2

� conditional PDF; likelihood function

f(rk|smk) =1√πN0

e−(rk−smk)2/N0 , k = 1, 2, ..., N

f(r|sm) =

N∏k=1

f(rk|smk) m = 1, 2, ...,M




Matched Filter Demodulator� Uses N Filters (instead of the correlators) with impulse

responseshk(t) = ψk(T − t), 0 ≤ t ≤ T

� Filter outputs

yk(t) =

∫ t

0

r(λ)hk(t− λ) dλ = ...

� Sampling at t = T

yk(T ) =

∫ T

0

r(λ)ψk(λ) dλ = rk, k = 1, 2, ..., N

identical to correlator outputs!




Matched Filter Demodulator(cont’d)

[Proakis 2002]





� Property of the matched filter: maximization of theoutput SNR at sampling time t = T

� Derivation: arbitrary impulse response h(t), 0 ≤ t ≤ Ty(t) = ...y(T ) = ...

� Signal-to-noise ratio (SNR)(S

N

)0

=y2s(T )

E{y2n(T )}

� Maximum SNR is found with Cauchy-Schwarz inequ.

h(t) = Cs(T − t) und

(S

N

)0

=2EsN0





Derivation of the Matched Filter Demodulator (cont’d)� SNR of the sampled filter output(

S

N

)0

=y2s(T )

E{y2n(T )}= ... =

[∫ T0 h(λ)s(T − λ) dλ]2

N0

2

∫ T0 h2(T − t) dt

� denominator is energy of the filter IR → hold constant� maximization of the numerator using Cauchy-Schwarz:[∫ ∞

−∞g1(t)g2(t) dt

]2≤

∫ ∞

−∞g21(t) dt

∫ ∞

−∞g22(t) dt

Equality (i.e. maximum) holds for g1(t) = Cg2(t)

This yields h(t) = Cs(T − t) to obtain max. SNRFund. of Digital CommunicationsChapter 5: Demodulation, Detection – p. 12/46



5-2 Optimum Detection(= Entscheidung)

� Signal model: received signal vector with i.i.d. noiser = sm + n

� sm ... points in N -dimensional signal space� n ... random vector with i.i.d. components

∼ N (0, N0/2)

� Find: decision minimizing the error probability, basedon the observed vector r

� Solution: decision rule based on a posterioriprobabilities:

P (signal sm was transmitted |r) m = 1, 2, ...,M

� choose sm that maximizes P (sm|r) for m = 1, 2, ...,M




Optimum Detection (cont’d)Maximizing the posterior probability:� Maximum a posteriori probability (MAP) criterion� finding the posterior probability using Bayes’ rule:

P (sm|r) = f(r|sm)P (sm)

f(r)

� f(r|sm) ... conditional PDF of r given sm wastransmitted (likelihood function)

� P (sm) ... a priori probability� PDF f(r) =

∑Mm=1 f(r|sm)P (sm) is independent of

sm, hence irrelevant� Metrik for MAP criterion: PM(r, sm) = f(r|sm)P (sm)




Optimum Detection (cont’d)Maximum Likelihood (ML) Criterion� equivalent to MAP; for equal a priori probabilities

P (sm) =1

M, ∀m = 1, 2, ...,M

� decision means finding the largest (the maximum)likelihood function f(r|sm) (at the observed point r bychoosing sm)




Optimum Detection (cont’d)� ML criterion for AWGN channel (logarithm of f(r|sm)):

ln[f(r|sm)] = −N2ln(πN0)− 1

N0

N∑k=1

(rk − smk)2

note: first term is irrelevant; 1/N0 too:� ML criterion means minimizing the distance!

D(r, sm) =

N∑k=1

(rk − smk)2 = ||r− sm||2

� D(r, sm) ... distance metric� Equivalently: maximizing the correlation metric:

C(r, sm) = 2rT sm − ||sm||2




Optimum Detection – ExampleGiven: binary, antipodal PAM� s1(t) = −s2(t) = gT (t)

� s1 = −s2 =√Eb (Eb ... energy per bit)

� P (s1) = p, P (s2) = 1− p

Find: metrics for MAP detection in AWGN� Result: threshold detection can be performed!

r ≷s1s2 γ




Optimum Detection – Example(cont’d)

Likelihood functions for binary antipodal PAM:

[Proakis 2002]




5-3 Error Probability� Assumption: binary antipodal signals with

P (s1) = P (s2) = 0.5 and s1 = −s2 =√Eb

� therefore we can use a threshold detector withthreshold γ = 0

� error probability for s1(t)

P (e|s1) =∫ 0

−∞f(r|s1) dr = ... = Q

(√2EbN0

)

Q(x) = 1√2π

∫∞x e−x2/2 dx ... integral over Gaussian

function




Error Probability (cont’d)� average error probability per bit:

Pb = P (s1)P (e|s1) + P (s2)P (e|s2)

=1

2P (e|s1) + 1

2P (e|s2)

= Q(√

2Eb/N0

)� using the distance between signal constellations:

Pb = Q

⎛⎝√

d2122N0

⎞⎠

holds for any binary signals with P (s1) = P (s2) = 0.5

� e.g. bin. antipodal: d12 = 2√Eb; orthogonal: d12 =

√2Eb




Error Probability (cont’d)

0 2 4 6 8 10 12 1410−6

10−5

10−4

10−3

10−2

10−1

SNR/Bit (Eb/N0) [dB]

Pb; B

it−E

rror

−Rat

e (B

ER

)

for antipodal,binary Signals

for orthogonalbinary Signals

3 dB

� Pb for antipodal and orthogonal binary signalsFund. of Digital CommunicationsChapter 5: Demodulation, Detection – p. 21/46



Error Probability (cont’d)� Approximate symbol error rate (SER) for higher-order

modulation schemes

PM ≈ NeQ

⎛⎝√d2min

2N0

⎞⎠

where� Ne ... average number of nearest neighbors (in

signal space)� dmin ... (min.) distance to those neighboring signal

vectors




Error Probability (cont’d)� for M -ary PAM

Ne =2M − 1

M

dmin =2

√3 log2M

M2 − 1

√Eb

PM ≈2M − 1

M

×Q

(√6 log2M

M2 − 1

√EbN0

)

� BER for Gray codingPb ≈ PM/k




Error Probability (cont’d)� for M -ary PSK

Ne =2

dmin =2√

Es sin π

M

=2√kEb sin π

M

PM ≈2Q

(√2kEbN0

sinπ

M

)

� BER for Gray codingPb ≈ PM/k




Error Probability (cont’d)� for M -ary orthogonal sign.

Ne =M − 1

dmin =√

2Es =√

2kEb

PM ≤(M − 1)Q

(√kEbN0

)

� Upper bound:PM < e−k(Eb/N0−2 ln 2)/2

i.e. PM → 0 for k → ∞ ifEb/N0 > 2 ln 2 = 1.4 dB

� in fact: Eb/N0 > −1.6 dBsufficient: Shannon limit




Comparison of modu-lation schemes:� shows R/W :

transmission ratenormalized withbandwidth

� as a function of SNRper bit: Eb/N0

� to reach SER of 10−5

� Shannon limit is ap-proached for orthog-onal signals if k → ∞

[Proakis 2002]




Channel Capacity[Proakis02, Sec. 9.3]� Channel capacity for AWGN channel (Shannon)

C = W log2

(1 +

P

N0W

)

� C ... channel capacity [bit/s]� W ... bandwidth [Hz]� P ... signal power [W]� N0 ... noise PSD [W/Hz]

� increase power P → (slow) increase of capacity� no limit but lots of power required




Channel Capacity (cont’d)� increase bandwidth W� at fixed power P

C =W log2

(1 +

P

N0W

)

� consider limit W → ∞

limW→∞

C =P

N0log2 e = 1.44

P

N0

� there is a hard limit� need to increase P too




Channel Capacity (cont’d)� in practice we must

have R < C

R < W log2

(1 +

P

N0W

)

define� r = R/W “spectral rate”� Eb = P/R energy per bit

we obtain

r < log2

(1 + r

EbN0

)Fund. of Digital CommunicationsChapter 5: Demodulation, Detection – p. 29/46



5-4 Equalization

[Proakis 2002, Section 8.6]� Nyquist criterion must be

fulfilled for ISI-freetransmission

� Cascade of linear filters

GT (f)C(f)GR(f) = Xrc(f)

� Xrc(f) ... Fourier transformof raised-cosine pulse (ISIfree!)




Channel Distortion (cont’d)

Design considerations; assume channel C(f) known� Cascade GT (f)C(f)GR(f) = Xrc(f) fulfills Nyquist� Noise n(t) is AWGN (flat spectrum)→ Receiver filter GR(f) should be matched to cascade

GT (f)C(f)

GT (f) =

√Xrc(f)

C(f)e−j2πft0 GR(f) =

√Xrc(f)e

−j2πftr




Equalization� Channel is unknown or changes with time

� e.g. telephone channel – routing changes� e.g. mobile radio – multipath propagation

→ design GT (f) and GR(f) as for AWGN channel:root raised cosine frequency responseGT (f) =

√Xrc(f)e

−j2πft0 |GR(f)| · |GT (f)| = Xrc(f)

� Output of receiver filter

r(t) = y(t) + n(t) =

∞∑k=−∞

a[k]x(t− kT ) + n(t)

a[k] ∈ {am}Mm=1 ... PAM (or QAM) symbolsx(t) = gT (t) ∗ c(t) ∗ gR(t) ... cascaded system response




Equalization (cont’d)� Equivalent output with periodic sampling:

y[k] = y(kT ) =∑L2

l=−L1xla[k − l] (noise-free)

xk = x(kT ) ... equiv. FIR discrete-time channel filter




Equalization (cont’d)Maximum likelihood sequence detection (MLSD)� optimum detector (for AWGN: r[k] = y[k] + n[k]):

choose information sequence a := {a[k]}Kk=1

based on the received sequence r := {r[k]}K+L−1k=1

� each a[k] ∈ {am}Mm=1,hence there are MK possible sequences {a(m)}ML

m=1

� joint likelihood function: (w/ y(m)[k] = a(m)[k] ∗ xk)

f(r|a(m)) =

K+L−1∏k=1

f(r[k]|a(m)[k])

=

K+L−1∏k=1

1√πN0

e(r[k]−y(m)[k])2/N0




Maximum likelihood sequencedetection (MLSD) (cont’d)

joint likelihood function (cont’d):

f(r|a(m)) =1

(πN0)K+L

2

exp

[− 1

N0

K+L−1∑k=1

(r[k]− y(m)[k])2

]

equivalent distance metric (squared distance)

D(r, a(m)) =

K+L−1∑k=1

(r[k]− y(m)[k])2; y(m)[k] =

L2∑l=−L1

xla(m)[k − l]

� MLSD: channel {xk} is known; find for m = 1, 2, ...,MK

m̂ = argmaxm

f(r|a(m)) = argminm

D(r, a(m))




Viterbi (MLSD)can be done sequentially using the Viterbi algorithm

� distances are computed recursively: D(r(1:k), a(m)(1:k)

) =

D(r(1:k−1), a(m)(1:k−1)

) + (r[k]−∑L2

l=−L1xla

(m)[k − l])2

� Complexity:� symbols a[k] are M -ary PAM symbols� FIR channel has memory L− 1 = L1 + L2

→ compute ML branch metrics at each stage k,yielding ML−1 surviving sequences (trellis diagram)

→ complexity: (K + L− 1)ML (scales linearly with K)

� while ML �MK , still M and L must be small!




Viterbi example (MLSD)� Channel model; L = 2 (1 memory); M = 2 (binary)

r[k] = x0a[k] + x1a[k − 1] + n[k]; a[k] ∈ {−1,+1}� state variable (memory) a[k − 1] ∈ {−1,+1}

� information sequence: {a[k]}Kk=1 = a; each a[k] ∈ {±1};i.e. 2K possible sequences denoted a(m)

� observation sequence: {r[k]}K+1k=1 = r

� likelihood functions (for m = 1, ...,MK)

f(r|a(m)) ∝ exp

[− 1

N0

K+1∑k=1

(r[k]− x0a(m)[k]− x1a

(m)[k − 1])2

]




Viterbi example (MLSD) (cont’d)

� MLSD maximizes likelihood fct. by testing any a(m),which is equivalent to minimizing:D(r, a(m)) =

∑K+L−1k=1 (r[k]− x0a

(m)[k]− x1a(m)[k − 1])2

� at stage kD(r(1:k), a

(m)(1:k)

) =∑k−1

l=1 (r[l]− x0a(m)[l]− x1a

(m)[l − 1])2

+(rk − x0a(m)[k]− x1a

(m)[k − 1])2

= D(r(1:k−1), a(m)(1:k−1)

) + branch metric

� Viterbi requires:� ML = 22 = 4 branch metrics at each state� ML−1 = 21 = 2 surviving paths at each state→ (K + 1)4 �MK = 2K metric computations




Linear Equalizers� Use linear filters to compensate for channeldistortion

� filter has adjustable parameters� preset equalizers: measure channel; set

parameters� adaptive equalizers: update parameters during

data transmission




Zero-Forcing Linear Equalizer

How to select equalizer GE(f)?� GR(f) is matched to GT (f) to fulfill zero-ISI� Hence GE(f) must compensate channel distortion

GE(f) =1

C(f)=

1

|C(f)|e−jΘc(f)

→ inverse channel filter� forces ISI to zero (at sampling times)

detector input: r[k] = y[k] + n[k]

� noise variance (for AWGN); noise gain!

σ2n =

∫ ∞

−∞Sn(f)|GR(f)|2|GE(f)|2df =

N0

2

∫ W

−W

|Xrc(f)||C(f)|2 df




Linear transversal filterPractical implementation of the linear equalizer� use FIR filter with adjustable taps {cn}� delays τ : τ = T ... symbol spaced; aliasing, if 1/T < 2W

� τ < T ... fractionally-spaced equalizer (often: τ = T/2)




Zero-Forcing Equalizer (cont’d)

How to determine taps {cn} for the zero-forcing equalizer?� Equalized output pulse

q(t) =

N∑n=−N

cnx(t− nτ)

for x(t) = gT (t) ∗ c(t) ∗ gR(t); number of taps 2N + 1 ≥ L

� enforce zero-ISI at sampled output:

q(kT ) =

N∑n=−N

cnx(kT − nτ) =

{1, k = 0

0, k = ±1,±2, ...,±N

� system of 2N + 1 linear equations: q = Xc

for coefficients {cn} → approx. solution c = X†q




Minimum mean-square-errorequalizer

� ZF equalizer ignores noise! → noise gain� solution:

� relax zero-ISI condition� consider residual ISI plus noise at equalizer output

→ use minimum mean-square-error (MMSE) criterion� sampled equalizer output

z(kT ) =

N∑n=−N

cnr(kT − nτ) = cT r[k]

� mean-square-error (MSE)MSE = E{(z(kT )− a[k])2} = E{(cT r[k]− a[k])2}




MMSE equalizer (cont’d)Find the equalizer c that minimizes the MSE

E{[cT r[k]− a[k]]2} = E{[cT r[k]− a[k]][rT [k]c− a[k]]}= E{cT r[k]rT [k]c} − 2E{cT r[k]a[k]} + E{a[k]2}= cTKrc− 2cTkar + E{a[k]2}

� with Kr = E{r[k]rT [k]} and kar = E{r[k]a[k]}� minimization: find derivative with c; set to zero

Krc− kar = 0 → c = Kr

−1kar

� correlation matrix Kr and correlation sequence kar

must be estimated→ training sequences




0 1 2 3 4 5 6−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

1.2

n (samples)

Am

plitu

de

Impulse Response

channel impulse reponsezero−forcing equalizerMMSE equalizer

0 0.2 0.4 0.6 0.8 1−600

−400

−200

0

Normalized Frequency (×π rad/sample)

Pha

se (d

egre

es)

0 0.2 0.4 0.6 0.8 1−20

−10

0

10

Normalized Frequency (×π rad/sample)

Mag

nitu

de (d

B)

channel frequency reponsezero−forcing equalizerMMSE equalizer

0 10 20 30 40 50−2

−1

0

1

2data and distorted data

−2 −1 0 1 20

1000

2000

3000

4000

5000

6000

0 10 20 30 40 50−3

−2

−1

0

1

2

3received signal samples

−4 −2 0 2 40

100

200

300

400

500

0 10 20 30 40 50−3

−2

−1

0

1

2

3ZF equalizer

−4 −2 0 2 40

100

200

300

400

500

0 10 20 30 40 50−2

−1

0

1

2MMSE equalizer

−4 −2 0 2 40

100

200

300

400

500




Adaptive equalizers� perform the MMSE equalization recursively� search along gradient for MSE (e.g. LMS algorithm)

� discussed in detail in VL: Adaptive Systems


Documents

Fund. of Digital Communications Chapter 5: Demodulation ...GRAZ UNIVERSITY OF TECHNOLOGY Signal Processing and Speech Communications Lab Optimum Detection (cont’d) Maximum Likelihood