ADVANCED SIGNAL PROCESSING TECHNIQUES

FOR WIRELESS COMMUNICATIONS

Erdal Panayırcı Electronics Engineering Department

IŞIK University

OUTLINE

IntroductionKnowledge Gaps in GeneralThe essential of EM algorithmThe Sage algorithmSome Application AreasSequential Monte Carlo Method

(SMC)Knowledge Gaps in SMC

INTRODUCTION

Future generation wireless commun. systems are confronted with new challenges mainly due toHostile channel characteristicsLimited bandwidthVery high data rates

Advanced Signal Proc. techniques such asThe Expectation-Maximization

algorithmThe SAGE algorithmThe Baum-Welch algorithmSequential Monte Carlo TechniquesKalman filters and their extensionsHidden Markov modelingStochastic approximation algorithms

In collaboration withInexpensive and Rapid

computational power provide powerful tools to overcome the limitations of current technologies.

Applications of advanced signal processing algorithms, include, but are not limited toJoint/Blind/AdaptiveSequence (data) detectionFrequency, Phase ,timing

synchronizationEqualizationChannel Estimation techniques.

These techniques are employed in advanced wireless communication systems such asOFDM/OFDMACDMAMIMO, Space-time-frequency CodingMulti-User detection

Especially, development of the suitable algorithms for wireless multiple access systems inNon-stationaryInterference-rich

environments presents major challenges to us.

Optimal solutions to these problems mostly can not be implemented in practice mainly due tohigh computational complexity

Advanced signal processing tools, I mentioned before, have provided a promising route for the design of low complexity algorithms with performances approaching the theoretical optimum forFast, and Reliable

communication in highly severe and dynamic wireless environment

Over the past decade, such methods have been successfully applied in several communication problems.

But many technical challenges remain in emerging applications whose solutions will provide the bridge between the theoretical potential of such techniques and their practical utility.

The Key Knowledge GapsTheoretical performance and convergence

analysis of these AlgorithmsSome new efficient algorithms need to be

worked out and developed for some of the problems mentioned above

Computational complexity problems of these algorithms when applied to on-line implementations of some algorithms running in the digital receivers must be handled.

Implementation of these algorithms based on batch processing and sequential (adaptive) processing depending on how the data are processed and the inference is made has not been completely solved for some of the techniques mentioned above.

Some class of algorithms requires efficient generation of random samples from an arbitrary target probability distribution, known up to a normalizing constant. So far two basic types of algorithms, Metropolis algorithm and Gibbs sampler have

been widely used in diverse fields. But it is known that they are substantially complex and difficult to apply for on-line applications like wireless communications.

There are gaps for devising new types of more efficient algorithms that can be effectively employed in wireless applications.

THE EM ALGORITHM

The EM algorithm was popularized in 1977 An iterative “algorithm” for obtaining ML

parameter estimates Not really an algorithm, but a procedure Same problem has different EM

formulations Based on definition of complete and

incomplete data

L. E. Baum, T. Petrie, G. Soules and N. Weiss, A Maximization Technique in Statistical Estimation for Probabilistic Functions of Markov Chains, Annals of Mathematical Statistics, pp. 164-171, 970.

A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum-Likelihood from Incomplete Data Via the EM Algorithm, Journal, Royal Statistical Society, Vol. 39, pp. 1-17, 1977.

C. F. Wu, On the Convergence Properties of the EM Algorithm, Annals of Statistics, Vol. 11, pp. 95-103, 1983.

Main References

The Essential EM Algorithm

Consider estimating parameter vector s from data y (“incomplete” data):

( ) nzsFy += ,

Parameters to be estimated Random parameters

Then, the ML estimate of s is:

( ) ( )[ ]zsypzEsypCsml

s ,maxargˆ =∈

=

Thus, obtaining ML estimates may require:

An Expectation Often analytically intractable

A MaximizationComputationally intensive

The EM Iteration

Define the complete data x

( )xyx → Many-to-one mapping

having conditional density ( )sxf

The EM iteration at the i-th step:

E-step:

M-step:

€

Q s ˆ s i ⎛ ⎝ ⎜

⎞ ⎠ ⎟≡ E log f x s( ) y, ˆ s i

⎡ ⎣ ⎢

⎤ ⎦ ⎥

€

ˆ s i+1 = arg maxs∈C

Q s ˆ s i ⎛ ⎝ ⎜

⎞ ⎠ ⎟

Convergence Properties

At each iteration the likelihood-function is monotonically non-decreasing

If the likelihood-function is bounded, then the algorithm converges

Under some conditions, the limit point coincides with the ML estimate

EM Algorithm Extensions

J. A. Fessler and A. O. Hero, Complete-data spaces and generalized EM algorithms, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-93), Vol. 4, pp. 1-4, 1993.

J. A. Fessler and A. O. Hero, Space-alternating generalized EM algorithm, IEEE Transactions and Signal Processing, October 1994.

The SAGE Algorithm

The SAGE algorithm is an extension of EM algorithm

It provides much faster convergence than EM

Algorithm alternates several hidden data spaces rather than just using one complete data space, and

Updates only a subset of elements of the parameters in each itteration

Some Application Areas

Positron-Emission-Tomography (PET) Genetics Neural Networks Radar Imaging Image / Speech processing Communications

Channel Estimation / Equalization Multiuser detection Squence estimation Interference rejection

SEQUENTIAL MONTE CARLO TECHNIQUE (SMC)

Emerged in the field of statistics, J. S. Liu and R. Chen, “Sequential Monte

Carlo Methods for Dynamics Systems”, J. American Stat. Assoc., Vol. 93, pp. 1032-1044, 1998.

Recently, SMC has been successfully applied to several problems in wireless communications, such as,Blind equalizationDetection/decoding in fading channels

It is basically based on approximating the expectation operation by means of sequentially generated Monte Carlo samples from either unknow state variables or system parameters.

Main Advantages

SMC is self adaptive and no training/pilot symbols or decision feedback are needed

Tracking of fading channels and the estimation of the data sequence are naturally in integrated

Channel noise can be either Gaussian on Non-Gaussian

It is suitable for MAP receiver design

If the system employs channel coding, the coded signal structure can be easily exploited to improve the accuracy of both channel and data estimation

SMC is suitable for high-speed parallel implementation using VLSI

Does not require iterations like in EM algorithm

Updating with new data can be done more efficiently

SMC Method

Let denote the parameter vector of interest Let denote the complete data

so that is assumed to be simple is partially observed It can be partitioned as where

denotes the observed part

( )Tt21t xxxX ,...,,=( )tX p θ

tX( )ttt SYX ,=

( )tt yyyY ,...,, 21=

( )tt sssS ,...,, 21= denotes in the incomplete data or unobservable or missing data.

θ

Example

1. Fading channel

,...2,1,1

=+=∑=

− tnshyL

ititit

Problem: Joinly estimate the data signal and the unknown channel

parameters

{ } Lihi ,...,2,1, =

,...2,1 , =tst

2. Joint Phase Offset and SNR Estimation

,...2,1 , tnesy t

j

tt

θ

θ is unknown phase offset

2 is unknown noise variance

t21t s s sS ,...,, is the data to be transmitted

Problem: Estimate 21 p θ , based on

complete data ttt SYX , where

tt yyyY ,...,, 21 observed part

tt sssS ,...,, 21 incomplete data

MAP SOLUTION USING SMC METHOD

MAP solution of the unknown parameter vector θ is

θθθθθθ

d Y p Y ttMAP ˆ

tttttS

t dS YS p SY pY pt

,θθ

Where p(θ |Yt) can be computed by

means of incomplete data sequence as

Substituting this in the above, we have

tt

tttS

tMAP

Y S

dS YS p S t

θ

ˆ

To implement SMS, we need to draw m independent samples (Monte Carlo samples)

m

j

j

ts1

from the conditional distribution of

yyysss pYS p t21t21tt ,...,,,...,,

Usually, directly drawing samples from this distribution is difficult.

But, drawing samples from some trial-distribution tt YSq is easy.

In this case, we can use the idea of importance sampling as follows:

Suppose a set of samples mjs j

t ,...,2,1, is drawn

from the trial distribution . YS q tt By associating

weight

t

j

t

t

j

tj

t Ys q

Ys pw

to the sample ,j

ts

jt

m

1j

j

t

t

ttMAP w S W

1Y S

θ̂

where,

m

1j

j

jt w W

The pair mjwS jt

jt ,...,2,1,, is called

a properly weighted sample. w. r. t. distribution . YS p tt

We can now estimate MAPθ̂ as follows;

By properly choosing the trial distribution q(.), the weighted samples

wS j

t

j

t ,

can be generated sequentially. That is, suppose a set of properly weighted samples

m

1j

j

1t

j

1t w S ,

Then SMC algorithm generates from this set, a new one

m

1j

j

t

j

t w S , at time t.

at time t-1 is given.

1. Draw samples j

t

j

1t

j

t s SS ,

jts from the trial distribution

q(.) and let

2. Compute the important weight jtw from

j1tw sequentially.

3. Compute the MAP estimate

jt

m

1j

j

tMAP w S m1

θ̂

As a summary SMC algorithm is given as follows for j = 1, 2,..., m

KNOWLEDGE GAPS IN SMC Coosing the effective sample size m (empirically

usually ,20 < m < 100). The sampling weights measures the

“quality” of the corresponding drawn data sequence

Small weights implies that these samples do not really represent the distribution from which they are drawn and have small contribution in the final estimation

Resampling procedure was developed for it. It needs to be improved for differential applications

50m j

tw

jtS

Delay Estimation Problem:

Since the fading process is highly correlated, the future received signals contain information about current data and channel state.

A delay estimate seems to be more efficient and promising than the present estimate summarized above.

In delay estimation:

Instead of making inference on (St, θ) with posterior density p(θ, St|Yt), we

delay this inference to a later time (t+) with the distribution p(θ, St|Yt+)

Note: Such a delay estimation method does not increase computational cost but it requires some extra memory.

Knowledge Gap: Develop computationally efficient delayed-sample estimation techniques which will find applications in channel with strong memory (ISI channel).

Turbo Coding Applications

Because, SMC is soft-input and soft-output in nature, the resulting algorithms is capable of exchanging extrinsic information with the MAP outher channel decoder and sucessively improving the overall receiver performance. Therefore blind MAP decoder in turbo receivers can be worked out.