Random matrices and their applications to Multi-Input ...signal.ece.utexas.edu/seminars/dsp_seminars/02spring/UTpresentation.pdf · Random matrices and their applications to Multi-Input

Random matrices and their applications

to Multi-Input Multi Output

Communication SystemsAnna Scaglione

School of Electrical & Computer Engineering

Cornell University

March 6, 2002

2002 Telecomm. Seminar 1

Why should we be interested in this problem

• Percentage of papers appeared in 2001 that have at least one

matrix defined: 89% IT, 65% Com., 99.999999% SP

• We all use Matlab, we all own our personal copy of “Matrix

Computations” [Golub, Van Loan]. Tons of matrices are

decomposed everyday by ECE students

• Random Matrices are practical, useful and have beautiful

properties

• Random Matrices are good for you: they make you realize that

your knowledge of Calculus is at the level of a Mickey Mouse

Cartoon

Anna Scaglione


Random Matrices in Communication Systems

• Multiple sources, Multiple Sensors

• System with transmit and receive diversity

I Random Fading

I Random Space-Time Codes

• Symbol Synchronous CDMA system

• Data Vectors with Random Covariance

Anna Scaglione


MIMO Systems

x (t) y (t)

h (t) k,r

x (t)

x (t)

y (t)

y (t)1

2

1

2

m n

x(t) = (x1(t), . . . , xn(t))T

• Multiple Access, Array Processing: x(t) from different sources

• Transmit Diversity: x(t) =∑+∞

n=−∞ x[n]gT (t− nT ) n× 1

• Space-time code c = (x[n1], . . . , x[nl]) n× l

Anna Scaglione


MIMO Channel Output Model

y(t) = (y1(t), . . . , yn(t))T

y[k] := y(kT ) signals samples, T ≈ 1/W , W= Bandwidth

• Narrowband Channel (Flat fading):

y[k] = H [k]x[k] + n[k] m× 1

• Broadband Channel (Frequency selective):

y[k] =∞∑

n=−∞H [k − n]x[n] + n[k]. m× 1

H [k] is m× n

Anna Scaglione


Symbol Synchronous CDMA system

• Multiple sources, One sensor, Multiple samples

y(t) =K∑

k=1

Akbksk(t) + n(t)

the vector {y}1,p , y(pTc),

y = SAb + n

{b}1,k , bk, Users symbols

A , diag(A1, . . . , AK), Users amplitudes

{S}k,p , sk(pTc), Users signatures sampled at the chip rate

Tc ≈ 1/W

• This model was used in [Hanly,Tse’99] to compare the

performances of Linear Multiuser Detectors

Anna Scaglione


Vectors with Random Covariances

• Sensor network

x1

x23x

xi

• The covariance matrix of x[k] = (x1[k], . . . , xn[k]) depends on the

relative distances between nodes di,j which is random.

• We can study the rate distortion function of the data as a whole

Anna Scaglione


Relevant Performance Measures

MIMO channel ⇒ H random, CDMA ⇒ S random

• MIMO channel Capacity and CDMA aggregate Capacity:

C = log |σ2I + HHH | C = log |σ2I + SAAHSH |• MIMO channel MMSE for LMMSE receiver

MMSE = Tr((I + HHHσ−2)−1)

• LMMSE User SIR

SIRi = sHi (SAAHSH + σ2I)−1si

• Decorrelating receiver User SIR

SIRi = {σ2(SAAHSH)−1}−1i,i

• Differential Entropy of Gaussian sensor data

H(X) =1

2log(2πe)n|Rxx|

Anna Scaglione


Random Eigenvalues −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10

5

10

15

• Field initiated by the pioneering work by Eugene Paul Wigner

• He searched for the asymptotic empirical density of the ordered

random eigenvalues of an n× n symmetric X(ω):

µω(x) =1

n

n∑i=1

δ(x− λii(X(ω)))

• Now we can run this 3 line worth Matlab experiment

>> n=600; B=randn(n); A=(B+B’)/(2*sqrt(2*n)); hist(eig(A),60)

and see what Wigner derived with pencil and paper (semicircle law)

Anna Scaglione


Asymptotic distribution we care about

Theorem [Machenko-Pastur ’67]

Let X = 1nBHB and B m× n and such that α = m/n:

(a) The elements of Bn are i.i.d. random variables ∈ C with

E{[B]i,j} = 0, V ar{[B]i,j} = 1 and E{|[B]i,j|4} < ∞.

µω(x) converges weakly as n →∞ to the Machenko-Pastur

distribution

µα(x) = max(1− α, 0)δ(x) +

√(x− a)(b− x)

2πx1[a,b](x)

where 1[a,b](x) = 1 for a ≤ x ≤ b and is zero elsewhere, δ(x) is a

Dirac delta and:

a := (√

α− 1)2 , b := (√

α + 1)2

• More general asymptotic results are available (used for CDMA

systems).

Anna Scaglione


MIMO channel- Some Asymptotic results

γ :=P0σ2

H

σ2n

The closed form expression for the normalized Capacity is

C(γ)=1

log(2)

(log(γ w) +

1− α

αlog

(1

1− v

)− v

α

)

where:

w =1

2

(1 + α + γ−1 +

√(1 + α + γ−1)2 − 4α

)

v =1

2

(1 + α + γ−1 −

√(1 + α + γ−1)2 − 4α

)

Anna Scaglione


Asymptotic results

The expressions of MSE and Pe are:

MSE(γ) =1

2 αγ

(−√

a b γ +√

1 + a γ√

1 + b γ − 1)

Pe(γ) ≤ exp (−(1 + α)γ)

(1

2√

αI1(2

√αγ)

1 + α

4αI0(2

√αγ)

)

−10 −5 0 5 10 15 20 25 30

10−3

10−2

10−1

γ(dB)

MM

SE

ζ = 0.3

ζ = 0.5

ζ = 0.8

Num.unif. Num.optim. theo.unif. theo.optim.

−10 −5 0 5 10 15 20

10−10

10−8

10−6

10−4

10−2

Pe

γ(dB)

ζ = 0.3

ζ = 0.5

ζ = 0.1

Num.unif. Num.optim. theo.unif. theo.optim.

Anna Scaglione


Derivation of the Statistics

• Asymptotic results are ready to use (Examples later)

• Derivation of the Statistics for the finite case

Matrix decompositions are Transformations of Random Variables!

I First step: deriving the Jacobian of the change of variables

from the original matrix to its factors

I Verify the uniqueness: true in the case of EVD (almost true),

QR or LU (lower-upper) decompositions and Cholesky

decomposition

• One way of doing it: Exterior differential Calculus

Anna Scaglione


Exterior Differential Calculus

• Seminal work of Elie Cartan

• Based on the concept of exterior product , ∧, introduced by

Hermann Gunter Grassmann in 1844

Axioms of Grassman Exterior Algebra:

I α ∧ α = 0

I α ∧ β = − β ∧ α

I (aα) ∧ β = a(α ∧ β).

The axioms are sufficient to establish that:

(Aα) ∧ β = |A|(α ∧ β).

Amenity: Grassman at the age of 53 grew frustrated with the lack of interest in his

mathematical work and turned to Sanskrit studies, writing a widely used dictionary.

Anna Scaglione


What kind of product is dxdy? Is dx ∧ dy

dx

dy

dx dy^

• The product of differentials dxdy behaves like dx ∧ dy and we can

use on it the axioms of Grassman Exterior Algebra

• To complete the description of Cartan’s differential forms:

Axiomatic definition of the d operator

I d(r-form) = (r + 1)-form

I d(dx) = 0 (Poincare Lemma)

• These rules are systematic and the results are simpler to grasp

than the theory of manifolds

Anna Scaglione


The Jacobian recipe

F dA matrix of differentials

F (dA) the exterior product of the independent entries in dA:

I for an arbitrary A, (dA) = ∧i ∧j daij

I if A is diagonal (dA) = ∧idaii

I if A = AT or A is lower triangular (dA) = ∧1≤i≤j≤ndaij

I for Q unitary . . . (not nice)

• Select the arbitrary unique matrix factorization, for Ex. X = AB

• Apply the d operator → dX = dAB + AdB

• Evaluate (dAB + AdB), ∧ product of all the independent

differentials

Warning: This last task requires the description of the group of

matrices by mean of their independent parameters

Anna Scaglione


The Stiefel Manifold

• A unitary Q is described by n2 smooth functions that can be

integrated over nice enough intervals (Stiefel Manifold)

• Clearly, the independent parameters of the Stiefel Manifold are not

the real and imaginary parts of the elements of Q

• n out of the n2 parameters are redundant (in the sense that the

decomposition is unique up to n parameters), hence we can

assume:

(a) The diagonal elements of Q are real

• QQH = I → QdQH = −dQQH

• Under (a) the diagonal elements of QdQH are zero and QdQH is

antisymmetric

(dQ) ≡ (QHdQ) = ∧i>jqHi dqj

Anna Scaglione


The Stiefel Manifoldfactors of

• The uniform p.d.f. in the Stiefel group of orthogonal or unitary

matrices is called Haar distribution

The volume of (QHdQ) integrated over QHQ = I, for Q unitary,

when the diagonal elements of Qm,n are constrained to be real:

V ol(Qm,n) ,∫

QHQ=I

(QHdQ) =(π)(m−1)n−n(n−1)/2

∏n−1i=0 Γ(m− i)

Anna Scaglione


The statistics of A = BHB

• A = BHB with pA(A) and pB(B) the pdfs of the random matrices A and B

• Trick: Use QR and Cholesky decompositions first

B = QR , A = RHR.

with (dR) = ∧i<j(drij)

(dA) = 2nn∏

i=1

(|rii|2)n+1−i(dR) =⇒ pA(A)(dA) = pA(RHR)n∏

i=1

2n(|rii|2

)n+1−i(dR)

(dB) =n∏

i=1

(|rii|2)m+1−i(dR)(dQ) =⇒ pB(B)(dB) = pB(QR)n∏

i=1

(|rii|2)m+1−i

(dR)(dQ)

where (dQ) = (QHdQ) is the element of volume of the Stiefel manifold

Anna Scaglione


Generalized Wishart Density

pA(A) = 2−n|A|m−n

∫pB(Q

√A)(QHdQ)

• When the p.d.f. pB(B) = pB(BHB) then:

I Q and R in the QR decomposition B = QR, are independent

I The p.d.f. of Q has Haar distribution

I The p.d.f. of A is:

pA(A) = 2−n|A|m−npB(A)V ol(Qm,n)

Anna Scaglione


The statistics of the EVD A = BHB

(dA) = (dUΛUH + UdΛUH + UHΛdU )

(dA) ≡ (UHdAU ) = (UHdUΛ−ΛUHdU + dΛ)

=n∏

1≤i<k≤n

(λk − λi)2(dΛ)(UHdU ).

In the general case of A = BHB:

pΛ(Λ) = 2−n

n∏

1≤i<k≤n

(λk − λi)2Ψ(λ1, . . . , λn)

Ψ(λ1, . . . , λn) ,∫

pA(UΛUH)(dU ).

{B}i,j ∼ N (0, σ2) =⇒ Ψ(λ1, . . . , λn) = (∏n

i=1 λi)m−n

e−P

i λiσ2

Anna Scaglione


MIMO frequency selective channel

Let’s use all this machinery!

a1. The noise is AWGN with variance σ2n = 1

a2. {H[l]}∗r,t are spatially uncorrelated circularly symmetric N (0, 1)

(Rayleigh fading) with RH [l1, l2, r1, r2, t1, t2] , E{{H[l1]}∗r1,t1

{H[l2]}r2,t2} = δ(t1 − t2) δ(r1 − r2)RH(l2, l1)

a3. n , min(NT , NR) , m , max(NT , NR)

C = log |I + γHH

H|

H , diag(H[d]) , d , (0, . . . , L),

H[k] is the MIMO transfer function at the kth frequency bin:

H[k] =∑L

l=0 H[l]e−j2π klK

Anna Scaglione


Average Capacity

frequency

transmitantennas

receiveantennas

channel response:one matrix (array manifold)per frequency bin

H[0]

H[N-1]

E{C} =K−1∑

k=0

NT∑

l=1

E {log(1 + γλl[k])}

Under a1, a2, the average Capacity for any (n,m) is:

E{C} =K−1∑

k=0

∫ ∞

0

log(1 + γσ2

H [k]x)µm−n

n (x)dx

with α = m− n (Lαk (x) the Laguerre polynomials):

µαn(x) =

1n

n−1∑

k=0

φαk (x)2 φα

k (x) ,[

k!Γ(k + α + 1)

xαe−x

]1/2

Lαk (x)

Anna Scaglione


Characteristic function of C

ΦC(s) = E{esC} = E

{K−1∏

k=0

|I + γH[k]HH[k]|s}

a3. The number of frequency bins K = Q(L + 1).

Choosing p = (0, Q, . . . , QL), since e−j 2πQ(L+1)

lQd = e−j 2π(L+1)

ld, W L+1 is

unitary

a4. RH(l1, l2) = RH(l2 − l1)

ΦC(s) ≈ γQsn

(L∏

l=0

(σ2[lQ])Qs+m−n(n+1)2 χ1(l)

)n∏

i=1

(Γ(i)Γ(m− n + Qs + i))L+1,

I Outage Capacity through Chernoff bound

Anna Scaglione


Numerical versus Theory plots

0 0.2 0.4 0.6 0.8 1 1.2 1.410

−35

10−30

10−25

10−20

10−15

10−10

10−5

100

105

1010

The plot of characteristic function of C

S

phiC

(S)

Using approximations Using Monte Carlo simK=8,L=7,NR=2,NT=2

0 0.2 0.4 0.6 0.8 1 1.2 1.410

−20

10−15

10−10

10−5

100

105

1010

1015

1020

1025

1030

The plot of characteristic function of C

S

phiC

(S)

Using approximations Using Monte Carlo sim

K=8,L=4,NR=3,NT=2

Anna Scaglione


Conclusion

I The study of Random Matrices has produced several beautiful

results that have immediate application in Communication systems

analysis

Anna Scaglione

Documents

Random matrices and their applications to Multi-Input ...signal.ece.utexas.edu/seminars/dsp_seminars/02spring/UTpresentation.pdf · Random matrices and their applications to Multi-Input