Random Processes i

8/13/2019 Random Processes i

http://slidepdf.com/reader/full/random-processes-i 1/35

ECNG 6700 - Stochastic Processes, Detection and

EstimationRandom Processes - Part I

Sean Rocke

September 26th & October 3rd , 2013

ECNG 6700 - Stochastic Processes, Detection and Estimation 1 / 35

http://find/



Outline

1 Probability & Random Variables

2 Random Vectors

3 Random (Stochastic) Processes

4 Stationarity

5

Power Spectral Densities for Real WSS Processes

6 Conclusion


http://find/



Probability & Random Variables

Bounds and Approximations

In many applications where we calculate probabilities we canpotentially run into two problems:

1 We do not know the underlying distributions completely . . . all we

have are sample moments such as E [X ], var (X ) and

higher–order moments E [(X −

µ)k ], k > 2.

2 We know the distributions, but integration in closed form is not

possible (e.g., Gaussian pdf)

Question:

How do we solve this?

We use approximation techniques to establish upper and\or lower

bounds on probabilities. . .


http://find/





At–Home Activity:

Read up on the following inequalities\bounds:

Markov inequality

Tchebycheff (Chebyshev) inequality

Chernoff inequality

Strong Law of Large Numbers (SLLN)

Weak Law of Large Numbers (WLLN)

Central Limit Theorem (CLT)


http://find/





Food for thought:

1 When would you use one approximation technique over another?

2 Which gives a tighter bound: Markov, Tchebycheff or Chernoff?

3 What is the difference between the SLLN and WLLN?

4 Express the CLT in your own words.

5 SLLN, WLLN, and CLT assume IID RVs. What if the RVS are

dependent or not identical?

6 Can you recognize when to use the approximation techniques if

given a problem to solve?

7 Can you apply the appropriate technique(s) when you recognize

that one is necesary?


http://find/

http://goback/



Random Vectors

Linear Algebra Review

At–Home Activity:

Review the following matrix operations:

1 Transpose

2 Matrix sum3 Matrix product

4 Trace of a matrix

5 Norm of a vector

6 Vector inner & outer products7 Block matrices & operations


http://find/



Random Vectors

Random Vectors & Matrices

Random Vector:A vector whose entries are RVs

Random Matrix:

A matrix whose entries are RVs

Important Parameters:

Expectation of a vector\matrix

Correlation matrix for a random vector

Covariance matrix for a random vectorCross–correlation matrix for two random vectors

Cross–covariance matrix for two random vectors


R d V

http://find/



Random Vectors

Random Vectors & Matrices

Example 1:

Write out the correlation, covariance, cross–correlation &cross–covariance matrices for the n–dimensional random vectors

X = [X 1, . . . , X n ]T and Y = [Y 1, . . . , Y n ]

T .


R d (St h ti ) P

http://find/

http://goback/



Random (Stochastic) Processes

Modelling Uncertainty in Random Signals

A random signal has some element of uncertainty, and thus wecan never determine its exact value at any given time.

Can describe signal probabilistically (e.g., in terms of average

properties, or probability that signal exceeds a given value)

Random Process: The probabilistic model used to describe suchrandom signals

Conceptual Definition of Random (Stochastic) Processes:

Mathematical model of an empirical process whose development isgoverned by probability laws . . .

Let’s look at some examples. . .



http://find/




RP Example: Modelling Temperature Anomalies



http://find/




RP Example: Modelling Global Precipitation



http://find/

http://goback/




RP Example: Modelling Stock Prices



http://find/




RP Example: Modelling Annual Rainfall



http://find/




RP Example: Modelling Network Traffic



http://find/



( )

RP Example: Modelling Network Traffic



http://find/



So formally, what is a Random Process?

Definition:

Family of RVs, {X (t ), t ∈ T } defined on a given probability space,S , indexed by the parameter, t , where t varies over the index set,

T Function of two arguments, {X (t , ζ ), t ∈ T , ζ ∈ S}

Questions:

1 For each fixed t = t k , what is X (t k , ζ )?

2 For a fixed sample point, ζ = ζ i , what is X (t , ζ i )?

3 For fixed t = t k & ζ = ζ i , what is X (t k , ζ i )?4 Be sure that you know the definitions of the following terms for

random processes: ensemble, member function, sample function,

realization



http://find/



Random Processes: Classification



http://find/



Random Processes: Classification

Stationarity:

Stationary (Strictly, Wide Sense)

Cyclostationary

Non–stationary

Real vs Complex–valued:

Real–valued bandpass RP - Z (t ) = A(t )cos [2πf c t + θ(t )]

Z (t ) =

{A(t )e j Θ(t )e j 2πf c t

}=

{W (t )e j 2πf c t

}Complex envelope,W (t ) = A(t )cos Θ(t ) + jA(t )sin Θ(t ) = X (t ) + jY (t )



http://find/



Random Processes: Methods of Description

Joint Distribution:F X (t 1),...,X (t n )(x 1, . . . , x n ) = P (X (t 1) ≤ x 1, . . . , X (t n ) ≤ x n )

Analytical Description using RVs:


Average Values:

Mean - µX (t ) = E [X (t )]

Autocorrelation - R XX (t 1, t 2) = E [X ∗(t 1)X (t 2)]

Autocovariance - C XX (t 1, t 2) = R XX (t 1, t 2) − µX ∗(t 1)µX (t 2)

Correlation coefficient - r XX (t 1, t 2) = C XX (t 1,t 2)√ C XX (t 1,t 1)C XX (t 2,t 2)



http://find/



Random Processes: 2 or more RPs

Joint Distribution:F X (t 1),...,X (t n ),Y (t ‘1),...,Y (t ‘m )(x 1, . . . , x n , y 1, . . . , y m )= P (X (t 1) ≤ x 1, . . . , X (t n ) ≤ x n , Y (t ‘1) ≤ y 1, . . . , Y (t ‘m ) ≤ y m )

Analytical Description using RVs:


Average Values:

Autocorrelation - R XY (t 1, t 2) = E [X ∗(t 1)Y (t 2)]

Autocovariance - C XY (t 1, t 2) = R XY (t 1, t 2) − µX ∗(t 1)µY (t 2)

Correlation coefficient - r XY (t 1, t 2) = C XY (t 1,t 2)√ C XX (t 1,t 1)C YY (t 2,t 2)



http://find/



Random Processes: 2 or more RPs

Properties:

For 2 uncorrelated RPs,

C XY (t 1, t 2) = 0, t 1, t 2 ∈ T

For 2 orthogonal RPs,

R XY (t 1, t 2) = 0, t 1, t 2 ∈ T

Independent

P [X (t 1) ≤ x 1, . . . , X (t n ) ≤ x n , Y (t ‘1) ≤ y 1, . . . , Y (t ‘m ) ≤ y m ]

=n

i =1{P [X (t i ) ≤ x i ]} m

j =1{P [Y (t ‘ j ) ≤ y j ]}

for all n , m & t 1, . . . , t n , t ‘1, . . . , t ‘m ∈ T



http://find/



Random Processes: Worked Examples

Example 2:

Let’s use MATLAB to define and investigate a Bernoulli (p ) randomprocess. . .

Example 3:

Consider the amplifier of a radio receiver. Because all amplifiers

internally generate thermal noise, even if the radio is not receiving anysignal, the voltage at the output of the amplifier is not zero but is well

modeled as a Gaussian random variable each time it is measured.

Suppose we measure this voltage once per second and denote the n th

measurement by Z n . Assume that the amplifier gain is 5 and an inputsignal, x (t ) = sin (2πft ), is applied.

Let’s investigate this further in MATLAB . . .



http://find/



Special Random Processes

Markov Processes

Gaussian (Normal) Processes

Independent increments (e.g., Wiener Process, Poisson Process)

You should really just get a better idea of the key characteristics of

these processes!!!


Stationarity

S i S i i (SSS)

http://goforward/

http://find/

http://goback/



Strict–sense Stationarity (SSS)

Definition:

1

The distribution function describing the process is invariant undera translation of time/space

2 For all t 1, . . . , t k , t 1 + τ , . . . , t k + τ ∈ T & all k = 1, 2, . . .,

P [X (t 1) ≤ x 1, . . . , X (t k ) ≤ x k ] = P [X (t 1 + τ ) ≤ x 1, . . . , X (t k + τ ) ≤x k ]

3 If stationary for k ≤ N but not k > N then X (t ) is a N th order

stationary process

SSS Properties:

Constant mean over index: E [X (t )] = µX = constant

Autocorrelation only depends on time difference, not actual time

samples: E [X ∗(t 1)X (t 2)] = R XX (t 2 − t 1)

Jointly SSS:

Joint distributions of X (t ) and Y (t ) are invariant to shifts in time


Stationarity

Wid St ti it (WSS)

http://find/



Wide–sense Stationarity (WSS)

WSS:

Less restrictive form of stationarity than SSSOnly based upon the mean and autocorrelation functions, µX (t )and R XX (t 1, t 2).

WSS Properties:

Constant mean over index: E [X (t )] = µX = constant

Autocorrelation only depends on time difference, not actual time

samples: E [X ∗(t 1)X (t 2)] = R XX (t 2 − t 1) or alternatively,

E [X ∗(t )X (t + τ )] = R XX (τ )

Jointly WSS:

E [X ∗(t )Y (t + τ )] = R XY (τ )

SSS implies WSS, but WSS does not imply SSS


Stationarity

Oth F f St ti it

http://goforward/

http://find/

http://goback/



Other Forms of Stationarity

Asymptotical stationarity:

Distribution of X (t 1 + τ ), . . . , X (t n + τ ) does not depend on τ whenτ is large

Stationary on an interval:

X (t ) is SSS for all τ for which t 1 + τ, . . . , t k + τ lie in an interval

γ ⊂ T Stationary increments:

Increments X (t + τ ) − X (t ) form a stationary process for every τ

Cyclostationarity/Periodic stationarity:

X (t ) is stationary for a constant shift T 0, or integer multiples of T 0

Food for Thought:

When a complete description of X (t ) is unavailable, but some process

samples are available, how would you test for stationarity?



A t l ti F ti (ACF) f WSS P

http://goforward/

http://find/

http://goback/



Autocorrelation Function (ACF) of a WSS Process

Properties:1 Average power, R XX (0) = E [X 2(t )] ≥ 0

2 R XX (τ ) is an even function of τ : R XX (τ ) = R XX (−τ )

3

|R XX (τ )

| ≤ R XX (0)

4 If X (t ) contains a periodic component, then R XX (τ ) will also

5 If limτ →∞ R XX (τ ) = C , then C = µ2X

6 If R XX (T 0) = R XX (0) for some T 0 = 0, then R XX (τ ) is periodic with

period T 07 If R XX (0) < ∞ and R XX (τ ) is continuous at τ = 0, then R XX (τ ) is

continuous for every τ



Cross correlation Function of a WSS Process

http://goforward/

http://find/

http://goback/



Cross–correlation Function of a WSS Process

Properties:

1 R XY (τ ) = R YX (−τ )

2

|R XY (τ )| ≤ R XX (0)R YY (0)

3 |R XY (τ )| ≤ 12 [R XX (0) + R YY (0)]

4 R XY (τ ) = 0 if X (t ) and Y (t ) are orthogonal

5 R XY (τ ) = µX µY if X (t ) and Y (t ) are independent



Power Spectral Density (PSD) of a WSS Process

http://find/



Power Spectral Density (PSD) of a WSS Process

Definition (Wiener-Khinchine Relation):

S XX (f ) = F{R XX (τ )} = ∞−∞ R XX (τ )e − j 2πf τ d τ

F{} is the Fourier transform operator

Autocorrelation can be retrieved using inverse transform on PSD

R XX (τ ) =

F −1

{S XX (f )

}= ∞−∞ S XX (f )e j 2πf τ df

Properties:

1 S XX (f ) is real & nonnegative

2 Average power in X (t ), E [X 2(t )] = R XX (0) = ∞−∞ S XX (f )df

3 For X (t ) real, R XX (τ ) is even and hence S XX (f ) is also even,S XX (f ) = S XX (−f )

4 If X (t ) has periodic impulses, then S XX (f ) will have impulses



Power & Bandwidth

http://goforward/

http://find/

http://goback/



Power & Bandwidth

Power in band [f 1, f 2],

P X [f 1, f 2] = f 2

f 1S XX (f )df +

−f 1−f 2

S XX (f )df = 2 f 2

f 1S XX (f )df



Cross power Spectral Density (CPSD)

http://find/



Cross–power Spectral Density (CPSD)

Definition:

S XY (f ) = F{R XY (τ )} = ∞−∞ R XY (τ )e − j 2πf τ d τ

Cross–correlation can be retrieved using inverse transform on

CPSD

R XY (τ ) = F −1{S XY (f )} = ∞−∞ S XY (f )e j 2πf τ df

Properties:

1 S XY (f ) is generally complex valued

2 S XY (f ) = S ∗YX (f )

3

{S

XY (f

)}is an even function of f and

{S

XY (f

)}is an odd

function of f

4 S XY (f ) = 0 if X (t ) and Y (t ) are orthogonal

5 S XY (f ) = µX µY δ (f ) if X (t ) and Y (t ) are independent



RP Examples

http://find/



RP Examples

Questions:

1 In a communication system, the carrier signal at the receiver is

modeled by X t = cos (2πft + Θ), where Θ ∼ Uniform [−π, π]. Find

the mean function and the correlation function of X t .

HINT: cosAcosB = 12 [cos (A + B ) + cos (A − B )].

2 Determine the stationarity of the above function.

3 What happens if the frequency or amplitude are random, as

opposed to the phase?4 What happens if more than one are random?



RP Examples

http://goforward/

http://find/

http://goback/



RP Examples

Questions:

1 Find the PSD of the RP with autocorrelation function,

R XX (τ ) =

1 − |τ |

T , |τ | < T

0, else

2 Find the PSD and effective bandwidth of the RP with

autocorrelation function, R XX (τ ) = Ae −α|τ |, A, α > 0

3 The PSD of a zero mean Gaussian RP is given by,

S XX (f ) =

1, |f | < 500Hz

0, else

Find R XX (τ ) and show that X (t ) and X (t + 1) are uncorrelated

and hence independent.

Uncorrelated Gaussian RVs are also independent!


Conclusion

Conclusion

http://goforward/

http://find/

http://goback/



Conclusion

We covered:

Concluded Random Variable FundamentalsIntroduction to Random Processes

Recommended Reading:

Kay - Sections 9.1–9.3, 9.8, 11.8, 15.1–15.5, 16.1–16.7, 17.1–17.4,17.6–17.8

Your goals for next class:

Continue working with MATLAB and Simulink

Revise in class exercises based on today’s discussions and askquestions in the next class

Start HW3 and ask questions in the next class

Review notes on RPs Part II in prep for next class


Q & A

Thank You

http://goforward/

http://find/

http://goback/



Thank You

Questions????


http://goforward/

http://find/

http://goback/

Documents

Random Processes i