On Minimax Estimation Problems for Periodically Correlated Stochastic ...paper.uscip.us/cms/CMS.2014.1001.pdf · minimax extrapolation problem for stationary processes was formulated

Columbia International Publishing

Contemporary Mathematics and Statistics

(2014) Vol. 2 No. 1 pp. 1-24

doi:10.7726/cms.2014.1001

Review

_____________________________________________________________________________________________________________________________

*Corresponding e-mail: [email protected]

1 Department of Probability Theory, Statistics and Actuarial Mathematics, Taras Shevchenko National

University of Kyiv, Kyiv 01601, Ukraine

2* Department of Probability Theory, Statistics and Actuarial Mathematics, Taras Shevchenko National

University of Kyiv, Kyiv 01601, Ukraine

1

On Minimax Estimation Problems for Periodically

Correlated Stochastic Processes

Iryna Dubovets’ka1 and Mikhail Moklyachuk 2*

Received 7 December 2013; Published online 8 March 2014

© The author(s) 2014. Published with open access at www.uscip.us

Abstract The aim of this article is to overview the problem of mean square optimal estimation of linear functionals

which depend on unknown values of periodically correlated stochastic process. Estimates are based on

observations of this process and noise. These problems are investigated under conditions of spectral

certainty and spectral uncertainty. Formulas for calculating the main characteristics (spectral characteristic,

mean square error) of the optimal linear estimates of the functionals are proposed. The least favorable

spectral densities and the minimax-robust spectral characteristics of optimal estimates of the functionals are

presented for given sets of admissible spectral densities.

Keywords: Periodically Correlated Process; Spectral Characteristic; Mean Square Error; Minimax Estimate;

Minimax-Robust Spectral Characteristic

1. Introduction

Periodically correlated processes are those signals whose statistics vary almost periodically, and

they are present in numerous physical and man-made processes. A comprehensive listing most of

the existing references up to the year 2005 on periodically correlated processes and their

applications was proposed by Serpedin et al. (2005). See also a review by Antoni (2009). For more

details see survey paper by Gardner (1994) and book by Hurd and Miamee (2007). Note, that most

of authors investigate properties of periodically correlated sequences while only few publications

deal with investigation of periodically correlated processes. Note also, that in the literature

periodically correlated processes are named in multiple different ways such as cyclostationary,

periodically nonstationary or cyclic correlated processes.

Iryna Dubovets’ka and Mikhail Moklyachuk / Contemporary Mathematics and Statistics

(2014) Vol. 2 No. 1 pp. 1-24

2

Periodically correlated processes can be defined as stochastic processes with a periodic structure.

In papers by Gladyshev (1961, 1963) investigation of periodically correlated processes was started.

Analysis of properties of correlation function and representations of periodically correlated

processes were presented. Relations between periodically correlated processes and stationary

processes were investigated by Makagon (1999a, 2001). Relations of periodically correlated

sequences with simpler stochastic sequences are proposed by Makagon (1999b, 2011), Makagon

and Miamee (2013), Hurd and Miamee (2007).

Methods of solution of problems of estimation of unknown values of stationary stochastic processes

(extrapolation, interpolation and filtering problems) were developed by Wiener (1966), Yaglom

(1987), Kolmogorov (1992). Estimation problems for stationary vector sequences were

investigated by Rozanov (1967). The proposed methods are based on the assumption that spectral

densities of processes are exactly known. In practice, however, it is impossible to have complete

information on the spectral density in most cases. To solve the problem one finds parametric or

nonparametric estimates of the unknown spectral density or selects a density by other reasoning.

Then the classical estimation method is applied provided that the estimated or selected density is

the true one. This procedure can result in significant increasing of the value of error as Vastola and

Poor (1983) have demonstrated with the help of some examples. This is a reason to search

estimates which are optimal for all densities from a certain class of admissible spectral densities.

These estimates are called minimax since they minimize the maximal value of the error. A survey of

results in minimax (robust) methods of data processing can be found in the paper by Kassam and

Poor (1985). The paper by Ulf Grenander (1957) should be marked as the first one where the

minimax extrapolation problem for stationary processes was formulated and solved. Franke and

Poor (1984), Franke (1984, 1985) investigated the minimax extrapolation and filtering problems

for stationary sequences with the help of convex optimization methods. This approach makes it

possible to find equations that determine the least favorable spectral densities for various classes of

admissible densities. For more details see, for example, books by Kurkin et al. (1990), Moklyachuk

(2008), Moklyachuk and Masyutka (2012). In papers by Moklyachuk (1994–2008) the minimax

approach was applied to extrapolation, interpolation and filtering problems for functionals which

depend on the unknown values of stationary processes and sequences. Methods of solution the

minimax-robust estimation problems for vector-valued stationary sequences and processes were

developed by Moklyachuk and Masyutka (2006-2011). Luz and Moklyachuk (2012–2013)

investigated the minimax estimation problems for linear functionals which depends on unknown

values of stochastic sequence with stationary n th increments. Minimax estimation problems for

linear functionals which depend on the unknown values of periodically correlated sequences and

processes were studied in works by Dubovetska, Masyutka, Moklyachuk (2011 –2013). These

problems are natural generalization of the extrapolation, interpolation and filtering problems for

functionals which depend on the unknown values of stationary processes and sequences.

In this article we consider the problem of optimal linear estimation of functionals ( 1)

0( ) ( )i N

N

TA a t t dt

+= ∫ζ ζ ,

0( ) ( )eA a t t dtζ ζ

∞= ∫ ,

0( ) ( )fA a t t dtζ ζ

∞= −∫

which depend on the unknown values of a periodically correlated process ( )tζ based on

observations of the process ( ) ( )t tζ θ+ at points of time \ [0, ( 1) ]t N T∈ +ℝ (for estimation iNA ζ ), at points of time 0t < (for estimation

eA ζ ), at points of time 0t ≤ (for estimation fA ζ ).


(2014) Vol. 2 No. 1 pp. 1-24

3

Here ( )tθ is an uncorrelated with ( )tζ periodically correlated stochastic process. We propose a

transition procedure from continuous periodically correlated stochastic processes ( ), t tζ ∈ℝ

and ( ), t tθ ∈ℝ to the corresponding infinite dimensional vector-valued stationary sequences

, j jζ ∈ℤ and , j jθ ∈ℤ which allows us to reduce the estimation problems for continuous

periodically correlated stochastic processes to the corresponding problems for stationary vector-

valued sequences.

Formulas for calculating the mean square errors and spectral characteristics of the optimal linear

estimates of the corresponding functionals are proposed in the case of spectral certainty where

spectral densities of generated stationary sequences , j jζ ∈ℤ and , j jθ ∈ℤ are exactly known.

The least favorable spectral densities and the minimax-robust spectral characteristics of the

optimal linear estimates are found in the case of spectral uncertainty where spectral densities are

not exactly known, but concrete classes of admissible densities are given. It is shown, for example,

that one-sided moving average sequence gives the greatest value of the mean square error of the

optimal estimate of the functional eA ζ .

2. Periodically correlated continuous processes and generated

vector-valued stationary sequences Definition 2.1 (Gladyshev, 1963) Mean square continuous stochastic process

2: ( , , )H L F Pζ → = Ωℝ , ( ) 0E tζ = , is called periodically correlated (PC) with period T , if its

correlation function ( , ) ( ) ( )K t s E t s= ζ ζ for all ,t s ∈ℝ and some fixed 0T > is such that

( , ) ( , ).K t s K t T s T= + +

Let ( ), t tζ ∈ℝ and ( ), t tθ ∈ℝ be uncorrelated PC stochastic processes with period T . We

construct the following sequences of stochastic functions

( ) ( ), [0, ), ,j u u jT u T jζ ζ= + ∈ ∈ℤ (1)

( ) ( ), [0, ), .j u u jT u T jθ θ= + ∈ ∈ℤ (2)

Sequences (1) and (2) form 2([0, ); )L T H -valued stationary sequences , j jζ ∈ℤ and , j jθ ∈ℤ ,

respectively, with the correlation functions

( , ) , ( ) ( ) ( ( ) , ) ( ),T T

l j H o oB l j E u lT u jT du K u l j T u du B l jζ ζ ζζ ζ ζ ζ= = + + = + − = −∫ ∫

( , ) , ( ) ( ) ( ( ) , ) ( ),T T

l j H o oB l j E u lT u jT du K u l j T u du B l jθ θ θθ θ θ θ= = + + = + − = −∫ ∫

where ( , ) ( ) ( ),K t s E t sζ ζ ζ= ( , ) ( ) ( )K t s E t sθ θ θ= are correlation functions of PC processes

( )tζ and ( )tθ . Consider in the space 2([0, ); )L T ℝ the following orthonormal basis

2 (1) /21 , 1,2,...,

k ki u T

ke e kT

π − = =ɶ , ,k j jke e δ=ɶ ɶ


(2014) Vol. 2 No. 1 pp. 1-24

4

Making use properties of this basis, stationary sequences , j jζ ∈ℤ and , j jθ ∈ℤ can be

represented in the forms

1,j kj kk

eζ ζ∞

==∑ ɶ

2 (1) /2

0

1, ( ) ,

k kT i u T

kj j k je v e dvT

πζ ζ ζ− −

= = ∫ɶ (3)

1,j kj kk

eθ θ∞

==∑ ɶ

2 (1) /2

0

1, ( ) .

k kT i u T

kj j k je v e dvT

πθ θ θ− −

= = ∫ɶ (4)

We call these sequences , j jζ ∈ℤ , , j jθ ∈ℤ and corresponding to them vector-valued

sequences ( , 1,2,...) , j kj k jΤ= = ∈

ℤζ ζ , ( , 1,2,...) , ,j kj k jΤ= = ∈

ℤθ θ generated (by

( ), t tζ ∈ℝ , ( ), t tθ ∈ℝ , respectively) vector-valued stationary sequences.

Components , 1,2,...kj kζ = and , 1,2,...kj kθ = of generated stationary sequences , j jζ ∈ℤ

and , j jθ ∈ℤ are such that (Kallianpur and Mandrekar, 1971; Moklyachuk, 1981)

0,kjEζ = 2 2

1|| || | | ,j H kjk

E Pζζ ζ∞

== = < ∞∑ ( ) , ,kl nj k nE R l j e eζζ ζ = −

0,kjEθ = 2 2

1|| || | | ,j H kjk

E Pθθ θ∞

== = < ∞∑ ( ) , ,kl nj k nE R l j e eθθ θ = −

where , 1,2,...ke k = is a basis of the space 2ℓ . Correlation functions ( )R jζ and ( )R jθ of

stationary sequences , j jζ ∈ℤ and , j jθ ∈ℤ are correlation operator functions in the space

2ℓ . Correlation operators (0)R Rζ ζ= , (0)R Rθ θ= are kernel operators:

2

1, || || ,k k j Hk

R e e Pζ ζζ∞

== =∑

2

1, || || .k k j Hk

R e e Pθ θθ∞

== =∑

The stationary sequences , j jζ ∈ℤ and , j jθ ∈ℤ have spectral densities

, 1( ) ( ) ,kn k nf fλ λ ∞== , 1( ) ( ) ,kn k ng gλ λ ∞

==

which are positive operator-valued functions in 2ℓ of the variable [ , )λ π π∈ − , if theirs correlation

functions ( )R jζ and ( )R jθ can be represented in the form

1( ) , ( ) , ,

2ij

k n k nR j e e e f e e dπ λ

ζ πλ λ

π −= ∫

1( ) , ( ) , ,

2ij

k n k nR j e e e g e e dπ λ

θ πλ λ

π −= ∫ , 1,2,....k n =

For almost all [ , )λ π π∈ − spectral densities ( ),f λ ( )g λ are kernel operators with the integrable

kernel norms

2

1 1

1( ) , , || || ,

2 k k k k j Hk kf e e d R e e P

π

ζ ζπλ λ ζ

π∞ ∞

= =−= = =∑ ∑∫

2

1 1

1( ) , , || || .

2 k k k k j Hk kg e e d R e e P

π

θ θπλ λ θ

π∞ ∞

= =−= = =∑ ∑∫

We will use representations (3), (4) for finding solutions to the mean square estimation problems

for continuous periodically correlated stochastic processes.


(2014) Vol. 2 No. 1 pp. 1-24

5

3. Hilbert space projection method of estimation of PC

processes 3.1 Interpolation problem

Consider the problem of optimal linear estimation of the functional ( 1)

0( ) ( )i N

N

TA a t t dt

+= ∫ζ ζ

which depends on the unknown values of the mean square continuous PC stochastic process ( )tζ

based on observations of the process ( ) ( )t tζ θ+ at points of time \ [0, ( 1) ]t N T∈ +ℝ . The noise

process ( )tθ is an uncorrelated with ( )tζ PC stochastic process. To be sure that the functional iNA ζ is well defined we will suppose that the function ( ), ,a t t +∈ℝ satisfies the natural necessary

condition ( 1)

0| ( ) | .

N Ta t dt

+< ∞∫

With the help of transformation (1) of the process ( )tζ we can represent the functional iNA ζ in the

form

00 0( ) ( ) ( ) ( ) ,

N TNiN j jj

A a t t dt a u u du=

= =∑∫ ∫ζ ζ ζ

where ( ) ( ),ja u a u jT= + ( ) ( ),j u u jTζ ζ= + [0, ).u T∈ Making use the decomposition (3) of the

generated stationary sequence , j jζ ∈ℤ and solutions of the equation

[ ] [ ]( 1) 2 ( 1) 2 0k nk n− + − =

of two variables ( , )k n , which are given by pairs (1,1), (2 1,2 )l l+ and (2 ,2 1)l l + for 2,3,...,l = the

functional can be represented in the form

0 0( ) ( )

TNiN j jj

A a u u du=

= =∑ ∫ζ ζ

[ ] ( ) [ ] ( )0 1 10

1exp 2 ( 1) 2 exp 2 ( 1) 2

TN k nkj njj k n

a i k u T a i n u T duT

∞ ∞

= = == − − =∑ ∑ ∑∫ π π

[ ] [ ] 0 1 1 0

1exp 2 ( 1) 2 ( 1) 2

TN k nkj kjj k n

a i k n u T duT

∞ ∞

= = == − + − =∑ ∑ ∑ ∫ζ π

0 1 0,

N N

kj kj j jj k ja a

∞ Τ= = =

= =∑ ∑ ∑

ζ ζ

where vectors 1 3 , 2 2 1, 2 ,( , 1,2,...) ( , ,..., , ,...) ,j kj j j j k j k ja a k a a a a aΤ Τ+= = =

, .kj j ka a e= ɶ

Assume that vectors , 0,1,..., ja j N=

satisfy the following conditions

2 2

1|| || , || || | | , 0,1,... .j j kjk

a a a j N∞

=< ∞ = =∑

(5)

It follows from condition (5) that the functional iNA ζ has finite second moment.

Let spectral densities ( )f λ and ( )g λ of the generated stationary sequences , j jζ ∈ℤ and

, j jθ ∈ℤ be such that the minimality condition is satisfied

1[( ( ) ( )) ] .Tr f g dπ

πλ λ λ−

−+ < ∞∫ (6)


(2014) Vol. 2 No. 1 pp. 1-24

6

The minimality condition (6) is necessary and sufficient in order that the error-free estimation of

unknown values of the sequence , j j j+ ∈ℤζ θ is impossible (Rozanov, 1967).

Denote by 2( )L f the Hilbert space of vector–functions 1( ) ( )k kb bλ λ ∞== , which are integrable with

respect to the measure with density ( )f λ

, 1( ) ( ) ( ) ( ) ( ) ( ) .k kn nk n

b f b d b f b d∞Τ

=− −= < ∞∑∫ ∫

π π

π πλ λ λ λ λ λ λ λ

Denote by 2 ( )NL f g− + the subspace of 2( )L f g+ generated by vector-functions ,ijke λδ

\ 0,1,..., ,j N∈ℤ 1,2,...,k = where knδ is the Kronecker symbol: 1kkδ = and 0kn =δ for k n≠ .

Every estimate ˆ iNA ζ of the functional

iNA ζ based on observations of the process ( ) ( )t tζ θ+ at

points \ [0, ( 1) ]t N T∈ +ℝ is characterized by its spectral characteristic 2( ) ( )i Nh e L f g−∈ +λ and

the orthogonal stochastic measure 1( ) ( )k kZ Zζ θ ζ θ+ + ∞=∆ = ∆ of the sequence , j j jζ θ+ ∈ℤ and

has the following form

1ˆ ( )( ( )) ( )( ( )).i i i

N k kkA h e Z d h e Z d

∞Τ + +=− −

= = ∑∫ ∫π πλ ζ θ λ ζ θπ π

ζ λ λ (7)

The mean square error of this estimate ˆ iNA ζ is calculated by the formula

2ˆ( ; , ) | |i iN Nh f g E A A∆ = − =ζ ζ

( )1[ ( ) ( )] ( )[ ( ) ( )] ( ) ( ) ( ) ,

2i i i i i i

N NA e h e f A e h e h e g h e dΤ Τ

−= − − +∫

π λ λ λ λ λ λπ

λ λ λπ

(8)

0( ) .

Ni ijN jj

A e a e=

=∑λ λ

The spectral characteristic ( , )h f g of the optimal linear estimate ˆ iNA ζ for given spectral densities

( ), ( )f gλ λ minimizes the value of the mean square error

2

2

ˆ( )

ˆ( , ) ( ( , ); , ) min ( ; , ) min | | .N i

N

i iN N

Ah L f gf g h f g f g h f g E A A

−∈ +∆ = ∆ = ∆ = −

ζζ ζ (9)

The optimal linear estimate ˆ iNA ζ is a solution of the optimization problem (9). To find the spectral

characteristic ( , )h f g and the mean square error ( , )f g∆ of the optimal linear estimate ˆ iNA ζ we

use the Hilbert space orthogonal projection method proposed by Kolmogorov (1992). According to

the method ˆ iNA ζ is a projection of

iNA ζ on the subspace ( )NH − +ζ θ generated in the space

( )H +ζ θ by values , \ 0,1,..., .j j j N+ ∈ℤζ θ The optimal estimate ˆ iNA ζ is determined by two

conditions:

1) ˆ ( ),i NNA H −∈ +ζ ζ θ 2) ˆ( ) ( ).i i N

N NA A H −− ∈ +ζ ζ ζ θ

The second condition gives the formula for the spectral characteristic ( , )h f g of the optimal

estimate ˆ iNA ζ

( ) 1( , ) ( ) ( ) ( ) [ ( ) ( )]i iN Nh f g A e f C e f gΤ Τ λ Τ λ −= λ − λ + λ =

( ) 1( ) ( ) ( ) ( ) [ ( ) ( )] ,i i iN N NA e A e g C e f gΤ λ Τ λ Τ λ −= − λ + λ + λ (10)


(2014) Vol. 2 No. 1 pp. 1-24

7

0( ) .

Ni ijN jj

C e c e=

=∑λ λ

The first condition leads to the equation for unknown coefficients 0 NN j jc c ==

1 ,N N N Nc B D a−= (11)

where 0 NN j ja a ==

is a vector, block-matrices , 0 ( , ) ,NN N l jB B l j == , 0 ( , ) N

N N l jD D l j == are

determined by elements

1 ( )1( , ) ( ( ) ( )) ,

2i j l

NB l j f g e dπ Τ− − λ

−π = λ + λ λ π ∫

1 ( )1( , ) ( )( ( ) ( )) .

2i j l

ND l j f f g e dπ Τ− − λ

−π = λ λ + λ λ π ∫

Taking into account formula (8) and the derived relations (10), (11), the mean square error of the

optimal estimate ˆ iNA ζ can be calculated by the formula

( , ) ( ( , ); , ) , , ,N N N N N Nf g h f g f g a R a c B c∆ = ∆ = + (12)

where ,⋅ ⋅ is the scalar product in 2ℓ , the block-matrix , 0 ( , ) NN N l jR R l j == is determined by

elements

1 ( )1( , ) ( )( ( ) ( )) ( ) .

2i j l

NR l j f f g g e dπ Τ− − λ

−π = λ λ + λ λ λ π ∫

Thus our results can be summarized in the following statements. For more details see article by

Dubovetska and Moklyachuk (2012c).

Theorem 3.1.1 Let ( ), t tζ ∈ℝ and ( ), t tθ ∈ℝ be uncorrelated PC stochastic processes such that

the generated stationary sequences , j jζ ∈ℤ and , j jθ ∈ℤ have spectral densities ( )f λ and

( )g λ , respectively, which satisfy the minimality condition (6). Let coefficients , 0,1,..., ja j N=

, that

determine the functional iNA ζ , satisfy condition (5).

The spectral characteristic ( , )h f g and the mean square error ( , )f g∆ of the optimal linear

estimate of the functional iNA ζ based on observations of the process ( ) ( )t tζ θ+ at points of time

\ [0, ( 1) ]t N T∈ +ℝ are calculated by formulas (10) and (12). The optimal linear estimate ˆ iNA ζ is

determined by the formula (7).

For the problem of optimal linear estimation of the functional iNA ζ based on observations of the

process ( )tζ without noise we have the following corollary from Theorem 3.1.1.

Corollary 3.1.1 Let ( ), t tζ ∈ℝ be PC stochastic process such that the generated stationary

sequence , j jζ ∈ℤ has the spectral density ( )f λ , which satisfy the minimality condition

1[( ( )) ] .Tr f dπ

πλ λ−

−< ∞∫ (13)

Let coefficients , 0,1,..., ja j N=

, that determine the functional iNA ζ , satisfy condition (5).


(2014) Vol. 2 No. 1 pp. 1-24

8

The spectral characteristic ( )h f and the mean square error ( )f∆ of the optimal linear estimate of

the functional iNA ζ based on observations of the process ( )tζ at points of time \ [0, ( 1) ]t N T∈ +ℝ

are calculated by formulas 1( ) ( ) ( )[ ( )] ,i i

N Nh f A e C e fΤ Τ λ Τ λ −= − λ (14)

( ) , ,N Nf c a∆ = (15)

where 0 NN j jc c ==

, 1 ,N N Nc B a−= block-matrix , 0 ( , ) N

N N l jB B l j == is determined by elements

1 ( )1( , ) ( ( )) .

2i j l

NB l j f e dπ Τ− − λ

−π = λ λ π ∫

The optimal linear estimate ˆ iNA ζ of the functional

iNA ζ is determined by the formula

1ˆ ( )( ( )) ( )( ( )).i i i

N k kkA h e Z d h e Z d

∞Τ=− −

= = ∑∫ ∫π πλ ζ λ ζπ π

ζ λ λ (16)

3.2 Extrapolation problem

Consider the problem of optimal linear estimation of the functional


∞= ∫

that depends on the unknown values of PC stochastic process ( )tζ based on observations of the

process ( ) ( )t tζ θ+ at points of time 0t < . The noise ( )tθ is an uncorrelated with ( )tζ PC

stochastic process. The function ( ), ,a t t +∈ℝ satisfies condition0

| ( ) | .a t dt∞

< ∞∫

Taking into consideration transformation (1) of the process ( )tζ and decomposition (3) of the

generated stationary sequence , ,j jζ ∈ℤ the functional eA ζ can be represented in the form

0 0 1 00 0( ) ( ) ( ) ( ) ,

Tej j kj kj j jj j k j

A a t t dt a u u du a a∞ ∞ ∞ ∞ ∞ Τ

= = = == = = =∑ ∑ ∑ ∑∫ ∫

ζ ζ ζ ζ ζ

where vector 1 3 , 2 2 1, 2 ,( , 1,2,...) ( , ,..., , ,...) .j kj j j j k j k ja a k a a a a aΤ Τ+= = =

Assume that coefficients , 0,1,...ja j =

satisfy conditions

0|| || ,jj

a∞

=< ∞∑

2

0( 1) || || .jj

j a∞

=+ < ∞∑

(17)

It follows from the first condition from (17) that the functional eA ζ has finite second moment. The

second condition provides compactness of operators defined further.

Let spectral densities ( )f λ and ( )g λ of the generated stationary sequences , j jζ ∈ℤ and

, j jθ ∈ℤ be such that the minimality condition (6) is satisfied.

Denote by 2 ( )L f− the subspace of 2( )L f generated by vector–functions ,ij

ke λδ 0,j <

1 ,k kn nδ δ ∞== 1,2,...,k = where knδ is the Kronecker symbol: 1kkδ = and 0kn =δ for k n≠ .

Every linear estimate ˆ eA ζ of the functional eA ζ based on observations of the process ( ) ( )t tζ θ+

at points of time 0t < is determined by spectral characteristic 2( ) ( )ih e L f g−∈ +λ and by formula

(7).


(2014) Vol. 2 No. 1 pp. 1-24

9

The classical Kolmogorov projection method (1992) allows us to find the value of the mean square

error ( , )f g∆ and spectral characteristic ( , )h f g of the optimal linear estimate of the functional eA ζ under the condition that spectral densities ( ), ( )f gλ λ of generated stationary sequences

, j jζ ∈ℤ , , j jθ ∈ℤ are known. Applying the same considerations as in the case of

interpolation problem we can verify validity of the following statements. For more details see

article by Dubovetska and Moklyachuk (2013b).



( )g λ , respectively, which satisfy the minimality condition (6). Let coefficients , 0,1,...ja j =

, that

determine the functional eA ζ , satisfy conditions (17).


estimate of the functional eA ζ based on observations of the process ( ) ( )t tζ θ+ at points of time

0t < are calculated by formulas

( ) 1( , ) ( ) ( ) ( ) [ ( ) ( )]i ih f g A e f C e f gΤ Τ λ Τ λ −= λ − λ + λ =

( ) 1( ) ( ) ( ) ( ) [ ( ) ( )] ,i i iA e A e g C e f gΤ λ Τ λ Τ λ −= − λ + λ + λ (18)

( , ) ( ( , ); , ) , , ,f g h f g f g a Ra c Bc∆ = ∆ = + (19)

where 0

( ) ,i ijjj

A e a e∞

==∑

λ λ

0( ) ,i ij

jjC e c e

∞

==∑

λ λ vector 0 j ja a ∞

==

is given, vector of unknown

coefficients 0 j jc c ∞==

is determined by the equation 1 ,c B Da−= block-matrices , 0 ( , ) ,l jB B l j ∞

==

, 0 ( , ) ,l jD D l j ∞== , 0 ( , ) l jR R l j ∞

== are determined by elements

1 ( )1( , ) ( ( ) ( )) ,

2i j lB l j f g e d

π Τ− − λ

−π = λ + λ λ π ∫

1 ( )1( , ) ( )( ( ) ( )) ,

2i j lD l j f f g e d

π Τ− − λ

−π = λ λ + λ λ π ∫

1 ( )1( , ) ( )( ( ) ( )) ( ) , , 0,1,...

2i j lR l j f f g g e d l j

π Τ− − λ

−π = λ λ + λ λ λ = π ∫

The optimal linear estimate ˆ eA ζ of the functional eA ζ is determined by the formula (7).

For the problem of optimal linear estimation of the functional eA ζ based on observations of the

process ( )tζ without noise we have the following statement, which is a corollary from Theorem

3.2.1.

Corollary 3.2.1 Let ( ), t tζ ∈ℝ be PC stochastic process such that the generated stationary

sequence , j jζ ∈ℤ has spectral density ( )f λ , which satisfy the minimality condition (13). Let

coefficients , 0,1,...ja j =

, that determine the functional eA ζ , satisfy conditions (17).


(2014) Vol. 2 No. 1 pp. 1-24

10

The spectral characteristic ( )h f and the mean square error ( )f∆ of the optimal linear estimate

ˆ eA ζ of the functional eA ζ based on observations of the process ( )tζ at points of time 0t < are

calculated by formulas 1( ) ( ) ( )[ ( )] ,i ih f A e C e fΤ Τ λ Τ λ −= − λ (20)

( ) , ,f c a∆ = (21)

where 0 j jc c ∞==

, 1 ,c B a−= block-matrix , 0 ( , ) l jB B l j ∞

== is determined by elements

1 ( )1( , ) ( ( )) .

2i j lB l j f e d

π Τ− − λ

−π = λ λ π ∫

The optimal linear estimate ˆ eA ζ of the functional eA ζ is determined by the formula (16).

Note that Kolmogorov proposed a method of solving the problem of interpolation of stationary

sequence (i. e. finding spectral characteristic and mean square error of the optimal linear estimate

of one missed observation of the sequence) using the Fourier coefficients of the function 1 f .

Theorem 3.2.1 shows that the Fourier coefficients of some functions from spectral densities can be

used to find spectral characteristics and the mean square error of optimal linear estimates of

functionals of stationary sequences for problems of extrapolation and interpolation based on

observations without noise as well as on observations with noise.

The form of the spectral characteristics and the form of the mean square error of the optimal linear

estimate are convenient for finding the least favourable spectral densities and minimax spectral

characteristics of optimal estimates for the problems of extrapolation and interpolation based on

observations without noise as well as on observations with noise.

To solve the problem of extrapolation of stationary sequences Kolmogorov (see also Kailath (1974),

Rozanov (1967), Wiener (1966), Yaglom (1987)) proposed a method based on factorization of

spectral density. This method is suitable for solving problems of extrapolation based on

observations without noise whereas Theorem 3.2.1 describes the method of solving problem of

extrapolation based on observations with noise.

Let apply the method based on factorization of the spectral density to the problem of estimation of

the functional from observations without noise. For more results see articles by Moklyachuk

(1995,1996) and book by Moklyachuk (2008).

Definition 3.2.1 Denote by ( )H nζ the closed linear subspace of the Hilbert space 2( , , )H L F P= Ω

generated by random variables , 1, .kj k j nζ ≥ ≤ The sequence , j jζ ∈ℤ is called regular if

( ) .nH nζ =∅∩ If ( )

nH n Hζ =∩ then the sequence , j jζ ∈ℤ

is called singular.

The following result allows simplifying the problem of optimal linear estimation of unknown values

of stationary sequence.

Theorem 3.2.2 A stationary sequence , j jζ ∈ℤ admits a unique representation in the form


(2014) Vol. 2 No. 1 pp. 1-24

11

r sj j j= +ζ ζ ζ

where , rj j ∈ℤζ is a regular sequence and , s

j j ∈ℤζ is a singular sequence. Moreover, the

sequences rjζ and snζ are orthogonal for all ,j n ∈ℤ .

Since the unknown values of components of singular stationary sequence has error-free estimate, we

will consider the estimation problem only for regular stationary sequences.

The regular stationary sequence , j jζ ∈ℤ admits the canonical moving average representation of

components (Kallianpur and Mandrekar, 1971; Moklyachuk, 1981)

1

( ) ( ),j M

kj km mu md j u uζ ε

=−∞ == −∑ ∑

(22)

where ( ), 1,..., ,m u m M uε = ∈ℤ are mutually orthogonal sequences in H with orthonormal

values; M is the multiplicity of , ;j jζ ∈ℤ ( ), 1, 2,..., 1,..., , 0,1,...,kmd u k m M u= = = are

matrix-valued sequences such that 2

0 1 1| ( ) | .

M

kmu k md u Pζ

∞ ∞

= = ==∑ ∑ ∑

As a consequence of representation (22) the optimal linear estimate of components of stationary

sequence , j jζ ∈ℤ can be represented in the form

1

1ˆ ( ) ( ).

M

kj km ms mj sd sζ ε−

=−∞ == −∑ ∑

(23)

The spectral density ( )f λ of regular stationary sequence , j jζ ∈ℤ admits the canonical

factorization *( ) ( ) ( ),f P Pλ λ λ=

0( ) ( ) ,iu

uP d u e λλ ∞ −

==∑ (24)

where matrices 1,1,

( ) ( ) m Mkm k

d u d u == ∞= , 0,u ≥ are determined by coefficients of the canonical

representation (22).

Taking into account decompositions (22), (23) and factorization (24) we can verify validity of the

following result.

Theorem 3.2.3 Let ( ), t tζ ∈ℝ be a PC stochastic process such that the generated stationary

sequence , j jζ ∈ℤ has spectral density ( )f λ , which satisfy the minimality condition (13) and

admits the canonical factorization (24). Let coefficients , 0,1,...ja j =

, that determine the functional

eA ζ , satisfy conditions (17).

The spectral characteristic ( )h f and the mean square error ( )f∆ of the optimal linear estimate of

the functional eA ζ based on observations of the process ( )tζ at points of time 0t < are calculated

by formulas

( ) ( ) ( ) ( ) ,i ih f A e S e QΤ Τ λ λ= − λ (25)

2( ) ,f Ad∆ = (26)


(2014) Vol. 2 No. 1 pp. 1-24

12

where 0

( ) ( ) ,i illl

S e Ad eλ λ∞

==∑ ( ) ( ), 0,l jj l

Ad a d j l l∞ Τ

== − ≥∑

the matrix function

1,

1,( ) ( ) k

mk m MQ qλ λ = ∞

== satisfies equation ( ) ( ) ,MQ P Iλ λ = 22

0( ) .ll

Ad Ad∞

==∑ The optimal

linear estimate ˆ eA ζ of the functional eA ζ is determined by formula (16).

Similar reasoning can be applied to find the optimal estimate of the functional ( 1)

0 0 1 00 0( ) ( ) ( ) ( ) .

N T TN N NeN j j kj kj j jj j k j

A a t t dt a u u du a a+ ∞ Τ

= = = == = = =∑ ∑ ∑ ∑∫ ∫

ζ ζ ζ ζ ζ

The following corollary from Theorem 3.2.3 holds true.

Corollary 3.2.2 Let ( ), t tζ ∈ℝ be a PC stochastic process such that the generated stationary

sequence , j jζ ∈ℤ has spectral density ( )f λ , which satisfy the minimality condition (13) and

admits the canonical factorization (24). Let coefficients , 0,1,..., ja j N=

, that determine the

functional eNA ζ , satisfy condition (5). The spectral characteristic ( )Nh f and the mean square error

( )N f∆ of the optimal linear estimate of the functional eNA ζ based on observations of the process

( )tζ at points of time 0t < are calculated by formulas

( ) ( ) ( ) ( ) ,i iN N Nh f A e S e QΤ Τ λ λ= − λ (27)

2 2

0( ) ( ) ,N N l Nl

f A d A d∞

=∆ = =∑ (28)

where 0

( ) ,Ni ij

N jjA e a eλ λ

==∑

0

( ) ( ) ,Ni il

N N llS e A d eλ λ

==∑ ( ) ( ).

N

N l jj lA d a d j lΤ

== −∑

The optimal linear estimate ˆ eNA ζ of the functional

eNA ζ is determined by the formula (16).

For a specified class of PC processes we can calculate the greatest values of the mean square error 2ˆ ˆ( , ) | |e e eA E A Aζ ζ ζ∆ = − of estimate ˆ eA ζ of the functional eA ζ and of the mean square error

2ˆ ˆ( , ) | |e e eN N NA E A A∆ = −ζ ζ ζ of estimate ˆ e

NA ζ of the functional eNA ζ . For proof of the following

results see the article by Dubovetska and Moklyachuk (2013a).

Denote by Λ the set of all linear estimates of the functional eA ζ based on observation of the

process ( )tζ at points of time 0t < . Let Y denotes the class of mean square continuous PC

processes ( )tζ such that ( ) 0E t =ζ and 2| ( ) | .E t P T≤ ζζ The following theorem holds true.

Theorem 3.2.4 Let coefficients , 0,1,..., ja j N=

which determine the functional eNA ζ satisfy

condition (5). The function ˆ( , )eNA∆ ζ has a saddle point on the set × ΛY and the following equality

holds true 2

ˆ ˆˆ ˆmin max ( , ) max min ( , ) · ,

e eN N

e eN N N

A AA A P

∈ ∈∈Λ ∈Λ∆ = ∆ =

Y Yζζ ζ

ζ ζ ν

where 2Nν is the greatest eigenvalue of the self-adjoint compact operator , 0 ( , )N p q

NNQ Q p q == in the

space 2ℓ determined by block-matrices , 1( , ) ( , )NN kn k nQ p q Q p q ∞

== with elements


(2014) Vol. 2 No. 1 pp. 1-24

13

( , )

, ,0( , ) · ,

min N p N qNkn k s p n s qs

Q p q a a− −

+ +==∑

, 1,2, , , 0,1, , .k n p q N= … = …

The least favorable stochastic sequence generated by PC process from the class Y for the optimal

estimate of the functional eNA ζ is a one-sided moving average sequence of order N of the form

( ) ( ),j

j u j Nd j u u

= −= −∑

ζ ε

where 0( ( ))NN pd d p == is the eigenvector, that corresponds to

2Nν , which is constructed from matrices

1,

1,( ) ( ) m M

km kd p d p =

= ∞= and is determined by condition 2 2

0|| || || ( ) ||

N

N pd d p P

== =∑ ζ ,

1( ) ( ) Mm mu u ==ε ε is a vector-valued stationary stochastic sequence with orthogonal values.

Theorem 3.2.5 Let coefficients , 0,1,...ja j =

which determine the functional eA ζ satisfy

conditions (17). The function ˆ( , )eA∆ ζ has a saddle point on the set ×ΛY and the following equality

holds true 2

ˆ ˆˆ ˆmin max ( , ) max min ( , ) · ,

e e

e e

A AA A P

∈ ∈∈Λ ∈Λ∆ = ∆ =

Y Yζζ ζ

ζ ζ ν

where 2ν is the greatest eigenvalue of the self-adjoint compact operator , 0 ( , ) p qQ Q p q =

∞= in the

space 2ℓ determined by block-matrices , 1( , ) ( , )kn k nQ p q Q p q ∞== with elements

, ,0( , ) · ,kn k s p n s qs

Q p q a a∞

+ +==∑

, 1,2, , , 0,1, .k n p q= … = …

The least favorable stochastic sequence generated by PC process from the class Y for the optimal

estimate of the functional eA ζ is an one-sided moving average sequence of the form

( ) ( ),j

j ud j u u

=−∞= −∑

ζ ε

where 0( ( )) pd d p =∞=

is the eigenvector, that corresponds to

2ν , it is constructed from matrices

1,

1,( ) ( ) m M

km kd p d p =

= ∞= and is determined by the condition 2 2

0|| || || ( ) || .

pd d p P

∞

== =∑ ζ

3.3 Filtering problem

Consider the problem of optimal linear estimation of the functional

0( ) ( )fA a t t dt

∞= −∫ζ ζ

which depends on the unknown values of a PC stochastic process ( )tζ based on observations of

the process ( ) ( )t tζ θ+ at points of time 0t ≤ . The function ( ), ,a t t +∈ℝ satisfies the condition

0| ( ) | .a t dt

∞< ∞∫

With the help of transformations (1), (3) of the process ( )tζ we can represent the functional fA ζ

in the form


(2014) Vol. 2 No. 1 pp. 1-24

14

,0 0 1 00 0( ) ( ) ( ) ( ) ,

Tfj j kj k j j jj j k j

A a t t dt a u u du a a∞ ∞ ∞ ∞ ∞ Τ

− − −= = = == − = − = =∑ ∑ ∑ ∑∫ ∫

ζ ζ ζ ζ ζ

where ( ) ( ),ja u a u jT= + ( ) ( ),j u u jTζ ζ− − = − − [0, ),u T∈ and vector

1 3 , 2 2 1, 2 ,( , 1,2,...) ( , ,..., , ,...) .j kj j j j k j k ja a k a a a a aΤ Τ+= = =

Assume that coefficients , 0,1,...ja j =

satisfy condition

0|| || .jj

a∞

=< ∞∑

(29)

Condition (29) guarantees finite second moment of the functional fA ζ .

Let spectral densities ( )f λ and ( )g λ of generated stationary sequences , j jζ ∈ℤ and

, j jθ ∈ℤ be such that minimality condition (6) is satisfied.

Every linear estimate ˆ fA ζ of the functional fA ζ is determined by the spectral characteristic

02( ) ( )ih e L f g−∈ +λ

and by formula (7). Denote by 0

2 ( )L f g− + the subspace of the space

2( )L f g+ generated by vector-functions ,ijke λδ 0,j ≤ 1 ,k kn nδ δ ∞

== 1,2,... .k =

The mean square error of the estimate ˆ fA ζ is calculated by the formula

2ˆ( ; , ) | |f fh f g E A A∆ = − =ζ ζ

( )1[ ( ) ( )] ( )[ ( ) ( )] ( ) ( ) ( ) ,

2i i i i i iA e h e f A e h e h e g h e dΤ Τ

− −−= − − +∫

π λ λ λ λ λ λ

πλ λ λ

π (30)

0( ) .i ij

jjA e a e

∞ −− =

=∑λ λ

The spectral characteristic ( , )h f g of the optimal linear estimate ˆ fA ζ for the given densities

( ), ( )f gλ λ minimizes the value of the mean square error

02

2

ˆ( )

ˆ( , ) ( ( , ); , ) min ( ; , ) min | | .f

f f

Ah L f gf g h f g f g h f g E A A

−∈ +∆ = ∆ = ∆ = −

ζζ ζ (31)

The optimal linear estimate ˆ fA ζ is a solution of the optimization problem (31). The classical

Kolmogorov projection method (1992) allows us to find the spectral characteristic ( , )h f g and the

value of the mean square error ( , )f g∆ of the optimal linear estimate of the functional fA ζ on

condition that the spectral densities ( )f λ and ( )g λ are known. In this case

( ) 1( , ) ( ) ( ) ( ) [ ( ) ( )]i ih f g A e f D e f gΤ Τ λ Τ λ −−= λ − λ + λ =

( ) 1( ) ( ) ( ) ( ) [ ( ) ( )] ,i i iA e A e g D e f gΤ λ Τ λ Τ λ −− −= − λ + λ + λ (32)

1( ) ,i ij

jjD e d e

∞

==∑

λ λ

( , ) ( ( , ); , ) , , ,f g h f g f g a Wa d Ud∆ = ∆ = + (33)

where vector 0 j ja a ∞==

, unknown coefficients 1

1 ,j jd d U Va∞ −== =

block-matrices

, 1 ( , ) ,l jU U j l ∞==

0,1,

( , ) ,l

jV V j l = ∞

= ∞= ɶɶ , 0

( , )l j

W W j l ∞==

ɶ ɶ

ɶɶ are determined by elements


(2014) Vol. 2 No. 1 pp. 1-24

15

1 ( )1( , ) ( ( ) ( )) , , , 1,2,...,

2i l jU j l f g e d l j

π Τ− − λ

−π = λ + λ λ = π ∫

1 ( )1( , ) ( )( ( ) ( )) , 1,2,..., 0,1,...,

2i l jV j l f f g e d j l

π Τ− − + λ

−π = λ λ + λ λ = = π ∫

ɶɶ ɶ

1 ( )1( , ) ( )( ( ) ( )) ( ) , , 0,1,....

2i l jW j l f f g g e d l j

π Τ− − λ

−π = λ λ + λ λ λ = π ∫

ɶ ɶɶ ɶɶ ɶ

Thus our results can be summarized in the following statement.



( )g λ , respectively, which satisfy the minimality condition (6). Let coefficients , 0,1,...ja j =

, that

determine the functional fA ζ , satisfy condition (29).


estimate of the functional fA ζ based on observations of the process ( ) ( )t tζ θ+ at points of time

0t ≤ are calculated by formulas (32) and (33), respectively. The optimal linear estimate of the

functional fA ζ is determined by formula (7).

4. Minimax-robust estimation method The proposed in Section 3 formulas for calculating spectral characteristics and mean square errors

of optimal linear estimates of functionals ,iNA ζ ,eA ζ ,e

NA ζ fA ζ can be used only in the case

where spectral density matrices ( )f λ and ( )g λ of the generated stationary sequences , j jζ ∈ℤ

and , j jθ ∈ℤ are exactly known. In the case where the spectral density matrices ( )f λ and ( )g λ

are not exactly known, but a set f gD D D= × of admissible spectral densities is specified, we find

estimates that minimize the mean square error for all spectral densities from a given class D

simultaneously. Such approach to the estimation problem of functionals of the unknown values of

stochastic processes is called minimax (robust) (Moklyachuk, 2008).

Definition 4.1 For a given class of spectral densities f gD D D= × spectral densities 0( ) ,ff Dλ ∈

0( ) gg Dλ ∈ are called least favorable in D for the optimal linear estimatation of the functional Aζ

if 0 0 0 0 0 0

( , )( , ) ( ( , ); , ) max ( ( , ); , ).

f g Df g h f g f g h f g f g

∈∆ = ∆ = ∆

Definition 4.2 For a given class of spectral densities f gD D D= × the spectral characteristic 0( )h λ

of the optimal linear estimate of the functional Aζ is called minimax-robust if

0 *2( , )

( ) ( ),D f g Dh H L f g

∈∈ = +∩λ

0

( , ) ( , )min max ( ; , ) max ( ; , ).

Dh H f g D f g Dh f g h f g

∈ ∈ ∈∆ = ∆


(2014) Vol. 2 No. 1 pp. 1-24

16

Here *2( )L f g+ denotes the subspace 2 ( )L f g− + in the case of extrapolation problem, 2 ( )NL f g− + in

the case of interpolation problem, and 0

2 ( )L f g− + in the case of filtering problem.

Taking into consideration these definitions and Theorems 3.1.1, 3.2.1, 3.2.3, 3.3.1 we can verify that

the following lemmas hold true.

Lemma 4.1 Spectral densities 0( ) ,ff Dλ ∈

0( ) gg Dλ ∈ , which satisfy the minimality condition (6),

are least favorable in the class f gD D D= × for the optimal linear estimation of the functional iNA ζ if

the Fourier coefficients of matrix functions 0 0 1( ( ) ( )) ,f g −+λ λ

0 0 0 1( )( ( ) ( )) ,f f g −+λ λ λ 0 0 0 1 0( )( ( ) ( )) ( )f f g g−+λ λ λ λ

determine matrices 0 0 0, ,N N NB D R , which give a solution of the extremum problem

( )1 0 0 1 0 0

( , )max , ( ) , , ( ) , .N N N N N N N N N N N N N N N Nf g D

a R a B D a D a a R a B D a D a− −

∈+ = +

The minimax-robust spectral characteristic 0 0 0( , )h h f g= of the optimal linear estimate of the

functional iNA ζ is calculated by formula (10) if the condition

0 0( , ) Dh f g H∈ holds true.

Lemma 4.2 Spectral densities 0( ) ,ff Dλ ∈

0( ) gg Dλ ∈ , which satisfy the minimality condition (6),

are least favorable in the class f gD D D= × for the optimal linear estimation of the functional eA ζ if

the Fourier coefficients of matrix functions 0 0 1( ( ) ( )) ,f g −+λ λ

0 0 0 1( )( ( ) ( )) ,f f g −+λ λ λ 0 0 0 1 0( )( ( ) ( )) ( )f f g g−+λ λ λ λ

determine matrices 0 0 0, ,B D R , which give a solution of the extremum problem

( )1 0 0 1 0 0

( , )max , , , ( ) , .f g D

a Ra B Da Da a R a B D a D a− −

∈+ = +


functional eA ζ is calculated by formula (18) if the condition


Lemma 4.3 Spectral density 0( ) ff D∈λ is least favorable in the class fD for the optimal linear

estimation of the functional eA ζ based on observations of the process ( )tζ for 0t < , if it admits the

canonical factorization

( )( )*0 0 0

0 0( ) ( ) ( ) ,iu iu

u uf d u e d u e

∞ ∞− −= =

= ∑ ∑λ λλ

where 0 0 ( ), 0,1,...d d u u= = is a solution of the conditional extremum problem

2max,Ad → ( )( )*

0 0( ) ( ) ( ) .iu iu

fu uf d u e d u e D

∞ ∞− −= =

= ∈∑ ∑λ λλ

The minimax-robust spectral characteristic 0 0( )h h f= of the optimal linear estimate of the

functional eA ζ is calculated by formula (25) if the condition

0( )fDh f H∈ holds true.


(2014) Vol. 2 No. 1 pp. 1-24

17

Lemma 4.4 Spectral densities 0( ) ff Dλ ∈ and

0( ) gg Dλ ∈ which satisfy the minimality condition

(6), are least favorable in the class D for the optimal estimation of the functional fA ζ , if the Fourier

coefficients of matrix functions 0 0 1( ( ) ( )) ,f g −+λ λ

0 0 0 1( )( ( ) ( )) ,f f g −+λ λ λ 0 0 0 1 0( )( ( ) ( )) ( )f f g g−+λ λ λ λ

determine matrices 0 0 0, ,U V W , which give a solution of the extremum problem

( )1 0 0 1 0 0

( , )max , , , ( ) , .f g D

a Wa U Va Va a W a U V a V a− −

∈+ = + (34)


functional fA ζ is calculated by formula (32) if the condition


Remark 4.1 The least favorable spectral densities 0( ) ff Dλ ∈ ,

0( ) gg Dλ ∈ and the minimax-

robust spectral characteristic 0 0( , ) Dh f g H∈ form a saddle point of the function ( ; , )h f g∆ on the

set DH D× . The saddle point inequalities hold true if 0 0 0( , )h h f g= ,

0 0( , ) Dh f g H∈ and

0 0( , )f g is a solution to the conditional extremum problem

0 0( ( , ); , ) max,h f g f g∆ → ( , ) ,f g D∈ (35)

where the functional 0 0( ( , ); , )h f g f g∆ is defined as

0 0 0 0 0 0 11( ( , ); , ) ( ) ( ) ( ( )) ( ( ) ( )) ( )

2i ih f g f g A e g C e f g f

π Τ λ λ Τ −

−π ∆ = λ + λ + λ λ × π ∫

0 0 1 0 0 0 01( ( ) ( )) ( ) ( ) ( ) ( ) ( ) ( ( ))

2i i i if g g A e C e d A e f C e

π− λ λ Τ λ λ Τ

−π λ + λ λ + λ + λ − × π ∫

0 0 1 0 0 1 0 0( ( ) ( )) ( )( ( ) ( )) ( ) ( ) ( )i if g g f g f A e C e d− − λ λ λ + λ λ λ + λ λ − λ

for the extrapolation problem;

0 0 0 0 0 0 11( ( , ); , ) ( ) ( ) ( ( )) ( ( ) ( )) ( )

2i i

N Nh f g f g A e g C e f g fπ Τ λ λ Τ −

−π ∆ = λ + λ + λ λ × π ∫

0 0 1 0 0 0 01( ( ) ( )) ( ) ( ) ( ) ( ) ( ) ( ( ))

2i i i i

N N N Nf g g A e C e d A e f C eπ− λ λ Τ λ λ Τ

−π λ + λ λ + λ + λ − × π ∫

0 0 1 0 0 1 0 0( ( ) ( )) ( )( ( ) ( )) ( ) ( ) ( )i iN Nf g g f g f A e C e d− − λ λ λ + λ λ λ + λ λ − λ

for the interpolation problem;

0 0 0 0 0 0 11( ( , ); , ) ( ) ( ) ( ( )) ( ( ) ( )) ( )

2i ih f g f g A e g D e f g f

π Τ λ λ Τ −−−π

∆ = λ + λ + λ λ × π ∫

0 0 1 0 0 0 01( ( ) ( )) ( ) ( ) ( ) ( ) ( ) ( ( ))

2i i i if g g A e D e d A e f D e

π− λ λ Τ λ λ Τ− −−π

λ + λ λ + λ + λ − × π ∫

0 0 1 0 0 1 0 0( ( ) ( )) ( )( ( ) ( )) ( ) ( ) ( )i if g g f g f A e D e d− − λ λ−

λ + λ λ λ + λ λ − λ

for the filtering problem.


(2014) Vol. 2 No. 1 pp. 1-24

18

In the case of interpolation (extrapolation) problem for the corresponding functional from

observations of the process ( )tζ without noise the conditional extremum problem (35) can be

rewritten as 0( ( ); ) max,h f f∆ → ,ff D∈ (36)

where the functional 0( ( ); )h f f∆ is defined as

0 0 0 1 0 1 01( ( ); ) ( ( )) ( ( )) ( )( ( )) ( )

2i ih f f C e f f f C e d

π λ Τ − − λ

−π∆ = λ λ λ λ

π ∫

for the extrapolation problem,

0 0 0 1 0 1 01( ( ); ) ( ( )) ( ( )) ( )( ( )) ( )

2i i

N Nh f f C e f f f C e dπ λ Τ − − λ

−π∆ = λ λ λ λ

π ∫

for the interpolation problem.

5. Minimax-robust spectral characteristics for given classes D

5.1 Interpolation problem in the class MD−

Consider the minimax estimation problem for the functional ( 1)

0( ) ( )i N

N

TA a t t dt

+= ∫ζ ζ

based on observations of the process ( )tζ at points of time \ [0, ( 1) ]t N T∈ +ℝ under the

condition that the spectral density ( )f λ of the generated vector stationary sequence , j j ∈ℤζ

belongs to the class

11( ) ( )cos( ) ( ), 0,1,..., ,

2MD f f m d P m m M− −

−

= = =

∫π

πλ λ λ λ

π

where the sequence , 1( ) ( ) , 0,1,..., ,kn k nP m p m m M∞== = is such that

*( ) ( )P m P m= − and the

matrix function ( )M im

m MP m e

=−∑λ is a nonnegative matrix with nonzero determinant. With the help

of Lagrange multipliers method we can find that solution 0( )f λ of the conditional extremum

problem (36) satisfies the relation

( ) ( )( )**0 1 0 0 0 1 0 1 0 1

0 0[( ( )) ] ( ) ( ) [( ( )) ] [( ( )) ] [( ( )) ] ,

M Mi i im imN N m mm m

f C e C e f f e e f− Τ λ λ − Τ − Τ λ λ − Τ= =

λ λ = λ α α λ∑ ∑

where , 0,1,..., ,m m Mα =

are Lagrange multipliers. The last equality holds if

0 0.

N Mij imj mj m

c e eλ λ= =

= α∑ ∑

Consider the following cases: M N≥ and M N< .

Let M N≥ . In this case the Fourier coefficients of the matrix function 0 1( ( ))f −λ determine the

matrix 0 .NB Thus, the extremum problem (36) is degenerate, and Lagrange multipliers

1 ... 0N M+α = = α =

.


(2014) Vol. 2 No. 1 pp. 1-24

19

We take 1 ... 0N M+α = = α =

, find 0,..., Nα α

from the equation 0

0 ,NN NB a=α where

0 0( ,..., )NN

Τα = α α

and come to conclusion that the least favorable density 0( )f λ satisfies the

relation

( ) ( ) ( )11 *

0

0 0( ) ( ) .

M M Mim ij ijj ju M j j

f P m e A e A eλ λ λλ−−

− −=− = =

= =

∑ ∑ ∑ (37)

So 0( )f λ is spectral density of the vector autoregressive stochastic sequence of order M

0.

M

j l j ljA −=

=∑

ζ ε (38)

Let M N< . In this case the matrix NB is determined by the Fourier coefficients of the matrix

function 1( ( ))f −λ . Matrices ( ), 0,..., ,P m m M= are known and matrices ( ), 1,..., ,P m m M N= +

are unknown. The unknown Lagrange multipliers , 0,..., ,m m Mα =

and matrices

( ), 1,..., ,P m m M N= + can be found from the equation

0M

N NB a=α , 0 0( ,..., ,0,...,0) .MM

Τα = α α

If the sequence of matrices ( ), 0,1,..., ,P m m N= form a positive definite matrix function

( )N im

m NP m e

=−∑λ with nonzero determinant, the spectral density

( ) ( )( )11 *

0

0 0( ) ( )

N N Nim ij ijj ju N j j

f P m e A e A e−−

− −=− = =

= =

∑ ∑ ∑λ λ λλ (39)

is least favorable and determines the vector autoregressive stochastic sequence of order N

0.

N

j l j ljA −=

=∑

ζ ε (40)

Thus our results can be summarized in the following statement. For more details see the article by

Dubovetska and Moklyachuk (2012c).

Theorem 5.1.1 Spectral density (37) of the vector autoregressive stochastic sequence (38) of order

M is least favorable in the class MD− for the optimal linear estimation of the functional

iNA ζ if

.M N≥ In the case where M N< and solutions ( ), 1,..., ,P m m M N= + of the equation

0M

N NB a=α with coefficients ( ), 0,1,..., ,P m m M= form a positive definite matrix function

( )N im

m NP m e

=−∑λ with nonzero determinant, spectral density (39) of the vector autoregressive

stochastic sequence (40) of order N is least favorable in .MD−

The minimax-robust spectral characteristic 0( )h f is given by (14).

5.2 Minimax extrapolation in the class 10D

Consider the minimax approach to the problem of estimation of the functional


∞= ∫


(2014) Vol. 2 No. 1 pp. 1-24

20

based on observations of the process ( )tζ at points of time 0t < under the condition that the

spectral density ( )f λ of the generated vector stationary sequence , j j ∈ℤζ admits the

canonical factorization (24) and belongs to the class

10

1( ) ( ) ,

2D f f d P

−

= =

∫π

πλ λ λ

π

where , 1 kn k nP p ∞== is a given nonnegative Hermitian matrix. According to remark 4.1, the least

favorable spectral density in the class 10D gives solution to the problem

0 0 0 0 0 10( ( ); ) ( ( )) ( ) ( ) ( ) ( ) max,i ih f f S e Q f Q S e d f D

π λ Τ λ

−π∆ = λ λ λ λ → ∈∫ . (41)

With the help of Lagrange multipliers method we can find that solution 0( )f λ of the conditional

extremum problem (41) satisfies the relation 0 0 0 * 0[ ( )] ( ) [ ( )] ( ),i iS e S e P Pλ Τ λ Τ= λ α α λ

(42)

where α is the Lagrange multiplier, 0 0

0( ) ( ) ,i il

llS e Ad e

∞

==∑

λ λ

0 0( ) ( ),l jj lAd a d j l

∞ Τ=

= −∑

0,l ≥ 0 0

0( ) ( ) .iu

uP d u e

∞ −=

=∑λλ Relation (42) holds true if the sequence of matrices

0 0 ( ), 0,1,...d d u u= = satisfies the system of equations

0( ) ( ), 0,1,....r p s pp o s

a a d s d r r∞ ∞ Τ Τ

+ += == αα =∑ ∑

(43)

Restrictions of the class 10D lead to the following condition on the sequence

0( ), 0,1,...d u u =

*( ) ( ) .u o

d u d u P∞

==∑ (44)

Then the following theorem holds true.

Theorem 5.2.1 Spectral density

( )( )*0 0 0

0 0( ) ( ) ( )iu iu

u uf d u e d u e

∞ ∞− −= =

= ∑ ∑λ λλ

of one-sided moving average sequence of the form (22) is least favorable in the class 10D for the

optimal linear estimation of the functional eA ζ . The sequence of matrices 0 0 ( ), 0,1,...d d u u= =

satisfies relations (43) and condition (44). The minimax-robust spectral characteristic 0( )h f is

calculated by formula (25).

5.3 Filtering problem in the class 20D D× ε

Consider the minimax estimation problem for the functional

0( ) ( )fA a t t dt

∞= −∫ζ ζ

based on observations of the process ( ) ( )t t+ζ θ at points of time 0t ≤ , under the condition that

spectral densities ( )f λ , ( )g λ of the generated vector stationary sequences , j j ∈ℤζ ,

, j j ∈ℤθ belong to classes


(2014) Vol. 2 No. 1 pp. 1-24

21

20

1( ) | ( ) ,

2D f Tr f d P

−

= =

∫π

ςπλ λ λ

π

1 2

1( ) | ( ) ( ) (1 ) ( ), ( ) ,

2D g Tr g g g Tr g d P

−

= = ⋅ + − ⋅ =

∫π

ε θπλ λ ε λ ε λ λ λ

π

where 1( ) 0g ≥λ is an unknown function, and 2( )g λ is a given function.

With the help Lagrange multipliers method we find that solution 0 0( , )f g of the conditional

extremum problem (35) satisfies relations

( )( ) ( )20 0 0 0 2 0 0( ) ( ) ( ) ( ) ( ) ( ( )) ( ) ( ) ,i i i ig A e D e A e g D e f gΤ Τ− −+ + = +λ λ λ λλ λ α λ λ (45)

( )( ) ( )20 0 0 0 2 0 0( ) ( ) ( ) ( ) ( ) ( ( )) ( ( )) ( ) ( ) ,i i i if A e D e A e f D e f gΤ Τ− −− − = + +λ λ λ λλ λ β ϕ λ λ λ (46)

where 2 2,α β are Lagrange multipliers, the function ( ) 0≤ϕ λ and ( ) 0=ϕ λ if

02( ) (1 ) ( )Tr g g≥ − ⋅λ ε λ .

Theorem 5.3.1 Let spectral densities 0 2

0( ) ,f D∈λ 0( )g D∈ ελ satisfy the minimality condition (6).

Then spectral densities 0 0( ), ( )f gλ λ are least favorable in the class

20D D× ε for the optimal linear

estimation of the functional ,fA ζ if they are determined by relations (45), (46), give solution to

conditional extremum problem (34) and satisfy restrictions which determine the class 20D D× ε . The

minimax-robust spectral characteristic 0 0( , )h f g of the estimate ˆ fA ζ is calculated by the formula

(32). The value of the mean square error 0 0( , )f g∆ is calculated by formula (33).

6. Conclusions In this article we describe methods of solution of the problem of optimal linear estimation of

functionals which depend on unknown values of periodically correlated (PC) processes. We present

a transition procedure from PC processes ( ), t t ∈ℝζ and ( ), t t ∈ℝθ to the corresponding

generated vector stationary sequences , j j ∈ℤζ and , j j ∈ℤθ . Decomposition of stationary

sequences , j j ∈ℤζ and , j j ∈ℤθ with the help of a special basis in Hilbert space allows us to

reduce the estimation problem for PC processes to the corresponding problem for stationary

vector-valued sequences.

The Hilbert space projection method is exploited to finding optimal linear estimates of functionals

,iNA ζ ,eA ζ ,e

NA ζ fA ζ based on observations of the PC process ( ) ( )t tζ θ+ with the PC noise

process ( )tθ . Formulas for calculating mean square errors and spectral characteristics of the

optimal linear estimates of the corresponding functionals are proposed in the case of spectral

certainty. Formulas that determine the greatest value of mean square errors and the minimax

estimates of functionals eA ζ ,

eNA ζ are presented. It is shown that the least favorable sequence for


(2014) Vol. 2 No. 1 pp. 1-24

22

the optimal estimation of eA ζ and eNA ζ is one-sided moving average stationary sequence

generated by PC process from the class Y . The minimax approach to the problem of estimation of

linear functionals ,iNA ζ ,eA ζ ,e

NA ζ fA ζ is analyzed in the case of spectral uncertainty for

concrete classes D of spectral density matrices. Least favorable spectral densities and minimax-

robust spectral characteristics of the optimal estimates of functionals are determined.

References Antoni, J., 2009. Cyclostationarity by examples. Mechanical Systems and Signal Processing 23(4), 987–1036.

Dubovets'ka, I., 2011. Extrapolation problem for functionals from a periodically correlated sequence. Visn.,

Mat. Mekh., Kyiv. Univ. Im. Tarasa Shevchenka 25, 22-26.

Dubovets'ka, I.I. , Masyutka, O.Yu. and Moklyachuk, M.P. , 2012a. Interpolation of periodically correlated

stochastic sequences. Theory Probability and Mathematical Statistics 84, 43-56.

http://dx.doi.org/10.1090/S0094-9000-2012-00862-4

Dubovets'ka, I.I., Moklyachuk, M.P., 2012b. Filtering of linear functionals from periodically correlated

sequences. Theory Probability and Mathematical Statistics 86, 43-55.

Dubovets'ka, I.I., Moklyachuk, M.P., 2012c. The minimax interpolation of periodically correlated processes.

Scientific Bulletin of the Uzhgorod University. Series Mathematics and Informatics 23(2), 51-62.

Dubovets'ka, I.I. , Moklyachuk, M.P. , 2012d. Filtering of periodically correlated processes. Prykladna

Statystyka. Aktuarna ta Finansova Matematyka 2, 149-158.

Dubovets'ka, I.I. , Moklyachuk, M.P. , 2013a. Extrapolation of periodically correlated processes from

observations with noise. Theory Probability and Mathematical Statistics 88, 60-75.

Dubovets'ka, I.I. , Moklyachuk, M.P. , 2013b. Minimax estimation problem for periodically correlated

stochastic processes. Journal of Mathematics and System Science 3(1), 26-30.

Franke, J., 1984.On the robust prediction and interpolation of time series in the presence of correlated noise.

Journal of Time Series Analysis 5, 227-244.

http://dx.doi.org/10.1111/j.1467-9892.1984.tb00389.x

Franke, J., 1985. Minimax robust prediction of discrete time series. Z. Wahrscheinlichkeitstheor. Verw.

Gebiete 68, 337-364. http://dx.doi.org/10.1007/BF00532645

Franke, J., Poor, H.V., 1984. Minimax-robust filtering and finite-length robust predictors, In: Robust and

Nonlinear Time Series Analysis. Lecture Notes in Statistics, Springer-Verlag 26, 87-126.

http://dx.doi.org/10.1007/978-1-4615-7821-5_6

Gardner, W.A., 1994. Cyclostationarity in communications and signal processing. IEEE Press, New

York

Gladyshev, E. G., 1961. Periodically correlated random sequences. Sov. Math., Dokl. 2, 385-388.

Gladyshev, Е.G., 1963. Periodically and almost periodically random processes with continuous time

parameter. Theory Probab. Appl. 8, 173-177.

Grenander, U., 1957. A prediction problem in game theory. Arkiv fuer Matematik 3, 371-379.

Hurd, H.L., Miamee, A., 2007. Periodically correlated random sequences. John Wiley & Sons, Inc., Publication.

353.

Kallianpur, G., Mandrekar, V., 1971. Spectral theory of stationary H-valued processes. J. Multivariate Analysis

1, 1-16.

Kassam, S.A., Poor, H.V., 1985. Robust techniques for signal processing: A survey. Proceedings of the IEEE 73,

433-481. http://dx.doi.org/10.1109/PROC.1985.13167

Kolmogorov, A.N., 1992. Selected works of A. N. Kolmogorov. Vol. II: Probability theory and mathematical

statistics. Ed. by A. N. Shiryayev. Mathematics and Its Applications. Soviet Series. 26. Dordrecht etc.:

Kluwer Academic Publishers, Moskva, 584.


(2014) Vol. 2 No. 1 pp. 1-24

23

Kurkin, O.M., Korobochkin, Yu. V. and Shatalov, S.A., 1990. Minimax information processing. Energoatomizdat,

Moskva, 214.

Luz, M. M., Moklyachuk, M. P., 2012a. Interpolation of functionals of stochastic sequences with stationary

increments. Theory Probability and Mathematical Statistics 87, 94-108.

Luz, M. M., Moklyachuk, M. P., 2012b. Interpolation of functionals of stochastic sequences with stationary

increments for observations with noise. Prykladna Statystyka. Aktuarna Finansova Matematyka 2, 131-

148.

Luz, M. M., Moklyachuk, M. P., 2013. Minimax-robust filtering problem for stochastic sequence with stationary

increments. Theory Probability and Mathematical Statistics 89, 117-131.

Makagon, A., 1999a. Induced stationary process and structure of locally square integrable periodically

correlated processes. Studia Math. 136 (1), 71-86.

Makagon, A., 1999b. Theoretical prediction of periodically correlated sequences. Probability and

Mathematical Statistics 19 (2), 287–322.

Makagon, A., 2001. Characterization of the spectra of periodically correlated processes. Multivariate

Analysis 78, 1-10.

Makagon, A., 2011. Stationary sequences associated with a periodically correlated sequence.

Probability and Mathematical Statistics 31(2), 263–283.

Makagon, A., Miamee, A., 2013. Spectral representation of periodically correlated sequences. Probability and

Mathematical Statistics 33(1), 175–188

Moklyachuk, M., 1981. Estimation of Linear Functionals of Stationary Stochastic Processes and a Two-Person

Zero-Sum Game. Stanford University Technical Report No. 169. 82.

Moklyachuk, M. P., 1994. Stochastic autoregressive sequences and minimax interpolation. Theory Probability

and Mathematical Statistics 48, 95-103.

Moklyachuk, M. P., 1998. Extrapolation of stationary sequences from observations with noise. Theory

Probability and Mathematical Statistics 57, 133-141.

Moklyachuk, M. P., 2000. Robust procedures in time series analysis. Theory Stochastic Processes 6(3-4), 127-

147.

Moklyachuk, M. P., 2001. Game theory and convex optimization methods in robust estimation problems.

Theory Stochastic Processes 7(1-2), 253-264.

Moklyachuk, M. P., 2008. Robust estimations of functionals of stochastic processes. `Kyivskyi Universytet’,

Kyiv, 320.

Moklyachuk, M. P., Luz, M. M., 2013. Robust extrapolation problem for stochastic sequences with stationary

increments. Contemporary Mathematics and Statistics 1(3), 123-150. doi:10.7726/cms.2013.1009

Moklyachuk, M.P., Masyutka, O.Yu., 2006. Interpolation of multidimensional stationary sequences. Theory


http://dx.doi.org/10.1090/S0094-9000-07-00687-4

Moklyachuk, M.P., Masyutka, O.Yu., 2006. Extrapolation of multidimensional stationary processes. Random

Operators Stochastic Equations 14(3), 233-244.

http://dx.doi.org/10.1515/156939706778239819

Moklyachuk, M.P., Masyutka, O.Yu., 2007. On the problem of filtration of vector stationary sequences. Theory


http://dx.doi.org/10.1090/S0094-9000-08-00718-7

Moklyachuk, M.P., Masyutka, O.Yu. ,2008. Minimax prediction problem for multidimensional stationary

stochastic sequences. Theory Stochastic Processes 14(3-4), 89-103.

Moklyachuk, M.P., Masyutka, O.Yu. ,2011. Minimax prediction problem for multidimensional stationary

stochastic processes. Communications in Statistics. Theory and Methods 40, 3700-3710.

DOI:10.1080/03610926.2011.581190

Moklyachuk, M. P., Masyutka, O.Yu., 2012. Minimax-robust estimation technique for stationary stochastic

processes. LAP LAMBERT Academic Publishing, 296.

Pshenichnyi, B.N., 1982. Necessary conditions for an extremum. 2nd ed., Nauka, Moskva, 144.

Rozanov, Yu. A., 1990. Stationary stochastic processes. 2nd rev. ed. Nauka, Moskva, 272.


(2014) Vol. 2 No. 1 pp. 1-24

24

Serpedin, E., Panduru, F., Sari, I., Giannakis, G.B., 2005. Bibliography on cyclostationarity. Signal Processing 85,

2233–2303

Vastola, K. S., Poor, H. V., 1983. An analysis of the effects of spectral uncertainty on Wiener filtering.

Automatica 28, 289-293.

http://dx.doi.org/10.1016/0005-1098(83)90105-X

Wiener, N., 1966. Extrapolation, interpolation and smoothing of stationary time series. With engineering

applications. The M. I. T. Press, Massachusetts Institute of Technology, Cambridge, Mass., 163

Yaglom, A. M., 1987. Correlation theory of stationary and related random functions. Vol. 1: Basic results.

Springer Series in Statistics, Springer-Verlag, New York etc., 526.

Yaglom, A. M., 1987. Correlation theory of stationary and related random functions. Vol. 2: Supplementary

notes and references. Springer Series in Statistics, Springer-Verlag, New York etc., 258.

Documents

On Minimax Estimation Problems for Periodically Correlated Stochastic ...paper.uscip.us/cms/CMS.2014.1001.pdf · minimax extrapolation problem for stationary processes was formulated