Upload
others
View
32
Download
0
Embed Size (px)
Citation preview
Towards inference for skewed alpha stable Levy processes
Simon Godsill and Tatjana Lemke
Signal Processing and Communications Lab.
University of Cambridge
www-sigproc.eng.cam.ac.uk/~sjg
Overview
Motivation
Continuous-time models
Inference for models with jumps
Inference for alpha-stable Levy processes
Conclusions
Motivation
Traditional tracking applications run in discrete time – discrete time state space models:
– Dynamic models of behaviour:
– Sensor (observation) models:
Hidden state (position/velocity…)
Measurements (range, bearing, …)
It is often more natural to model in continuous time:
– Observations can arrive at arbitrary timings, and out of sequence
– Object manoeuvres are carried out in continuous time without reference to observation times
Real objects undergo rapid and random changes of regime, or `jumps’, occurring at unknown times on the continuous-time scale:
To appear
0 50 100 150 200 25099.5
99.6
99.7
99.8
99.9
100
100.1
100.2
100.3
Variable rate - C=60,
C=2,
T=100, N Particles=400
Can approximate this type of behaviour with jump Markov (or semi-Markov) discrete time systems, allowing a finite set of discrete time switching models (HMMs, etc….)
However, seems more natural to model directly in continuous time, without limiting regimes to a finite set (although can extend models to allow in addition regime switching between discrete states)
Bayesian Monte Carlo computational tools are well suited to inference in these continuous time models
Example: stochastic trend model with Gaussian jumps
0 50 100 150 200 25099.5
99.6
99.7
99.8
99.9
100
100.1
100.2
100.3
Variable rate - C=60,
C=2,
T=100, N Particles=400
Now assume that W(t) is comprised of (scaled) Brownian motion B(t) plus
a pure Gaussian jump processZ(t):
0 20 40 60 80 100 120 140 160 180 200-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
0 20 40 60 80 100 120 140 160 180 20085
90
95
100
105
110
115
See Christensen, Murphy and Godsill, 2012
Observation of the jump process
Now assume discrete-time observations of the continuous-time process, e.g.
Filtering/ smoothing and parameter estimation can now be carried out using Kalman-filter conditional likelihoods:
Fixed Interval Filtering and Smoothing
Particle filter-smoother, See Godsill, Doucet,
West (2004) and Bunch and Godsill (2012)
®-stable Levy processes
Previously modelled jumps as a finite activity process
Perhaps more realistic to model the jumps as an infinite collection of large/ small/tiny jumps occurring in each finite time interval
®-stable Levy processes provide a natural
way to achieve this – justified in many Communications, Signal Processing and Finance applications
Joint work with Tatjana Lemke,
see Lemke and Godsill Proc.
ICASSP 2011, ICASSP 2012
Approaches to inference can be made by scale mixtures of normals for symmetric case, and Euler discretisation – Godsill and Kuruoglu 1998,
Godsill and Yang 2006, Tsionas 1998. See also Buckle (1995)
Our current work focuses on a powerful series expansion for the general asymmetric case:
[Here stated in its simplest form for a random variable with
®<1. A similar form applies for the stochastic integral.]
[Samorodnitsky and Taqqu 1994]
iid random variables
satisfying some simple
moment conditions, e.g.
A Gaussian would do.
Arrival times of a unit rate Poisson process
now has a `physical’ interpretation as the scale of the
ith jump.
[Samorodnitsky and Taqqu 1994]
[See also Barndorff-Nielsen and Shephard (2001,2001)]
Now, taking Wi to be iid N(¹W,¾W2), we have
a Gaussian structure, conditional upon the terms ¡i
This can be readily handled using Bayesian Monte Carlo inference.
As a significant bonus, (¹W,¾W2) can be
analytically marginalised, leaving only ® to be
sampled
The trade-off is that we must also simulate an infinite series of ¡i terms – in practice we
do this by truncating the series and approximating the residual.
Truncation of the Series
In practice cannot compute the entire infinite series:
Instead, truncate at a limit, ¡i<c, and
approximate the residual with a Gaussian matched to first two moments of the residual (analytically computed).
Inference Schemes
Consider e.g. A MCMC inference algorithm for the parameters and the latent variables.
Parameters are ®, ¾W, ¹W, m,V
The challenging part is the latent variables (m,V), one for each data point.
Latent Variables (m,V)
Can sample directly from the prior p(m,V) and use rejection sampling (unbounded envelope, but can fix (condition on) the residual terms and make it bounded).
Not good enough for large values of X (slow in the tails!); instead, look at p(m,V):
ca
Observe that for large m, V is approximately m2
This means that we can force prior samples of m into the tails of the
distribution (®-stable and hence pareto) and perform rejection sampling
with likelihood constrained to the line V=m2
This is a rare-event tail approximation for large X values, but the approximation
appears very good and speeds up sampling dramatically.
Can tune accuracy vs. speed by selecting the switchover point between optimal (slow)
and tail approximation (fast).
State-space models and continuous time
Incorporation into discrete-time state-space models is fairly straightforward – likelihood computation can then be done by a mean- and scale-shifted Kalman filter
Continuous time is also a fairly straightforward extension – also need to sample the Vi
Conclusion
A general framework for inference of ®-stable
distribution parameters, linear state-space models and continuous-time Levy processes
Straightforward computations using conditionally Gaussian models (Shephard (1993 Biometrika), Carter and Kohn (1994 Biometrika)), and particle filters
Currently exploring Particle MCMC methods for parameter estimation
References
S. J. Godsill, J. Vermaak, W. Ng and J.F. Li. Variable rate particle filters for tracking applications
in Proceedings of the IEEE, Special Issue on Large Scale Dynamical Systems, 2007.
S. J. Godsill and J. Vermaak. Variable rate particle filters for tracking applications. In Proc. IEEE Stat. Sig. Proc., Bordeaux, 2005.
S. J. Godsill and J. Vermaak, Models and algorithms for tracking using trans-dimensional sequential Monte Carlo. In Proc. IEEE ICASSP 2004
Barndorff-Nielsen, Ole E. and Neil Shephard (2001) "Normal modified stable processes", Theory of Probability and Mathematical Statistics, 2001, 1-19.
Barndorff-Nielsen, Ole E. and Neil Shephard (2001) "Non-Gaussian Ornstein-Uhlenbeck-based models and some of their uses in financial economics", (with discussion) Journal of the Royal Statistical Society, Series B, 63, 167-241.
Buckle, D. J. (1995), Bayesian inference for stable distributions, JASA, 90:605-613.
S. J. Godsill. Inference in symmetric alpha-stable noise using MCMC and the slice sampler. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, volume VI, pages 3806-3809, 2000. ISBN 0-7803-6296-9.
S. J. Godsill. MCMC and EM-based methods for inference in heavy-tailed processes with alpha-stable innovations. In Proc. IEEE Signal processing workshop on higher-order statistics, June 1999. Caesarea, Israel.[ bib | .ps ]
S. J. Godsill and E. E. Kuruoglu. Bayesian inference for time series with heavy-tailed symmetric α-stable noise processes. In Proc. Applications of heavy tailed distributions in economics, engineering and statistics, June 1999. Washington DC, USA.[ bib | .ps ]
S.J. Godsill and L. Yang. Bayesian inference for continuous-time AR models driven by non-Gaussian lÉvy processes. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, Toulouse, France, May 2006.
Tatjana Lemke, Simon J. Godsill: Enhanced Poisson sum representation for alpha-stable processes. ICASSP 2011: 4100-4103
Tatjana Lemke, Simon J. Godsill: LINEAR GAUSSIAN COMPUTATIONS FOR NEAR-EXACT BAYESIAN MONTE CARLO INFERENCE IN SKEWED ALPHA-STABLE TIME SERIES MODELS, ICASSP 20112