University of Wollongong University of Wollongong
Research Online Research Online
University of Wollongong Thesis Collection 1954-2016 University of Wollongong Thesis Collections
2016
Reading the Waves: Volatility Analysis and the Hilbert-Huang Transform Reading the Waves: Volatility Analysis and the Hilbert-Huang Transform
Carson Drummond University of Wollongong
Follow this and additional works at: https://ro.uow.edu.au/theses
University of Wollongong University of Wollongong
Copyright Warning Copyright Warning
You may print or download ONE copy of this document for the purpose of your own research or study. The University
does not authorise you to copy, communicate or otherwise make available electronically to any other person any
copyright material contained on this site.
You are reminded of the following: This work is copyright. Apart from any use permitted under the Copyright Act
1968, no part of this work may be reproduced by any process, nor may any other exclusive right be exercised,
without the permission of the author. Copyright owners are entitled to take legal action against persons who infringe
their copyright. A reproduction of material that is protected by copyright may be a copyright infringement. A court
may impose penalties and award damages in relation to offences and infringements relating to copyright material.
Higher penalties may apply, and higher damages may be awarded, for offences and infringements involving the
conversion of material into digital or electronic form.
Unless otherwise indicated, the views expressed in this thesis are those of the author and do not necessarily Unless otherwise indicated, the views expressed in this thesis are those of the author and do not necessarily
represent the views of the University of Wollongong. represent the views of the University of Wollongong.
Recommended Citation Recommended Citation Drummond, Carson, Reading the Waves: Volatility Analysis and the Hilbert-Huang Transform, Doctor of Philosophy thesis, School of Mathematics and Applied Statistics, University of Wollongong, 2016. https://ro.uow.edu.au/theses/4831
Research Online is the open access institutional repository for the University of Wollongong. For further information contact the UOW Library: [email protected]
University of Wollongong
Doctoral Thesis
Reading the Waves: Volatility Analysisand the Hilbert-Huang Transform
Author:
Carson Drummond
Supervisor:
Pam Davy
Co-Supervisor:
Chandra Gulati
A thesis submitted in fulfilment of the requirements
for the degree of Doctor of Philosophy
in the
National Institute for Applied Statistics Research Australia
School of Mathematics and Applied Statistics
August 2016
Declaration of Authorship
I, Carson Drummond, declare that this thesis titled, ’Reading the Waves: Volatility
Analysis and the Hilbert-Huang Transform’ and the work presented in it are my own. I
confirm that:
This work was done wholly or mainly while in candidature for a research degree
at this University.
Where any part of this thesis has previously been submitted for a degree or any
other qualification at this University or any other institution, this has been clearly
stated.
Where I have consulted the published work of others, this is always clearly at-
tributed.
Where I have quoted from the work of others, the source is always given. With
the exception of such quotations, this thesis is entirely my own work.
I have acknowledged all main sources of help.
Where the thesis is based on work done by myself jointly with others, I have made
clear exactly what was done by others and what I have contributed myself.
Signed:
Date:
i
“I don’t pretend we have all the answers. But the questions are certainly worth
thinking about.”
Arthur C. Clarke
Abstract
Reading the Waves: Volatility Analysis and the Hilbert-Huang Transform
by Carson Drummond
Countless processes, both natural and man made are driven by seemingly random pro-
cesses. Measuring the randomness which dominates everything from the movements of
stock market prices to the recordings on a seismograph is the first step towards un-
derstanding it. This thesis focused on measuring the volatility of financial time series,
however the potential applications of the techniques developed herein are not limited to
the financial realm.
The Hilbert-Huang Transform (HHT) is a powerful new tool which is well suited for
the analysis of non-stationary and nonlinear time series. The literature exploring its po-
tential applications to the analysis of financial data is surprisingly sparse considering its
apparent suitability. This thesis developed and tested several techniques for estimating
time series volatility, all of which use the HHT to break down financial data into simple
wave like structures which facilitate analysis.
The estimation techniques developed were tested on low frequency data, namely ten
years worth daily data for the NASDAQ and All Ords Indices. A simulated study to
emulate high frequency data was also carried out to study volatility estimation in the
presence of microstructure noise. Finally, several estimation techniques were used to give
one step ahead predictions on the AUD/USD, GBP/USD and EUR/USD exchanges, a
simulated options market was then set up in which differing predictions compete against
one another.
The HHT based estimators were found to be competitive with the alternatives tested
in the low frequency realm, with the added advantage that the technique is intuitive
and can handle unevenly spaced data with ease. The HHT procedure was used as a low
pass filter in order to sift off market microstructure and effectively measure volatility
when the true price was obfuscated by market frictions. Finally, the construction of a
simulated options market operating on real high frequency foreign exchange data showed
that this filtering approach was also effective when real data was used.
. . .
Acknowledgements
First and foremost I would like to thank my primary supervisor, Pam Davy. Her enthu-
siasm for research into the unknown is contagious and her contributions to this thesis
are many. I would also like to thank my co-supervisor, Chandra Gulati. Chandra’s great
attention to detail and broad knowledge of statistical literature has been invaluable. The
guidance, encouragement and many words of wisdom from both of my supervisors has
kept me going and this thesis is a result of their labors as much as my own.
My thanks also go to Song-Ping Zhu for encouraging me to pursue a PhD and giving
me the freedom to go where my research interests took me.
Lastly, my sincerest thanks go to my parents, Carol and Gary Drummond, for their
unwavering love, understanding and support.
iv
Contents
Declaration of Authorship i
Abstract iii
Acknowledgements iv
Contents v
List of Figures vii
List of Tables ix
Abbreviations x
1 Introduction 1
1.1 Defining and Measuring Volatility . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Low and High Frequency Volatility Dynamics and Measurement . . . . . 5
1.2.1 Low Frequency Volatility Dynamics . . . . . . . . . . . . . . . . . 5
1.2.2 High Frequency Volatility Dynamics . . . . . . . . . . . . . . . . . 8
1.3 A New Tool in the Toolbox . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.4 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2 Background 16
2.1 Volatility Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.1.1 The Moving Average Volatility Model . . . . . . . . . . . . . . . . 16
2.1.2 The Exponentially Weighted Moving Average Model . . . . . . . . 17
2.1.3 The GARCH(1,1) Model . . . . . . . . . . . . . . . . . . . . . . . . 20
2.1.4 Basic Realized Volatility . . . . . . . . . . . . . . . . . . . . . . . . 21
2.1.5 Overnight Returns . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.1.6 Sparse Realized Volatility and the Averaged or Subsampled Real-ized Volatility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.1.7 Two Sample Realized Volatility . . . . . . . . . . . . . . . . . . . . 26
2.2 The Hilbert-Huang Transform . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.1 Empirical Mode Decomposition . . . . . . . . . . . . . . . . . . . 27
2.2.2 The HHT as a Variability Measure . . . . . . . . . . . . . . . . . . 30
v
Contents vi
2.2.3 Monte Carlo Analysis of the EMD Procedure . . . . . . . . . . . . 30
2.2.4 The Hilbert Transform . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.3 Comparison and Evaluation Methods . . . . . . . . . . . . . . . . . . . . . 41
3 Low Frequency Volatility Estimation & the HHT 44
3.1 Volatility Estimation using the HHT . . . . . . . . . . . . . . . . . . . . . 44
3.1.1 Generating a Proxy of the Returns . . . . . . . . . . . . . . . . . . 44
3.1.2 Extracting Volatility from the Returns Proxy . . . . . . . . . . . . 48
3.2 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.2.1 Data and Testing Methodology . . . . . . . . . . . . . . . . . . . . 50
3.2.2 Graphical Volatility Analysis . . . . . . . . . . . . . . . . . . . . . 52
3.2.3 Further Analysis of the HHT Procedure . . . . . . . . . . . . . . . 54
3.2.4 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.3 Chapter Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4 High Frequency Simulated Data and Microstructure Noise 66
4.1 Improved HHT Volatility Estimator . . . . . . . . . . . . . . . . . . . . . 66
4.2 Volatility Comparison Test . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.2.1 Test Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.2.2 Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.3 The Filtered-HHT Volatility Estimator for Microstructure Noise . . . . . 72
4.3.1 Adapting the HHT Estimator to the High Frequency Realm . . . . 73
4.4 Monte Carlo Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.4.1 Spectral Analysis and Hurst Parameter Estimation . . . . . . . . . 78
4.4.2 Results and Comparisons . . . . . . . . . . . . . . . . . . . . . . . 79
4.5 Chapter Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5 Simulated Options Market & Real FX Data 90
5.1 The Simulated Options Market and Clarification of Methods . . . . . . . 91
5.1.1 Simulated Option Market Construction . . . . . . . . . . . . . . . 91
5.1.2 Data Specification and Clarification of Methods . . . . . . . . . . . 93
5.1.3 Modified HHT Estimator With Market Microstructure . . . . . . . 95
5.1.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.2 Chapter Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
6 Discussion & Conclusion 112
6.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.3 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
Bibliography 119
List of Figures
2.1 All Ords Index prices from 1st Jan 2000 to 31st Dec 2004 (bold) and asimple MA volatility estimate with a window length of 50 days (dashed). 18
2.2 EWMA volatility estimates with decay rates λ = 0.94 & λ = 0.7 for theAll Ords data set covering 1st Jan 2000 - 31st Dec 2004. . . . . . . . . . . 19
2.3 Super imposed sinusoids (solid blue), the local maxima and connectingspline (red circle and dot-dash line respectively) and the local minimaand connecting spline (green circle and dashed line respectively). . . . . . 29
2.4 The first two IMFs of S(t). . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.5 IMF number vs log2 IMF period for GWN. . . . . . . . . . . . . . . . . . 31
2.6 Log mean period vs log2 variance for GWN (top) & IMF number vs log2
IMF variance for GWN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.7 Power Spectra of 10 IMFs on GWN. . . . . . . . . . . . . . . . . . . . . . 34
2.8 Rescaled Power Spectra of 10 IMFs on GWN. . . . . . . . . . . . . . . . 35
2.9 Power Spectra of 10 IMFs on fGn for three values of H. . . . . . . . . . . 36
2.10 Log period vs log variance for 3 values of H (top). IMF number vs logvariance for 3 values of H (bottom). . . . . . . . . . . . . . . . . . . . . . 38
2.11 Instantaneous amplitude (left) and frequency (right) given by a Hilberttransform of the IMFs displayed in Figure 2.4. . . . . . . . . . . . . . . . 41
3.1 Frequency vs K statistic of a K-S test during a typical search for a returnproxy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.2 Actual returns (top left). Proxy to the log returns P (t) (top right). QQplot of the the proxy P (t) against the actual returns (bottom left). CDFplot of the proxy P (t) against the actual returns (bottom right). . . . . . 49
3.3 HHT based and realized volatility estimates for the NASDAQ data set,1st Jan 2000 - 31st Dec 2004 (top). Corresponding GARCH(1,1) andrealized volatility estimates (middle). Corresponding EWMA and realizedvolatility estimates (bottom). . . . . . . . . . . . . . . . . . . . . . . . . 53
3.4 NASDAQ Index price for 1st Jan 2000 - 31st Dec 2001 and a measure ofthe local average eM(t). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.5 The high frequency components which form the proxy (top). The remain-ing components which are a form of local mean (bottom). . . . . . . . . 56
3.6 The ‘variability’ measure proposed by Huang applied to the NASDAQdata set. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.7 Amplitudes of the first 5 IMFs scaled by the data for 10 years of theNASDAQ Index data set. . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.8 Frequency information given by the Hilbert transform of the first 6 IMFsfor two years of the NASDAQ Index data set. . . . . . . . . . . . . . . . . 58
vii
List of Figures viii
3.9 First six IMFs for the log of the NASDAQ Index from 1st Jan 2000 - 31st
Dec 2001. The average period of each IMF is given in calendar days asopposed to trading days. . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.10 QQ plots of the volatility distribution for the HHT method (top), EWMA(middle) and GARCH (bottom) for the NASDAQ Index covering theperiod 1st Jan 2000 - 31st Dec 2004. . . . . . . . . . . . . . . . . . . . . . 60
3.11 CDF plots of volatility distribution for the HHT method (top), EWMA(middle) and GARCH (bottom) for the NASDAQ Index covering 1st Jan2000 - 31st Dec 2004. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.1 EWMA Response profiles for three values of λ. . . . . . . . . . . . . . . . 71
4.2 Response profiles for HHT estimator & EWMA with λ = 0.67 . . . . . . . 72
4.3 SNR vs response time for EWMA with 0.1 ≤ λ ≤ 0.98 (curve) and theHHT volatility estimate (circle). . . . . . . . . . . . . . . . . . . . . . . . 73
4.4 Volatility estimates for various sample rates across different levels of fil-tration. The ‘o’ symbol denotes the estimate provided by Equation (4.13).. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.5 Spectral analysis of IMFs from GARCH, Affine and Log volatility models.Top row: no microstructure noise. Middle row: 0.1% noise. Bottom row:0.5% noise. Note that the vertical axis is in terms of the Power DensitySpectra (PDS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.6 Volatility signature plot at 0.1% noise. M1 M2 M3 refers to the threevolatility models, GARCH, Two-Factor Affine and Log-Normal diffusionrespectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.7 Volatility vs Time plots for the GARCH (top), Two-Factor Affine (middle)and Log-Normal Diffusion (bottom) models at 0.1% noise. Each plotshows model volatility, the Adjusted-TSRV estimate and the Filtered-HHT volatility estimate. . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.8 Volatility vs Time plots for the GARCH (top), Two-Factor Affine (middle)and Log-Normal Diffusion (bottom) models at 0.5% noise. Each plotshows model volatility, the Adjusted-TSRV estimate and the Filtered-HHT volatility estimate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.1 Volatility estimates with sample intervals of ∆t(1, . . . , 8) = [0.2, 0.25, 0.5, 1, 1.5, 2, 3, 4],calculated across different levels of filtration using FX data. The ‘o’ sym-bol denotes the estimate provided by the procedure outlined in this chap-ter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.2 Instantaneous or spot volatility estimates for 24 hours of high frequencyAUD/USD data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.3 The ‘frictionless’ price (red), as extracted from 24 hours of high frequencyAUD/USD data (blue). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
List of Tables
3.1 Errors for various volatility estimators for the All Ords Index coveringthe period 1st Jan 2000 - 31st Dec 2004 . . . . . . . . . . . . . . . . . . . 63
3.2 Errors for various volatility estimators for the NASDAQ Index coveringthe 1st Jan 2000 - 31st Dec 2004 . . . . . . . . . . . . . . . . . . . . . . . 63
3.3 Errors for various volatility estimators for the All Ords Index coveringthe 1st Jan 2005 - 31st Dec 2009 . . . . . . . . . . . . . . . . . . . . . . . 63
3.4 Errors for various volatility estimators for the NASDAQ Index coveringthe 1st Jan 2005 - 31st Dec 2009 . . . . . . . . . . . . . . . . . . . . . . . 63
4.1 Hurst exponent estimates from the EMD procedure and their associatedstandard deviations for the three models at three noise levels. . . . . . . . 79
4.2 Sampling rate and equivalent sampling intervals in seconds. . . . . . . . . 81
4.3 Mean error analysis at γ = 0.1% noise. . . . . . . . . . . . . . . . . . . . . 83
4.4 Mean square error analysis at γ = 0.1% noise. . . . . . . . . . . . . . . . . 83
4.5 Mean absolute error analysis at γ = 0.1% noise. . . . . . . . . . . . . . . . 83
4.6 Mean error analysis at γ = 0.5% noise. . . . . . . . . . . . . . . . . . . . . 84
4.7 Mean square error analysis at γ = 0.5% noise. . . . . . . . . . . . . . . . . 84
4.8 Mean absolute error analysis at γ = 0.5% noise. . . . . . . . . . . . . . . . 84
5.1 Pairwise competitive trading results for AUD/USD Straddles . . . . . . . 103
5.2 Pairwise competitive trading results for EUR/USD Straddles . . . . . . . 103
5.3 Pairwise competitive trading results for GBP/USD Straddles . . . . . . . 104
5.4 Fair price trading results for AUD/USD Straddles . . . . . . . . . . . . . 104
5.5 Fair price trading results for EUR/USD Straddles . . . . . . . . . . . . . 105
5.6 Fair price trading results for GBP/USD Straddles . . . . . . . . . . . . . 105
5.7 AUD/USD Competitive pairwise trading results in full . . . . . . . . . . . 109
5.8 EUR/USD Competitive pairwise trading results in full . . . . . . . . . . . 109
5.9 GBP/USD Competitive pairwise trading results in full . . . . . . . . . . . 110
5.10 AUD/USD Fair price trading results in full . . . . . . . . . . . . . . . . . 110
5.11 EUR/USD Fair price trading results in full . . . . . . . . . . . . . . . . . 111
5.12 GBP/USD Fair price trading results in full . . . . . . . . . . . . . . . . . 111
ix
Abbreviations
All Ords All Ordinaries
ARMA AutoRegressive Moving Average
DST Discrete Sine Transform
DHT Discrete Hilbert Transform
EMD Empirical Mode Decomposition
EWMA Exponentially Weighted Moving Average
fBm fractional Brownian motion
FFT Fast Fourier Transform
fGn fractional Gaussian Noise
GARCH Generalised AutoRegressive Conditional Heteroskedasticity
GFC Global Financial Crisis
GWN Gaussian White Noise
HHT Hilbert-Huang Transform
IID Independent and Identically Distributed
IMF Intrinsic Mode Function
IV Integrated Volatility
MACD Moving Average Convergence Divergence
MAE Mean Absolute Error
MAPE Mean Absolute Percentage Error
ME Mean Error
MSE Mean Square Error
NASDAQ National Association of Securities Dealers Automated Quotations
PDS Power Density Spectra
PMF Proto Mode Function
RV Realized Volatility
SIRCA Securities Industry Research Centre of Asia-Pacific SNR Signal to Noise Ratio
TSRV Two Scale Realized Volatility
VIX CBOE Volatility Index
x
To My Parents
xi
Chapter 1
Introduction
Our lives are governed by random processes, whether it’s the stock market, weather pat-
terns or quantum behaviour we are surrounded by uncertainty at all times. This thesis
seeks to shed some light on the nature of randomness, with a focus on financial time
series volatility, through the lens of a powerful new tool, the Hilbert-Huang Transform
(HHT) developed by Huang et al. [1].
A system contains randomness if not all parts of it are fully deterministic, i.e. ex-
actly how a random process will behave can not be fully described in advance. Not fully
described —luckily this still leaves much room for some particularly useful descriptions.
The descriptive which most concerns the topic of this thesis is the volatility, best thought
of as the magnitude of the uncertainty. At this point it is important to distinguish be-
tween uncertainty and risk. While risk may be more concerned with a negative outcome,
i.e. a large loss made on an investment, the volatility is akin to the uncertainty and is
more closely related to the magnitude of the movement, not its direction. A measure
of risk that focuses more closely on negative returns was proposed by Markowitz [2]
however it has not been widely adopted. Although conceptually distinct, volatility and
risk are inextricably linked, thus to manage risk one must also understand volatility.
Following the stock market crash of 1987 the first Basel Accord made understanding
volatility a matter of policy for many financial institutions which were henceforth re-
quired to manage their exposure to risk in terms of volatility. Unfortunately, recent
1
Chapter 1. Introduction 2
events in stock markets around the world show that even the experts still have a thing
or two to learn about the nature of volatility.
One reason why there is so much interest in understanding volatility is because the
volatility of a stochastic system impedes on our ability to give accurate forecasts. The
more volatile a system, the more rapidly a forecast becomes addled by uncertainty. It
quickly becomes apparent that the confidence placed in any such forecast is closely linked
to an accurate measure of volatility. This is true for all manner of stochastic systems,
it is why long term weather forecasts are notoriously unreliable and economists’ predic-
tions are often taken as opinions rather than facts.
The type of stochastic system that is investigated over the course of this thesis is a
time series, this is simply a sequence of points or data which come in some order. Ex-
amples of a time series include the temperature of your city measured daily, the position
of a seismograph needle during an earthquake or the current and voltage output of your
computer’s power supply over the course of a few seconds. Thus, any useful insights
into the nature of time series volatility will not be restricted to financial mathematics,
but to a broad swath of scientific disciplines.
To examine volatility in a practical setting, a focus is placed on financial time series.
This is partly because there is a wealth of available data in the area, but also because it is
a place where forecasts on stochastic systems are tested and refined on a daily basis and
any contributions to this field are likely to have a maximal impact. Volatility estimates
are also very important in option pricing, where the price of the financial instrument
is directly affected by the perceived variability of an underlying stock. Nowhere is this
more evident than the VIX Index, which trades in perceived uncertainty itself.
Methods for determining local volatility fall into two categories, time series meth-
ods which base their estimate on the underlying historical data and implied volatility
methods, which calculate the theoretical volatility needed to account for the price of an
Chapter 1. Introduction 3
option under a particular choice of pricing model. The time series approach to volatility
modelling assumes that some important characteristics of the volatility can be extracted
from the underlying historical data and as Poon and Granger [3] put it, these charac-
teristics can provide some insight into forward or future volatility. Thus any accurate
means of measuring past volatility also has a place in volatility forecasting.
1.1 Defining and Measuring Volatility
Firstly, it is important to clarify that our objective is to measure the volatility of the
log returns of a financial time series. It is standard practice in much of the financial
literature to use the log returns and then treat them as normally distributed. The log
return at time t is given by:
rt = log(Pt)− log(Pt−1), (1.1)
where Pt is the price or value of some underlying asset at time t. For notational con-
venience, log(Pt) is denoted pt henceforth, so the log returns can also be expressed as:
rt = pt − pt−1.
If one is to assume that the volatility is constant over time, estimating the volatility
from N return observations would be a simple matter of finding the Sample Variance
SV σ2, which is given by:
SV σ2 =1
N − 1
N∑t=1
(rt − r)2 , (1.2)
where r is the arithmetic mean of the returns. Note that r is assumed to be zero for many
financial time series sampled daily or at higher frequencies. However, the assumption
of constant volatility is rarely justified and there has been much work towards finding
an accurate and meaningful measure of a time varying variance σ2t . It is worth noting
at this point that σ2 only represents the second moment of the data and that it is of
Chapter 1. Introduction 4
limited usefulness without considering the probability density function that it is asso-
ciated with. The assumption of a log-normal return distribution is also questioned in
much of the literature as the log returns are often observed to be fat tailed or leptokurtic.
To understand the continuous time analogue of Equation (1.2) we let the log of the
frictionless price follow
dpt = σtdW(1)t , (1.3)
where W(1)t is a standard Brownian motion, the superscript is merely to distinguish this
Brownian motion from those used to drive a stochastic volatility model described in
Chapter 4. The instantaneous or spot volatility, denoted σt, has a continuous sample
path. It is assumed that the σt and W(1)t processes are uncorrelated. One period (often
one trading day for convenience) of Integrated Volatility (IV) is defined as:
IVt ≡∫ t
t−1σ2τdτ. (1.4)
This definition of IVt is sometimes referred to as the quadratic return variation. If
the discretely sampled returns are serially uncorrelated and the sample path for σt is
continuous, it follows from the theory of quadratic variation described by Merton [4] &
Karatzas and Shreve [5], that
lim1/h→∞
∫ t
t−1σ2τdτ −
∑i=1,...,1/h
r2t−1+ih(h)
= 0, (1.5)
where h is the increment size between samples, i.e. a period of length 1 has 1/h many
return observations. The key result from this is that a variable volatility is theoretically
observable at a time t provided that the sampling frequency is high enough. Andersen
et al. [6, 7] have shown that even though the spot volatility is not a directly observable
variable, estimations of the integrated volatility can be closely estimated by the easily
calculable Realized Volatility (RV), which is covered in more detail in Chapter 2. Thus,
the goal of much of this thesis and much literature is to reconstruct the path of σ2t for a
better understanding of market dynamics, to improve volatility forecasts, manage risk
Chapter 1. Introduction 5
more effectively and price options with higher accuracy.
1.2 Low and High Frequency Volatility Dynamics and Mea-
surement
This thesis can roughly be split into two sections, volatility estimation in the low fre-
quency domain and volatility estimation in the high frequency domain. The reason for
this division is that the problems faced at each time scale differ, as do the approaches
needed to overcome them. The low frequency domain is largely restricted to data col-
lected on the daily scale, like the closing price and opening price of an index, stock or
currency etc. The high frequency realm on the other hand involves orders of magnitude
more data and usually includes samples anywhere from one every fifteen minutes to only
milliseconds apart.
1.2.1 Low Frequency Volatility Dynamics
There is an escalating struggle between those that discover interesting new market dy-
namics and those that try to model them. Like lepidopterists collecting butterflies,
financial practitioners delight in naming new market phenomena and challenging aca-
demics to accurately describe them before pinning them down in a useful model. The
first to understand any new market dynamic will often have an edge over their compe-
tition, this has led to considerable research into all manner of market dynamics and an
arms race between algorithmic trading strategies that is evolving daily.
As previously stated, volatility appears to change over time in financial markets,
rendering the standard deviation of past returns of limited use. One of the simplest
tools to overcome this is simply to take the standard deviation over a shorter period of
time and trade off an increase in sample error (due to a smaller sample size) against an
Chapter 1. Introduction 6
increase in contemporariness. This approach yields the simple moving average estimate
of volatility described in Chapter 2.
Many financial returns series also appear to have the property of long memories which
results in the autocorrelations of variances staying positive for a long time, i.e. shocks
in the volatility process tend to persist. Methods to deal with this phenomenon include
the popular Exponentially Weighted Moving Average (EWMA) model (see Chou et al.
[8], Ding and Meade [9], Wilmott [10]) and the more general form of Auto Regressive
Conditional Heteroskedasticity (ARCH) models, first described by Engle [11].
The phenomenon of volatility clustering, described by Mandelbrot [12] as “large
changes tend to be followed by large changes, of either sign, and small changes tend to
be followed by small changes” can be described by ARCH models with large lags but
is more parsimoniously described by the Generalized Auto Regressive Conditional Het-
eroskedasticity or GARCH model, see Bollerslev [13] for more on the origins of GARCH.
Another advantage of the GARCH model is that it easily extends to accommodate even
more interesting phenomena.
The leverage effect is a phenomenon described by Black [14] as the negative corre-
lation that exists between recent returns and volatility. Typically, increases in asset
prices lead to decreasing levels of volatility and vice versa. The effect is also usually
asymmetric, i.e. a drop in an asset price would lead to larger change in volatility than
a corresponding increase in asset price. Models built to take this into account include
the GJR-GARCH model by Glosten et al. [15] and the EGARCH model by Nelson [16].
There is also the ARCH-in-Mean type modification by Engle et al. [17] to take into
account a similar phenomenon known as the volatility feedback effect. Note that the
volatility feedback effect leads to the same observed correlation between asset prices and
volatility as the leverage effect, however as Ait-Sahalia et al. [18] put it, “[the] effect is
consistent with the same correlation but reverses the causality: increases in volatility
lead to future negative returns”. Indeed the catalogue of volatility dynamics and their
Chapter 1. Introduction 7
corresponding ARCH / GARCH formulations seem to be endless, for more information
see two excellent review articles on the subject by Poon and Granger [3] & Andersen
et al. [19].
For financial forecasters, one aspect that must be considered is not just the accuracy
of a volatility estimate but also its time scale, what might be an accurate account of daily
volatility levels might be of less use to a long term forecaster than a less accurate record
of monthly volatility estimates. This leads to an interesting property of financial time
series; what is considered noise and what is considered a genuine movement in volatility is
often just a matter of scale. This phenomenon is caused by volatility structures not being
preserved after intertemporal aggregation, i.e. volatility tends to lose its self similarity
when viewed at different time scales. This property’s effect on volatility forecasts using
daily vs monthly data was described by Figlewski [20] however it is a more dominant
phenomenon when dealing with high frequency returns and will be revisited in this
context during Chapters 4 & 5. This raises a perplexing question; at any particular
scale, which changes in volatility can be described as noise and which are legitimate
changes in variance?
While there are techniques available to determine volatility in the low frequency
realm, none of them are perfect. The main drawback of many existing and popular
techniques is that assumptions made on the underlying behaviour of the time series may
introduce artefacts which can artificially inflate or retard volatility estimates. Further-
more, many of the models described so far need large amounts of in-sample data for the
purposes of parameter fitting. There is room for more non-parametric methods which
require no lead in data for parameter fitting and make no assumptions on the underlying
process.
To address the points made above, one of the aims of this thesis is to construct a
parameter free volatility estimate which is less susceptible to the kinds of artefacts which
can result in misleading volatility estimates.
Chapter 1. Introduction 8
1.2.2 High Frequency Volatility Dynamics
High frequency volatility estimation is also an increasingly interesting topic for re-
searchers and of growing practical importance for market participants. Recent increases
in the availability of high frequency financial tick data has inexorably led to the high
frequency trading strategies that take advantage of it. There is an inherent risk in most
trading strategies, even if the exposure is for a matter of seconds. Some of the trading
strategies which expose themselves to short term risk include asset pricing, statistical ar-
bitrage, market making, trading signal generation and a growing plethora of autonomous
trading strategies. Of paramount importance to the successful implementation of these
strategies is a very recent and reliable measure of volatility.
Unfortunately in the high frequency realm, measuring volatility doesn’t necessar-
ily become easier as the frequency of the available data increases. For many kinds of
financial markets, e.g. stock indices and Foreign eXchange (FX) rates, as the sample fre-
quency increases the nature of the random process appears to change. This breakdown
of self similarity is due to price movements after long time intervals being driven by a
perceived change in the true price (or frictionless price) of the asset, while price move-
ments after short intervals are dominated by microstructure effects (market frictions).
These microstructure effects, or market frictions include:
• price discretisation; e.g. when asset prices are only being traded to the nearest
cent,
• bid ask bounce; anyone wanting to initiate a buy or sell order of an asset must
cross the ‘spread’ which is the difference between the lowest sell quote and the
highest buy quote,
• heterogeneous beliefs; traders acting on different, often opposite beliefs may cause
prices to swing rapidly,
• simultaneous quotations & trades occurring on different networks or markets; in-
formation collected on asset prices over different trading networks can show the
one asset with two prices simultaneously.
Chapter 1. Introduction 9
Typically, the whole collection of microstructure effects are accounted for in the literature
by adding the Independent and Identically Distributed (IID) noise term ut in (1.6), where
pt represents the unobservable frictionless log price and pt the noisy logarithmic price.
pt = pt + ut (1.6)
Up until this point no distinction has been made between the observable and the
frictionless price because at low frequencies market frictions contribute little towards
the observed returns and so pt ≈ pt under these circumstances. However, as the time
between observations gets smaller, market microstructure effects make up a larger pro-
portion of any observed change in price. Thus, the frictionless returns are increasingly
obfuscated under the observed returns at high frequencies. Hence, we are presented
with a dichotomy. Ideally, more observations would correspond to a lower stochastic
error within a volatility measurement; on the other hand, a higher sampling rate can
introduce biases due to microstructure effects.
Much progress has been made in the attempt to overcome the difficulties in high
frequency volatility estimation. Most significantly, quadratic variation theory allows us
to measure the integrated volatility using squared daily returns, see Andersen et al. [6].
While empirical analysis by Martens [21] has shown that this approach does yield better
estimates than that obtained by methods that use daily data, it does have a tendency
to break down when microstructure noise becomes significant, see Andersen et al. [22],
Bandi and Russell [23], Zhang et al. [24]. Corsi et al. [25] found that this breakdown in
the realized volatility measure at high frequencies can be explained by the autocorrela-
tion structure of the returns data. This phenomenon is most commonly observed in the
negative autocorrelation of high frequency returns which results in positive anomalous
scaling for volatility estimates in FX markets. Conversely, positive autocorrelation in
returns leads to negative anomalous scaling of volatility estimates in many stocks and
indices.
Chapter 1. Introduction 10
Several suggestions have been put forward to solve the problem of microstructure
noise, firstly Andersen et al. [6], [26] put forward the idea of sparse sampling, which ef-
fectively decreases the proportion of the return volatility which is due to noise. However,
it was shown by Zhang et al. [24] that this is not an adequate solution to the problem.
This led to the development of the Two Scale Realized Volatility (TSRV) estimator,
which was put forward by Zhang and later refined by Ait-Sahalia et al. [27]. There have
also been several proposals for consistent Kernel based estimators, most notable are the
contributions made by Barndorff-Nielsen et al. [28] whose kernel estimators were shown
to be robust to endogenous and dependent noise as well as endogenously spaced data.
The method of applying filters has also been applied to the quandary of microstructure
noise, in particular Hansen et al. [29] were able to produce a consistent estimator of the
integrated volatility using the Moving Average (MA(1)) structure outlined by Ander-
sen et al. [26] and Ebens [30]. Finally, a conceptually similar approach to the method
developed in this thesis was taken by Curci and Corsi [31] in which their Discrete Sine
Transform (DST) diagonalised a moving average process into an orthonormal basis, from
which the microstructure and actual price could be separated.
The HHT based volatility estimates proposed in this thesis are also useful in the
realm of high frequency volatility estimation. It is shown that the empirical mode de-
composition process which forms a key component of the HHT, is capable of filtering
out a proxy for the ‘true’ or frictionless price from the noisy observable price which
has been obfuscated by the effects of market microstructure. The proposed approach
is also capable of delivering spot volatility estimates as opposed to the estimations of
integrated volatility that RV based techniques provide, with further improvements this
feature may be of great interest to those with an interest in ultra high frequency volatility.
Chapter 1. Introduction 11
1.3 A New Tool in the Toolbox
The Hilbert-Huang transform is a powerful new tool for the analysis of non-stationary
and nonlinear time series. Its adaptive and data driven nature make it highly flexible,
this has driven its use in a growing number of fields. The HHT is comprised of two parts,
first the Empirical Mode Decomposition (EMD) breaks a signal down into simple wave
like components that are sorted by frequency and then the Hilbert transform is applied,
yielding instantaneous amplitudes and frequencies of the simple waves. One of the main
advantages of using this transform is the ease with which the information from the HHT
can be interpreted. Due to the nature of the frequency and amplitude information the
process extracts, it is natural to make a comparison to Fourier methods, as indeed many
have, Huang et al. [1], Donnelly [32] to name a few. While Fourier techniques are ex-
tremely useful and are perfect for transforming signals from the time-amplitude domain
into the amplitude-frequency domain, they suffer from the drawback that this transform
is done at the expense of the time domain. Thus, where the Fourier transform is lacking,
is in the identification of changes in frequency with respect to time.
While Fourier techniques transform a signal from the time domain into a frequency
domain signal they cannot transform back into the amplitude-time domain without as-
suming that the signal is periodic and stationary. Thus, the main advantage of the HHT
is that the time domain is preserved, allowing for not only the identification of different
frequencies in a signal, but also their temporal location. This property is particularly
attractive when analysing non-stationary time series which, by definition change their
properties over time. There are also Fourier methods which analyse only a small sliding
window of data at a time, this procedure can maintain the time domain to an extent,
however there is a costly trade off between frequency and temporal resolution which is
difficult to overcome.
Wavelet transforms are another tool which can yield frequency information without
Chapter 1. Introduction 12
compromising the time domain however they are described by Huang et al. [1] as suf-
fering from uniformly poor resolution at different time scales because it is restricted
by essentially the same dichotomy as windowed Fourier transforms, i.e. one may have
high temporal resolution or high frequency resolution but not both simultaneously. This
being said, the literature on wavelets is growing rapidly and new wavelet decomposition
techniques are challenging this unfortunate dichotomy regularly. Comparisons between
the HHT, or more strictly the EMD which underpins it and wavelet transforms seem
more apt, indeed Flandrin et al. [33, 34] has made a strong case for the the EMD be-
having like a diadic filter bank, resembling some wavelet decompositions.
One of the more appealing attributes of the HHT is its seemingly natural ability to
extract periodic signals even when they are hidden by high levels of noise, see Li and
Meng [35]. This ability to perform hidden feature extraction was given more weight by
the work of Wu and Huang [36] who were able to assign a statistical significance to the
these extracted trends.
Flandrin et al. [33, 34] & Wu and Huang [37] further explored the properties of the
EMD when applied to noisy and stochastic signals and much of their methodology is
employed over the course of this thesis to examine the spectral properties of the wave
like structures that are produced by the EMD process, Chapter 2 elaborates further on
these developments.
The HHT and the EMD process which underpins it, have found many and varied
applications in a number of fields. In mechanical engineering the HHT has been used by
Liu et al. [38] to identify faults in gearboxes by analysing the telltale changes in vibration
which may indicate a cracked tooth or a some other gearbox related malady. In electrical
engineering the HHT is becoming an increasingly popular method for monitoring power
quality, see Senroy et al. [39], Yalcin et al. [40]. Due to the adroit manner in which the
EMD process handles seasonal data it is has found use as an analytical tool in the area
of climate science, see Molla et al. [41], ocean waves Rao and Hsu [42], Hwang et al.
Chapter 1. Introduction 13
[43] and pathology Cummings et al. [44]. The EMD and HHT processes are also slowly
gaining in popularity as tools in the financial world due to the complex, occasionally
seasonal, and reliably chaotic nature of financial data. The first and most natural appli-
cation of these new tools is seasonality analysis, Zhang et al. [45], Crowley [46], Huang
et al. [47], all used these techniques to break down financial time series data into seasonal
components for easier analysis.
The HHT has even been used as a financial volatility indicator in the past, in par-
ticular by Rao and Hsu [42], Huang et al. [47]. These approaches are often referred to
as variability or fluctuation measures as they do not give a volatility measure which
can be interpreted in terms of the variance. Thus the volatility estimates they produce,
while being useful volatility indicators, are harder to fit into existing financial models
and so there exists an enticing gap in the literature where a usable HHT based volatility
estimator should be.
The HHT specialises in analysing nonlinear and non-stationary time series, two prop-
erties which make it an ideal candidate for the backbone of a volatility estimator. In this
thesis the HHT is used as an actual measure of financial time series variance for the first
time. The only HHT based volatility measure conceptually similar to the one introduced
in this thesis was created by Wen and Gu [48] for earthquake simulation and modelling,
however its full potential as a volatility estimator was left largely unexplored. In this
thesis the HHT is shown to be capable of extracting the spot volatility σ2t which is easily
interpreted, rather than the less tractable volatility indicators it had hitherto produced.
Since the HHT procedure has not been used in this context before, the goal of this thesis
is not to necessarily create the best volatility estimator possible, but rather to evaluate
the viability of the HHT procedure as a spot volatility estimator and to examine the
nature of financial time series dynamics through a new and powerful analytical tool.
Chapter 1. Introduction 14
1.4 Thesis Outline
• Chapter 2
This chapter goes into more detail regarding a selection of popular volatility es-
timation procedures which were mentioned in the introduction. The essential
background information needed to understand the Hilbert-Huang transform as
well as the volatility estimator derived from it are also discussed in detail. The
components of the HHT procedure are also used to perform some analysis on de-
terministic and stochastic time series in order to familiarise the reader with some
of the properties of the HHT procedure. This chapter also covers some of the
testing and comparison procedures and some of the difficulties faced therein.
• Chapter 3
This chapter introduces an intuitive means of estimating the low frequency (daily)
historical volatility using the HHT. The HHT procedure is used to generate volatil-
ity estimates which are then compared against estimates given by a handful of con-
temporary spot volatility estimators as well as the realized volatility which is used
as a proxy for the true volatility. The proposed procedure is applied to ten years
worth of data from the All Ordinaries (All Ords) and the National Association of
Securities Dealers Automated Quotations (NASDAQ) Indices.
• Chapter 4
This chapter contributes to both low and high frequency volatility measurement
by first proposing and testing an improved HHT based method of measuring the
volatility of a time series. Firstly a test is proposed in order to examine and
quantify the trade off between temporal resolution long term accuracy. The HHT
based volatility estimate is then adapted to handle time series with microstructure
noise and its effectiveness is thoroughly tested using both simulated and data in
this chapter. Monte Carlo techniques are employed to simulate three different
kinds of time varying volatility for different levels of market microstructure noise.
• Chapter 5
This chapter tests a variant of the procedure developed in Chapter 4, using real
Chapter 1. Introduction 15
high frequency FX data. The inherent difficulty of assessing the estimation of
a latent variable is overcome by setting up a simulated options market, which
enabled an effective comparison based evaluation under real market conditions.
• Chapter 6
This chapter concludes the thesis. There is a summary of progress made and a
some thoughts are shared on the possible directions of future work.
Chapter 2
Background
This chapter is split up into three sections, the first section covers some of the most
widely practiced methods for calculating volatility at both low and high frequencies.
The second section describes the Hilbert-Huang transform and some analysis is into
its properties is provided. Finally, section three discusses some of the problems faced
when analysing a latent variable such as volatility and presents some of the measures
commonly used to evaluate model accuracy.
2.1 Volatility Measures
This section introduces some of the problems faced when measuring historical volatility
and clarifies which variants of each model have been used in this study. A descrip-
tion of the Moving Average (MA), Exponentially Weighted Moving Average (EWMA),
Generalized Autoregressive Conditional Heteroskedasticity (GARCH(1,1)) and Realized
Volatility (RV) models are given.
2.1.1 The Moving Average Volatility Model
The MA volatility estimate is calculated by sliding a window of interest along a time
series and calculating the sample variance of points within this moving window. When
this technique is applied a sliding window length must be chosen, this parameter can
16
Chapter 2. Background 17
greatly alter the volatility observed at any point. While the MA method may be suit-
able for describing the volatility of some time series, it suffers from the drawback that
one may only increase long term accuracy with a decrease in temporal resolution. This
drawback gives rise to artefacts such as ghosting and plateauing, in which a short term
spike or trough in volatility will be equally represented along an entire window length.
Overly inflated or retarded volatility estimates persist until the short term disturbance
is no longer included within the sliding window. This plateauing effect is clearly visible
in Figure 2.1, where short term fluctuations in price are followed by a long period of
overly high volatility estimates. The issue with poor temporal resolution and plateauing
renders the MA method of limited use, Wilmott [10] suggests that its use is confined
to cases with slowly varying volatility. The MA variance of the log returns r, usually
calculated on market closing prices is given by:
MAσ2n(N) =
1
N − 1
N−1∑j=0
(rn−j − r)2, (2.1)
where r is the expected (or mean) return which is usually assumed to be zero when the
sampling rate is daily or shorter and MAσ2n(N) should be understood as the variance
on the nth day where N is the sliding window length . Note that the ∼ symbol above
a character is often used to denote estimates throughout this thesis and that the MA
standard deviation is simply the square root of (2.1).
2.1.2 The Exponentially Weighted Moving Average Model
EWMA is a popular and effective technique for calculating daily variances, it largely
overcomes the plateauing effects of the MA model by weighting the importance of returns
based on their time of arrival, with the most recent returns given higher weights. The
Chapter 2. Background 18
2000 2001 2002 2003 2004 2005
2000
3000
4000
5000
All
Ord
s P
rice
Date
All Ords Price and MA Volatility
0
0.1
0.2
0.3
MA
Vol
atili
ty σ
n
Figure 2.1: All Ords Index prices from 1st Jan 2000 to 31st Dec 2004 (bold) and asimple MA volatility estimate with a window length of 50 days (dashed).
EWMA procedure for estimating volatility is given by:
EWMAσ2n(λ) = (1− λ)
∞∑j=0
λjr2n−j ,
or, written recursively:
EWMAσ2n(λ) = λEWMAσ2
n−1(λ) + (1− λ)r2n. (2.2)
where EWMAσ2n−1(λ) is the last variance estimate, r2
n the most recent squared return and
λ the decay rate. For daily returns it is common to let λ = 0.94 and for monthly returns
it is common practice to let λ = 0.97 because these values were found to minimise mean
square errors when performing one step ahead volatility forecasts, this is often referred
to as the RiskMetrics model for volatility modelling, see Longerstaey and Spencer [49].
The EWMA approach does indeed decrease the persistence of volatility spikes how-
ever they are still observed to decay exponentially at a rate proportional to λ. This
brings one to the inevitable question, what is a suitable choice for λ? Whatever our
choice for the decay rate it is still a balancing act between a noisy volatility estimate
Chapter 2. Background 19
with high temporal resolution and a smooth estimate with better long term accuracy.
This in turn begs the question of just what is noise and what is a legitimate short term
change in volatility. Suppose you had a time series with rapidly changing volatility fluc-
tuating around a long term mean; when determining parameters such as the decay rate,
a choice must be made as to which property of the time series your volatility estimate is
designed to extract, i.e. the long term average or the short term fluctuations. Figure 2.2
shows how two different choices for λ result in one noisy but highly responsive measure
and one that is smooth and better suited to picking up long term trends. The latter of
these two parameter choices is more susceptible to a plateauing like effect, in which the
volatility gradually decays down to its new level, rather than producing the tabletop
like artefact the MA procedure generates. This effect is particularly evident after sharp
declines in volatility as seen around mid 2000 and late 2001 in Figure 2.2. The impact
of different choices for λ are explored further in Chapter 4.
2000 2001 2002 2003 2004 20050
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Date
Vol
atili
ty σ
n
All Ords EWMA Volatility
λ=0.7λ=0.94
Figure 2.2: EWMA volatility estimates with decay rates λ = 0.94 & λ = 0.7 for theAll Ords data set covering 1st Jan 2000 - 31st Dec 2004.
Chapter 2. Background 20
2.1.3 The GARCH(1,1) Model
The GARCH(1,1) model given by Equation (2.5) is widely used to model volatility
because of its relative simplicity and flexibility. It was designed to capture the volatil-
ity clustering behaviour, a phenomenon involving periods of high and low volatility that
have a tendency to group together, see Ruppert [50] for a more in depth look at volatility
clustering in financial markets. The GARCH(1,1) model is similar to the EWMA model
because it too depends on the last variance estimate the model returned (GARCH σ2n−1)
and the most recent squared return r2n, however in addition to this there is also the long
term average term average variance form, represented in its weighted form as ω below.
The disadvantage involved with using a GARCH(1,1) model is that it can require
a large amount of data in order to determine its three parameters, also like EWMA
methods GARCH(1,1) still shows some evidence of plateauing which can be observed
most prominently after a sharp decrease in volatility. The GARCH(1,1) model is usually
written in the following recursive form when used as a variance estimate:
rn = r + εn (2.3)
εn | Φn−1 ∼ N(0,GARCH σ2n) (2.4)
GARCH σ2n = ω + αε2n + βGARCH σ2
n−1, (2.5)
where rn is the daily log return, r is the mean return which is assumed to be zero for
sufficiently high frequency, Φn−1 contains all information up to day n-1, and the residu-
als εn follow a conditional normal distribution with mean zero and variance GARCH σ2n.
Also ω, α and β are parameters to be determined, with ω > 0; α ≥ 0; and for long
term stability it is required that α+β < 1 so that the long term average volatility stays
positive. More specifically, α is the weighting of the dependence on the most recent
innovation, β is the weight assigned to the last volatility estimate and ω is a constant.
Note that the unconditional variance of is given by σ2 = ω1−(α+β) .
Chapter 2. Background 21
The GARCH(1,1) model behaves similarly to the EWMA model, with β taking
the place of λ and α that of (1 − λ). The more general GARCH model is given as
GARCH(p,q) which uses the p most recent values of the returns r and q most recent
estimates of GARCH σ2. For more information on the GARCH procedure and its imple-
mentation see Bollerslev [13], Posedel [51] and Reinhard Hansen and Lunde [52].
2.1.4 Basic Realized Volatility
The realized volatility measure is a non-parametric, model free indicator and is generally
considered to be a very accurate means of extracting the volatility of a time series for
which intraday data is available. This section formalises and expands upon the realized
volatility discussion given in the introduction.
For the purposes of the RV measures we assume a price model like that described in
Andersen et al. [53] and Barucci et al. [54], i.e. the log of the frictionless price follows
Equation (1.3). The goal of the RV method is to measure the Integrated Volatility (IV)
over a certain period, for convenience, the unit for the time interval is usually set as one
day. Recall that one period of integrated volatility is defined by Equation (1.4). This
definition of IVt is sometimes referred to as the quadratic return variation. Although the
integrated volatility is not a directly observable variable it has been shown by Andersen
et al. [7] and Barndorff-Nielsen and Shephard [55] to be closely estimated by the realized
volatility, which is defined in Equation (2.7). First, we recall that the log return at time
t for step size h is defined as:
rt(h) = pt − pt−h. (2.6)
Then the realized volatility can be defined as:
BasicRV t(h) ≡1/h∑i=1
(rt−1+ih(h))2, (2.7)
which should be understood as the RV on day t for sample interval h, where 1/h is a
positive integer and corresponds to the daily sampling frequency. Note that when h is
at the smallest increment available, i.e. every sample point is used, the resulting RV
Chapter 2. Background 22
measure is denoted in this thesis as AllRV t.
Theoretically, without the presence of microstructure noise the realized volatility
estimate given by Equation (2.7) converges towards the integrated volatility given by
Equation (1.4) as h → 0 because the sample error approaches zero. In practice, when
microstructure noise is included into the observed log price, as in Equation (1.6), the
RV estimate can be greatly distorted as the the time between samples decreases. For
this reason, RV estimates usually use returns separated by intervals of between 5 and
15 minutes.
2.1.5 Overnight Returns
While many FX markets are open and trading frequently for 24 hours a day, it is typical
of most stock markets to only operate for a fraction of the day, usually between the
hours of 10am and 4pm local time. This cessation of trading usually results in a jump
(up or down) in asset prices when the market is reopened which leads to something
of a disconnect between the volatility measures that use strictly intraday trading data
and those which use daily data. Since this overnight change can be quite large, usually
following some important event, any volatility estimate that uses strictly intraday data
risks becoming a poor proxy to the daily volatility. To take into account this change
in overnight prices, a number of approaches are considered here, see Hol and Koopman
[56] or Tsay [57] for alternative approaches.
Since the more formal definitions of the RV model defined in Section 2.1.4 do not
easily lend themselves to the discussion of overnight returns, some simple notation is
introduced. Let the log prices on day t be defined as pt(k), with intraday values denoted
by k = 0 . . . 1/h. A value of k = 0 gives the opening price on day t and k = 1/h gives
the closing price on day t. Hence, a simple “close to open” log return is defined as
cort = pt(0)− pt−1(1/h).
Chapter 2. Background 23
One simple approach to to calculate the RV for a whole day is simply to add any
overnight return square to the RV estimated for that day, i.e.:
WDRV t(h) =co r2t +Basic RV t(h), (2.8)
where the superscript WD simply denotes that the estimate covers a whole day. The
problem with this approach is the noisy nature of cort which has a variance far greater
than that of BasicRV t(h), leading to an estimate of volatility for the whole day that is
relatively noisy.
When overnight return data is available, even if it is sparsely sampled then the
overnight volatility can be calculated by including the overnight returns at intervals h
and combined with the intraday RV to give:
ContRV2t (h, h) ≈
1/h∑j=1
(rt−1+jh(h))2 +
1/h∑i=1
(rt−1+ih(h))2, (2.9)
where the superscript Cont denotes continuous trading, 1/h is an integer, the summation∑1/hj=1 covers the sparsely sampled overnight (close to open) period and the summation∑1/hi=1 covers the (open to close) trading hours. Naturally, this reduces to Equation (2.8)
when the overnight sample period is very low. Conversely, when the sampling rate is
consistently high, this approach is equivalent to Equation (2.7) with a 24 hour period.
An alternative approach was developed by Martens [21] & Koopman et al. [58], it is
defined as follows:
WDRV2t (h) = (1 + c)BasicRV t(h), (2.10)
where WD simply denotes that the estimate covers a whole day and c is a positive
constant, i.e. taking into account overnight returns will only increase a variance estimate
that has only used data collected while the market was open. The scaling factor (1 + c)
is defined as:
1 + c =σ2oc + σ2
co
σ2oc
, (2.11)
Chapter 2. Background 24
where σ2oc and σ2
oc are variance estimates of the “open to close” and “close to open”
data, i.e.:
σ2oc =
1
N
N∑t=1
(ocrt)2, (2.12)
σ2co =
1
N
N∑t=1
(cort)2, (2.13)
where a simple “open to close” log return is defined as ocrt = pt(1/h) − pt(0) and cort
was defied earlier.
The observation that σ2oc is inherently noisier than RV oc led Hansen and Lunde
[59, 60] to develop their own whole day volatility estimate. Their approach proposes
that the adjustment factor of Equation (2.10) follows:
1 + c =
∑Nt=1(ccrt − r)2∑N
t=1BasicRV t(h)
, (2.14)
where ccrt denotes the “close to close” return on day t, i.e. ccrt = rt(1/h) − rt−1(1/h)
and r = 1N
∑Nt=1
ccrt. Equations (2.10) & (2.14) are used to calculate the daily returns
for the data examined in Chapter 3 of this thesis.
2.1.6 Sparse Realized Volatility and the Averaged or Subsampled Re-
alized Volatility
Consider a model with microstructure noise as given by Equation (1.6) with ut being
IID noise and the observed log returns given by:
rt(h) ≡ pt − pt−h, (2.15)
the noise is linked to the contaminated (noisy) observable returns rt(h) by:
rt(h) = rt(h) + et(h), (2.16)
Chapter 2. Background 25
where
et(h) = ut − ut−h. (2.17)
The sparse RV estimate is essentially the same as the basic RV measure however it
is not sampled at the highest available frequency, but rather at spacings h which are
some multiple (denoted nh) of the minimum sample interval, e.g. h = nhh where 1/h is
a positive integer. The sparse volatility estimator sparseRV is then defined as:
sparseRV t(h) ≡1/h∑i=1
(rt−1+ih(h))2. (2.18)
The advantage of this method is that it is less susceptible to the microstructure noise
that plagues higher frequency measurements. This is where an important compromise
must be made, one must choose between a low sample error and high microstructure
interference or high sample error and less microstructure effects.
Motivated by the desire to keep sample error at a minimum, while still retaining the
more robust nature of sparse sampling, the method of averaged or subsampled RV was
born. Firstly, an offset version of the subsampled method described above is given as:
OffsetRV t(h, k) ≡Nk∑i=1
(rt−1+ih+kh(h))2, (2.19)
with k = 0, . . . , nh − 1 and Nk = 1/h.
The AverageRV volatility estimator is then produced by taking the average of many
offset sparse RV estimates, i.e.
AverageRV t(h) =1
nh
nh−1∑k=0
OffsetRV t(h, k) (2.20)
The result of this procedure is a volatility estimator that can be resistant to market
microstructure noise when sampled sparsely and has a smaller sample error than the
standard sparse RV estimate.
Chapter 2. Background 26
2.1.7 Two Sample Realized Volatility
Unfortunately the sparse estimator and the averaged sparse estimator can still suffer
from a bias, especially if the sample rate is not chosen carefully. Fortunately though,
the size of this bias can be estimated and taken into account. This is achieved by the
Two Scale Realized Volatility (TSRV) procedure, which first uses an average sparse
estimator and then subtracts the theoretical bias.
˜TSRV t(h) =Average RV t(h)− (h/h)AllRV (h). (2.21)
A simple adjustment can be made to the above formula to account for the finite-sample
size of the two summands in Equation (2.21), this adjusted variant is given by:
Adjusted ˜TSRV = (1− h/h)−1 ˜TSRV t(h) (2.22)
While the family of RV methods are extremely useful, they may not be perfect for
every scenario. For example, Figlewski [20] showed that using daily volatility estimates
can lead to poorer long term forecasts than those produced from lower frequency data.
Realized volatility also requires a large amount of high frequency intraday data, this is
somewhat of a drawback since such high frequency data may not be available or may be
costly. Still, when high frequency data is available the Adjusted-TSRV method forms a
good benchmark and is widely used.
2.2 The Hilbert-Huang Transform
The HHT algorithm includes two separate procedures, firstly the EMD process breaks up
a signal into different frequency components known as Intrinsic Mode Functions (IMFs)
denoted by Ψ and secondly the Hilbert transform is used to calculate the instantaneous
amplitude and frequency for each separate IMF component. The IMFs are naturally
sorted from the highest to the lowest frequency, i.e. the first IMF is a time series con-
taining the rapidly changing components of a signal while later IMFs extract features
Chapter 2. Background 27
that vary more slowly in time.
The reader should be aware that there is also a slight change in notation at this
point. To be in keeping with the standard notation of the HHT, time will no longer be
denoted by a subscript as those positions are now reserved for IMF numbers. Also, a
resource that was of great help in understanding and performing the EMD process is
provided by Flandrin [61].
2.2.1 Empirical Mode Decomposition
For the HHT process to give meaningful information about a signal, the EMD process
will decompose a signal S(t) into IMFs which have the following properties
(i) Each IMF must have exactly one zero between any two consecutive local extrema.
(ii) At any point the mean value of the upper envelope (consisting of splines connecting
all of the maxima) and the lower envelope (consisting of splines connecting all of
the minima) is zero (within a tolerance).
The following is an outline of the steps needed to complete the EMD process, it
follows an outline described by Senroy et al. [39]:
1. Identify local maxima and minima of the signal S(t).
2. Create a cubic spline going through all of the maxima Cmax(t) and another going
through all the minima Cmin(t). This step is illustrated in Figure 2.3 where the
signal S(t) is clearly seen to be encased by the maximum and minimum splines.
3. Calculate the mean of the two splines.
Cmean(t) =Cmax(t) + Cmin(t)
2. (2.23)
Chapter 2. Background 28
4. Calculate the first potential IMF known as a Proto-Mode Function (PMF),
PMF1(t) = S(t)− Cmean(t). (2.24)
5. If PMF1(t) satisfies the conditions to be an IMF then Ψ1(t) = PMF1(t). If not
repeat steps 1 - 4 on PMF1 until it becomes an IMF.
6. Calculate the first residue Ω1(t),
Ω1(t) = S(t)−Ψ1(t). (2.25)
7. If the maximum amplitude of the residue is below a threshold or there are three
or fewer local maxima or minima, terminate the EMD process, otherwise repeat
steps 1 - 6 on the residue Ω1(t).
When the procedure has terminated, the result is a set of wave like functions as seen
in Figure 2.4 plus a leftover which can be recombined to give the original signal i.e.
S(t) =
n∑j=1
Ψj(t) + Ωn(t), (2.26)
where Ψj(t) are the wave like IMFs and Ωn is the leftover or residue. Also, as observed
in Wu and Huang [36], Equation (2.26) may be rewritten as:
S(t) =n∑j=1
Ψj(t) + Ωn(t) =m∑j=1
Ψj(t) + Ωm(t) =m+1∑j=1
Ψj(t) + Ωm+1(t), (2.27)
where m < n and Ωm(t) is the remainder of S(t) after m number of IMFs are extracted.
Also, it can be shown that each residue, with the exception of the last, can be written
as an IMF plus a lower frequency residue, i.e.
Ωj(t) = Ψj+1(t) + Ωj+1(t). (2.28)
As Wu and Huang [36] observe, Ωj+1(t) is a local mean of Ωj(t) which is valid for a
time approximately equal to the local period of Ψj+1(t), i.e. each successively lower
Chapter 2. Background 29
0 0.2 0.4 0.6 0.8 1−6
−4
−2
0
2
4
Time
S(t
)
Signal within Envelope
SignalMinMax
Figure 2.3: Super imposed sinusoids (solid blue), the local maxima and connectingspline (red circle and dot-dash line respectively) and the local minima and connecting
spline (green circle and dashed line respectively).
frequency residue forms a local mean to its higher frequency predecessor. This concept
of extracting a local mean in terms of residues and IMFs is used to construct a proxy to
the return series, as described in Section 3.1.1.
A simple example of the procedure follows. Consider a sinusoidal signal that contains
two waveforms, one at 5 Hz at amplitude 3 arbitrary units (arb.) and another at 15
Hz at an amplitude of 1 arb. which persists until t = 0.5, after which point the higher
frequency component increases from 15 to 25 Hz and persists until t = 1. This signal of
superimposed sinusoids can be seen in Figure 2.3 along with local maxima and minima
indicated as well as the splines joining those extrema points. Figure 2.4 shows the
IMFs extracted from the sinusoidal signal, note how the first IMF contains the two
high frequency components consecutively while the second IMF only contains the lower
frequency component. This demonstrates, in a simple manner, the frequency sorting
nature of the EMD process.
Chapter 2. Background 30
0 0.2 0.4 0.6 0.8 1−1.5
−1
−0.5
0
0.5
1
1.5First IMF
Time0 0.2 0.4 0.6 0.8 1
−4
−3
−2
−1
0
1
2
3
4
Time
Second IMF
Figure 2.4: The first two IMFs of S(t).
2.2.2 The HHT as a Variability Measure
The EMD process has been used by Rao and Hsu [42], Huang et al. [47] to estimate
historical volatility, the estimate is described by:
V ariabilityHHT (t) =|∑n
j=1 Ψj(t)|S(t)
, (2.29)
where n is the number of IMFs in the summation. In practice this means that n is a
parameter that determines the time scale to which the variability measure applies. In
effect this procedure gives the distance between the current value of the time series and
a variable local mean which is set by changing n. Perhaps the biggest disadvantage of
this method is the fact that the results it yields aren’t in terms of the variance and
are therefore not easily compatible with existing models. While this variability measure
does have several drawbacks, many described by Rao and Hsu [42], Huang et al. [47], it
does at least serve as a starting point from which more practical HHT based volatility
estimators may be built.
2.2.3 Monte Carlo Analysis of the EMD Procedure
In this section a quick study is carried out on the properties of the EMD procedure
when applied to both Gaussian White Noise (GWN) and the more complicated process
Chapter 2. Background 31
0 1 2 3 4 5 6 7 8−11
−10
−9
−8
−7
−6
−5
−4
−3
−2Semi−log plot of IMF Number vs Period
IMF Number
Log 2 P
erio
d
Figure 2.5: IMF number vs log2 IMF period for GWN.
of fractional Gaussian noise (fGn).
Some interesting properties the EMD procedure are revealed when it is applied to
simple Gaussian White Noise. In this simulated study a simple GWN is generated for
2048 points and repeated 1000 times. Applying the EMD procedure to this data set and
taking the average over the 1000 simulations it is possible to extract some interesting
features, namely the log-linear relationship between the IMF number and the log of the
average period of each IMF, as shown in Figure 2.5. From this figure we can also observe
that the period almost exactly doubles with each successive IMF, this is in line with the
observations made by Flandrin et al. [33] and Wu and Huang [37]. This period doubling
of successive IMFs is similar to the behaviour of dyadic wavelet filters, see Flandrin et al.
[33, 34].
Research done by Huang et al. [62] suggests that for white noise, the spread of energy
(or variance in our case, as seen in the top of Figure 2.6) within each IMF should follow a
χ2 distribution. Also, Huang proposes that deviations from this spread are evidence for
statistically significant deviations from random white noise. This finding has important
Chapter 2. Background 32
implications for seasonality analysis as it may allow for the identification of seasonal
activity, or trend extraction and provide a means for statistical significance testing.
Figure 2.7 depicts the Fourier power spectra of the first 10 IMFs, it illustrates another
important feature of the EMD process observed by Flandrin et al. [33] & Rilling et al.
[63], in that the first IMF acts as a high pass filter, i.e. only components of a signal
that are above some frequency threshold are admitted. This behaviour can be observed
in the right most part of the figure where the spectrum which corresponds to the first
IMF maintains a high power spectral density right through to the limit of observable
frequencies (the Nyquist frequency). The last IMF, visible on the far left of the figure
behaves in quite the opposite way, acting as a low pass filter. Contrary to the first and
last IMFs, every other IMF behave more like a band-pass filter, i.e. the frequencies
found in the IMFs are bound above and below.
The abscissa (horizontal position) of the IMF power spectra can also be shifted to
demonstrate that IMFs produced by the EMD procedure are highly self similar at dif-
ferent time scales. This property is depicted in Figure 2.8, note that with the exception
of the first and last IMF which have been removed from the plot, the remaining IMFs
largely lay on top of one another. This result, as with many others in this section con-
firm the results found by Flandrin et al. [33] & Rilling et al. [63], which support the
proposition that the EMD procedure has a dyadic filter-bank structure.
An interesting linear relationship exists between the log period of an IMF and the
log of the variance within that IMF, a similar relationship also exists between the IMF
number and the log variance. Both of these relationships can be seen in Figure 2.6. From
Figure 2.6 it is evident that there is a rough doubling of variance as the IMF number
increases and that a similar behaviour is being followed by the variance in relation to the
IMF number. These two properties, or rather the breakdown of these two properties,
is of practical importance during the self similarity analysis which is to follow. The
research carried out by Flandrin et al. [33] & Rilling et al. [63] also shows that the EMD
procedure can be used to investigate the self similarity of a time series at different scales
Chapter 2. Background 33
−10 −9 −8 −7 −6 −5 −4 −3 −2−10
−9
−8
−7
−6
−5
−4
−3
−2
−1
0
Log2 Mean Period
Log 2 V
aria
nce
IMF Period vs IMF Variance for Gaussian White Noise
0 1 2 3 4 5 6 7−10
−9
−8
−7
−6
−5
−4
−3
−2
−1
0
IMF Number
Log 2 V
aria
nce
IMF Number vs IMF Variance for Gaussian White Noise
Figure 2.6: Log mean period vs log2 variance for GWN (top) & IMF number vs log2IMF variance for GWN.
Chapter 2. Background 34
0 2 4 6 8 10−30
−25
−20
−15
−10
−5
log Frequency
log
PS
D (
dB)
Fourier Power Spectra of IMFs
Figure 2.7: Power Spectra of 10 IMFs on GWN.
by examining the change in relationship between the variance and the IMF number or
log period. This is of particular interest since it may provide a useful tool when exam-
ining the autocorrelation structure of market microstructure noise .
Fractional Gaussian noise (fGn) can be used to demonstrate different autocorrelation
structures by altering the Hurst exponent H. We now turn to the more complex model
of fGn to generate our data which is of length 2048 and has 1000 repeats. This simu-
lation model is more flexible and allows for changes in self similarity which will aid our
later examination of market microstructure noise. The Hurst exponent is of particular
interest because it yields information on the self similarity of the process at different
scales and the parameter is directly related to the fractal dimension of the series. This
is of interest because when the Hurst estimate is 0 < H < 0.5, there is evidence of a
negative autocorrelation which is a property that also leads to the anomalous scaling
caused by market microstructure. This negative autocorrelation means that the time
Chapter 2. Background 35
5 6 7 8 9 10−16
−15
−14
−13
−12
−11
−10
−9
−8
Shifted Frequency
log
PS
D (
dB)
Shifted Fourier Power Spectra of IMFs
Figure 2.8: Rescaled Power Spectra of 10 IMFs on GWN.
series will have a tendency to regress towards the mean quickly, i.e. any large jumps in
one direction are likely to be followed by large jumps in the opposite direction. Con-
versely if 0.5 < H < 1 this implies a positive autocorrelation, when this is the case any
movements of a time series in one particular direction are more likely to be followed my
more movements in the same direction, also known as clustering. Evidence of positive
autocorrelations in some financial time series returns have led to the rise of momentum
trading.
The change in behaviour of fGn with different Hurst exponents is well demonstrated
by its Fourier power spectra as see in Figure 2.9. Note that in the top plot when H = 0.25
the spectral peaks are increasing in height from left to right meaning that the short term
fluctuations have more spectral power. Conversely when H = 0.75 as depicted in the
bottom figure, the height of the peaks are decreasing from left to right which means that
lower frequency components are more powerful. Finally in the case when H = 0.5 the
Chapter 2. Background 36
0 1 2 3 4 5 6 7 8 9 10−35
−30
−25
−20
−15
−10
−5
log
PS
D (
dB)
H=0.25
0 1 2 3 4 5 6 7 8 9 10−35
−30
−25
−20
−15
−10
−5
log
PS
D (
dB)
H=0.5
0 1 2 3 4 5 6 7 8 9 10−30
−25
−20
−15
−10
−5
0
log
PS
D (
dB)
H=0.75
log Frequency
Figure 2.9: Power Spectra of 10 IMFs on fGn for three values of H.
spectral peaks are the same for each IMF which is to be expected since fGn is actually
GWN for this value of H and the spectra is identical to that in Figure 2.7.
Following the procedures for examining self similarity outlined by Flandrin et al.
[33, 34] and Rilling et al. [63], the Hurst exponent was calculated using slight changes
in the slope of the linear relationship between the log of the mean period of an IMF and
Chapter 2. Background 37
the log variance within each IMF. Alternatively the IMF number and log variance can
be used since the same linear relationship is observed, as shown in Figure 2.10. As the
figure shows, the slope of the linear regression between the log IMF period and the log
IMF variance is observed to vary in with changes in the Hurst exponent and it is this
property which allows for the approximation of H using the EMD procedure and the
simple formula H = 1 + slope2 .
While the actual values of H in Figure 2.10 were 0.250, 0.500 and 0.750 the estimates
yielded from this procedure were 0.343, 0.535, 0.773 using the change in gradient of the
IMF number vs variance, as seen in the right hand side of Figure 2.10. However, using
the change in gradient in the period vs variance as seen in the left hand side of Figure
2.10 yielded the estimates 0.361, 0.534, 0.765. Both of these estimates were made using
the least squares merit function to maximise the fit of a line on the mean values of the
log period or IMF number with the log variance. Due to the usage of the means in
each case, only one H estimate was obtained for each method over the whole simulation,
making some forms of error analysis difficult.
In an effort to give some means to assess the variability of these H estimates the
process was repeated without first taking the mean values of the data resulting in a H
estimate for every simulation so that its variability could be investigated. This process
yielded slightly different H estimates of 0.413, 0.531 & 0.712 with standard deviations
of 0.052, 0.054 & 0.059 respectively. The EMD procedure to calculate the Hurst expo-
nent is significantly biased for smaller values of H and moderately biased in the case
when H > 0.5, this was also found to be the case by Rilling et al. [63]. That be-
ing said, the Hurst exponent is still a useful indicator of long or short term dependence
in a time series and so having another procedure to estimate it may provide a useful tool.
Chapter 2. Background 38
−10 −9 −8 −7 −6 −5 −4 −3 −2 −1−12
−10
−8
−6
−4
−2
0
Log2 Mean Period
Log 2 V
aria
nce
IMF Period vs IMF Variance for Fractional Gaussian Noise
1 2 3 4 5 6 7 8−12
−10
−8
−6
−4
−2
0
IMF Number
Log 2 V
aria
nce
IMF Number vs IMF Variance for Fractional Gaussian Noise
H=0.75
H=0.50
H=0.50
H=0.25
H=0.25
H=0.75
Figure 2.10: Log period vs log variance for 3 values of H (top). IMF number vs logvariance for 3 values of H (bottom).
Chapter 2. Background 39
2.2.4 The Hilbert Transform
Now we move on to the Hilbert part of the Hilbert-Huang transform. Essentially the
Hilbert transform (denoted Hilb) is applied to each IMF to create what is known as an
analytic signal, which is like the IMF it was applied to, but in the complex plane and
out of phase with the original signal by π/2. From this, the instantaneous amplitude
and frequency can be calculated for each IMF.
More formally, the Hilbert transform Hilb(t) of a function S(t) of the continuous
variable t is defined as:
Hilb(t) =1
πP
∫ ∞−∞
S(η)
η − tdη, (2.30)
where P is the Cauchy Principal Value integral. Huang et al. [1] goes on to say that S(t)
and Hilb(t) form a complex conjugate pair, so the analytic signal Z(t) can be expressed
as:
Z(t) = S(t) + iHilb(t) = A(t)eiϕ(t) (2.31)
where i =√−1 with instantaneous phase ϕ and amplitude A(t) given by:
• Instantaneous Amplitude:
A(t) =
√Hilb2(t) + S2(t). (2.32)
• Instantaneous Phase:
ϕ(t) = arctan
(Hilb(t)
S(t)
). (2.33)
• The Instantaneous frequency f(t) is found using:
ϕ(t) = ω(t) = 2πf(t), (2.34)
or, as described by Deering and Kaiser [64] as:
ω(t) =d
dtarctan
Hilb(t)
S(t)=S(t) Hilb′(t)−Hilb(t)S′(t)
S2(t) + Hilb2(t). (2.35)
Chapter 2. Background 40
The Discrete Hilbert Transform (DHT) Hilb(k) of a function S(k) may be written in
the form:
Hilb(k) =
N−1∑m=0
h(k −m)S(m), (2.36)
where N is even and
h(k) =2
Nsin2(
πk
2) cot(
πk
N). (2.37)
As in Deering and Kaiser [64], a centered difference approximation was used on the
derivatives in Equation (2.35):
f(k) =S(k)(Hilb(k + 1)−Hilb(k − 1))−Hilb(k)(S(k + 1)− S(k − 1))
2ts(S2(k) + Hilb2(k)), (2.38)
where ts is the sampling time.
Finally, the frequency can be corrected by f(k) = fs2π arcsin( f(k)
fs), where fs is the
sampling rate. Therefore, by using the DHT on a signal S(t), a meaningful instanta-
neous magnitude, frequency and phase can be obtained. Figure 2.11 shows the result of
this Hilbert transform applied to the IMFs given in Figure 2.4. This gives a point by
point (instantaneous) amplitude and frequency for each part of the signal.
Returning for a moment to the simple example made by the superposition of sinusoids
described in Section 2.2.1. The time vs amplitude plot in Figure 2.11 clearly identifies
two distinct amplitudes at 3 and 1 arb. units. The time vs frequency plot in Figure
2.11 clearly shows that the dashed curve which represents the components of amplitude
1 arb. unit in the graph above clearly undergo a frequency change at the half way point,
changing from 15 to 25 Hz while the solid line of amplitude three in the graph above
stays at a constant frequency for the duration. Note that the edge effects are due to the
interpretation of the beginning and ends of the data as a local minima or maxima, these
edge effects are much less pronounced when there are multiple frequencies and only the
highest ones are of interest, the effect can also be mitigated with some prior or post
history which may be used as a lead in or out. The speed at which the HHT is able to
determine changes in frequency is the main advantage of using this method over other
techniques like the Fourier transform which lacks such fine temporal resolution, for a
Chapter 2. Background 41
comparison between the two methods see Donnelly [32].
The HHT hinges on the ability of the EMD process to decompose a signal into
separate components based on their frequencies, however the EMD process has been
found to have difficulties when multiple frequencies are close together which can result
in IMFs with mixed frequency and amplitude information. This phenomenon arises
because the two sinusoids superimposed on one another may be interpreted equivalently
as either the sum of two sinusoids or a single amplitude modulated sinusoid, this is
known as mode mixing and is the focus of much research, most notably by Deering and
Kaiser [64] and Rilling and Flandrin [65].
0 0.2 0.4 0.6 0.8 10.5
1
1.5
2
2.5
3
3.5
4
Time
Am
plitu
de
Amplitude Vs. Time
IMF1
IMF2
0 0.2 0.4 0.6 0.8 10
5
10
15
20
25
30
Time
F (
Hz)
Frequency Vs. Time
IMF1
IMF2
Figure 2.11: Instantaneous amplitude (left) and frequency (right) given by a Hilberttransform of the IMFs displayed in Figure 2.4.
2.3 Comparison and Evaluation Methods
There are many ways to compare the accuracy of volatility estimates. A few evaluation
measures that are used throughout this thesis are described by Equations (2.39)−(2.42):
• Mean Absolute Error (MAE):
MAE =1
N
N∑t=1
|σ2t − σ2
t |. (2.39)
Chapter 2. Background 42
• Mean Error (ME):
ME =1
N
N∑t=1
(σ2t − σ2
t). (2.40)
• Mean Square Error (MSE):
MSE =1
N
N∑t=1
(σ2t − σ2
t)2. (2.41)
• Mean Absolute Percent Error (MAPE):
MAPE =100
N
N∑t=1
|σ2t − σ2
t|σ2t
. (2.42)
Where σ2t is the ‘true’ volatility and σ2
t is the estimate under comparison.
Another useful test to compare the distribution of the return estimate to that of the
‘true’ volatility is the two-sample Kolmogorov-Smirnov (K-S) test which is a measure of
the maximum distance between the two cumulative distributions, for more on the K-S
test see Massey [66].
Measuring and comparing volatility estimates isn’t always straightforward. The prob-
lems of computing an accurate volatility estimate are compounded by the fact that
volatility is a latent or unobservable variable. In reality, the ‘true’ volatility is only
known during simulated studies, while simulated studies have provided many insights
into the nature of how microstructure noise affects high frequency volatility estimates,
the advantage of knowing the model volatility is obviously lost when real financial data
is considered, thus making any estimates of real market volatility difficult to accurately
assess.
When real market data is considered the advantage of being able to compare against
this ideal volatility is lost, as is the case in Chapters 3 & 5. Luckily though, there are
some workarounds. In Chapter 3 the problem of not knowing the true daily volatility was
Chapter 2. Background 43
overcome by using the RV which is theoretically much more accurate and has the advan-
tage of orders of magnitude more data. However, due to microstructure effects, one can
not simply increase the sampling frequency to yield better estimates ad infinitum. Thus,
another problem must be overcome when comparing between different high frequency
volatility estimates. A novel solution to this problem based on a virtual options trading
market was proposed by Engle et al. [67] and explored in the context of high frequency
financial forecasts by Bandi et al. [68]. This is the approach that has been adopted in
Chapter 5 for the evaluation of our proposed volatility estimates on high frequency FX
data obtained from the SIRCA database. In this approach, competing volatility esti-
mates are used to give one step ahead forecasts, these are then used to price short term
straddle options which are bought and sold among virtual traders, each with their own
short term forecast of future volatility. See Chapter 5 for more details on this procedure.
Chapter 3
Low Frequency Volatility
Estimation & the HHT
This chapter introduces a method to extract volatility from a time series of daily index
data using the HHT. The proposed method is suitable for non-stationary processes with
volatility changing over time, like the behaviour observed in most financial time series.
Firstly, a proxy to the log returns is constructed using the HHT process and a statistical
measure of fit. The second step is to calculate the variance of the wave like components
that make up the proxy. Volatility estimates produced by this method are then compared
against the EWMA and GARCH(1,1) procedures on NASDAQ and All Ords Index data
covering a ten year period.
3.1 Volatility Estimation using the HHT
This section explains in detail the procedure used to estimate volatility from a time
series using the HHT.
3.1.1 Generating a Proxy of the Returns
The first step towards using the HHT to extract a measure of volatility is to generate a
proxy to the log return series of S(t) using IMFs. Once a proxy is generated which is
44
Chapter 3. Low Frequency Volatility 45
statistically similar to the log returns the simple sinusoidal like nature of the IMFs can
be exploited to extract the volatility.
The proxy to the returns implemented in this thesis is based on fitting the distribution
of the log returns to the distribution of the proxy by the implementation of a cutoff
frequency and a statistical measure of fit. A more detailed explanation of the procedure
follows:
• Apply the HHT process on the log of the price data to obtain the IMFs Ψj(t) and
the instantaneous amplitudes and frequencies Aj(t) & fj(t), where j is the IMF
number. Recall that the log returns on the frictionless price under this notation
are given by:
r(t, h) = p(t)− p(t− h), (3.1)
where h is some time increment, considered to be one day in the context of this
chapter.
• We wish to construct a proxy to these log returns which is easy to analyse and has
a similar volatility. Our proxy to the returns P r(t) is of the form:
P r(t) = p(t)−M(t). (3.2)
where M(t) represents a local mean of the log price at time t. Conceptually it
helps to view the log returns as a relative difference between consecutive prices
while the proxy can be viewed as a measure of short term fluctuations away from
a “local mean”. Although they are different measures they can behave similarly
when an appropriate choice is made for the local mean. A simple measure for the
local mean which could be defined over a variable time scale is given by the residues
of Equation 2.28. However, one drawback with this approach is that filtering by
whole IMFs at a time acts as a rather coarse filter. Thus, a finer filter which is
able to take fractions of an IMF would allow for grater flexibility. Creating a local
mean that can use partial IMFs was achieved by implementing a cutoff frequency
which is chosen in such a way as to which maximise a measure of fit between the
Chapter 3. Low Frequency Volatility 46
proxy and the log returns. The statistical measure of fit used in this chapter was
the K statistic of a two sample Kolmogorov-Smirnov (K-S) test.
• The log price can be written in terms of high frequency fluctuations and low
frequency long terms trends as::
p(t) =n∑j=1
Ψj(t) + Ωn(t) =m∑j=1
Ψj(t)︸ ︷︷ ︸RapidF luctuations
+n∑
j=m+1
Ψj(t) + Ωn(t)︸ ︷︷ ︸LongTermTrend
, (3.3)
where j is the IMF number and m is an integer smaller than n representing the
divide between high and low frequencies. Similarly a specific frequency may be
viewed as a cutoff between high and low frequency, i.e.:
p(t) =n∑j=1
H(fj(t)− F )Ψj(t) +n∑j=1
(1−H (fj(t)− F )) Ψj(t) + Ωn(t), (3.4)
where F is a frequency cutoff separating high frequency rapid fluctuations from
low frequency long term trends and H is the unit step function defined below:
H(x) =
0, if x < 0.
1, otherwise.
(3.5)
• Applying the idea that low frequency IMFs can express a local mean allows Equa-
tion (3.2) to be rewritten in terms of the original signal minus the low frequency
components, i.e.
P r(t) =n∑j=1
Ψj(t)−M(t), (3.6)
where
M(t) =
n∑j=1
H(1− (fj(t)− F ))Ψj(t). (3.7)
It follows that a convenient approximation to the proxy is given simply by the
IMFs containing only the high frequency components, i.e.:
P r(t) =n∑j=1
H(fj(t)− F )Ψj(t). (3.8)
Chapter 3. Low Frequency Volatility 47
The frequency cutoff F is the only parameter needed for this model, it may be searched
for efficiently using one of many gradient descent methods that are capable of dealing
with noise however for this thesis a simple grid search in the vicinity of the average
frequency of the fourth IMF was sufficient for a proof of concept. Figure 3.1 shows a
typical search for the optimal cutoff frequency using the K-S test. Note how there is a
clear optimal frequency which minimises the K statistic of a K-S test.
If too many IMFs are included in the proxy, then the volatility is over estimated as
the “local mean” covers too wide a swath of time. Conversely if too few IMFs are chosen
for the proxy then the volatility is under estimated as some short term fluctuations are
interpreted as long term trends and disregarded because of an overly tight interpreta-
tion of the local mean. Currently K-S fitting is used to decide on a suitable frequency
cutoff and hence a frequency value for the local mean, however there are many possible
ways to construct a proxy to the returns using the information given by the HHT process.
Figure 3.2 demonstrates how closely the proxy actually does behave like the returns.
Note how visually similar the returns and the proxy is in the upper two plots of Figure
3.2 and how closely the distribution matches as shown by the Quantile-Quantile (QQ)
plot in the lower left and Cumulative Distribution Function (CDF) plot in the lower
right. The proxy selection process outlined in this chapter is merely one possibility and
is not purported to be optimal. All of the approaches to fitting the proxy explored in the
course of the research for this chapter share one common feature, they are all trying to
optimise some fitting criteria between a selection of IMFs and the return series. While
the procedure outlined above happens to optimise the fit between the distributions of the
returns and the proxy to the returns, others could be made to optimise other measures
of fit such as the mean error, mean absolute error, squared returns etc. Indeed some of
the methods could be entirely parameter free which may save on computation time.
Chapter 3. Low Frequency Volatility 48
0 5 10 15 20 250.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22Proxy to Returns KS Fitting
Frequency Cutoff
K V
alue
s
Figure 3.1: Frequency vs K statistic of a K-S test during a typical search for a returnproxy.
3.1.2 Extracting Volatility from the Returns Proxy
In this section the wave like nature of the proxy is exploited to yield a simple estimate
of the volatility. The Hilbert spectral representation of the proxy to the log returns is
given by:
P r(t) = Re
n∑j=1
H(fj(t)− F )Aj(t)ei(ϕj(t))
. (3.9)
where i =√−1. At this point a random phase is introduced in order to extract the
volatility of the system at a point. This is akin to considering each IMF at each point as
just one possibility within a ensemble consisting of waves with similar amplitudes and
frequency but differing phases. A similar approach was taken by Wen and Gu [48] in
order to simulate earthquake data. The uniformly distributed random phase is given by
φ, so we have:
P r(t) = Re
n∑j=1
H(fj(t)− F )Aj(t)ei(ϕj(t)+φ)
. (3.10)
The instantaneous variance of P r(t) is given by:
HHT σ2(t) =1
∆tEφ
((P r(t)− Eφ
(P r(t)
))2), (3.11)
Chapter 3. Low Frequency Volatility 49
2000 2001 2002 2003 2004 2005−0.1
−0.05
0
0.05
0.1Log Returns for NASDAQ Index
Date
Log
Ret
urn
2000 2002 2004 2000 2002 2004−0.1
−0.05
0
0.05
0.1Proxy Returns for NASDAQ Index
Date
Log
Ret
urn
−0.2 −0.1 0 0.1 0.2−0.15
−0.1
−0.05
0
0.05
0.1
0.15
0.2
X Quantiles
Y Q
uant
iles
QQ plot of Proxy against Log Returns
−0.2 −0.1 0 0.1 0.20
0.2
0.4
0.6
0.8
1
x
F(x
)
CDF plots of Proxy and Log Returns
Log ReturnsProxy
Figure 3.2: Actual returns (top left). Proxy to the log returns P (t) (top right). QQplot of the the proxy P (t) against the actual returns (bottom left). CDF plot of the
proxy P (t) against the actual returns (bottom right).
where ∆t is the time between discrete samples. Since Eφ(P r(t)
)= 0, we have:
HHT σ2(t) =1
∆tEφ
Re n∑j=1
H(fj(t)− F )Aj(t)ei(ϕj(t)+φ)
2=
1
∆tEφ
n∑j=1
H(fj(t)− F )Aj(t) cos (ϕj(t) + φ)
2 (3.12)
Note that the IMFs have been described as “orthogonal for all practical purposes” by
Huang Huang et al. [1] due to the frequency sorting nature of the EMD process, so any
Chapter 3. Low Frequency Volatility 50
cross products in the above equation are zero, resulting in the simpler expression:
HHT σ2(t) =1
∆tEφ
n∑j=1
H(fj(t)− F )A2j (t) cos2 (ϕj(t) + φ)
. (3.13)
Recall that the nth central moment is defined by E[(X−E(X))n] =∫ ba (x−µ)nf(x)dx,
where f(x) is the probability density function of the random variable x. For the uniform
random variable φ, f(φ) = 1b−a and since the function is periodic, we can let a = 0 &
b = 2π. Now, taking the expectation of the function given by Equation (3.13), with
respect to the random phase gives:
HHT σ2(t) =1
∆t
∫ 2π
0
n∑j=1
H(fj(t)− F )A2j (t) cos2 (ϕj(t) + φ)
1
2πdφ
=
1
∆t
n∑j=1
H(fj(t)− F )A2j (t)
(ϕj(t) + φ
2+
sin (2(ϕj(t) + φ))
4
)1
2π
2π
0
=1
2∆t
n∑j=1
H(fj(t)− F )A2j (t), (3.14)
which is our local volatility estimate of the log returns.
3.2 Results and Discussion
3.2.1 Data and Testing Methodology
The intraday data for the NASDAQ and All Ords indices was obtained from the SIRCA
database, covering the period from 1st January 2000 to 31st December 2009. The All
Ords Index consisted of 2,958,287 lines of high frequency data and the NASDAQ Index
contained 13,333,424 lines. The text files containing this data were 168 Mb and 745 Mb
respectively. To put the size of these arrays into context, Microsoft Excel is known to
crash and run out of resources after 65,536 lines. When text files grow to this length,
efficient data handling and cleaning the data becomes a necessity.
Chapter 3. Low Frequency Volatility 51
Matlab was used for much of the analysis carried out in this thesis and this program
requires that arrays of data are stored into contiguous RAM memory. Unfortunately
though, even a computer with 16 Gb of RAM memory will usually only have 512 Mb
to 1 Gb of contiguous memory at the best of times, depending on hardware. Worse
still, once the computer has been operational for a short while the memory will begin to
fragment and the size of the largest available contiguous memory reduces quickly. Thus,
care must be taken when handling data that stretches into the millions of lines and it
must be cleaned section by section and split into more manageable formats, i.e. date
strings formatted into date numbers which use less memory, etc.
Once considerations were made for the size of the data, it was cleaned by removing
simultaneous quotations and extraneous data that occasionally filled a cell where price
or date information should be. The data was also sorted into periods of high frequency
and low frequency, such delineations also marked the boundaries of when the market
was considered opened or closed. From these high frequency data sets the daily realized
volatility was extracted and the last value from each trading day was then used as the
daily closing price. Both ten year sets were then split in half and each half analysed
separately because of different market conditions in the first half of the decade as com-
pared to the the latter half, namely the Global Financial Crisis (GFC).
The volatility estimates obtained via the HHT volatility estimate as well as the
EWMA and GARCH(1,1) models are compared against the more accurate but also
more data intensive realized volatility method, allowing error measurements to be ob-
tained and comparisons drawn. The GARCH parameter estimation was done using the
garchfit procedure in Matlab 2010b for each five year period. For the EWMA method,
the exponential decay rate of 0.94 was used because it is popularly referred to in the
literature as the value J.P. Morgan use in their RiskMetrics EWMA volatility model,
see Bauwens et al. [69].
Chapter 3. Low Frequency Volatility 52
Analysis is split up into two parts, firstly the NASDAQ Index covering the period 1st
Jan 2000 - 31st Dec 2004 is analysed qualitatively via the graphical results and discus-
sion. Secondly, Section 3.2.4 provides some numerical error analysis for the four effective
data sets mentioned above.
The comparison methods used to evaluate the validity of the HHT procedure as a
volatility estimate are given by Equations (2.39)-(2.42). Note that the realized volatil-
ity estimate, obtained using intraday log returns, will be used as the true volatility θ2t ,
while the volatility estimate under comparison θt2, uses only daily returns. The stan-
dard K-S test was also used to determine how close the volatility distributions are to
one another. It is important to note that all error measures are only as reliable as the
realized volatility, which is used as the basis for comparison. The RV is often regarded
as a very accurate measure to use when intraday data is available, in this case Equations
(2.10) & (2.14) were used as the RV estimate with 15 minute returns with over night
effects taken into account.
3.2.2 Graphical Volatility Analysis
One apparent feature of the HHT volatility estimate is its high degree of temporal
resolution compared to the other methods examined, i.e. the proposed method adapts
quickly to large changes in volatility. As Figure 3.3 clearly indicates the HHT based
volatility estimate is capturing much of the the same volatility behaviour as the realized
volatility method, all be it with less noise and working with orders of magnitude less
data. This is most evident around mid to late 2000 and early 2001 in Figure 3.3, where
the realized volatility as well as the HHT measure both drop sharply, while the GARCH
and EWMA methods are observed to decay exponentially towards the new volatility
level, resulting in inflated GARCH and EWMA volatility estimates for this period.
The EWMA procedure exhibits similar artefacts to GARCH when the decay factor is
set at the commonly used value of λ = 0.94. This slowly decaying over estimation
can be largely reduced by placing more weight on the most recent returns, however
Chapter 3. Low Frequency Volatility 53
2000 2001 2002 2003 2004 20050
0.2
0.4
0.6
0.8
1
Date
Vol
atili
ty
HHT vs Realized: NASDAQ 1st Jan 2000 − 31st Dec 2004
RealizedHHT
2000 2001 2002 2003 2004 20050
0.2
0.4
0.6
0.8
1
Date
Vol
atili
ty
GARCH vs Realized: NASDAQ 1st Jan 2000 − 31st Dec 2004
RealizedGARCH(1,1)
2000 2001 2002 2003 2004 20050
0.2
0.4
0.6
0.8
1
Date
Vol
atili
ty
EWMA vs Realized: NASDAQ 1st Jan 2000 − 31st Dec 2004
RealizedEWMA
Figure 3.3: HHT based and realized volatility estimates for the NASDAQ data set,1st Jan 2000 - 31st Dec 2004 (top). Corresponding GARCH(1,1) and realized volatilityestimates (middle). Corresponding EWMA and realized volatility estimates (bottom).
this increase in temporal resolution comes at the cost of potentially yielding a noisier
volatility estimate. This relationship between noise and temporal resolution is examined
in more detail in Section 4.2.
Chapter 3. Low Frequency Volatility 54
3.2.3 Further Analysis of the HHT Procedure
The procedure used to generate the returns proxy also has the added benefit of pro-
ducing an interesting local mean as a by-product, this can be seen in Figure 3.4. This
local mean is made up of the information with a frequency too low to be included in the
proxy. Since the EMD procedure was applied to the log price, the exponential of the
running mean eM(t) can be taken to allow for comparison with the original price series,
such a comparison is given in Figure 3.4. While the high frequency part of the signal
makes up our proxy which is needed for our volatility estimates, the lower frequency
elements of the signal may also be of interest to some analysts as they represents a
centered mean that is scalable with time and which differs from that used in regular
Moving Average Convergence/Divergence (MACD) style analysis. Furthermore if the
local mean is defined in terms of the IMF number alone, then MACD style analysis could
be carried out using the EMD procedure. This would generate means that capture more
of inherent dynamics of the underlying system, because those means are made up of
intrinsic modes. Another advantage of the HHT as a local mean estimator is the fact
that it doesn’t appear to suffer from the plateauing effects similar to those described in
Section 2.1.1.
The proposed K-S based procedure for calculating the returns proxy usually resulted
in an optimal cutoff frequency somewhere between three and four IMFs. These IMFs
capture enough stochastic behaviour of the system to yield a reasonable estimate of
the local variance. Figure 3.5 is an alternative representation of the NASDAQ data
displayed in 3.4. Figure 3.5 shows the distinction between the components considered
high frequency (top) and those considered low frequency (bottom). The high frequency
component is effectively the proxy, converted into units of $ and is equivalent to the
difference between the two lines in Figure 3.4. The low frequency component is identical
to the dashed line if Figure 3.4, and forms a measure of the local mean to the original
index price. In this context the HHT is acting like a high pass filter and those combined
high frequency components are effectively the difference from the longer term trend at
that point to the current value thus they can be thought of as an alternative to the
Chapter 3. Low Frequency Volatility 55
2000 2001 20021000
1500
2000
2500
3000
3500
4000
4500
5000
Date
NASDAQ Index Price and HHT Based Mean
Inde
x P
rice
($)
Index PriceHHT Mean
Figure 3.4: NASDAQ Index price for 1st Jan 2000 - 31st Dec 2001 and a measure ofthe local average eM(t).
return series.
The scaling of the highest frequency IMF components by the data at that point was
first done by Huang et al. [47], however they then rectified this signal and used that
as their volatility or ‘variability’ measure described by Equation (2.29). Applying this
method to the NASDAQ data produces Figure 3.6. The problem with this approach is
that it is not only noisy but at each point when the combined IMF components, which
are visible in the upper plot of Figure 3.5, cross the x axis, the volatility measure drops
to zero. This property is undesirable in a volatility estimator since the variance hasn’t
become any smaller at this point, the random walk was just crossing a local average.
Thus its use is limited to that of a volatility indicator as it would be difficult to use as
a practical measure.
The key difference between Huang’s variability measure and the variance measure
proposed in this chapter is the usage of the amplitude information given by the Hilbert
transform on the IMFs. It is these amplitudes that are of value when measuring volatil-
ity since they capture the behaviour of the system without going to zero when an IMF
Chapter 3. Low Frequency Volatility 56
2000 2001 2002−400
−200
0
200
400High Frequency Components: NASDAQ Index
HF
com
pone
nt (
$)
2000 2001 20021000
2000
3000
4000
5000
Date
Low Frequency Components: NASDAQ Index
LF c
ompo
nent
($)
Figure 3.5: The high frequency components which form the proxy (top). The remain-ing components which are a form of local mean (bottom).
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20100
0.05
0.1
0.15
0.2Huang Variability measure
Date
Var
iabi
lity
Figure 3.6: The ‘variability’ measure proposed by Huang applied to the NASDAQdata set.
crosses a local mean and they form the backbone of the proposed HHT volatility esti-
mator. Figure 3.7 depicts the amplitudes of the first 5 IMFs of the NASDAQ data set,
scaled by the data. It can be seen from the figure that the lower frequencies are more
prone to edge effects, like the one visible towards the end of the fifth IMF. These edge
effects are due to the splining procedure used in the EMD process not having an extrema
point near the edge of the now filtered data and could be remedied by the inclusion of
ex post data. Furthermore, these edge effects are rarely of practical importance because
Chapter 3. Low Frequency Volatility 57
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20100
0.05
0.1HHT Amp Scaled by Data
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20100
0.05
0.1
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20100
0.05
0.1
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20100
0.05
0.1
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20100
0.1
0.2
Figure 3.7: Amplitudes of the first 5 IMFs scaled by the data for 10 years of theNASDAQ Index data set.
they have a tendency to effect the lower frequency IMF components which are discarded
in the proxy fitting procedure.
The HHT also yields the frequencies of the IMF components which allows us to look
for any underlying periodic dynamics like weekly, fortnightly and monthly trends. Figure
3.8 displays the frequencies extracted from the same five IMFs covering two years worth
of NASDAQ Index data. The mean frequency in cycles per year of the IMFs in Figure
3.8 were found to be 83.0, 43.5, 24.6, 14.7, 8.7 and 4.3 cycles per year respectively.
The EMD procedure was applied to two years of the NASDAQ Index data from 1st
Jan 2000 - 31st Dec 2001 resulting in nine IMFs and one residue, the first six IMFs are
given in Figure 3.9. From the figure it is evident that the EMD procedure is working
as intended for the financial data, separating it into intrinsic modes with periods of
increasing length as the IMF number increases. Note that the period is increasing by
just under a factor of 2, deviations from this trend or the appearance of a mode with an
unusually high amplitude may indicate the presence of seasonality in the modes. The
fifth IMF in Figure 3.9 may show some signs of seasonality as it has a comparatively
Chapter 3. Low Frequency Volatility 58
5
148HHT Frequency for NASDAQ Jan 2000 − Jan 2002
4
105
3
81
2
49
Fre
quen
cy
1
25
2000 2001 20021
10
Date
Figure 3.8: Frequency information given by the Hilbert transform of the first 6 IMFsfor two years of the NASDAQ Index data set.
high amplitude and a fairly steady quarterly period. Research carried out by Wu and
Huang [36] has put a quantitative measure on the statistical significant of IMFs gener-
ated from Gaussian white noise so that effects attributable to seasonal trends may be
distinguishable from stochastic noise with some confidence. Such research could give an
indication of the statistical significance of any apparent seasonality in financial systems,
such the apparent trends visible in Figure 3.9, however the application of such a measure
is out of the scope of this thesis.
The HHT volatility estimate also has a tendency to capture the distribution of the
realized volatility using only daily data more closely than the alternatives tested. As the
top Quartile-Quartile plots and cumulative frequency diagrams in Figure 3.10 demon-
strate, the distribution of the HHT volatility matches that of the realized more closely
than the EWMA and GARCH procedures. All three estimators shown are light or skinny
tailed when compared to the distribution of the realized volatility, this is expected as
Chapter 3. Low Frequency Volatility 59
−0.07
0.07 Mean Period=4.5 Days
IMFs for NASDAQ Jan 2000 − Jan 2002
−0.05
0.05 Mean Period=8.9 Days
−0.06
0.05 Mean Period=14.7 Days
−0.06
0.06 Mean Period=22.4 Days
−0.06
0.06 Mean Period=36.6 Days
2000 2001 2002−0.11
0.11 Mean Period=86.3 Days
IMF
Date
Figure 3.9: First six IMFs for the log of the NASDAQ Index from 1st Jan 2000 -31st Dec 2001. The average period of each IMF is given in calendar days as opposed to
trading days.
the lower frequency methods under comparison appear as less noisy than the RV method.
This apparent noise can be deceptive, the RV procedure is by all accounts more accu-
rate than the lower frequency procedures however this high accuracy on one time scale
can appear as noise on another. Therefore, the term ‘noise’ should be used with care
and one must be careful not to confuse apparent noise with a lack of smoothing. Note
that due to issues of readability, the log of the volatility was taken for Figure 3.10. The
cumulative frequency diagrams given in Figure 3.11 also indicate how the GARCH and
EWMA estimates deviate from the realized distribution and the close fit of the HHT
procedure for this data set. Note that the EWMA procedure can be tuned to capture
the distribution more accurately by decreasing λ which gives more weight to recent data
and less weight to past volatility estimates.
Chapter 3. Low Frequency Volatility 60
−6 −5 −4 −3 −2 −1 0 1−6
−4
−2
0
2
X Quantiles
Y Q
uant
iles
Log Volatility QQ plot for HHT vs Realized
−4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1 −0.5 0 0.5−6
−4
−2
0
2
X Quantiles
Y Q
uant
iles
Log Volatility QQ plot for EWMA vs Realized
−4 −3.5 −3 −2.5 −2 −1.5 −1 −0.5 0−6
−4
−2
0
2
X Quantiles
Y Q
uant
iles
Log Volatility QQ plot for GARCH vs Realized
Figure 3.10: QQ plots of the volatility distribution for the HHT method (top),EWMA (middle) and GARCH (bottom) for the NASDAQ Index covering the period
1st Jan 2000 - 31st Dec 2004.
Chapter 3. Low Frequency Volatility 61
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
x
F(x
)
CDF plots of HHT and Realized
HHTRealized
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
x
F(x
)
CDF plots of EWMA and Realized
EWMA λ=0.94Realized
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
x
F(x
)
CDF plots of GARCH and Realized
GARCH(1,1)Realized
Figure 3.11: CDF plots of volatility distribution for the HHT method (top), EWMA(middle) and GARCH (bottom) for the NASDAQ Index covering 1st Jan 2000 - 31st
Dec 2004.
Chapter 3. Low Frequency Volatility 62
3.2.4 Numerical Results
The volatility estimators under comparison were evaluated using a number of error mea-
sures which were described in Chapter 2. Since each measure is itself imperfect, each
measure only indicates the performance of a volatility estimator. The error measures
for several volatility measures are summarised in the Tables 3.1-3.4. For the data con-
sidered in Tables 3.1 -3.4 it appears as though the HHT based volatility estimates are
competitive with the EWMA and GARCH(1,1) volatility estimators. The 95% con-
fidence intervals are also given for most error measures in the tables. Note that the
method labelled Fast-HHT is a fully parameter free version of the HHT volatility esti-
mator which is described in Section 4.1. Its results have been included here for later
comparison.
The high accuracy of the RV method at one scale appears as noise against the more
slowly varying methods which use only daily closing prices. This apparent noise in the
more accurate measure which was used as the basis for comparison has led to high
MAPE errors and is a source of noise in the ME, MSE and MAE measures.
The numerical evidence provided by the K-S fit results in Tables 3.2, 3.3 support a
previous claim that for some data sets, the HHT procedure was capable of fitting the
realized volatility distribution more accurately than the GARCH(1,1) or popularly used
EWMA procedure in two of the four data sets that were examined.
3.3 Chapter Conclusion
This chapter has introduced a new and novel method for calculating local historical
volatility which is highly adaptable and likely to seed a variety of new HHT based
volatility measures. Previous attempts at using the HHT for a volatility estimator have
resulted in either volatility indicators like that developed by Huang et al. [47] which are
Chapter 3. Low Frequency Volatility 63
Table3.1:
Err
ors
for
vari
ous
vola
tili
tyes
tim
ato
rsfo
rth
eA
llO
rds
Ind
exco
veri
ng
the
per
iod
1st
Jan
2000
-31s
tD
ec2004
Met
hod
MA
EM
EM
SE
MA
PE
K-S
Fit
EW
MAλ
=0.
94.0
107±
.001
6.0
054±
.0016
9.0
6E
-04±
4.7
5E
-04
64.6981±
5.0042
0.0906
GA
RC
H.0105±
.0015
.0053±
.0016
8.56E-04±
4.38E-04
68.9
140±
5.2
138
0.1
950
HH
T.0
122±
.001
6.0
078±
.0017
9.8
7E
-04±
4.9
6E
-04
69.6
290±
4.1
322
0.1
904
Fas
t-H
HT
*.0
121±
.001
4.0035±
.0015
7.48E-04±
2.65E-04
86.1
083±
7.3
922
0.0432
Table3.2:
Err
ors
for
vari
ous
vola
tili
tyes
tim
ato
rsfo
rth
eN
AS
DA
QIn
dex
cove
rin
gth
e1st
Jan
2000
-31st
Dec
2004
Met
hod
MA
EM
EM
SE
MA
PE
K-S
Fit
EW
MAλ
=0.
94.1
053±
.0129
-.0034±
.0141
.0644±
.0469
66.5
741±
4.9
277
0.1
185
GA
RC
H.1
058±
.0129
-.0042±
.0142
.0650±
.0478
68.9
055±
5.1
517
0.1
257
HH
T.0978±
.0125
.0319±
.0135
.0601±
.0465
58.1922±
3.9149
0.0454
Fas
t-H
HT
*.1
198±
.0128
-.0265±
.0144
.0675±
.0335
75.7
504±
5.2
459
0.0
515
Table3.3:
Err
ors
for
vari
ous
vola
tili
tyes
tim
ato
rsfo
rth
eA
llO
rds
Ind
exco
veri
ng
the
1st
Jan
2005
-31s
tD
ec2009
Met
hod
MA
EM
EM
SE
MA
PE
K-S
Fit
EW
MAλ
=0.
94.0212±
.0025
-.0053±
.0027
2.41E-03±
8.77E-04
78.7745±
5.2724
0.1
449
GA
RC
H.0
220±
.002
6-.
0069±
.0028
2.6
5E
-03±
9.1
6E
-04
87.6
885±
5.8
322
0.2
070
HH
T.0
267±
.002
9-.0032±
.0033
3.5
1E
-03±
1.0
9E
-03
98.0
982±
8.2
663
0.0657
Fas
t-H
HT
*.0
287±
.003
9-.
0097±
.0042
5.7
0E
-03±
2.2
8E
-03
93.4
691±
7.1
633
0.1
250
Table3.4:
Err
ors
for
vari
ous
vola
tili
tyes
tim
ato
rsfo
rth
eN
AS
DA
QIn
dex
cove
rin
gth
e1st
Jan
2005
-31st
Dec
2009
Met
hod
MA
EM
EM
SE
MA
PE
K-S
Fit
EW
MAλ
=0.
94.0
357±
.004
4.0012±
.0049
7.6
6E
-03±
3.5
0E
-03
67.0
557±
8.3
383
0.0948
GA
RC
H.0337±
.0043
.0030±
.0047
7.15E-03±
3.52E-03
67.8
501±
7.8
068
0.1
695
HH
T.0
391±
.004
9.0
215±
.0052
9.1
8E
-03±
3.9
7E
-03
64.6630±
6.2877
0.1
645
Fas
t-H
HT
*.0
424±
.005
9-.
0063±
.0063
1.2
9E
-02±
6.1
9E
-03
75.3
533±
7.1
483
0.0409
Chapter 3. Low Frequency Volatility 64
difficult to interpret as they are not in terms of variance or in the case of Wen and Gu
[48] were not fully explored.
For the first time the HHT has been used to give volatility estimates in terms of
the local variance which is easily understood and, as the the tables in Section 3.2.4
show, the degree of accuracy is favorably comparable to the popularly used EWMA and
GARCH(1,1) methods.
As Section 3.2.2 shows, for the data set considered, the HHT volatility estimate does
not appear to suffer from the same kinds of artefacts which can render GARCH and
EWMA unreliable after sharp increases or decreases in volatility. The reason for this
apparent advantage of the HHT volatility estimate is the high temporal resolution of
the HHT procedure which underpins it.
From the numerical evidence provided here there is no clear winner in terms of
overall performance. All three methods are fitting the RV to a similar degree of success.
What makes an assessment of performance difficult is the absence of a “true” account of
volatility, with only the RV standing in as a proxy. In comparison to methods which rely
on daily data, the RV volatility measure can be perceived as noisy, this brings one back
to a question posed in the introduction; which changes in volatility can be described as
noise and which can be attributed to legitimate changes in variance? There is no one
answer, what is considered the best volatility estimate appears to be dependent on the
the intended use of the estimator. This apparent noise of the RV method has led to
relatively large standard errors and consequently quite broad 95% confidence intervals.
Thus, care must be exercised when judging the performance of volatility estimators that
have access to only daily data against the those measure which uses intraday data. This
is again a matter of scale, while the former estimators might be suitable for weekly
forecasts, the latter is better suited to daily forecasts
.
Chapter 3. Low Frequency Volatility 65
Note that while there are many different possible implementations of the GARCH,
EWMA or RV procedures, there is also a growing number of ways to implement the
HHT process. One such variant of the HHT process proposed by Hu [70] forces IMFs
into being completely orthogonal by a process known as the Orthogonal Hilbert-Huang
Transform (OHHT). This OHHT process was implemented and tested over the course
of this chapter but results varied so little from those obtained by the standard HHT
procedure that they were omitted from the results and further discussion. This close
resemblance between the HHT and the OHHT are not surprising since, as previously
described the IMFs may be considered orthogonal for all practical purposes.
The volatility estimator proposed in this chapter also has much scope for development
as it is based on the highly flexible HHT procedure. Applications of this method also
include the analysis of unevenly spaced time series data as the HHT has no requirement
for evenly spaced data. As briefly explored in Section 3.2.3, the EMD procedure also
gives information which may be used for seasonality analysis and MACD style technical
analysis. Further work has also been done towards making the HHT volatility procedure
completely parameter free for greater simplicity and computational efficiency. The first
such method which was seeded by the work first undertaken in this chapter is denoted
Fast-HHT in Tables 3.1-3.4, this method is parameter free and therefore much faster, it
forms a core component in the analysis which is to follow.
Chapter 4
High Frequency Simulated Data
and Microstructure Noise
In this chapter an improved version of the HHT volatility estimator is proposed and
evaluated. Its simpler and parameter free nature make it computationally more efficient
than the variant proposed in Chapter 3. This faster, simpler approach has enabled a
Monte Carlo study and allowed for further refinements which can even handle market
microstructure noise. In this chapter, two simulated studies are carried out, the first
compares the improved HHT volatility estimator against the EWMA procedure and the
second tests a variant of the HHT procedure in the presence of market microstructure.
4.1 Improved HHT Volatility Estimator
In this section the wave like nature of the IMFs is exploited to yield a simple estimate of
the volatility. Its main advantage over the procedure which was outlined in Chapter 3,
is that it requires no parameter fitting at all, making it substantially faster and able to
be investigated more thoroughly through detailed Monte Carlo simulations. The main
difference is that there is no need to create a proxy to the returns as the procedure is
applied directly to the log returns.
The HHT procedure can be used to give the Hilbert spectral representation of the
66
Chapter 4. HF Simulated Data and Microstructure 67
log returns as:
r(t,∆t) = Re
n∑j=1
Aj(t,∆t)eiϕj(t,∆t)
, (4.1)
where ∆t is the period between returns. Note that IMFs Ψj(t) are actually continuous
because of the splining nature of the EMD algorithm and that discrete returns can be
seen as a subset of these continuous functions. Also, the zero centered nature of the
IMFs implies that all of them satisfy Et (Ψj(t)) = 0, however this does not hold true for
the final residue Ωn(t).
Now the concept a random phase is introduced in order to extract the volatility of
the system at any point. This is akin to considering each IMF at each point as just
one possibility within a virtual ensemble consisting of waves with similar amplitudes
and frequency but differing phases. A similar approach was taken by Wen and Gu [48]
in order to simulate earthquake data, however the focus was on simulating different
volatility levels rather than estimating them. The uniformly distributed random phase
is given by φ, so we have:
r(t,∆t, φ) = Re
n∑j=1
Aj(t,∆t)ei(ϕj(t,∆t)+φ)
, (4.2)
where r(t,∆t, φ) represents a whole ensemble of waves. The instantaneous variance of
r(t,∆t, φ) is given by:
σ2(t,∆t) =1
∆tEφ
((r(t,∆t)− Eφ (r(t,∆t)))2
). (4.3)
Note that when IMFs are constructed using log return data, the IMF volatility drops
off exponentially with the IMF number, so any long term trends contained in the last
residue can be safely ignored for the purposes of this volatility analysis. Now we have:
σ2(t,∆t) =1
∆tEφ
Re n∑j=1
Aj(t,∆t)ei(ϕj(t,∆t)+φ)
2=
1
∆tEφ
n∑j=1
Aj(t,∆t) cos (ϕj(t,∆t) + φ)
2 . (4.4)
Chapter 4. HF Simulated Data and Microstructure 68
Note that the IMFs have been described as “orthogonal for all practical purposes” by
Huang et al. [1] due to the frequency sorting nature of the EMD process, so any cross
products in the above equation are zero, resulting in the simpler expression:
σ2(t,∆t) =1
∆tEφ
n∑j=1
A2j (t,∆t) cos2 (ϕj(t,∆t) + φ)
. (4.5)
Now, taking the expectation with respect to the uniform random phase between 0 and
2π gives:
σ2(t,∆t) =1
∆t
∫ 2π
0
n∑j=1
A2j (t,∆t) cos2 (ϕj(t,∆t) + φ)
1
2πdφ
=
1
2∆t
n∑j=1
A2j (t,∆t), (4.6)
which is our instantaneous and parameter free volatility estimate for the log returns.
Note that in a similar fashion to the way in which sparse and average RV estimates
were described in Chapter 2, the sparse HHT estimator is described as Equation (4.6)
with sample spacing being some integer multiple of the smallest available step size, i.e.
∆t = n∆t and for ease of notation let 1∆t be a positive integer representing the number
of sparse subsamples in one period. The average HHT can then be described by a mean
of sparse estimates with different offsets, i.e. each offset is given by:
offsetHHTσ2(t,∆t, k) =1
2∆t
n∑j=1
A2j (t+ k∆t,∆t). (4.7)
where k = 0, . . . , n − 1. Then the average HHT based estimator is given by a sum of
these sparse estimations as:
AverageHHTσ2(t,∆t) =1
n
n−1∑k=0
offsetHHTσ2(t,∆t, k) (4.8)
The estimate described by Equation (4.6) was used to give the values described as “Fast-
HHT” in Section 3.2.4. The numerical results given in Chapter 3 support the claim that
the algorithm described in this section is as accurate, if not more accurate than the
Chapter 4. HF Simulated Data and Microstructure 69
procedure described in Chapter 3. The main advantage is the the method described by
Equation (4.6) requires no fitting of a proxy to the returns and is therefore completely
parameter free and therefore much faster to run. This advantage of speed has enabled
more specialised versions of the HHT estimator to be developed which are comprised
of multiple estimates such as that described by Equation (4.8) and later the Filtered-
HHT procedure which can deal with market microstructure. It should also be noted
that, while the Fast-HHT procedure is much faster than the proxy dependent variant
described in Chapter 3, it is still substantially more complex, and thus slower than a
simple realized volatility measure.
4.2 Volatility Comparison Test
The results given in Chapter 3, particularly the graphical volatility analysis of Figure
3.3 appear to show that the HHT is capable of maintaining low levels of noise while also
adapting to changes in volatility more rapidly than the popular EWMA and GARCH
volatility methods. In the absence of any test to verify these claims in a straightforward
manner, one has been developed specifically to quantify this kind of behaviour. As
discussed in Section 2.1.2, there is often a compromise that must be made between the
ability of a method to capture short term or long term trends. This test seeks to quantify
the previous statement by examining this tradeoff in more detail.
4.2.1 Test Design
The test consists of two main parts, the calculation of the Signal to Noise Ratio (SNR)
and the response time.
I Calculate the Signal to Noise Ratio
• A Geometric Brownian Motion (GBM) time series of length m is simulated
with constant volatility, i.e. σ2(t) = c for t = 1, . . . ,m.
Chapter 4. HF Simulated Data and Microstructure 70
• Competing instantaneous volatility methods are used to approximate the volatil-
ity over the length of the time series. The estimates are denoted σ2(t) for
t = 1, . . . ,m.
• The SNR is then calculated using:
SNR =σ2(t)
1m
∑mt=1 (|σ2(t)− σ2(t)|)
(4.9)
This should then be repeated many times for each variable parameter within
the estimation method, i.e. λ in the case of EWMA.
II Determine the Response Time
• Secondly, a GBM time series is simulated with a distinct step in volatility mid
way through the data, i.e. σ2(t) = c for t = 1, . . . ,m/2 which then switches to
σ2(t) = c/2 for t = m/2 + 1, . . . ,m. The time it takes for competing volatility
estimate to approach to within a threshold (within ±20% of c/2 was used for
this research) of the new volatility level is then recorded. As before, this step
should be repeated for different values of any parameters in the estimation
method.
III Compare the Signal to Noise Ratio to the Response Time.
• A useful comparison can be made by graphing the SNR vs response time results
across across each variable parameter, this procedure produced Figure 4.3.
Note that the EWMA estimator gives results which can vary over λ and so the
EWMA estimator yields a curve as its properties change with this parameter.
On the other hand, the HHT procedure is parameter free and so the SNR vs
response plot for this method yields only a single point for comparison.
4.2.2 Test Results
Figure 4.1 shows how the EWMA procedure adjusts to the sharp drop in volatility levels
for several values of the parameter λ. The estimator can be seen to approach the new
volatility level more slowly as the decay rate gets closer to one. Figure 4.2 selects one
Chapter 4. HF Simulated Data and Microstructure 71
0 50 100 1500.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22
Time
Vol
atili
ty
Response Test for the EWMA Procedure
λ=0.1λ=0.86λ=0.98
Figure 4.1: EWMA Response profiles for three values of λ.
of the EWMA estimates, corresponding to λ = 0.67 and compares it against the HHT
volatility estimator. The value of λ = 0.67 was chosen because it has a similar SNR
to the HHT method. The HHT volatility estimate descends towards the new volatility
level faster than the EWMA estimate. Note that the HHT volatility estimate actually
dips before the actual volatility drops, this is because the EMD process which underpins
the HHT procedure uses cubic splines and ex-post data, it is in fact an artefact and does
not indicate a hidden predictive ability.
Importantly, Figure 4.3 shows that this single point produced by the HHT procedure
is below the line formed by considering all of the parameter choices for the EWMA pro-
cedure. This implies that the HHT volatility estimator (4.6) is adapting to new volatility
levels faster, while maintaining a lower SNR than the EWMA procedure under test con-
ditions, which is the claim that this test was designed to investigate.
Chapter 4. HF Simulated Data and Microstructure 72
0 50 100 1500.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22
Time
Vol
atili
ty
Response Test for the EWMA & HHT Methods
HHT VolEWMA λ=0.67Actual
Figure 4.2: Response profiles for HHT estimator & EWMA with λ = 0.67
4.3 The Filtered-HHT Volatility Estimator for Microstruc-
ture Noise
The HHT volatility estimate can also provide consistent estimates even in the presence
of market microstructure noise. This is achieved by effectively filtering off the higher
frequency components which constitute the unwanted noise. The main advantage of
the HHT procedure over any of the RV techniques is that it provides an instantaneous
volatility measure so that finer scale volatility characteristics are revealed. Note that
this method of filtering off excess noise is only suitable for the kind of microstructure
which results in positive anomalous scaling of volatility, such as the kind of noise found
in high frequency FX markets.
Chapter 4. HF Simulated Data and Microstructure 73
2 4 6 8 10 12 14 16 180
10
20
30
40
50
60
70
80
90
100
Signal−Noise Ratio
Res
pons
e T
ime
EWMA and HHT Signal−Noise vs Response Time Comparison
EWMAHHT
Figure 4.3: SNR vs response time for EWMA with 0.1 ≤ λ ≤ 0.98 (curve) and theHHT volatility estimate (circle).
4.3.1 Adapting the HHT Estimator to the High Frequency Realm
The procedure to calculate a consistent high frequency volatility estimator based on the
HHT method is as follows:
1. EMD Step:
Apply the EMD procedure to the observed log price series p, note that the pro-
cedure works equally well when applied to the regular price series. The IMFs are
then passed to the next step. Also, recall that in Chapter 3 the observed price
was given by p, however this was the low frequency case where market frictions
are negligible and the frictionless price is effectively observable. In this chapter
the observed log price which includes market microstructure is denoted as p, with
the frictionless p being unobservable.
2. Filtration Step:
If this is the first iteration of the filtration step, skip it and pass all IMFs through
to the next step. Otherwise, pass on the residue and all but the highest frequency
Chapter 4. HF Simulated Data and Microstructure 74
IMF to the next step. This step is essentially using the EMD process a low pass
filter.
3. Volatility and Averaging Step:
Now, using the IMFs passed on from the previous step, the HHT volatility mea-
sure described by Equation (4.8) is used at various sampling rates. For exam-
ple, in this chapter the simulated time series was sub-sampled at ∆t(1, . . . , 12) =
[0.4, 0.6, 0.8, 1, 2, 3, 4, 5, 6, 7, 8, 9] minute intervals and for convenience, the time
scale for one period is 24 hours, or 1440 minutes so that the daily sample rate
is given by: fs(j) = 1440∆t(j) , j = 1, . . . , 12. The sample rates should be chosen so
that the they capture both high frequency inflationary noise as well as some sparse
samples which are relatively unaffected by market microstructure. Note that the
method proposed above gives the spot volatility at each sample point in time and
for each sample rate, which results in 12 volatility estimates spanning the length
of the simulation for each filtration. However in the comparisons to follow, we
wish to compare this instantaneous or spot volatility to a measure of integrated
volatility and so the mean of the spot volatility should be taken over a length
comparable to that in the integrated volatility procedure (one day). Let this mean
or integrated volatility at sample rate fs(j), day t and filtration step i be denoted
by HHTIV (t, fs(j), i).
4. Iteration Step:
Now, iterate the previous two steps until only the residual of the signal is left.
5. Optimisation Search Step:
• Calculate the mean volatility with respect to time at every sample rate and
filtration step. This mean volatility is given by:
meanHHTIV (fs(j), i) =1
∆t(j)
1/∆t(j)∑t=1
HHTIV (t, fs(j), i). (4.10)
This provides a good indication of overall inflation when compared to means
at other sample rates and filtration steps.
Chapter 4. HF Simulated Data and Microstructure 75
• While one approach would be to simply determine which filtration step yields
the most consistent volatility estimates over multiple sample rates, experience
from Chapter 3 has taught that filtering by whole IMFs can be too coarse a
filter. One alternative to this coarse filter approach is a finer frequency filter,
such as the step function approach taken by chapter 3. An alternative, which
is used during this chapter, is to use a linear combination of under filtered
and over filtered volatility estimates to yield the final result.
More specifically, mean volatility measures at neighboring filtration steps
(i − 1 & i) are linearly combined in such a way as to maximise the flatness
of the combined volatility profile over all sample rates. Note that depar-
tures from flatness in the volatility profile are quantified by the gradient of
meanHHTIV (fs(j), i) with respect to changes in fs. Let the gradient at each
sample rate fs(j), j = 1, . . . , k and filtration step i, be defined using simple
numerical differentiation as:
Γ(fs(j), i) =
meanHHTIV (fs(j+1),i)−meanHHTIV (fs(j),i)
fs(j+1)−fs(j) for j = 1,
meanHHTIV (fs(j+1),i)−meanHHTIV (fs(j−1),i)fs(j+1)−fs(j−1) for j = 2, . . . , k − 1,
meanHHTIV (fs(j−1),i)−meanHHTIV (fs(j),i)fs(j)−fs(j−1) for j = k.
(4.11)
• The idea now is to search for a linear combination of neighboring filtration
steps which give the minimum absolute gradient sum, i.e. the cost function
f(C) is minimised by searching for an optimal C (for a proof of concept a
simple grid search was used).
f(C) =k∑j=1
|(C)Γ(fs(j), i) + (1− C)Γ(fs(j), i− 1)| , (4.12)
where 0 ≤ C ≤ 1 and k is the number of sample rates.
6. Calculation of Volatility Step:
Finally, once the pair of neighboring IMFs which lead to the flattest volatility
profile have been found, the filtered volatility estimate is given by a linear combi-
nation of these over filtered and under filtered estimates at various sampling rates
Chapter 4. HF Simulated Data and Microstructure 76
as:
σ2(t, fs(j)) = (C)HHTIV (t, fs(j), i) + (1− C)HHTIV (t, fs(j), i− 1). (4.13)
This procedure is best understood with the aid of Figure 4.4, depicting the character-
istic volatility profile or volatility signature plot for the Two-Factor Affine model at
the γ = 0.1 microstructure level. This figure demonstrates how changes in sample rate
can effect volatility estimates. In this figure the flattening of the blue lines indicate
clear that successive applications of the filtration process are resulting in a reduction of
upwards bias at shorter sampling intervals. The relative flatness of the Filtered-HHT
volatility estimation is also evident in Figure 4.4. This demonstrates that the goal of
a consistent estimator across different sample rates is also being achieved. Since the
proposed estimate yields volatility estimates across several different sampling rates, the
best result is obtained by averaging across several of these estimates. Further, the figure
also shows why this procedure can be considered as a combination of under and over
filtered components.
Ideally, taking the average over every sample rate would be optimal, however due to
occasional over filtering at the fastest sample rates it was found that averaging over the
flattest region of the volatility signature plot provided the most consistent results. This
was found to be around the last 5 sampling rates and this is the procedure that was
used in Section 4.4.2.
4.4 Monte Carlo Analysis
In this section, Monte Carlo analysis is used to test the performance of the Filtered-HHT
volatility estimation procedure proposed above. Following the models used by Andersen
et al. [53] and Barucci et al. [54] in similar investigations of market microstructure, a
log price with frictions is considered. Specifically, the underlying price model follows
Equation (1.3): dpt = σtdW(1)t with additive microstructure noise given by Equation
Chapter 4. HF Simulated Data and Microstructure 77
0 1 2 3 4 5 6 7 8 90
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8Characteristic Volatility Profile
Sample interval in minutes (simulated)
Dai
ly V
olat
ility
Successive filtrationsOptimised
Figure 4.4: Volatility estimates for various sample rates across different levels offiltration. The ‘o’ symbol denotes the estimate provided by Equation (4.13).
(1.6): pt = pt + ut, where ut is IID noise with mean zero and the variance is propor-
tional to the expected volatility for the frictionless log price, i.e. V ar[u(t)] = γE[IVt]
with γ = 0%, 0.1%, 0.5%. The observable price is then given by Equation (2.15):
rt(h) ≡ pt − pt−h.
The volatility, σt in Equation (1.3) is then set to one of three models:
• Model I: GARCH
dσ2t = k(θ − σ2
t )dt+ cGσ2t dW
(2)t , (4.14)
with k = 0.6, θ = 0.636 and cG = 0.1439, noting that E[IVt] = θ. Note that this
model has a strong mean reversion for this choice of parameters.
Chapter 4. HF Simulated Data and Microstructure 78
• Model II: Two-Factor Affine
σ2t = σ2
1,t + σ22,t (4.15)
dσ2i,t = ki(θi − σ2
i,t)dt+ ciσ2i,tdW
(i,2)t , (4.16)
with k1 = 0.5708, k2 = 0.0757, θ1 = 0.3257, θ2 = 0.1786, c1 = 0.2286, c2 = 0.1096,
noting that E[IVt] = θ1 + θ2. This model has a very volatile first factor with
strong mean reversion, and a less volatile component which is only weakly mean
reverting.
• Model III: Log-Normal Diffusion
d log σ2t = k(θ − log σ2
t )dt+ cLNdW(2)t , (4.17)
with k = 0.0136, θ = −0.8382 and cLN = 0.1148, noting that E[IVt] = eθ+cLN/4k.
All three of these models are Ornstein-Uhlenbeck process, they are widely used in the
literature to simulate market microstructure noise.
4.4.1 Spectral Analysis and Hurst Parameter Estimation
Before we delve into volatility analysis, some of the tools that were outlined in Chapter
2 are revisited, specifically, the spectral structures and Hurst exponents are examined
for the three stochastic volatility models described above.
One of the most powerful tools available in spectral analysis is the Fast Fourier Trans-
form (FFT). Using the FFT it is possible to transform a signal from the time-amplitude
domain into the frequency-power domain. When the EMD procedure is performed first,
a FFT can be applied to each of the IMFs separately, this procedure was undertaken for
the three models described above. The simulation ran for 250 virtual trading days with
one trade every 10 seconds for six hours giving 2160 trades each day and was repeated
at three microstructure noise level, namely 0%, 0.1% and 0.5%. The resulting spectra
Chapter 4. HF Simulated Data and Microstructure 79
are shown in Figure 4.5. The first thing to notice is that with zero noise, the spectra are
flat. This is consistent with a Gaussian white noise process with stochastic volatility.
When a small amount of microstructure noise of γ = 0.1% is introduced, it becomes
evident that there is an increase in the power of the IMFs with higher frequencies. This
effect becomes more pronounced as the noise is increased to γ = 0.5%, where the high
frequency IMFs are of considerably higher power in all three models.
One can not help but notice that the IMF power spectra plots of Figure 4.5 bear a
striking similarity to those produced by the examination of a fractional Gaussian noise
process as seen in Figure 2.9, namely the ones withH < 0.5. This link has been examined
in more detail by applying the EMD procedure for calculating the Hurst exponent for
all three models at the same noise levels as above, the results are given in Table 4.1.
As hinted by the spectral information in Figure 4.5, the Hurst exponents do indeed fall
below H = 0.5 and as the noise parameter γ increases the H estimates decrease.
Table 4.1: Hurst exponent estimates from the EMD procedure and their associatedstandard deviations for the three models at three noise levels.
γ = 0% Std γ = 0.1% Std γ = 0.5% Std
H GARCH 0.5186 0.0518 0.3428 0.0473 0.2980 0.0477
H Two-Factor Affine 0.5242 0.0505 0.4011 0.0508 0.3217 0.0517
H Log Normal Diffusion 0.5210 0.0543 0.3497 0.0499 0.3096 0.0498
4.4.2 Results and Comparisons
In this section the Monte Carlo models described above are used to generate 250 sim-
ulated trading days, each of duration 6 hours with one trade every second. Firstly, to
show the effects of even low levels of market microstructure on a volatility estimate,
the RV was calculated at different levels of daily sampling rates 1/h (available in Table
4.2) and the result is displayed in the volatility signature profile Figure 4.6. From this
one can observe that even a small amount of market microstructure noise can dominate
volatility estimates at high sampling rates. This emphasises the importance of using a
volatility estimator that is specifically designed to handle such inflationary noise.
Chapter 4. HF Simulated Data and Microstructure 80
24
68
1012
−30
−28
−26
−24
−22
−20
Log 2(F
requ
ency
)
Log PSD (dB)G
AR
CH
Spe
ctra
, 0%
Noi
se
24
68
1012
−30
−28
−26
−24
−22
−20
Affi
ne S
pect
ra, 0
% N
oise
Log 2(F
requ
ency
)2
46
810
12−
30
−28
−26
−24
−22
−20
Log 2(F
requ
ency
)
Log
Spe
ctra
, 0%
Noi
se
24
68
1012
−30
−28
−26
−24
−22
−20
−18
Log 2(F
requ
ency
)
Log PSD (dB)
GA
RC
H S
pect
ra, 0
.1%
Noi
se
24
68
1012
−30
−28
−26
−24
−22
−20
−18
Affi
ne S
pect
ra, 0
.1%
Noi
se
Log 2(F
requ
ency
)2
46
810
12−
30
−28
−26
−24
−22
−20
−18
Log 2(F
requ
ency
)
Log
Spe
ctra
, 0.1
% N
oise
24
68
1012
−30
−25
−20
−15
Log 2(F
requ
ency
)
Log PSD (dB)
GA
RC
H S
pect
ra, 0
.5%
Noi
se
24
68
1012
−30
−25
−20
−15
Affi
ne S
pect
ra, 0
.5%
Noi
se
Log 2(F
requ
ency
)2
46
810
12−
30
−25
−20
−15
Log 2(F
requ
ency
)
Log
Spe
ctra
, 0.5
% N
oise
Figure 4.5: Spectral analysis of IMFs from GARCH, Affine and Log volatility models.Top row: no microstructure noise. Middle row: 0.1% noise. Bottom row: 0.5% noise.
Note that the vertical axis is in terms of the Power Density Spectra (PDS)
Chapter 4. HF Simulated Data and Microstructure 81
Table 4.2: Sampling rate and equivalent sampling intervals in seconds.
1/h 2160 1440 720 360 288 96 48 24 12 6Seconds 10 15 30 60 75 225 450 900 1800 3600
2 3 4 5 6 7 8 90
0.5
1
1.5
2
2.5
3
3.5
4RV Volatility Signature Plot 0.1% Noise
log Sample Interval (seconds)
Vol
atili
ty
M1M2M3
Figure 4.6: Volatility signature plot at 0.1% noise. M1 M2 M3 refers to the threevolatility models, GARCH, Two-Factor Affine and Log-Normal diffusion respectively.
One of the benefits of using simulated data is that the actual frictionless volatility
of each process is known throughout the simulation and so a simple error analysis using
the procedures outlined by Equations 2.39- 2.41 in Chapter 2 is all that is needed to
determine the accuracy of each volatility measure. One year of simulated data at the
high frequency rate of one point per second is generated for three models at two noise
levels, γ = 0.1% and γ = 0.5%. From this the mean error, mean square error and mean
absolute errors are calculated for each volatility measure over the course of the simulated
year and the results are given in Tables 4.3 − 4.8. Several RV-based volatility estimation
methods were included in this study to give a good basis of comparison to the proposed
HHT method. As a guide to understand what variant of each method was implemented,
a short description of each follows:
• RV all
This will serve as a baseline for a poor measure in the presence of microstructure
Chapter 4. HF Simulated Data and Microstructure 82
noise. It is defined as the RV given by Equation (2.7) when the highest available
frequency of data is used. This procedure is not designed to be used under the
conditions of market microstructure.
• RV averaged
As described by Equation (2.18), this volatility estimate is simply a sparsely sam-
pled RV measure which has been averaged over many offsets. The sparse sampling
was done at a rate of one sample every 5 minutes. This procedure should have
some resistance to market microstructure.
• TSRV
This is the first real measure designed specifically to deal with market microstruc-
ture, defined by Equation (2.21)
• TSRV adj
Short for Adjusted−TSRV . A slight improvement on TSRV that adjusts for sam-
ple size difference which make up the two components in the summands, defined
by Equation (2.21)
• HHT averaged
This is the HHT equivalent of RV averaged and is the average of many offset HHT
volatility estimates sampled 5 minutes apart, defined by Equation (4.8)
• HHT filtered
This is the volatility estimate proposed in Section 4.3.1 and is defined therein.
The mean error results given in Tables 4.3 & 4.6 show that the Filtered-HHT method
is yielding similar results to the Adjusted-TSRV procedure, with each achieving the low-
est mean error 3 of 6 times. The good performance of the Filtered-HHT procedure in
this test is most likely due to the averaging process over several sample rates, since
the average of several unbiased estimators is likely to have a lower bias than each es-
timate individually. A variant of the TSRV procedure that does use an average over
many sample rates is known as the MSRV and was proposed by Zhang [71], this proce-
dure is likely to have a very low bias for the kind of simulated data tested in this chapter.
Chapter 4. HF Simulated Data and Microstructure 83
Table4.3:
Mea
ner
ror
an
aly
sis
atγ
=0.
1%
nois
e.
Mod
elRVall
RVaverage
TS
RV
TSRVadj
HHTaverage
HHTfiltered
GA
RC
H-1
0.9
6±
.0493
-0.3
552±
.011
90.
0320±
.011
90.0106±
.0123
-0.3
744±
.026
0-0
.026
4±
.018
4
AF
F-2
.385±
.0125
-0.0
641±
.009
40.
0334±
.009
40.
0160±
.009
6-0
.056
1±
.013
50.0110±
.0112
LO
GN
-9.5
20±
.0429
-0.3
061±
.005
60.
0213±
.005
70.
0116±
.005
8-0
.309
2±
.015
3-0.0073±
.0082
Table4.4:
Mea
nsq
uare
erro
ran
aly
sis
atγ
=0.
1%
nois
e.
Mod
elRVall
RVaverage
TS
RV
TSRVadj
HHTaverage
HHTfiltered
GA
RC
H120
.3±
1.0
873
0.13
53±
.008
90.
0102±
.001
80.0099±
.0018
0.18
36±
.023
10.
0225±
.003
8
AF
F5.
697±
.059
60.
0097±
.001
70.
0068±
.001
10.0062±
.0011
0.01
49±
.003
20.
0082±
.001
8
LO
GN
90.7
5±
.817
90.
0958±
.003
50.
0025±
.000
50.0023±
.0005
0.11
07±
.010
60.
0044±
.000
9
Table4.5:
Mea
nab
solu
teer
ror
an
aly
sis
atγ
=0.
1%
nois
e.
Mod
elRVall
RVaverage
TS
RV
TSRVadj
HHTaverage
HHTfiltered
GA
RC
H10
.96±
.049
30.
3552±
.011
90.
0794±
.007
70.0786±
.0076
0.37
51±
.025
80.
1223±
.010
8
AF
F2.
385±
.012
50.
0784±
.007
50.
0654±
.006
20.0632±
.0059
0.09
27±
.009
90.
0706±
.007
0
LO
GN
9.52
0±
.042
90.
3061±
.005
60.
0382±
.004
10.0364±
.0039
0.30
94±
.015
30.
0505±
.005
4
Chapter 4. HF Simulated Data and Microstructure 84
Table4.6:
Mea
ner
ror
an
aly
sis
atγ
=0.
5%
nois
e.
Mod
elRVall
RVaverage
TS
RV
TSRVadj
HHTaverage
HHTfiltered
GA
RC
H-5
5.1
1±
.2485
-1.7
944±
.014
10.
0629±
.012
90.0441±
.0133
-1.8
555±
.062
5-0
.059
9±
.014
4
AF
F-1
1.9
0±
.0540
-0.3
764±
.008
30.
0352±
.008
30.
0209±
.008
5-0
.364
4±
.019
5-0.0041±
.0122
LO
GN
-47.6
5±
.2129
-1.5
427±
.015
90.
0701±
.014
70.0471±
.0152
-1.5
703±
.057
8-0
.050
2±
.018
8
Table4.7:
Mea
nsq
uare
erro
ran
aly
sis
atγ
=0.
5%
nois
e.
Mod
elRVall
RVaverage
TS
RV
TSRVadj
HHTaverage
HHTfiltered
GA
RC
H30
41±
27.4
010
3.23
3±
.050
80.
0146±
.002
30.0133±
.0022
3.69
35±
.247
20.
0169±
.002
9
AF
F14
1.7±
1.28
300.
1461±
.006
50.
0057±
.000
90.0051±
.0008
0.15
72±
.016
00.
0096±
.001
9
LO
GN
2273±
20.3
422
2.39
6±
.049
50.
0188±
.003
10.0171±
.0030
2.68
01±
.202
10.
0252±
.006
0
Table4.8:
Mea
nab
solu
teer
ror
an
aly
sis
atγ
=0.
5%
nois
e.
Mod
elRVall
RVaverage
TS
RV
TSRVadj
HHTaverage
HHTfiltered
GA
RC
H55
.11±
.248
51.
794±
.014
10.
0982±
.008
80.0935±
.0084
1.85
55±
.062
50.
1039±
.009
8
AF
F11
.90±
.054
00.
3764±
.008
30.
0625±
.005
30.0590±
.0051
0.36
53±
.019
20.
0777±
.007
4
LO
GN
47.6
5±
.212
91.
543±
.015
90.
1104±
.010
10.1035±
.0100
1.57
03±
.057
80.
1204±
.012
9
Chapter 4. HF Simulated Data and Microstructure 85
The mean square error and mean absolute error information given in Tables 4.4, 4.5,
4.7 & 4.8 show that the Filtered-HHT procedure is a great improvement over the RV and
unfiltered HHT processes, however it appears to slightly under performing in this area
compared to the TSRV and its adjusted variant. It is worth pointing out though, while
the errors are larger than the TSRV-based methods they are still relatively small, they
are of the same magnitude and usually only slightly larger. This suggests that while
the Filtered-HHT method is non biased, it does vary further from the true volatility
more often than its counterparts, although it does so almost equally both above and
below the true volatility. This can be seen in Figure 4.7 and Figure 4.8 which show the
volatility estimates and the true model volatility for all three models and the two noise
levels of γ = 0.1% and γ = 0.5%. Also Figures 4.7 & 4.8 provide a clearer perspective of
how closely the Filtered-HHT procedure and the Adjusted-TSRV procedures are to one
another as well as the model’s true volatility. While there is a small difference between
the two estimates, both methods are capturing much of the model volatility with some
accuracy.
The errors in Tables 4.3 - 4.5 indicate the 95% confidence intervals for each measure.
A larger simulation could have been performed to reduce the uncertainty around these
errors, however due to the high computational load involved in repeatedly performing
the the EMD algorithm for each Filtered-HHT estimate, the simulation was kept to one
simulated year, with trades occurring every 10 seconds. The Filtered-HHT procedure
is not currently optimised for computational efficiency however in its current form it is
sufficient for a proof of concept, as was the size of the simulation carried out.
Such results are promising, as the TSRV and Adjusted-TSRV methods were designed
to deal with the specific kind of additive microstructure noise which was examined in
this section. The Filtered-HHT procedure on the other hand, made no assumptions on
the kind of microstructure which was present and the filtering algorithm successfully
yielded a consistent estimator. Not being tied to a specific kind of microstructure noise
Chapter 4. HF Simulated Data and Microstructure 86
0 50 100 150 200 2500.2
0.4
0.6
0.8
1
1.2GARCH Volatility γ=0.1
σ2
Model Vol
TSRVadj
HHTfiltered
0 50 100 150 200 2500.2
0.4
0.6
0.8
1
1.2Two−Factor Affine Volatility γ=0.1
σ2
Model Vol
TSRVadj
HHTfiltered
0 50 100 150 200 2500
0.2
0.4
0.6
0.8 Log−Normal Diffusion Volatility γ=0.1
Time (simulated days)
σ2
Model Vol
TSRVadj
HHTfiltered
Figure 4.7: Volatility vs Time plots for the GARCH (top), Two-Factor Affine (middle)and Log-Normal Diffusion (bottom) models at 0.1% noise. Each plot shows model
volatility, the Adjusted-TSRV estimate and the Filtered-HHT volatility estimate.
Chapter 4. HF Simulated Data and Microstructure 87
0 50 100 150 200 2500
0.5
1
1.5GARCH Volatility γ=0.5
σ2
Model Vol
TSRVadj
HHTfiltered
0 50 100 150 200 2500.2
0.4
0.6
0.8
1Two−Factor Affine Volatility γ=0.5
σ2
Model Vol
TSRVadj
HHTfiltered
0 50 100 150 200 2500
0.5
1
1.5
2 Log−Normal Diffusion Volatility γ=0.5
Time (simulated days)
σ2
Model Vol
TSRVadj
HHTfiltered
Figure 4.8: Volatility vs Time plots for the GARCH (top), Two-Factor Affine (middle)and Log-Normal Diffusion (bottom) models at 0.5% noise. Each plot shows model
volatility, the Adjusted-TSRV estimate and the Filtered-HHT volatility estimate.
may lead to the Filtered-HHT procedure being a more flexible estimator of HF volatil-
ity than those which are designed to account for a particular type of microstructure noise.
Chapter 4. HF Simulated Data and Microstructure 88
4.5 Chapter Conclusion
While no direct comparison is made between the improved HHT volatility measure pro-
posed in Section 4.1 with its predecessor proposed in Section 3.1, the improved estimator
effectively supplants the older one. It is theoretically more accurate since it’s using the
log returns directly and doesn’t have to create a proxy for them, furthermore it’s entirely
parameter free, making it much faster and enabling the Monte Carlo analysis carried
out in this chapter.
In Section 4.2 a test was designed specifically to test the claim that the HHT volatility
estimate was able to pick up on changes in volatility faster than EWMA methods while
maintaining a lower signal to noise ratio. The test results have shown that under certain
conditions this is indeed the case, namely the HHT estimates were able to respond to
changes volatility faster. The real price paid by the HHT based method is that it is best
suited to ex-post data, while no such requirement is placed on the EWMA procedure.
This early uptake of changes to volatility is likely due to the two sided ex-post nature
of the EMD process in which the cubic splining procedures at the heart of the EMD
procedure fitting data based on past and future information relative to each point. This
two sided nature of the EMD process can lead to a short lived artefact in which volatility
spikes can effect neighboring data on either side of the spike. An interesting investiga-
tion that may shed some light on how rapid changes or spikes propagate through the
EMD procedure was carried out by Rilling et al. [63].
Spectral analysis of the three simulated models performed in Section 4.4.1 showed
that when microstructure noise was introduced, there was an increase in the power level
of high frequency components and a simultaneous decrease in the Hurst exponents ex-
tracted from the data. This shares many parallels with the spectral analysis of the
fractional Gaussian noise conducted in Chapter 2. In effect, increasing the microstruc-
ture noise is leading to a rougher time series, with negative autocorrelations at the high
frequencies. Also, we have shown that the procedure proposed by Rilling et al. [63] to
Chapter 4. HF Simulated Data and Microstructure 89
estimate the Hurst exponent using the EMD process may be a useful means to analyse
inflationary market microstructure in high frequency data.
The Filtered-HHT method proposed in this chapter to deal with market microstruc-
ture noise appears to be functioning as intended. Figure 4.4 and Tables 4.3 − 4.8
clearly show that the EMD filtering approach taken clearly reduces the effect of market
microstructure on variance estimates. The low mean error results given by the tables
indicate that the Filtered-HHT approach has a comparably low bias with the TSRV-
based procedures which are also designed to handle market microstructure noise. The
Filtered-HHT procedure gave slightly higher mean square errors and mean absolute er-
rors than the TSRV and Adjusted-TSRV methods, however it is worth reiterating that
these procedures were specifically designed to handle the kind of uncorrelated additive
microstructure noise which was present in the simulation. Conversely, the Filtered-HHT
approach made no assumptions on the structure of the microstructure noise, other than
the fact that it was inflationary at higher frequencies. Although it is not made clear in
this chapter, the greater generality arising from a lack of assumptions on the underlying
structure can be a strength, just how much of an advantage it may be is more evident
in the following chapter.
Chapter 5
Simulated Options Market &
Real FX Data
In this chapter, a simulated options market in created in order to assess the performance
of several volatility estimators using real high frequency FX data for three currencies.
This approach of a simulated options market was developed by Engle et al. [67] to evalu-
ate the performance of volatility forecasting algorithms (ARMA, GARCH(1,1) etc.) for
low frequency financial data. More recently this approach was implemented by Bandi
et al. [68] to evaluate different volatility estimates (RV, TSRV, etc) at high frequencies.
The key feature of this approach is that it allows for the direct comparison of different
volatility estimates in the absence of the ‘true’ volatility.
ARMA(1,1) models are used to generate short term, one step ahead volatility fore-
casts for several different volatility estimating techniques. These different forecasts are
each represented by a virtual agent which buys and sells straddle options on $1 US of
the underlying (the currency) to other virtual agents which have differing volatility fore-
casts. These straddle options are then held to maturity and the profits and losses are
calculated. This chapter also presents a slightly modified version of the Filtered-HHT
procedure which reduces computational load.
90
Chapter 5. Simulated Options Market & Real FX Data 91
5.1 The Simulated Options Market and Clarification of
Methods
In this section the proposed volatility measure for dealing with HF microstructure noise
is tested with real FX data, namely the: AUD/USD, EUR/USD & GBP/USD exchange
rates. Volatility is a latent variable, therefore there is no true volatility measure one
can use as a basis for comparison as there would be in simulated data, thus we turn
to a competitive simulated options market and compare outcomes. The details of this
approach follow.
5.1.1 Simulated Option Market Construction
In this section we specify the simulated options market where one step ahead volatility
forecasts are pitted against one another. Every volatility estimator is considered to have
its own virtual trading agent that buys and sells straddle options to other agents on a
pairwise basis, based on whether or not they view the straddle as being over or under
priced by their pairwise partner. For simplicity, each option is priced on a $1 US share in
the underlying, or the exchange in our case with a strike price of $1 and a risk free rate
of zero. Note that over such short time intervals this zero expected return is entirely
plausible for stocks and even more so for FX markets. Under these assumptions the
Black-Scholes European call price (and also the put price) is given by:
Ct = 2Φ
(1
2σ(t)∆t0.5
)− 1, (5.1)
where Φ is the cumulative normal distribution and σ(t) is the specific volatility forecast
in terms of daily standard deviation at time t and ∆t = 124 since the option expires in
one hour. Note that a straddle is made up of a call and a put option purchased at the
same spot price and at the same strike price. A straddle will increase in value if the
underlying asset (or rate) either increases or decreases, thus trading straddles is akin to
trading in volatility.
Chapter 5. Simulated Options Market & Real FX Data 92
The following procedure outlines the virtual market trading place.
1. Estimate past volatility
Historical volatility estimates (σ(t)) are calculated using a number of methods.
All volatility estimates are constructed from high frequency intra-day data in one
hour chunks.
2. Forecast future volatility
The one step ahead out of sample forecasts were computed using an ARMA(1,1)
model with parameters computed over a 500 hour sliding window on the log of the
historical volatility. Parameter fitting was performed on the log volatility because
the leptokurtic nature of the raw volatility distribution led to inflated volatility
forecasts, this in turn was due to the least squares parameter fitting procedure
that was used to find the ARMA(1,1) model parameters. The exponential of the
forecasted log volatility was then used to price the option.
3. Price a straddle option
For a given pair of volatility forecasts, each method computes their perceived fair
price for a one hour straddle option using Equation (5.1).
4. Trade with partner
Trades then take place between each pair at a price calculated using the midpoint
of the two volatility estimates. Note this differs from the literature slightly where
the trade is executed at the midpoint of the two perceived fair prices. The agent
with the higher volatility forecast of the two will perceive the option as being
under priced and take a long position in the straddle option (buy). Conversely,
the agent with the lower volatility forecast will perceive the option as being over
priced and take a short position (sell). If the two agents have the exact same
volatility estimate at any point then no trade is made.
5. Hedge
Finally, the two positions are hedged to minimise exposure to risk. Since a straddle
consists of a call and a put option, with corresponding hedge ratios of Φ(
12σt∆t
0.5)
short and 1−Φ(
12σt∆t
0.5)
long, the total hedge ratio is 1−2Φ(
12σt∆t
0.5). Hence,
Chapter 5. Simulated Options Market & Real FX Data 93
the hourly profit for the agent who buys a straddle is given by:
|RFXratet |−2Ct +RFXratet
(1− Φ
(1
2σt∆t
0.5
))(5.2)
where RFXratet is the return on the $1 US spent on the currency at time t − 1.
Conversely, the seller of the straddle makes the hourly profit given by:
2Ct − |RFXratet |−RFXratet
(1− Φ
(1
2σt∆t
0.5
)). (5.3)
When it comes to assessing the performance of each forecaster, the most obvious, but
not the only useful metric, is the total profits or losses incurred after engaging in com-
petitive trading against virtual agents representing other volatility forecasts. This test
metric is described throughout this chapter as the pairwise competitive trading test.
There is also much to be learned by a parallel market in which each agent buys or
sells the straddle to the other party at what they consider to be their own fair price, i.e.
no midpoint is agreed on and each trader enters into the trade almost independently of
the other, with only their position determined by the others price. In this case the profit
from one will no longer be equal in magnitude to the corresponding loss of its trading
counterpart. Also, it is significant to note that the the most successful trader according
to this test, is not the one with the highest profit, but rather the one with the smallest
magnitude of profit or loss. This change in the assessment metric is due to the fact that
each agent is effectively trading at its own perceived fair price, thus only the agents with
the fairest price will result in a profit or loss that is small in magnitude. This test metric
is described throughout this chapter as the fair price trading test.
5.1.2 Data Specification and Clarification of Methods
The FX rate used for analysis was at the midpoint of the bid rate and the ask rate.
Actual time was used as opposed to tick or business time and the data was of suffi-
ciently high frequency that there were always points sufficiently close to the desired
Chapter 5. Simulated Options Market & Real FX Data 94
sampling point. For the AUD/USD exchange rates there were 1565 total high frequency
trading hours considered between the 1st March 2014 and 31st May 2014, 1576 for the
EUR/USD exchange and 1552 for the GBP/USD exchange. Each of these hours was
considered high frequency if it had at least one trade very ten seconds, although most
hours had substantially more than this.
The FX quotes are provided around the clock throughout weekdays so there is no
consideration taken for overnight effects. Weekends were handled simply by closing all
trading positions before a weekend starts so that no options or stocks are held over the
weekend.
Nine distinct methods were used to provide historical volatility estimates as well as
three supplementary estimates of Min, Max and Average which are useful diagnostic
tools. The historical volatility estimation methods used were:
1. The RV method with a sample interval of one minute given by Equation (2.18)
and denoted here as ‘RV 1min’
2. The RV method with a sample interval of five minutes given by Equation (2.18)
and denoted here as ‘RV 5min’
3. The Averaged-RV method with a sample interval of one minute given by Equation
(2.20) and denoted here as ‘Av RV 1 min’
4. The Averaged-RV method with a sample interval of five minutes given by Equation
(2.20) and denoted here as ‘Av RV 1 min’
5. The Adjusted-TSRV method with a sample interval of one minute given by Equa-
tion (2.22) and denoted here as ‘Adj TSRV 1 min’
6. The Adjusted-TSRV method with a sample interval of five minutes given by Equa-
tion (2.22) and denoted here as ‘Adj TSRV 5 min’
7. The Average-HHT method with a sample interval of one minute given by Equation
(4.8) and denoted here as ‘HHT 1 min’
Chapter 5. Simulated Options Market & Real FX Data 95
8. The Average-HHT method with a sample interval of five minutes given by Equation
(4.8) and denoted here as ‘HHT 5 min’
9. The Filtered-HHT method, described in Section 5.1.3 and denoted here as ‘HHT
Filtered’.
Finally, the minimum, maximum and mean of all of the above methods was taken at
each point as a diagnostic tool. These supplementary estimates are given as:
10. Average: Constructed by taking the arithmetic mean of the above forecasts on a
point-by-point basis
11. Max: Constructed by taking the highest volatility forecast from the above methods
on a point-by-point basis
12. Min: Constructed by taking the lowest volatility forecast from the above methods
on a point-by-point basis.
5.1.3 Modified HHT Estimator With Market Microstructure
This version of the HHT volatility estimator differs slightly from that outlined in Chap-
ter 4. Due to differences in the data and overall computational load, a slightly more
efficient procedure was used. The main difference is a change in the stopping criteria for
the iterative filtration process. The subsample times were also changed to capture the
dynamics of the underlying FX market more succinctly. As a guide, the range of sam-
ple intervals should include enough high frequency points to capture some inflationary
behaviour, as well as some relatively sparse samples which have not been significantly
effected by market microstucture. A step by step outline of the modified volatility
estimation procedure follows.
1. EMD Step:
Apply the EMD procedure to the observed raw price series P (t,∆t), where ∆t is
the highest available sample rate and the observed raw price with market frictions
Chapter 5. Simulated Options Market & Real FX Data 96
is given by P . Note that this differs from the low frequency realm, where the ob-
served price was essentially the same as the frictionless price, which was denoted
P . Also, recall that the observed price can be reconstructed by a summation of
all IMFs, i.e. P (t,∆t) =∑n
l=1 Ψl(t,∆t) + Ωn(t,∆t).
2. Filtration Step:
Next, filter the observed price using the EMD procedure to give the filtered price
Fi(t,∆t), defined by:
Fi(t,∆t) =
n∑l=1+i
Ψl(t,∆t) + Ωn(t,∆t), (5.4)
for i = 0, . . . , n, where i is the number of IMFs filtered off, so that when i = 0,
F0(t,∆t) is simply the observed price. Note that the ith filtered signal is actually
equivalent to the ith residual, i.e.
Ωi(t,∆t) = P (t,∆t)−i∑l=0
Ψl(t,∆t) =
n∑l=i+1
Ψl(t,∆t) + Ωn(t,∆t). (5.5)
This observation is consistent provided that Ω0 is the considered original observed
price and Ψ0 is empty.
3. Subsampling Step:
Subsample Fi(t,∆t) at various time intervals to give Fi(t,∆t), where ∆t = n∆t.
For example, the numerical results given in this chapter used ∆t(1, . . . , 6) =
[0.25, 0.5, 1, 1.5, 2, 3] minute intervals. The daily sample rate is then given as
fs(j) = 1440∆t(j) , for j = 1, . . . , 6.
4. Volatility and Averaging Step:
Now, starting at the 0th filtration step, the procedure outlined in Section 4.1 is
used to estimate the volatility of Fi(t,∆t) for each ∆t. The the level of filtration
is increased with each successive iteration until the stopping criteria is met. Note
that this procedure gives the spot volatility, and for the purposes of comparison,
we wish to convert this into integrated volatility to make hourly estimates. The
Chapter 5. Simulated Options Market & Real FX Data 97
mean of each of the resulting volatility measures is equivalent to the integrated
volatility over the period the mean was taken, which is one hour in this case. Let
this mean or integrated volatility at sample rate fs, hour t and filtration step i be
given by HHTIV (t, fs(j), i), defined by Equation (4.10).
5. Iteration and Stopping Criteria Step:
Iterate the previous step until the characteristic volatility curve changes the sign
of its gradient at the highest sample rates (fs(1)), i.e. the characteristic volatility
curve changes from decreasing as the sample interval increases, to increasing as
the sample interval increases. This can also be defined in terms of the gradient
changing with respect to changes in sample rate, i.e. Γ(fs(1), i− 1)) > 0 changes
to Γ(fs(1), i) < 0 (note that fs(j) decreases as j increases) where Γ is defined in
Section 4.11, with k = 6.
6. Optimisation Search & Calculation of Volatility Steps:
Once the stopping criteria is met, the last two filtrations of the volatility measure
are linearly combined in such a way as to maximise the flatness of the combined
volatility profile over all sample rates. This is achieved by minimising the cost
function described by Equation (4.12) in Chapter 4. Finally, Equation (4.13) is
used to calculate the Filtered-HHT volatility estimate.
The procedure is best illustrated through Figure 5.1, where it evident that successive
filtrations have removed much of the market microstructure. In Figure 5.1 it is also
possible to see that a change of gradient has occurred at the highest sampling rates in
the last filtration. This is the condition which triggered the stopping criteria, as the
change in gradient signals the point at which the filtered signal has changed from being
under filtered to over filtered.
The end result of this is a volatility measure which should be roughly the same
whether it was sampled at 0.5 minutes or 3 minute intervals, thus any of the values can
be chosen or better still, a mean over the different sample rates can be chosen as the
final integrated volatility estimate. A mean of all sample rates what the procedure used
Chapter 5. Simulated Options Market & Real FX Data 98
0 0.5 1 1.5 2 2.5 3 3.5 42
4
6
8
10
12
14x 10
−5
Sample interval in minutes
Dai
ly V
olat
ility
Characteristic Volatility Profile
Successive filtrationsOptimised
Figure 5.1: Volatility estimates with sample intervals of ∆t(1, . . . , 8) =[0.2, 0.25, 0.5, 1, 1.5, 2, 3, 4], calculated across different levels of filtration using FX data.The ‘o’ symbol denotes the estimate provided by the procedure outlined in this chapter.
in this chapter.
Importantly, once the optimal choice for the parameter C is known, the volatility
estimates which used the highest frequency data can be used to describe the instan-
taneous volatility down to the smallest time scales. Figure 5.2 shows this in practice.
This spot volatility estimate makes it possible to see how volatility is evolving over the
course of a minute, making daily volatility estimates seem low frequency by comparison.
Unfortunately, the results from the previous chapter also suggest that the Filtered-HHT
procedure can be quite noisy, rendering volatility estimates at the tick scale of limited
use at the current time, however with further refinements this approach could lead to
better short term forecasts.
Chapter 5. Simulated Options Market & Real FX Data 99
0 6 12 18 24−12
−11
−10
−9
−8
−7
−6
−5Average Daily Volatility Cycle
Hours
Log
Vol
atili
ty
Figure 5.2: Instantaneous or spot volatility estimates for 24 hours of high frequencyAUD/USD data.
Furthermore, by effectively de-noising a signal in order to create a consistent volatility
measure there is an added bonus of being able to either examine the frictionless price or
the noise separately. Figure 5.3 depicts AUD/USD FX data riddled with microstructure
noise as well as the frictionless price, which can easily be extracted once the optimal
multiplier C is known. It was also observed during the course of this research that once
this frictionless price is extracted, inconsistent integrated volatility estimates such as RV
also become consistent when applied to the filtered price. Figure 5.3 also shows how the
EMD process has been used as a smoother and a low pass filter, as the jagged move-
ments and jumps which are apparent at the highest frequencies have been smoothed out.
5.1.4 Results
The aim of this section is to see how the volatility estimates proposed above compare
against the commonly used RV and Adjusted-TSRV methods. Note that the return
Chapter 5. Simulated Options Market & Real FX Data 100
Figure 5.3: The ‘frictionless’ price (red), as extracted from 24 hours of high frequencyAUD/USD data (blue).
profits were annualised assuming 252 trading days in a year, each day consisting of 24
trading hours.
For the competitive option market in which the highest profits is the metric of suc-
cess, Tables 5.1-5.3 show that the proposed Filtered-HHT method provides the best
volatility estimates for the EUR/USD and GBP/USD exchanges and achieves a rank
of second place for the AUD/USD exchange. Conversely, the realized volatility formed
by averaging multiple five minute estimates earns itself first place in the AUD/USD
exchange and second in both the EUR/USD and GBP/USD exchanges. Surprisingly
the Adjusted-TSRV method, which is designed specifically to handle high frequency
noise appears to be over adjusting for the inflationary noise and under estimating the
volatility. Unsurprisingly all of the volatility estimates which make no adjustment for
the presence of high frequency noise perform very poorly at higher sampling rates when
the effect of high frequency noise is at its greatest.
In the second performance test, straddle options were valued and bought or sold at
Chapter 5. Simulated Options Market & Real FX Data 101
each virtual trader’s opinion of a fair price. Thus, the object under this measure is not
to have the largest profit, but the smallest magnitude profit or loss, since it implies the
fairest pricing. The results for this test are given in Tables 5.4-5.6 . Under this test the
rankings are dominated by the same two methods that performed well in the pairwise
trading test. The Filtered-HHT procedure achieving first place for the AUD/USD and
GBP/USD exchanges and second for the EUR/USD exchange. Conversely, the realized
volatility formed by averaging multiple five minute estimates earns itself first place in
the EUR/USD exchange and second in both the AUD/USD and GBP/USD exchanges.
As expected the methods that make no adjustment for noise are performing the worst
at the highest sampling rate when the inflationary bias is at its worst. Again, the
Adjusted-TSRV method appears to be biased below, resulting in poorer than expected
performance.
The annualised standard deviation for the percentage returns is also given in Tables
5.1-5.3 for the pairwise competitive test and in Tables 5.4-5.6 for the fair price trading
test. Similar annualised standard deviations are observed within each market because
of the way each estimator interacts with each other estimator, the profit from one is the
loss of the same magnitude for another. This observation of only a small change in the
standard deviation of profits within a market is consistent with observations made by
Bandi et al. [68].
The Sharpe ratios are also given for each method during the competitive trading
test in each currency in Tables 5.7-5.9. It should be noted that values given for this
measure are not particularly informative, as so much of the result is based on the choice
of methods present in the simulation. The values given in Tables 5.7-5.9 have to be
considered in the right context, these Sharpe ratios are entirely dependent on the mod-
els that were chosen to compete in the simulated market, some of which were not well
suited to market microstructure. Thus, the results given by this metric may not be
representative of performance in a real market because of its dependence on the models
chosen for the simulated study. As expected, the ranking results of the Sharpe ratios
Chapter 5. Simulated Options Market & Real FX Data 102
follow the ranking of competitive trading test very closely because the standard devia-
tion of profits varies little across different methods. Sharpe ratios were omitted from the
test results in which each estimation procedure traded at its own perceived fair price be-
cause they could be misleading and are no more informative than the rankings provided.
Under both performance metrics it may be of some interest to examine the potential
profits if all trades were unhedged. However, the unhedged results were calculated but
are not displayed because the time interval was so short that there was no significant
difference between the hedged and unhedged results and no difference in the performance
rankings in this case. If the options were held for longer periods or were exposed to more
risk in some other way then the importance of hedging would become more apparent.
The estimates denoted as Min, Max and Average were used as diagnostic tools when
setting up the simulated market participants. Due to the inclusion of some volatility
estimation techniques which were not designed to handle market microstructure noise,
such as RV 1 min and HHT 1 min, it is reasonable to assume that there are going to
be some very inflated volatility estimates. The presence of these highly biased volatility
estimates has led to the Max volatility forecast being substantially worse than the Min
forecast. Such a result was expected as there is no mechanism present which was likely
to yield an extremely low volatility estimates, additionally the maximum volatility was
unbounded and thus had far greater room for error.
The presence of systematically biased estimates has also led to the Average forecaster
being biased above, however it is worth noting that if the competing set of forecasters
was made up of independent estimates without such a systematic bias, then the average
of these forecasts would be expected to perform quite well.
More detailed tables that describe how each method performed when trading against
every other method for both performance tests can be found in the appendix to this
chapter.
Chapter 5. Simulated Options Market & Real FX Data 103
Table 5.1: Pairwise competitive trading results for AUD/USD Straddles
Method Av Annual Return % Std of Return% Return Rank
RV 1 min -31.06 4.69 9
Adj TSRV 17.12 4.68 6
Av RV 1 min -37.34 4.71 10
RV 5 min 27.35 4.69 3
Adj TSRV 5 min 10.03 4.53 7
Av RV 5 min 34.44 4.70 1
HHT 1 min -37.92 4.62 11
HHT 5 min 25.71 4.67 4
HHT Filtered 32.16 4.70 2
Max -69.75 4.48 12
Min 4.44 4.45 8
Average 24.82 4.70 5
Table 5.2: Pairwise competitive trading results for EUR/USD Straddles
Method Av Annual Return % Std of Return% Return Rank
RV 1 min -31.47 3.25 10
Adj TSRV 8.86 3.25 7
Av RV 1 min -15.57 3.28 9
RV 5 min 20.66 3.28 3
Adj TSRV 5 min 12.26 3.21 6
Av RV 5 min 22.45 3.29 2
HHT 1 min -34.75 3.26 11
HHT 5 min 17.48 3.27 5
HHT Filtered 30.76 3.27 1
Max -55.59 3.13 12
Min 6.19 3.13 8
Average 18.73 3.29 4
5.2 Chapter Conclusion
Assessing the performance of estimations on an unobservable variable is inherently a
difficult task. Without resorting to simulated models, which are highly specification
dependent, we were successfully able to use a virtual option market to compare a new
procedure for high frequency estimation with several popular methods.
The volatility estimate proposed in this chapter is designed to overcome the prob-
lem of inflated volatility estimates caused by microstructure noise. The Filtered-HHT
Chapter 5. Simulated Options Market & Real FX Data 104
Table 5.3: Pairwise competitive trading results for GBP/USD Straddles
Method Av Annual Return % Std of Return% Return Rank
RV 1 min -9.79 3.13 10
Adj TSRV 11.18 3.14 7
Av RV 1 min -4.70 3.11 9
RV 5 min 18.65 3.13 3
Adj TSRV 5 min 13.02 3.05 6
Av RV 5 min 21.25 3.13 2
HHT 1 min -54.51 3.00 11
HHT 5 min 14.14 3.07 5
HHT Filtered 27.51 3.12 1
Max -62.00 2.96 12
Min 8.46 2.97 8
Average 16.81 3.13 4
Table 5.4: Fair price trading results for AUD/USD Straddles
Method Av Annual Return % Std of Return% Return Rank
RV 1 min -74.19 4.70 9
Adj TSRV -20.94 4.70 6
Av RV 1 min -81.29 4.72 10
RV 5 min -6.78 4.70 3
Adj TSRV 5 min -36.52 4.53 7
Av RV 5 min 0.71 4.70 2
HHT 1 min -82.70 4.63 11
HHT 5 min -13.99 4.68 5
HHT Filtered -0.50 4.71 1
Max -116.87 4.50 12
Min -45.62 4.46 8
Average -7.46 4.70 4
estimate proposed in this thesis achieved the best overall performance when considering
all three exchanges and both performance tests so there is strong evidence to suggest
that the aim of constructing a consistent volatility estimator using the HHT has been
achieved. The reduction in high frequency bias after successive filtrations as depicted
Figure 5.1 also provides some evidence supporting the argument that the proposed
method is acting as a consistent estimator. Also, the apparent dipping of the volatility
estimate with a sample interval of four minutes in Figure 5.1 is likely due to the higher
sample error of the more sparsely sampled estimates. This apparent dipping is specific to
this randomly selected profile and highlights why simply choosing a large sample width
Chapter 5. Simulated Options Market & Real FX Data 105
Table 5.5: Fair price trading results for EUR/USD Straddles
Method Av Annual Return % Std of Return% Return Rank
RV 1 min -62.92 3.26 10
Adj TSRV -21.03 3.28 6
Av RV 1 min -45.40 3.28 9
RV 5 min -5.48 3.30 4
Adj TSRV 5 min -23.50 3.22 7
Av RV 5 min -3.32 3.29 1
HHT 1 min -66.30 3.25 11
HHT 5 min -11.95 3.29 5
HHT Filtered 4.20 3.26 2
Max -89.82 3.14 12
Min -33.63 3.15 8
Average -4.97 3.29 3
Table 5.6: Fair price trading results for GBP/USD Straddles
Method Av Annual Return % Std of Return% Return Rank
RV 1 min -37.90 3.12 10
Adj TSRV -16.76 3.15 5
Av RV 1 min -32.43 3.11 9
RV 5 min -6.86 3.13 4
Adj TSRV 5 min -20.77 3.06 7
Av RV 5 min -4.91 3.13 2
HHT 1 min -93.02 3.02 11
HHT 5 min -17.04 3.08 6
HHT Filtered 2.39 3.11 1
Max -101.52 2.98 12
Min -28.77 2.97 8
Average -6.29 3.12 3
is an inadequate solution to the problem of market microstructure.
Perhaps the main advantage of the Filtered-HHT approach is that there are no un-
derlying assumptions on the structure of the microstructure noise, only that the noise
is causing inflationary effects at higher frequencies. The lack of assumptions, combined
with an algorithmic approach to achieve consistency over different time scales, appears
to have made the HHT based estimator flexible enough to handle real data effectively.
Chapter 5. Simulated Options Market & Real FX Data 106
Better volatility estimates aren’t the only useful outcome of the filtering procedure
outlined in Section 5.1.3. With filtering, the high frequency noise that usually obfus-
cates our view of the frictionless price can be stripped away one layer at a time until the
elusive true price is revealed. Furthermore, as Figure 5.2 demonstrates, the proposed
estimator is also capable of providing spot or instantaneous volatility estimates as op-
posed to just the integrated volatility over a period of time. This ultra high frequency
volatility estimate may led to better short term predictions as well as a more thorough
understanding of high frequency dynamics.
It is also worth noting that asymmetries in the Black-Scholes pricing formula would
also tend to penalise under estimates differently than over estimates, as the option
prices do not vary linearly with volatility due to the convexity of the Black-Scholes pric-
ing formula and the effect of Jensen’s Inequality. This raises an important point about
volatility assessment in general, using a more realistic and complicated cost function to
determine the superiority of one estimator over another may yield to completely different
conclusions than assessments of the ME, MSE, MAPE etc. This is due to asymmetries
in the cost function placing more weight on some characteristics than others, i.e. under
estimating volatility may be more costly than over estimating it or a low bias might be
more important to some cost functions than a low noise.
It should be noted that at such high frequencies, assumptions required for the Black-
Scholes equation start to break down, namely the requirements for log-normally dis-
tributed returns and continuous sample paths. At such high frequencies, market frictions
(microstructure effects) lead to a highly discretised return structure which may be more
suited to a binomial or trinomial option pricing model with variable volatility, such as
the one developed by Haahtela [72]. Although we can acknowledge that the B-S model
has its drawbacks, so long as results gained from it are put into their correct context,
there is still a lot of information that may be garnered from its use. Furthermore, the
wide adoption of the B-S model makes any new insights more accessible and relevant.
Alternative, perhaps more appropriate option pricing schemes may have a non-trivial
Chapter 5. Simulated Options Market & Real FX Data 107
impact on the evaluation of volatility estimators, however this line of research falls out-
side of the scope of this thesis and is an area of future research.
Chapter Appendix
The tables contained in this appendix give a more comprehensive account of how each
volatility estimator and its associated virtual trader fared against its competitors on
a pairwise basis. Note that the tables for the competitive trading results should be
symmetric about the diagonal in magnitude, but opposite in sign. For the competitive
trading case in Tables 5.7 – 5.9, the result in cell (i,j) should be seen as the profit or loss
resulting in a trade with its partner in cell position (j,i). For the fair trading tests in
Tables 5.10 – 5.12, the tables no longer have any symmetry as different virtual traders
are no longer in direct competition. Although they are no longer in competition, the are
not completely independent of one another, this is why results still vary among different
trading partners during the fair pricing test. What is driving the difference in results is
simply the fact that its position to buy or sell a straddle (at its own perceived fair price)
is still dependent on its trading partner.
From the full tables, the performance of individual methods becomes more apparent.
Some of the features which stand out the most in all of the tables is the fact that the
composite volatility estimate obtained by taking the maximum volatility forecast on
a point by point basis is clearly grossly over estimating the volatility, resulting in the
poorest overall performance. This is closely followed by the three volatility estimators
which are not designed to handle market microstructure, namely the “RV 1 min”, “Av
RV 1 min” and “HHT 1 min” procedures.
The composite volatility estimate generated by taking the minimum volatility fore-
cast at each point also has very poor performance across the board. While not quite as
Chapter 5. Simulated Options Market & Real FX Data 108
bad as the composite maximum or the three procedures which are particularly suscepti-
ble to market microstructure noise, the composite minimum was the next worst estimate
in every test. This should not come as a surprise, as the estimate should be biased below
the true volatility, unless every other forecaster is also biased above the true volatility.
The fact that all of the estimates which account for market microstructure in some way
are between the composite maximum and the composite minimum is a reassuring sign
for the validity of the test. It implies that the true volatility lies between these two
extreme estimates.
The composite “Average” volatility estimate is performing relatively well considering
that it is being biased above quite heavily by the three procedures which are known to
over estimate volatility in the presence of market microstructure. Note that this pro-
cedure is generated from the 9 different volatility estimates which does not include the
two composite methods of the minimum and maximum. With the removal of the three
procedures known to be heavily biased above, the Average estimate would actually be
expected to have the best performance, since an average of independent unbiased esti-
mates is often better than any individual estimate.
Surprisingly, the “Adj TSRV” and “Adj TSRV 5 min” procedures aren’t dominat-
ing the forecast performance tables, as the results from Chapter 4 would indicate they
should. The TSRV procedures appear to be under estimating the true volatility, as the
“RV 5 min” and “Av RV 5 min” have, by definition a higher volatility estimate than
the TSRV procedures and yet they have are performing better under this simulation.
What is left to discuss are the sparsely sampled RV and HHT estimates as well as
the Filtered-HHT procedure. The more sparsely sampled RV and HHT based estimates
are all performing relatively well in both the competitive trading and fair pricing tests.
Sampling at five minute intervals has largely reduced the effects of market microstruc-
ture, however not the sparse sampling didn’t quite meet the performance of the filtering
procedure which has the best overall performance.
Chapter 5. Simulated Options Market & Real FX Data 109
Table5.7:
AU
D/U
SD
Com
pet
itiv
ep
air
wis
etr
ad
ing
resu
lts
infu
ll
Meth
od
RV
1min
AdjTSRV
Av
RV
1min
RV
5min
AdjTSRV
5min
Av
RV
5min
HHT
1min
HHT
5min
HHT
Filte
red
Max
Min
Avera
ge
RV
1min
08.59E-0
5-2
.73E-0
51.01E-0
47.29E-0
51.01E-0
4-2
.15E-0
58.39E-0
51.08E-0
4-1
.33E-0
46.72E-0
51.27E-0
4AdjTSRV
-8.59E-0
50
-9.14E-0
52.53E-0
5-1
.87E-0
53.80E-0
5-9
.34E-0
52.48E-0
52.93E-0
5-1
.04E-0
4-2
.66E-0
5-9
.07E-0
6Av
RV
1min
2.73E-0
59.14E-0
50
1.05E-0
47.73E-0
51.05E-0
42.20E-0
68.83E-0
51.13E-0
4-1
.34E-0
47.17E-0
51.32E-0
4RV
5min
-1.01E-0
4-2
.53E-0
5-1
.05E-0
40
-2.65E-0
52.17E-0
5-1
.09E-0
45.86E-0
61.60E-0
5-1
.16E-0
4-3
.36E-0
5-2
.50E-0
5AdjTSRV
5min
-7.29E-0
51.87E-0
5-7
.73E-0
52.65E-0
50
3.52E-0
5-7
.99E-0
55.16E-0
52.64E-0
5-8
.80E-0
5-2
.18E-0
5-9
.64E-0
7Av
RV
5min
-1.01E-0
4-3
.80E-0
5-1
.05E-0
4-2
.17E-0
5-3
.52E-0
50
-1.08E-0
4-2
.94E-0
5-7
.60E-0
6-1
.15E-0
4-3
.95E-0
5-2
.61E-0
5HHT
1min
2.15E-0
59.34E-0
5-2
.20E-0
61.09E-0
47.99E-0
51.08E-0
40
9.07E-0
51.15E-0
4-1
.34E-0
47.43E-0
51.34E-0
4HHT
5min
-8.39E-0
5-2
.48E-0
5-8
.83E-0
5-5
.86E-0
6-5
.16E-0
52.94E-0
5-9
.07E-0
50
1.49E-0
5-9
.87E-0
5-5
.71E-0
5-1
.09E-0
5HHT
Filte
red
-1.08E-0
4-2
.93E-0
5-1
.13E-0
4-1
.60E-0
5-2
.64E-0
57.60E-0
6-1
.15E-0
4-1
.49E-0
50
-1.23E-0
4-3
.09E-0
5-1
.60E-0
5M
ax
1.33E-0
41.04E-0
41.34E-0
41.16E-0
48.80E-0
51.15E-0
41.34E-0
49.87E-0
51.23E-0
40
8.24E-0
51.41E-0
4M
in-6
.72E-0
52.66E-0
5-7
.17E-0
53.36E-0
52.18E-0
53.95E-0
5-7
.43E-0
55.71E-0
53.09E-0
5-8
.24E-0
50
5.36E-0
6Avera
ge
-1.27E-0
49.07E-0
6-1
.32E-0
42.50E-0
59.64E-0
72.61E-0
5-1
.34E-0
41.09E-0
51.60E-0
5-1
.41E-0
4-5
.36E-0
60
Hourly
Retu
rn-5
.14E-0
52.83E-0
5-6
.17E-0
54.52E-0
51.66E-0
55.69E-0
5-6
.27E-0
54.25E-0
55.32E-0
5-1
.15E-0
47.35E-0
64.10E-0
5Hourly
Retu
rnStD
6.03E-0
46.02E-0
46.06E-0
46.03E-0
45.82E-0
46.04E-0
45.94E-0
46.01E-0
46.05E-0
45.76E-0
45.72E-0
46.05E-0
4Sim
ple
AnnualRetu
rn%
-31.06
17.12
-37.34
27.35
10.03
34.44
-37.92
25.71
32.16
-69.75
4.44
24.82
AnnualRetu
rnStd
%4.69
4.68
4.71
4.69
4.53
4.70
4.62
4.67
4.70
4.48
4.45
4.70
Hourly
Sharp
eRatio
-0.085
0.047
-0.102
0.075
0.029
0.094
-0.105
0.071
0.088
-0.200
0.013
0.068
AnnualSharp
eRatio
-6.62
3.66
-7.93
5.83
2.22
7.33
-8.20
5.50
6.84
-15.56
1.00
5.27
Retu
rnRankin
g9
610
37
111
42
12
85
Sharp
eRankin
g9
610
37
111
42
12
85
Table5.8:
EU
R/U
SD
Com
pet
itiv
ep
air
wis
etr
ad
ing
resu
lts
infu
ll
Meth
od
RV
1min
AdjTSRV
Av
RV
1min
RV
5min
AdjTSRV
5min
Av
RV
5min
HHT
1min
HHT
5min
HHT
Filte
red
Max
Min
Avera
ge
RV
1min
06.84E-0
56.11E-0
58.36E-0
56.32E-0
58.05E-0
5-7
.77E-0
67.04E-0
57.96E-0
5-8
.33E-0
55.70E-0
59.97E-0
5AdjTSRV
-6.84E-0
50
-6.03E-0
52.02E-0
57.01E-0
62.99E-0
5-7
.30E-0
51.69E-0
55.42E-0
5-8
.74E-0
5-5
.83E-0
65.60E-0
6Av
RV
1min
-6.11E-0
56.03E-0
50
8.13E-0
55.73E-0
57.51E-0
5-8
.50E-0
56.73E-0
57.65E-0
5-1
.35E-0
45.10E-0
59.53E-0
5RV
5min
-8.36E-0
5-2
.02E-0
5-8
.13E-0
50
-7.04E-0
69.24E-0
6-8
.39E-0
5-4
.82E-0
61.54E-0
5-9
.51E-0
5-1
.74E-0
5-7
.03E-0
6AdjTSRV
5min
-6.32E-0
5-7
.01E-0
6-5
.73E-0
57.04E-0
60
1.75E-0
5-6
.45E-0
51.97E-0
52.31E-0
5-7
.47E-0
5-1
.82E-0
5-5
.48E-0
6Av
RV
5min
-8.05E-0
5-2
.99E-0
5-7
.51E-0
5-9
.23E-0
6-1
.75E-0
50
-8.35E-0
5-2
.12E-0
53.78E-0
5-9
.41E-0
5-2
.34E-0
5-1
.17E-0
5HHT
1min
7.77E-0
67.30E-0
58.50E-0
58.39E-0
56.45E-0
58.35E-0
50
7.38E-0
58.28E-0
5-8
.47E-0
55.83E-0
51.04E-0
4HHT
5min
-7.04E-0
5-1
.69E-0
5-6
.73E-0
54.82E-0
6-1
.97E-0
52.12E-0
5-7
.38E-0
50
3.39E-0
5-8
.36E-0
5-3
.29E-0
5-1
.32E-0
5HHT
Filte
red
-7.96E-0
5-5
.42E-0
5-7
.65E-0
5-1
.54E-0
5-2
.31E-0
5-3
.78E-0
5-8
.28E-0
5-3
.39E-0
50
-9.33E-0
5-2
.56E-0
5-3
.72E-0
5M
ax
8.33E-0
58.74E-0
51.35E-0
49.51E-0
57.47E-0
59.41E-0
58.47E-0
58.36E-0
59.33E-0
50
6.82E-0
51.12E-0
4M
in-5
.70E-0
55.83E-0
6-5
.10E-0
51.74E-0
51.82E-0
52.34E-0
5-5
.83E-0
53.29E-0
52.56E-0
5-6
.82E-0
50
-1.51E-0
6Avera
ge
-9.97E-0
5-5
.60E-0
6-9
.53E-0
57.03E-0
65.48E-0
61.17E-0
5-1
.04E-0
41.32E-0
53.72E-0
5-1
.12E-0
41.52E-0
60
Hourly
Retu
rn-5
.20E-0
51.46E-0
5-2
.57E-0
53.42E-0
52.03E-0
53.71E-0
5-5
.75E-0
52.89E-0
55.09E-0
5-9
.19E-0
51.02E-0
53.10E-0
5Hourly
Retu
rnStD
4.18E-0
44.18E-0
44.22E-0
44.22E-0
44.13E-0
44.23E-0
44.19E-0
44.21E-0
44.20E-0
44.03E-0
44.02E-0
44.23E-0
4Sim
ple
AnnualRetu
rn%
-31.47
8.86
-15.57
20.66
12.26
22.45
-34.75
17.48
30.76
-55.59
6.19
18.73
AnnualRetu
rnStd
%3.25
3.25
3.28
3.28
3.21
3.29
3.26
3.27
3.27
3.13
3.13
3.29
Hourly
Sharp
eRatio
-0.125
0.035
-0.061
0.081
0.049
0.088
-0.137
0.069
0.121
-0.228
0.025
0.073
AnnualSharp
eRatio
-9.68
2.72
-4.75
6.29
3.82
6.83
-10.67
5.34
9.41
-17.75
1.98
5.69
Retu
rnRankin
g10
79
36
211
51
12
84
Sharp
eRankin
g10
79
36
211
51
12
84
Chapter 5. Simulated Options Market & Real FX Data 110
Table5.9:
GB
P/U
SD
Com
pet
itiv
ep
air
wis
etr
ad
ing
resu
lts
infu
ll
Meth
od
RV
1min
AdjTSRV
Av
RV
1min
RV
5min
AdjTSRV
5min
Av
RV
5min
HHT
1min
HHT
5min
HHT
Filte
red
Max
Min
Avera
ge
RV
1min
02.73E-0
52.30E-0
55.61E-0
54.02E-0
55.73E-0
5-1
.04E-0
44.39E-0
56.26E-0
5-1
.31E-0
43.70E-0
56.59E-0
5AdjTSRV
-2.73E-0
50
-2.36E-0
5-3
.33E-0
62.56E-0
61.15E-0
5-9
.71E-0
59.44E-0
72.96E-0
5-1
.08E-0
4-6
.24E-0
71.16E-0
5Av
RV
1min
-2.30E-0
52.36E-0
50
5.42E-0
53.83E-0
55.77E-0
5-1
.32E-0
44.31E-0
56.20E-0
5-1
.42E-0
43.58E-0
56.76E-0
5RV
5min
-5.61E-0
53.33E-0
6-5
.42E-0
50
-8.48E-0
6-4
.27E-0
6-1
.05E-0
4-8
.15E-0
62.91E-0
5-1
.09E-0
4-1
.50E-0
5-1
.20E-0
5AdjTSRV
5min
-4.02E-0
5-2
.56E-0
6-3
.83E-0
58.48E-0
60
1.84E-0
5-8
.53E-0
5-2
.31E-0
61.71E-0
5-9
.06E-0
5-1
.29E-0
5-8
.58E-0
6Av
RV
5min
-5.73E-0
5-1
.15E-0
5-5
.77E-0
54.27E-0
6-1
.84E-0
50
-1.01E-0
4-2
.58E-0
52.97E-0
5-1
.05E-0
4-2
.31E-0
5-2
.10E-0
5HHT
1min
1.04E-0
49.71E-0
51.32E-0
41.05E-0
48.53E-0
51.01E-0
40
8.94E-0
51.07E-0
4-3
.02E-0
58.09E-0
51.20E-0
4HHT
5min
-4.39E-0
5-9
.44E-0
7-4
.31E-0
58.15E-0
62.31E-0
62.58E-0
5-8
.94E-0
50
1.19E-0
5-9
.33E-0
5-2
.12E-0
5-1
.36E-0
5HHT
Filte
red
-6.26E-0
5-2
.96E-0
5-6
.20E-0
5-2
.91E-0
5-1
.71E-0
5-2
.97E-0
5-1
.07E-0
4-1
.19E-0
50
-1.11E-0
4-1
.67E-0
5-2
.35E-0
5M
ax
1.31E-0
41.08E-0
41.42E-0
41.09E-0
49.06E-0
51.05E-0
43.02E-0
59.33E-0
51.11E-0
40
8.48E-0
51.24E-0
4M
in-3
.70E-0
56.24E-0
7-3
.58E-0
51.50E-0
51.29E-0
52.31E-0
5-8
.09E-0
52.12E-0
51.67E-0
5-8
.48E-0
50
-4.79E-0
6Avera
ge
-6.59E-0
5-1
.16E-0
5-6
.76E-0
51.20E-0
58.58E-0
62.10E-0
5-1
.20E-0
41.36E-0
52.35E-0
5-1
.24E-0
44.79E-0
60
Hourly
Retu
rn-1
.62E-0
51.85E-0
5-7
.77E-0
63.08E-0
52.15E-0
53.51E-0
5-9
.01E-0
52.34E-0
54.55E-0
5-1
.03E-0
41.40E-0
52.78E-0
5Hourly
Retu
rnStD
4.02E-0
44.03E-0
44.00E-0
44.02E-0
43.92E-0
44.02E-0
43.86E-0
43.94E-0
44.01E-0
43.81E-0
43.81E-0
44.02E-0
4Sim
ple
AnnualRetu
rn%
-9.79
11.18
-4.70
18.65
13.02
21.25
-54.51
14.14
27.51
-62.00
8.46
16.81
AnnualRetu
rnStd
%3.13
3.14
3.11
3.13
3.05
3.13
3.00
3.07
3.12
2.96
2.97
3.13
Hourly
Sharp
eRatio
-0.040
0.046
-0.019
0.077
0.055
0.087
-0.233
0.059
0.114
-0.269
0.037
0.069
AnnualSharp
eRatio
-3.13
3.57
-1.51
5.96
4.27
6.79
-18.15
4.61
8.83
-20.94
2.85
5.37
Retu
rnRankin
g10
79
36
211
51
12
84
Sharp
eRankin
g10
79
36
211
51
12
84
Table5.10:
AU
D/U
SD
Fair
pri
cetr
ad
ing
resu
lts
infu
ll
Meth
od
RV
1min
AdjTSRV
Av
RV
1min
RV
5min
AdjTSRV
5min
Av
RV
5min
HHT
1min
HHT
5min
HHT
Filte
red
Max
Min
Avera
ge
RV
1min
0-2
.71E-0
5-4
.09E-0
58.79E-0
7-6
.86E-0
57.75E-0
7-3
.73E-0
5-4
.00E-0
51.76E-0
5-1
.45E-0
4-8
.30E-0
56.20E-0
5AdjTSRV
-1.81E-0
40
-1.90E-0
4-1
.48E-0
6-5
.34E-0
51.10E-0
5-1
.94E-0
4-1
.87E-0
63.34E-0
6-2
.09E-0
4-6
.00E-0
5-5
.03E-0
5Av
RV
1min
1.35E-0
5-2
.61E-0
50
8.79E-0
7-6
.86E-0
57.75E-0
7-6
.63E-0
6-4
.00E-0
51.89E-0
5-1
.43E-0
4-8
.30E-0
56.20E-0
5RV
5min
-1.87E-0
4-5
.31E-0
5-1
.94E-0
40
-6.46E-0
56.32E-0
6-2
.00E-0
4-2
.03E-0
5-4
.65E-0
6-2
.13E-0
4-7
.79E-0
5-5
.54E-0
5AdjTSRV
5min
-1.87E-0
4-1
.27E-0
5-1
.95E-0
4-8
.26E-0
60
1.42E-0
6-1
.99E-0
43.20E-0
5-1
.60E-0
5-2
.13E-0
4-2
.92E-0
5-6
.20E-0
5Av
RV
5min
-1.87E-0
4-6
.59E-0
5-1
.95E-0
4-3
.70E-0
5-7
.17E-0
50
-1.99E-0
4-5
.11E-0
5-2
.63E-0
5-2
.13E-0
4-8
.34E-0
5-5
.59E-0
5HHT
1min
5.58E-0
6-2
.63E-0
5-1
.11E-0
52.10E-0
6-6
.86E-0
57.75E-0
70
-4.00E-0
51.89E-0
5-1
.40E-0
4-8
.30E-0
56.20E-0
5HHT
5min
-1.87E-0
4-5
.07E-0
5-1
.95E-0
4-3
.05E-0
5-7
.24E-0
58.81E-0
6-1
.99E-0
40
-1.52E-0
5-2
.13E-0
4-7
.95E-0
5-5
.95E-0
5HHT
Filte
red
-1.86E-0
4-5
.68E-0
5-1
.95E-0
4-3
.73E-0
5-7
.38E-0
5-1
.17E-0
5-1
.99E-0
4-4
.72E-0
50
-2.13E-0
4-8
.49E-0
5-3
.85E-0
5M
ax
1.20E-0
4-2
.31E-0
51.25E-0
41.31E-0
6-6
.86E-0
57.75E-0
71.27E-0
4-4
.00E-0
51.89E-0
50
-8.30E-0
56.20E-0
5M
in-1
.87E-0
4-3
.07E-0
6-1
.95E-0
4-6
.35E-0
61.49E-0
5-3
.64E-0
7-1
.99E-0
43.65E-0
5-1
.71E-0
5-2
.13E-0
40
-6.20E-0
5Avera
ge
-1.87E-0
4-3
.60E-0
5-1
.95E-0
4-7
.55E-0
6-6
.86E-0
5-5
.64E-0
6-1
.99E-0
4-4
.25E-0
5-7
.67E-0
6-2
.13E-0
4-8
.30E-0
50
Hourly
Retu
rn-1
.23E-0
4-3
.46E-0
5-1
.34E-0
4-1
.12E-0
5-6
.04E-0
51.17E-0
6-1
.37E-0
4-2
.31E-0
5-8
.30E-0
7-1
.93E-0
4-7
.54E-0
5-1
.23E-0
5Hourly
Retu
rnStD
6.04E-0
46.04E-0
46.07E-0
46.04E-0
45.82E-0
46.04E-0
45.96E-0
46.01E-0
46.05E-0
45.78E-0
45.73E-0
46.05E-0
4Sim
ple
AnnualRetu
rn%
-74.19
-20.94
-81.29
-6.78
-36.52
0.71
-82.70
-13.99
-0.50
-116.87
-45.62
-7.46
AnnualRetu
rnStd
%4.70
4.70
4.72
4.70
4.53
4.70
4.63
4.68
4.71
4.50
4.46
4.70
Retu
rnRankin
g9
610
37
211
51
12
84
Chapter 5. Simulated Options Market & Real FX Data 111
Table5.11:
EU
R/U
SD
Fair
pri
cetr
ad
ing
resu
lts
infu
ll
Meth
od
RV
1min
AdjTSRV
Av
RV
1min
RV
5min
AdjTSRV
5min
Av
RV
5min
HHT
1min
HHT
5min
HHT
Filte
red
Max
Min
Avera
ge
RV
1min
0-1
.22E-0
54.91E-0
59.86E-0
6-4
.46E-0
54.87E-0
6-2
.17E-0
5-1
.97E-0
53.24E-0
6-9
.20E-0
5-6
.12E-0
55.08E-0
5AdjTSRV
-1.34E-0
40
-1.23E-0
4-1
.17E-0
5-3
.10E-0
5-1
.62E-0
6-1
.39E-0
4-1
.49E-0
52.30E-0
5-1
.59E-0
4-4
.18E-0
5-2
.58E-0
5Av
RV
1min
-7.30E-0
5-1
.53E-0
50
1.31E-0
5-4
.46E-0
55.46E-0
6-9
.45E-0
5-1
.69E-0
55.89E-0
6-1
.48E-0
4-6
.12E-0
55.17E-0
5RV
5min
-1.44E-0
4-5
.20E-0
5-1
.38E-0
40
-3.92E-0
5-4
.72E-0
6-1
.46E-0
4-2
.74E-0
5-5
.44E-0
6-1
.63E-0
4-5
.64E-0
5-2
.95E-0
5AdjTSRV
5min
-1.45E-0
4-3
.87E-0
5-1
.35E-0
4-2
.14E-0
50
-9.05E-0
6-1
.47E-0
4-2
.81E-0
7-3
.85E-0
6-1
.63E-0
4-2
.73E-0
5-4
.98E-0
5Av
RV
5min
-1.43E-0
4-6
.12E-0
5-1
.33E-0
4-2
.31E-0
5-4
.69E-0
50
-1.47E-0
4-3
.71E-0
52.16E-0
5-1
.63E-0
4-6
.06E-0
5-3
.48E-0
5HHT
1min
-6.25E-0
6-7
.77E-0
67.52E-0
58.72E-0
6-4
.45E-0
56.56E-0
60
-1.72E-0
55.57E-0
6-9
.25E-0
5-6
.12E-0
55.43E-0
5HHT
5min
-1.42E-0
4-4
.64E-0
5-1
.35E-0
4-1
.62E-0
5-4
.14E-0
56.32E-0
6-1
.46E-0
40
1.47E-0
5-1
.63E-0
4-5
.70E-0
5-4
.63E-0
5HHT
Filte
red
-1.42E-0
4-8
.48E-0
5-1
.35E-0
4-3
.62E-0
5-5
.37E-0
5-5
.43E-0
5-1
.46E-0
4-5
.46E-0
50
-1.63E-0
4-6
.27E-0
5-6
.11E-0
5M
ax
7.39E-0
5-1
.30E-0
61.21E-0
41.11E-0
5-4
.39E-0
58.49E-0
67.65E-0
5-1
.68E-0
57.05E-0
60
-6.12E-0
55.30E-0
5M
in-1
.45E-0
4-2
.26E-0
5-1
.36E-0
4-1
.61E-0
51.01E-0
5-8
.97E-0
6-1
.47E-0
41.18E-0
5-6
.07E-0
6-1
.63E-0
40
-5.30E-0
5Avera
ge
-1.43E-0
4-4
.02E-0
5-1
.35E-0
4-1
.77E-0
5-4
.78E-0
5-1
.35E-0
5-1
.49E-0
4-2
.43E-0
51.07E-0
5-1
.63E-0
4-6
.12E-0
50
Hourly
Retu
rn-1
.04E-0
4-3
.48E-0
5-7
.51E-0
5-9
.06E-0
6-3
.89E-0
5-5
.50E-0
6-1
.10E-0
4-1
.98E-0
56.94E-0
6-1
.49E-0
4-5
.56E-0
5-8
.22E-0
6Hourly
Retu
rnStD
4.19E-0
44.22E-0
44.21E-0
44.24E-0
44.15E-0
44.24E-0
44.18E-0
44.23E-0
44.19E-0
44.03E-0
44.05E-0
44.23E-0
4Sim
ple
AnnualRetu
rn%
-62.92
-21.03
-45.40
-5.48
-23.50
-3.32
-66.30
-11.95
4.20
-89.82
-33.63
-4.97
AnnualRetu
rnStd
%3.26
3.28
3.28
3.30
3.22
3.29
3.25
3.29
3.26
3.14
3.15
3.29
Retu
rnRankin
g10
69
47
111
52
12
83
Table5.12:
GB
P/U
SD
Fair
pri
cetr
ad
ing
resu
lts
infu
ll
Meth
od
RV
1min
AdjTSRV
Av
RV
1min
RV
5min
AdjTSRV
5min
Av
RV
5min
HHT
1min
HHT
5min
HHT
Filte
red
Max
Min
Avera
ge
RV
1min
0-3
.01E-0
51.12E-0
54.31E-0
6-4
.16E-0
55.05E-0
7-1
.39E-0
4-3
.14E-0
51.29E-0
5-1
.67E-0
4-5
.21E-0
53.41E-0
5AdjTSRV
-7.93E-0
50
-7.46E-0
5-3
.81E-0
5-4
.09E-0
5-2
.53E-0
5-1
.65E-0
4-3
.93E-0
5-4
.21E-0
6-1
.75E-0
4-4
.91E-0
5-1
.65E-0
5Av
RV
1min
-3.48E-0
5-3
.27E-0
50
3.15E-0
6-4
.25E-0
52.06E-0
6-1
.67E-0
4-3
.10E-0
51.36E-0
5-1
.78E-0
4-5
.23E-0
53.74E-0
5RV
5min
-1.01E-0
4-2
.79E-0
5-9
.80E-0
50
-3.82E-0
5-1
.85E-0
5-1
.78E-0
4-3
.35E-0
59.46E-0
6-1
.85E-0
4-4
.96E-0
5-3
.29E-0
5AdjTSRV
5min
-1.06E-0
4-3
.76E-0
5-1
.03E-0
4-1
.88E-0
50
-4.46E-0
6-1
.77E-0
4-1
.77E-0
5-1
.26E-0
5-1
.85E-0
4-2
.19E-0
5-5
.04E-0
5Av
RV
5min
-1.06E-0
4-4
.39E-0
5-1
.05E-0
4-9
.45E-0
6-4
.25E-0
50
-1.78E-0
4-4
.47E-0
51.31E-0
5-1
.85E-0
4-5
.20E-0
5-4
.53E-0
5HHT
1min
6.23E-0
51.50E-0
59.09E-0
51.24E-0
5-3
.76E-0
52.32E-0
60
-2.71E-0
51.82E-0
5-3
.33E-0
5-5
.23E-0
55.24E-0
5HHT
5min
-1.06E-0
4-3
.45E-0
5-1
.04E-0
4-1
.51E-0
5-1
.32E-0
58.06E-0
6-1
.78E-0
40
-1.22E-0
5-1
.85E-0
4-3
.46E-0
5-5
.08E-0
5HHT
Filte
red
-1.06E-0
4-6
.02E-0
5-1
.05E-0
4-4
.95E-0
5-5
.04E-0
5-4
.75E-0
5-1
.78E-0
4-3
.86E-0
50
-1.85E-0
4-5
.47E-0
5-4
.24E-0
5M
ax
8.93E-0
52.45E-0
59.93E-0
51.24E-0
5-3
.58E-0
52.31E-0
62.70E-0
5-2
.71E-0
51.82E-0
50
-5.23E-0
55.24E-0
5M
in-1
.07E-0
4-3
.71E-0
5-1
.05E-0
4-1
.51E-0
55.33E-0
6-2
.66E-0
6-1
.78E-0
49.32E-0
6-1
.58E-0
5-1
.85E-0
40
-5.24E-0
5Avera
ge
-9.55E-0
5-4
.03E-0
5-9
.56E-0
5-1
.11E-0
5-4
.04E-0
5-6
.18E-0
6-1
.78E-0
4-2
.91E-0
52.76E-0
6-1
.85E-0
4-5
.23E-0
50
Hourly
Retu
rn-6
.27E-0
5-2
.77E-0
5-5
.36E-0
5-1
.13E-0
5-3
.43E-0
5-8
.12E-0
6-1
.54E-0
4-2
.82E-0
53.96E-0
6-1
.68E-0
4-4
.76E-0
5-1
.04E-0
5Hourly
Retu
rnStD
4.02E-0
44.05E-0
44.00E-0
44.03E-0
43.93E-0
44.03E-0
43.88E-0
43.96E-0
44.00E-0
43.84E-0
43.82E-0
44.02E-0
4Sim
ple
AnnualRetu
rn%
-37.90
-16.76
-32.43
-6.86
-20.77
-4.91
-93.02
-17.04
2.39
-101.52
-28.77
-6.29
AnnualRetu
rnStd
%3.12
3.15
3.11
3.13
3.06
3.13
3.02
3.08
3.11
2.98
2.97
3.12
Retu
rnRankin
g10
59
47
211
61
12
83
Chapter 6
Discussion & Conclusion
6.1 Discussion
The world is full of complex systems and processes that change their stochastic behaviour
over time. By restricting our attention to financial time series the implications of this
research have not been fettered or narrowed, the financial setting is merely a focal point
for the methodology developed herein.
The goal of this thesis was essentially to explore low and high frequency volatility
estimation in the light of a new tool, the HHT, which had hitherto been largely neglected
in this context. To that end, clear contributions have been made in the realms of low and
high frequency volatility estimation using the HHT procedure, which has again proved
its worth and demonstrated that its true value lies in its flexibility.
The HHT was successfully used to break down complicated financial time series infor-
mation into simpler wave like structures which were more amenable to intuitive analysis.
This ability to decompose complicated signals is demonstrated by the spectral analysis
components of Sections 2.2.3 & 4.4.1. In these sections, the EMD process at the heart of
the HHT procedure can be seen to act as an effective band-pass filter and the spacings
between frequencies clearly indicate that it performs as a dyadic filter, a result which can
112
Chapter 6. Discussion & Conclusion 113
also be achieved with some wavelet transforms. This parallel with wavelets may lead
to further developments and a more complete mathematical description of the EMD
process which has so far proved an intractable problem for researchers due to the adap-
tive nature of the EMD algorithm. Furthermore, this filter bank structure allows for a
convenient method of estimating the Hurst exponent which can describe the autocorre-
lation structure of the data. In particular, a change in the Hurst exponent at different
sampling rates indicates that a change in autocorrelation structure has occurred and the
self similarity of the time series may not be preserved through intertemporal aggregation.
In Chapter 3, the filtering nature of the EMD process was also put to use as a means
to generate a proxy to the returns by subtracting a local mean (made up of low frequency
components) from the time series. Again, the filtration structure was used in Chapters
4 & 5 to effectively sift off high frequency noise from a return series. This generated an
approximation to the “true” or frictionless price as well as a volatility estimate which
was free from the inflationary effects of market microstructure.
A parametric HHT based procedure for estimating low frequency volatility is also
proposed and tested in Chapter 3. The procedure is further refined by the completely
non-parametric approach taken in Section 4.1. The later HHT based method needs
no in-sample data for parameter fitting and makes no assumptions on the stochastic
process driving the time series. Also, in the low frequency realm the HHT method is
able to extract key volatility features with a high temporal resolution while maintaining
relatively low levels of noise, a claim which is substantiated in Section 4.2. Furthermore,
perhaps the best feature of the methods developed over the course of this thesis is their
intuitive simplicity. This elegant simplicity comes from the ability of the HHT to rep-
resent seemingly chaotic signals in terms simpler wave like structures that combine to
form more complex ones.
In Chapter 4, a benchmark test was developed specifically to test this claim that
the HHT procedure could adapt quickly to changes in the level of volatility level, while
Chapter 6. Discussion & Conclusion 114
maintaining a low signal to noise ratio. Simulated data and this benchmark were used
to test the claim against the EWMA procedure for many values of the decay parameter
λ. To some satisfaction, the results can all be summarised in one simple plot given by
Figure 4.3, which does indeed show the claim to be true under test conditions.
As an introduction to market microstructure and as an exercise in spectral analysis,
the HHT procedure was then used to estimate the Hurst exponent at different levels
of market microstructure for the three kinds of stochastic volatility processes used in
Chapter 4. A drop in the Hurst exponent would indicate an increasingly rough time
series with a short term memory, just like the fGn studied in Chapter 2. This is exactly
what was observed, with (H < 0.5) indicating the presence of inflationary market mi-
crostructure noise. Markets with alternative kinds of market microstructure noise have
also been considered over the course of this research, some of which have a retarding or
dampening effect on volatility estimates at higher frequencies. These are often associ-
ated with long term memory processes (H > 0.5). Typically this deflationary behaviour
is observed in index data, such as the All Ords Index covering 1st Jan 2000- 31st Dec
2009. The ability of the HHT to estimate the Hurst exponent opens the possibility for
it to be used to indicate the presence and extent of market microstructure noise.
Perhaps the largest contribution the HHT procedure has to make to the study of
volatility estimation is its inherent flexibility. There is no requirement for evenly spaced
data and the approach described by Equation (4.6) is completely parameter free, requir-
ing no lead in data or parameter fitting. The improved speed this approach has over its
predecessor developed in Chapter 3 has enabled more complex methods to be developed
which can deal with the problems of high frequency volatility estimation.
The Filtered-HHT method was then proposed to deal with market microstructure
noise, comparisons with TSRV based methods in a simulated study followed. Results
indicated that the the proposed Filtered-HHT procedure was capable of adjusting for
high levels of market microstructure noise. While most of the error metrics were slightly
Chapter 6. Discussion & Conclusion 115
higher for the HHT based procedure, it is stipulated that this was largely due to the
fact that the TSRV methods were specifically designed to adjust for the kind of additive
noise used in the simulated data studied in Chapter 4, while the HHT based approach
remains more general.
In an effort to test the Filtered-HHT procedure with real high frequency data, a sim-
ulated options market was constructed in Chapter 5. This simulated market was based
on several competing volatility forecasts and used real high frequency exchange data.
This approach of using a simulated options market is a novel and effective way to assess
forecast performance on an unobservable variable. The purpose of this test was to see
how the different volatility estimates performed when real market microstructure was
encountered, microstructure which isn’t the simple additive noise used in the simulations
of Chapter 4. Another interesting aspect of this chapter is the more complex evaluation
criteria, while other chapters used measures like ME, MSE, MAPE, etc, this simulated
study involved a nonlinear pricing formula (the Black-Scholes equation), which added an
element of realism to the testing. The results for this chapter also indicate that the HHT
based approach combined with the frequency filtering capability of the EMD process
was able to accurately extract volatility levels in the presence of market microstructure.
Furthermore, this approach met with more success than its RV based competitors for
these simulations, perhaps owing to the lack of assumptions made by the HHT approach.
One possible drawback of the simulated options market approach is that the per-
formance indicators were entirely dependent on the methods that were chosen to be
included in the market. While the results obtained still provided an effective and infor-
mative rankings and pairwise comparisons, they are harder to quantify outside of the
context of this simulated market. For example, the average profits and losses for each
method are entirely dependent on the presence of the other eleven simulated market
participants, some of which (like the one denoted as RV 1 min) have no place in a high
frequency market and were only included to show how poorly they perform under such
Chapter 6. Discussion & Conclusion 116
conditions.
6.2 Future Work
The global financial markets are experiencing a paradigm shift in the way trading is
conducted. The trend is shifting away from speculators calling their brokers to make
long term investments and towards automated systems which trade against one another
in an environment where mere nanoseconds matter. In such a marketplace, having the
most up to date volatility estimate is a clear advantage.
One of the key features of the HHT procedure is that it can yield instantaneous,
rather than integrated volatility estimates. For clarification, the term ‘instantaneous
volatility’ isn’t referring to its calculation time, rather that volatility estimates are given
for every point of a time series. Furthermore, due to the cubic splining procedure used
in the EMD phase of the HHT process, IMFs produced and hence volatility estimates
are actually continuous functions. While the procedures outlined in this thesis are ca-
pable of producing such a high frequency estimate, as demonstrated by Figure 5.2, it is
a goal of future work to improve upon these instantaneous measurements and test their
performance as short term forecasters. There are two approaches currently undergoing
development. The first approach is to rescale IMFs so that the power spectra produced
are of equal power, the second involves making assumptions on the underlying auto-
correlation structure of the noise and taking it into account in a similar manner to the
TSRV method.
At the high frequencies considered for the simulated options market of Chapter 5,
many of the assumptions underlying the Black-Scholes model become invalid. In particu-
lar, the assumption that prices come from a continuous distribution becomes particularly
strained at scales when market microstructure starts to dominate and returns become
highly discretised. To account for this, an option pricing formula based on more discrete
Chapter 6. Discussion & Conclusion 117
stock movements such as a binomial or trinomial method could be used and the simu-
lated options market study could be redone under a more discretised framework which
may be better suited to the behaviour of markets at high frequency. The trinomial
model developed by Haahtela [72] would be particularly well suited to this as it allows
for changing volatility estimates over time.
There is also much interest in the relationship between volume, volatility and price.
A higher dimensional version of the EMD algorithm such as the one proposed by Sinclair
and Pegram [73] could be used to explore the relationship between volume, volatility
and price. This would allow for the creation of a three dimensional surface which, it
is hoped, will shed some light on how these variables interact with one another and
perhaps lead to better forecasting models.
Some research was also done into potential improvements in the way overnight re-
turns are handled with low frequency data. The research explored the idea of using the
opening and closing prices of a stock or index to split the information into two corre-
lated time series which could then be recombined to give a more accurate result. While
promising, further research into the idea was halted after the the work done by Bertram
[74] was found to cover the concept in sufficient depth.
Assessing the performance of the Filtered-HHT procedure on different kinds of mar-
ket microstructure noise is also an area of interest. Further work in this area would help
substantiate claims made on the flexibility of the methods developed herein. One line
of inquiry which was partially explored over the course of this thesis, are the random
processes generated by the Karhunen-Loeve transform. There are many similarities be-
tween the way this transform constructs random walks and the way in which the HHT
procedure breaks them back down again and so this could be the starting point in
an investigation into long memory processes with microstructure noise that attenuates
volatility estimates rather than inflates them.
Chapter 6. Discussion & Conclusion 118
6.3 Concluding Remarks
This thesis has provided an in depth look at the potential for the HHT procedure as a
volatility estimator. A procedure which has largely been ignored by financial practition-
ers has now been built up to the point where it could be considered an effective tool in
both the high and low frequency realms.
The results of several simulated experiments and two studies using real financial data
all indicate that the HHT based volatility estimators are competitively accurate with
popular alternatives and are worthy of further research. Furthermore, the interpretation
of volatility as a sum of wave amplitudes has a certain intuitive and elegant simplicity
which is appealing in its own right.
Bibliography
[1] Norden E. Huang, Zheng Shen, Steven R. Long, Manli C. Wu, Hsing H. Shih, Qua-
nan Zheng, Nai-Chyuan Yen, Chi Chao Tung, and Henry H. Liu. The empirical
mode decomposition and the Hilbert spectrum for nonlinear and non-stationary
time series analysis. Proceedings of the Royal Society of London. Series A: Mathe-
matical, Physical and Engineering Sciences, 454(1971):903–995, 1998.
[2] Harry M. Markowitz. Portfolio Selection. Blackwell Publishers, Malden, MA, 2nd
edition, 1991.
[3] Ser-Huang Poon and Clive W.J. Granger. Forecasting Volatility in Financial Mar-
kets: A Review. Journal of Economic Literature, 41:478–539, 2003.
[4] Robert C. Merton. On estimating the expected return on the market : An ex-
ploratory investigation. Journal of Financial Economics, 8(4):323–361, December
1980.
[5] Ioannis Karatzas and Steven E. Shreve. Brownian Motion and Stochastic Calculus.
Springer US, 1988.
[6] Torben G. Andersen, Tim Bollerslev, Francis X. Diebold, and Paul Labys. Modeling
and Forecasting Realized Volatility. Econometrica, 71(2):579–625, March 2003.
[7] Torben G. Andersen, Tim Bollerslev, Francis X. Diebold, and Paul Labys. (Under-
standing, Optimizing, Using and Forecasting) Realized Volatility and Correlation.
New York University, Leonard N. Stern school finance department working paper
seires, 1999.
119
Bibliography 120
[8] Shyan-Rong Chou, Andy Chien, Chang-Cheng Changchien, and Chien-Hui Wu. A
Comparison of the Forecasting Volatility Performance on EWMA Family Models.
International Research Journal of Finance and Economics, 54:19–28, 2010.
[9] Jie Ding and Nigel Meade. Forecasting accuracy of stochastic volatility, GARCH
and EWMA models under different volatility scenarios. Applied Financial Eco-
nomics, 20:771–783, 2010.
[10] P. Wilmott. Paul Wilmott on Quantitative Finance, volume 3. Springer, 2006.
[11] Robert F. Engle. Autoregressive Conditional Heteroskedasticity with Estimates of
the Variance of United Kingdom Inflation. Econometrica, 50:987–1007, 1982.
[12] Benoit Mandelbrot. The Variation of Certain Speculative Prices. The Journal of
Business, 36:394, 1963.
[13] Tim Bollerslev. Generalized Autoregressive Conditional Heteroskedasticity. Journal
of Econometrics, 31:307–327, 1986.
[14] F. Black. Studies of stock price volatility changes. In Proceedings of the 1976
Meeting of the Business and Economic Statistics Section. American Statistical As-
sociation, Washington, D.C, 1976.
[15] Lawrence R. Glosten, Ravi Jagannathan, and David E. Runkle. On the relation
between the expected value and the volatility of the nominal excess return on stocks.
Staff Report 157, Federal Reserve Bank of Minneapolis, 1993.
[16] Daniel B. Nelson. Conditional Heteroskedasticity in Asset Returns: A New Ap-
proach. Econometrica, 59(2):347–70, March 1991.
[17] Robert F. Engle, David M. Lilien, and Russell P. Robins. Estimating Time Varying
Risk Premia in the Term Structure: The ARCH-M Model. Econometrica, 55(2):
391–407, March 1987.
[18] Yacine Ait-Sahalia, Jianqing Fan, and Yingying Li. The Leverage Effect Puzzle:
Disentangling Sources of Bias at High Frequency. Working Paper 17592, National
Bureau of Economic Research, November 2011.
Bibliography 121
[19] Torben G. Andersen, Tim Bollerslev, and Francis X. Diebold. Parametric and
Nonparametric Volatility Measurement. Working Paper 279, National Bureau of
Economic Research, August 2002.
[20] Stephen Figlewski. Forecasting Volatility. Financial Markets, Institutions & In-
struments, 6(1):1–88, 1997.
[21] M. Martens. Measuring and Forecasting S&P 500 Index-Futures Volatility Using
High-Frequency Data. Journal of Futures Markets, 22:497–518, 2002.
[22] Torben G. Andersen, Tim Bollerslev, and Francis X. Diebold. Parametric and
Nonparametric Volatility Measurement. NBER Technical Working Papers 0279,
National Bureau of Economic Research, Inc, August 2002.
[23] F. M. Bandi and J. R. Russell. Microstructure Noise, Realized Variance, and Op-
timal Sampling. The Review of Economic Studies, 75(2):339–369, 2008.
[24] Lan Zhang, Per A. Mykland, and Yacine Ait-Sahalia. A Tale of Two Time Scales:
Determining Integrated Volatility With Noisy High-Frequency Data. Journal of the
American Statistical Association, 100:1394–1411, December 2005.
[25] Fulvio Corsi, Gilles Zumbach, Ulrich Muller, and Michel Dacorogna. Consistent
High-Precision Volatility from High-Frequency Data. Economic Notes, 30:183–204,
2001.
[26] Torben G. Andersen, Tim Bollerslev, Francis X. Diebold, and Paul Labys. The Dis-
tribution Of Realized Exchange Rate Volatility. Journal of the American Statistical
Association, 96(453):42–55, March 2001.
[27] Yacine Ait-Sahalia, Per A. Mykland, and Lan Zhang. Ultra high frequency volatility
estimation with dependent microstructure noise. Journal of Econometrics, 160(1):
160–175, January 2011.
[28] Ole E. Barndorff-Nielsen, Peter Reinhard Hansen, Asger Lunde, and Neil Shephard.
Designing Realized Kernels to Measure the ex post Variation of Equity Prices in
the Presence of Noise. Econometrica, 76(6):1481–1536, 2008.
Bibliography 122
[29] Peter R. Hansen, Jeremy Large, and Asger Lunde. Moving Average-Based Estima-
tors of Integrated Variance. Econometric Reviews, 27(1-3):79–111, 2008.
[30] H. Ebens. Realized stock index volatility. Working Paper 420, Department of
Economics Johns Hopkins University, Baltimore, 1999.
[31] G. Curci and F. Corsi. A discrete sine transform approach for realized volatility
measurement. Working paper, University of Southern Switzerland, 2004.
[32] D. Donnelly. The Fast Fourier and Hilbert-Huang Transforms: A Comparison.
In Proceedings of the Multiconference on Computational Engineering in Systems
Applications, pages 84–88, Beijing, China, 2006.
[33] Patrick Flandrin, Paulo Goncalves, and Gabriel Rilling. EMD equivalent filter
banks, from interpretation to applications. In Hilbert-Huang Transform and Its
Applications, pages 57–73. World Scientific Publishing Co. Pte. Ltd., 2005.
[34] P. Flandrin, G. Rilling, and P. Goncalves. Empirical mode decomposition as a filter
bank. Signal Processing Letters, IEEE , 11(2), 2004.
[35] H.G. Li and G. Meng. Detection of harmonic signals from chaotic interference by
empirical mode decomposition. Chaos, Solitons & Fractals , 30(4):930–935, 2006.
[36] Zhaohua Wu and Norden E. Huang. Statistical significance test of intrinsic mode
functions. In Hilbert-Huang Transform and Its Applications, pages 107–127. World
Scientific Publishing Co. Pte. Ltd., 2005.
[37] Zhaohua Wu and Norden E. Huang. A study of the characteristics of white noise
using the empirical mode decomposition method. Proceedings of the Royal Society
of London A: Mathematical, Physical and Engineering Sciences, 460(2046):1597–
1611, 2004.
[38] Bao Liu, S. Riemenschneider, and Y. Xu. Gearbox fault diagnosis using empir-
ical mode decomposition and Hilbert spectrum. Mechanical Systems and Signal
Processing, 20(3):718–734, 2006.
Bibliography 123
[39] N. Senroy, S. Suryanarayanan, and P.F. Ribeiro. An Improved Hilbert-Huang
Method for Analysis of Time-Varying Waveforms in Power Quality. IEEE Trans-
actions on Power Systems , 22:1843–1850, 2007.
[40] T. Yalcin, O. Ozgonenel, and U. Kurt. Feature vector extraction by using empirical
mode decomposition for power quality disturbances. In Proceedings of the 10th
International Conference on Environment and Electrical Engineering , pages 1–4,
May 2011.
[41] Md Khademul Islam Molla, Akimasa Sumi, and M. Sayedur Rahman. Analysis of
temperature change under global warming impact using empirical mode decompo-
sition. International Journal of Information Technology, 3(2):131–139, 2006.
[42] A.R. Rao and E.-C. Hsu. Hilbert-Huang Transform Analysis of Hydrological and
Environmental Time Series. Springer, 2008.
[43] Paul A. Hwang, Norden E. Huang, David W. Wang, and James M. Kaihatu. Hilbert
Spectra of Nonlinear Ocean Waves. In Hilbert-Huang Transform and Its Applica-
tions, pages 211–225. World Scientific Publishing Co. Pte. Ltd., 2005.
[44] Derek A.T. Cummings, Rafael A. Irizarry, Norden E. Huang, Timothy P. Endy,
Ananda Nisalak, Kumnuan Ungchusak, and Donald S. Burke. Travelling waves
in the occurrence of dengue haemorrhagic fever in Thailand. Nature, 427(6972):
344–347, 2004.
[45] Xun Zhang, KK Lai, and Shou-Yang Wang. A new approach for crude oil price
analysis based on Empirical Mode Decomposition. Energy economics, 30(3):905–
918, 2008.
[46] P. Crowley. How Do You Make A Time Series Sing Like a Choir? Extracting
Embedded Frequencies from Economic and Financial Time Series using Empirical
Mode Decomposition. Studies in Nonlinear Dynamics & Econometrics, 16(5), 2012.
[47] Norden E. Huang, Man-Li Wu, Wendong Qu, Steven R. Long, and Samuel S. P.
Shen. Applications of Hilbert-Huang transform to non-stationary financial time
series analysis. Applied Stochastic Models in Business and Industry, 19(3):245–268,
2003.
Bibliography 124
[48] Y. Kwei Wen and Ping Gu. The Hilbert-Huang Transform in Engineering. CRC
Press, 2005.
[49] Jacques Longerstaey and Martin Spencer. RiskMetrics Technical Document. Mor-
gan Guaranty Trust Company of New York: New York, 1996.
[50] D. Ruppert. Statistics and Data Analysis for Financial Engineering. Springer Texts
in Statistics. Springer, 2010.
[51] Petra Posedel. Properties and Estimation of GARCH(1,1) Model. metodoloski
zvezki, 2(2):243–257, 2005.
[52] P. Reinhard Hansen and A. Lunde. Does anything beat a GARCH(1,1)? A com-
parison based on test for superior predictive ability. In Proceedings of the IEEE
International Conference on Computational Intelligence for Financial Engineering,
pages 301–307, 2003.
[53] Torben G. Andersen, Tim Bollerslev, and Nour Meddahi. Realized volatility fore-
casting and market microstructure noise. Journal of Econometrics, 160(1):220–234,
January 2011.
[54] Emilio Barucci, Davide Magno, and Maria Elvira Mancino. Fourier volatility fore-
casting with high-frequency data and microstructure noise. Quantitative Finance,
12(2):281–293, September 2012.
[55] Ole E. Barndorff-Nielsen and Neil Shephard. Econometric analysis of realized
volatility and its use in estimating stochastic volatility models. Journal of the
Royal Statistical Society: Series B (Statistical Methodology), 64(2):253–280, 2002.
[56] Eugenie Hol and Siem J. Koopman. Stock Index Volatility Forecasting with High
Frequency Data. Tinbergen Institute, Tinbergen Institute Discussion Papers, 2002.
[57] Ruey S. Tsay. Analysis on Financial Time Series. Wiley, Hoboken, NJ, 3 edition,
2010.
[58] Siem J. Koopman, B. Jungbacker, and Eugenie Hol. Forecasting daily variability of
the S&P 100 stock index using historical, realised and implied volatility measure-
ments. Journal of Empirical Finance, 12:445, 2005.
Bibliography 125
[59] Peter Reinhard Hansen and Asger Lunde. A realized variance for the whole day
based on intermittent high-frequency data. Journal of Financial Econometrics, 3
(4):525–554, 2005.
[60] Peter R. Hansen and Asger Lunde. A forecast comparison of volatility models: does
anything beat a GARCH(1,1)? Journal of Applied Econometrics, 20(7):873–889,
2005.
[61] Patrick Flandrin. EMD code, 2007. URL http://perso.ens-lyon.fr/patrick.
flandrin/emd.html.
[62] Norden E. Huang, Man-Li C. Wu, Steven R. Long, Samuel S.P. Shen, Wendong
Qu, Per Gloersen, and Kuang L. Fan. A confidence limit for the empirical mode
decomposition and Hilbert spectral analysis. Proc. R. Soc. Lond., 459(2037), 2003.
[63] G. Rilling, P. Flandrin, and P. Goncalves. Empirical mode decomposition, frac-
tional Gaussian noise and Hurst exponent estimation. In Proceedings of the IEEE
International Conference on Acoustics, Speech, and Signal Processing , volume 4,
pages 489–492, March 2005.
[64] R. Deering and J.F. Kaiser. The use of a masking signal to improve empirical mode
decomposition. In Proceedings of the IEEE International Conference on Acoustics,
Speech, and Signal Processing , volume 4, pages 485–488, 2005.
[65] R. Rilling and P. Flandrin. One or Two Frequencies? The Empirical Mode De-
composition Answers. IEEE Transactions on Signal Processing, 56 Issue:1:85–95,
2008.
[66] Jr. Massey, Frank J. The Kolmogorov-Smirnov Test for Goodness of Fit. Journal
of the American Statistical Association, 46(253):68–78, 1951.
[67] Robert F. Engle, Che-Hsiung Hong, and Alex Kane. Valuation of Variance Forecast
with Simulated Option Markets. NBER Working Papers 3350, National Bureau of
Economic Research, Inc, 1990.
[68] Federico M. Bandi, Jeffrey R. Russell, and Chen Yang. Realized volatility forecast-
ing and option pricing. Journal of Econometrics, 147(1):34–46, 2008.
Bibliography 126
[69] Luc Bauwens, Sebastien Laurent, and Jeroen V. K. Rombouts. Multivariate
GARCH models: a survey. Journal of Applied Econometrics, 21(1):79–109, 2006.
[70] Canyang Hu. Simulation of nonstationary processes based on orthogonal Hilbert-
Huang transform. In Proceedings of the 3rd International Congress on Image and
Signal Processing, volume 7, pages 3111–3115, Oct 2010.
[71] Lan Zhang. Efficient estimation of stochastic volatility using noisy observations: a
multi-scale approach. Bernoulli, 12(6):1019–1043, December 2006.
[72] T. Haahtela. Recombining Trinomial Tree for Real Option Valuation with Changing
Volatility. In Proceedings of the 14th Annual International Conference on Real
Options, Rome, Italy, 2010.
[73] S. Sinclair and G. G. S. Pegram. Empirical Mode Decomposition in 2-D space and
time: a tool for space-time rainfall analysis and nowcasting. Hydrology and Earth
System Sciences, 9:127–137, 2005.
[74] William K. Bertram. An empirical investigation of Australian Stock Exchange data.
Physica A: Statistical Mechanics and its Applications, 341(C):533–546, 2004.