Oded Luria | Deep Learning Meetup | Dec 2015

Pricing illiquid assets –

A Deep Learning approach

“Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction.

These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics…

Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.“

Deep Learning in Nature (May 2015)

• see also Jürgen Schmidhuber, Critique of Paper by "Deep Learning Conspiracy" (Nature 521 p 436). http://people.idsia.ch/~juergen/deep-learning-conspiracy.html

Takeuchi and Lee, 2013

• Examine whether deep learning techniques can discover features in time series of stock prices that can successfully predict future returns

• Main idea:

Autoencoder reduces inputs to 4 dimensional features

Classifier outputs probabilities of two classes (returns below/above the median)

• Performance: Overall accuracy rate of 53%

• Open questions:

Use separate autoencoders for different categories of features

The impact of updating the weights over time.

• See also: Gilberto Batres-Estrada, 2015

http://cs229.stanford.edu/proj2013/TakeuchiLee-ApplyingDeepLearningToEnhanceMomentumTradingStrategiesInStocks.pdf

4

http://robusttechhouse.com/list-of-funds-or-trading-firms-using-artificial-intelligence-or-machine-learning/

Everyone is exploring(*from public sources)

5

“If you have a deep learning architecture that can identify those patterns across different time scales, you can arguably use that information to better forecast what will happen next”

“Binatix’s software doesn’t just learn from static data points, but incorporates “temporal signals,” essentially how the information continually changes over time”

Nadav Ben-Efraim (left) and Itamar Arel (right)

Finance and Deep Learning ?

• Weak ‘spatial’ correlation/ parameter tying

• Requires transparent model• Conservative industry

• Lots of features• lots of examples• Complex data• Versatile regimes• Evolving market conditions• Competitive advantage?

(*from public sources)

PROS CONS

Pricing illiquid assets

price uncertainty increases

8

Price uncertainty

1. Accurate pricing2. Prediction confidence3. Transparency

Business Requirements:

trade

theoretical price curve

Time

Pri

ce

9

Data preparation

• Mixture of numerical, boolean & categorical features

• Filtering outliers• Scaling• Filling missing values• Splitting categorical variables• Some degree of feature engineering

Experimental settings

10

Feature engineering?

http://blog.kaggle.com/2012/11/01/deep-learning-how-i-did-it-merck-1st-place-interview/

*Molecular Activity Challenge

*

• Trends• Cross-sectional information• Information about other bonds?

Deep Leaning aspects

12

Research

methodology

This (supervised regression) problem can be approached using a number of methods:

1. Using discriminative models (supervised learning)Use Trace spreads as target labels for classification \ regression problem

2. Using Hybrid models (combine supervised and unsupervised learning)Discrimination assisted with outcomes of generative \ unsupervised networks

3. More options?

Open questions

Classification \Regression?

Type of units?

Use Ensemble learning?

Types of optimizers?

Network depth?

13

Regression Vs Classification for price prediction

regression classification

Direct approach

Cost function: mean square error

Difficult in estimation of example error*

Unimodal model

Ordered classes

Cost function: categorical cross entropy

Performance vs. resolution tradeoff

Good error estimation for each example

Supports multi-modal decisions

14

How deep should the network be?

Shallow networks:

Easier to train

Could suffer from high bias error

Deep networks:

Superior when the data is complex

have more parameters

More difficult to train

More prone to overfit?

*units removed from figure

15

Are the Deep Learning features better?

Ran

do

m F

ore

sts

cla

ss p

red

ictions

raw

fe

atu

res

raw

fe

atu

res

cla

ss p

red

ictions

Ran

do

m F

ore

sts

cla

ssifie

r

Random Forests applied on raw features

Random Forests applied on Deep Learning features

*units removed from figure

16

Are the DL features better?

PCA applied on Deep Learning features

570

-60

spread

low confidencehigh error

high confidencelow error

17

Assessing prediction quality (classification net) Ordinal classification -> smooth probability distribution Indicates spread prediction certainty Similarity to bid-offer spread The predictive power of this index needs to be verified

Pro

ba

bili

ty

Spread [BPS]

18

Training process

Ca

teg

orical C

ross-e

ntr

op

y

Epoch

Bin

Cove

rage

[%

]

19

Ensemble Learning

• Avoids being trapped in saddle points/ bad minima

• Handles initialization problems

• Improves resolution

5 networks1 network

“A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions” (Hinton)

“Another way to exploit computing power to push performance imodel averaging... After training them, the outputs of different networks can be averaged” (text modified from Yoshua Bengio)

Integration into business processes

21

Conclusions

• Applying Deep Learning on Financial datasets is not trivial

• Use it when: Many features Many examples Data has complex relationships

between variables Many regimes Transparency is not required (mostly)

• Think of ways to assess your error

• Use benchmark methods to assess the Deep Learning contribution

oded.luria@citi.com

Thank you