Embed Size (px)
Citation preview
Real-Time Market DataTechnology Overview
Zoltan Radvanyi
Morgan Stanley
Morgan Stanley
• What is market data? Basic terms used in market data world
• Market data processing systems
• Real time requirements
• Designing performance of real-time market data processing software
• Hardware based market data processing systems
• Q&A
Session Outline
Morgan Stanley
Financial markets
Marketplaces to trade financial products like stocks, currencies, etc.
Example: Stock exchanges
(http://en.wikipedia.org/wiki/File:NYSE127.jpg) (https://picasaweb.google.com/razolt/WallStreet#5110261011978568802)
Morgan Stanley
Financial instruments
The things that are traded on a financial market ie. bought by buyers / sold by sellers.
Examples:
• Stock exchanges: stocks
(www.bet.hu)
Morgan Stanley
Orders and trades
• Order: An order made by someone to buy or sell some financial instruments.
• Buy order: someone’s intention to buy a defined number of instruments at a defined price / below a defined price
• Sell order: someone’s intention to sell a defined number of instruments at a defined price / above a defined price
• An order represents a market position of a trader
• Trade: The event when a buyer buys some instruments (e.g. stocks) from a seller at an agreed price
Morgan Stanley
Order book
The list of buy/sell orders investors entered for a financial instrument.
• Buy side: contains the buy orders
• Sell side: contains the sell orders
Order books are maintained electronically by the exchanges
Morgan Stanley
Order book example – buy orders
Individual orders
Buy size Buy price
150 120
360 120
256 120
4 112
12 112
20 100
5 95
1000 95
86 90
Morgan Stanley
Buy size Buy price
150 120
360 120
256 120
4 112
12 112
20 100
5 95
1000 95
86 90
Prices in descending order from the top to the bottom
Order book example – buy orders
Morgan Stanley
Buy size Buy price
150 120
360 120
256 120
4 112
12 112
20 100
5 95
1000 95
86 90
Order book example – buy orders
Price levels
Morgan Stanley
Buy size Buy price
150 120
360 120
256 120
4 112
12 112
20 100
5 95
1000 95
86 90
Order book example – buy orders
Buy size Buy price
766 120
16 112
20 100
1005 95
86 90
1 price – 1 entrySizes are aggregated
Aggregated order book
Full order book
1 order – 1 entry
Morgan Stanley
Order book example – sell orders
Individual orders
Sell size Sell price
2000 122
13 125
145 125
630 125
25 131
300 135
550 140
1 140
12 140
Morgan Stanley
Order book example – sell orders
Sell size Sell price
2000 122
13 125
145 125
630 125
25 131
300 135
550 140
1 140
12 140
Prices in ascending order from the top to the bottom
Morgan Stanley
Order book example – sell orders
Sell size Sell price
2000 122
13 125
145 125
630 125
25 125
300 135
550 140
1 140
12 145
Sell size Sell price
2000 122
813 125
300 135
551 140
12 145
Aggregated order book
Full order book
Morgan Stanley
Order book example – aggregated buy + sell side together
Buy size Buy price
766 120
16 112
20 100
1005 95
86 90
Sell price Sell size
122 2000
125 813
135 300
140 551
145 12
Best buy price = highest price to buy Best sell price = lowest price to sell
Morgan Stanley
Trades
A trade happens when a buy order’s price is equal to or greater than the lowest sell price
or
a sell order’s price is equal to or less than the highest buy price
Morgan Stanley
Trades - example
Buy size Buy price
766 120
16 112
20 100
1005 95
86 90
Sell price Sell size
122 2000
125 813
135 300
140 551
145 12
Morgan Stanley
Trades - example
Buy size Buy price
3000 122
766 120
16 112
20 100
1005 95
86 90
Sell price Sell size
122 2000
125 813
135 300
140 551
145 12
Someone entered an order to buy 3000 pieces of stock for a price of 122 EUR
There are 2000 stocks available for 122 EUR on the sell side
Morgan Stanley
Trades - example
Buy size Buy price
1000 122
766 120
16 112
20 100
1005 95
86 90
Sell price Sell size
125 813
135 300
140 551
145 12
1000 stocks remained unfilled on the buy side at price 122 EUR
All stocks at 122 EUR got filled on the sell sideThe new best sell price is now 125 EUR
Result: 2000 stocks got traded at trade price of 122 EUR.
Morgan Stanley
Market data
(www.buydearborn.com)
Market prices (trade prices)
Order books
Morgan Stanley
Market data
(http://www.artremis.com/article/ing_trading_floor.html)
Market data is important for traders to know what is going on the markets
Morgan Stanley
Market data distribution
Exchange
Network
Market data processing engines
Message distribution network
Traders & trading engines
Morgan Stanley
Morgan Stanley
Market data messages
• Describe market events like price changes
• Generated by exchanges in real-time
• Each message describes an event for one instrument (e.g. company stock) or a group of instruments
• Distributed to consumers via different types of network connections (TCP/IP, UDP multicast)
• Message formats are specific to an exchange
Morgan Stanley
Market data messages - example
A market data message describing one level of the order book of one instrument may have the below structure:
MessageType (2 bytes)
InstrumentName (4 bytes)
OrderBookLevel (2 bytes)
SellPrice (8 bytes)
SellSize (8 bytes)
BuyPrice (8 bytes)
BuySize (8 bytes)
struct OrderBookMessage{
char MessageType[ 2 ];char InstrumentName[ 4 ];char OrderBookLevel[ 2 ];char SellPrice[ 8 ];char SellSize[ 8 ];char BuyPrice[ 8 ];char BuySize[ 8 ];
};
12
OTPB
2
3400
795
3395
625
Specification Example representation in a C program
Real message
Morgan Stanley
Market data messages – protocol stack
Market datamessages
Market data messages
TCPUDP
(mainly multicast)
IP
Data link layer
Physical layer
Morgan Stanley
Real-time: low data latency & high throughput
• Latency: Time it takes for data to arrive at traders from the exchange
• Measured in microseconds
• Ideally less than 50µs
• Throughput: The average number of messages processed in a unit of time.
• Measured in messages/second
• Market peeks around 1 million messages/sec
• Goal : keep latency as low and throughput as high as possible
Morgan Stanley
Why real time?
• Algorithmic trading: trading computers (engines) automatically entering orders on the market
• Computers can react to market events much faster than human traders
• Latency matters! The time between a new order appears on the market, trading decision is made and action is taken may happen below 50 - 100µs
• High frequency trading
• Arbitrage: taking advantage of a price difference between two or more markets
Morgan Stanley
Algorithmic trading example - Arbitrage
Buy size
Buy price
Sellprice
Sell size
10 95 100 30
20 90 105 40
30 85 110 50
20 80 120 60
50 75 130 70
Buy size
Buyprice
Sell price
Sell size
41 125 130 78
97 95 135 300
125 75 140 87
231 70 145 42
2 60 150 96
Market1 Market2
Morgan Stanley
Algorithmic trading example - Arbitrage
Buy size
Buy price
Sellprice
Sell size
10 95 100 30
20 90 105 40
30 85 110 50
20 80 120 60
50 75 130 70
Buy size
Buyprice
Sell price
Sell size
41 125 130 78
97 95 135 300
125 75 140 87
231 70 145 42
2 60 150 96
Market1 Market2
I am going to buy 30 stocks for 100 EUR on Market1and sell all of them for125 EUR on Market2
Morgan Stanley
Algorithmic trading example - Arbitrage
Buy size
Buy price
Sellprice
Sell size
30 100 100 30
10 95 105 40
20 90 110 50
30 85 120 60
20 80 130 70
50 75
Buy size
Buyprice
Sell price
Sell size
41 125 130 78
97 95 135 300
125 75 140 87
231 70 145 42
2 60 150 96
Market1
Market2
Buy 30 stocks for 100 EUR
Morgan Stanley
Algorithmic trading example - Arbitrage
Buy size
Buy price
Sellprice
Sell size
10 95 105 40
20 90 110 50
30 85 120 60
20 80 130 70
50 75
Buy size
Buyprice
Sell price
Sell size
41 125 130 78
97 95 135 300
125 75 140 87
231 70 145 42
2 60 150 96
Market1 Market2
I have now 30 stocks, let’s sell it quickly!
Morgan Stanley
Algorithmic trading example - Arbitrage
Buy size
Buy price
Sellprice
Sell size
10 95 105 40
20 90 110 50
30 85 120 60
20 80 130 70
50 75
Buy size
Buyprice
Sell price
Sell size
41 125 125 30
97 95 130 78
125 75 135 300
231 70 140 87
2 60 145 42
150 96Market1
Market2
Sell 30 stocks for 125 EUR
Morgan Stanley
Algorithmic trading example - Arbitrage
Buy size
Buy price
Sellprice
Sell size
10 95 105 40
20 90 110 50
30 85 120 60
20 80 130 70
50 75
Buy size
Buyprice
Sell price
Sell size
11 125 130 78
97 95 135 300
125 75 140 87
231 70 145 42
2 60 150 96
Market1 Market2
I made a profit of 25 EUR per stock
Morgan Stanley
Market data processing engines
Exchange
Network
Market data processing engines
Message distribution network
Traders & trading engines
Morgan Stanley
Morgan Stanley
Market data processing engines
• Purpose:– Receive market data messages in the exchange’s message
format– Extract information from the messages– Create messages in the firm’s internal message format
containing the extracted information and send them to the message distributor network
• Challenge:– How to process the incoming messages to gain low latency and
high throughput
Morgan Stanley
Market data processing engines
Two main categories of solutions:
• Single threaded message processing
• Multithreaded message processing
Morgan Stanley
Single threaded solution
• Incoming messages are processed sequentially
• Can be used where message rate is low (some thousand messages/sec) and where latency is not so critical (data is only used by human traders)
• Easy to maintain and debug
• For reasonable performance code must be micro-tuned –parsing of one message should not take more than 10 –20 µs
Morgan Stanley
Multithreaded solution using worker threads
• Messages for the same financial instrument have to be processed in the same order as the messages come in
• Can be implemented with a pool of worker threads consuming messages from dedicated input queues
• Messages for the same financial instrument always have to be put in the same queue
Morgan Stanley
Thread1 Thread2 Thread3 Thread4
. . .
Multithreaded solution using worker threads
A2 B1 A3 C1 A4 A5 D1 B2 B3 D2A1
An instrument’s messages will always be processed by the same thread – e.g. messages for instrument “A” will always be put in Thread1’s queue
Morgan Stanley
Issues using worker thread approach
• An instrument may be more actively traded than an other one (e.g. OTP vs. TVK) � uneven distribution of update rates for different instruments leads to uneven distribution of workload among worker threads
• Assigning messages to threads happens on a message basis without having a higher level view of the work that needs to be done
• Hard to define the number of worker threads
• Does not scale well with high number of CPUs
Morgan Stanley
Worker threads
Thread activity –dark green shows active threads
Issues using worker thread approach
Concurrency level - how many threads were simultaneously busy and for how long during the runtime of the process
Test conditions: 16 core box (Intel Nehalem CPUs) running 64 bit RedHat4 OS
Time
Morgan Stanley
Task based parallelism using Intel® TBB threading library
• No need for managing system threads directly
• Developer only deals with creating / grouping logical tasks to execute
• The library deals with the execution of tasks ensuring high level of parallelism− Task stealing: load balancing algorithm to increase CPU
utilization. When a thread has finished with the tasks in its queue it can “steal” tasks from an other thread.
Morgan Stanley
Algorithm of processing incoming messages using TBB:
• Split up a block of incoming messages to groups which can be processed in parallel: messages for one instrument will get into the same group
• Messages in one group will be processed sequentally (in their chornological order) but the different groups will be executed in parallel
• One logical task will be responsible for processing one message group
• Let the TBB library schedule the execution of the tasks
Task based parallelism using Intel® TBB threading library
Morgan Stanley
Thread1 Thread2 Thread3 Thread4
. . .A2 B1 A3 C1 A4 A5D1 B2 E1A1
Task based parallelism using Intel® TBB threading library
F1 E2
Morgan Stanley
Thread1 Thread2 Thread3 Thread4
. . .A2 B1A3 C1A4 D1B2A1
Task based parallelism using Intel® TBB threading library
Task1 Task 2 Task 3 Task 4
TBB scheduler
E1 E2
Task 5
F1
Task 6
Morgan Stanley
How does solution using TBB perform?
Concurrency level
Worker threads
Thread activity
Time
Morgan Stanley
Tools for performance analysis
• Intel® VTune -Thread Profiler
• Intel® VTune – Sampling and Call Graph
• Intel Purify/Quantify
• Valgrind
• Internally developed tools for latency and throughput measurements
Morgan Stanley
FPGA accelerated low-latency market data processing
• Can perfectly used for market data processing
• Flexible
• High throughput with low latency by eliminating the operating system's network stack
• Constant latency, irrespective of throughput ~ 4µs
• Very expensive
• FPGA - Field-Programmable Gate Array
• Configuration: Hardware Description Language (HDL)
• Wide range of applications
Morgan Stanley
Thank you for your attention!
Q&A
