Non-Functional Testing at London Stock Exchange5 April 2017Iosif Itkin, Exactpro CEO & Co-FounderLondon Stock Exchange Group

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 2460, +1 415 830 38 49www.exactpro.com

2

Clients Worldwide:

Software Quality Assurance experts for global financial markets enjoying steady growth thanks to expertise and unparalleled service

Exactpro is:

3

London Stock Exchange Group

1/8 of employees working in QA

128m of transactions per day near Brexit and US elections

84 microseconds equity markets roundtrip times

4

No Trust No Fear No Begging

Question Everything. Keep Calm. Be Confident.

5

Build Software to Test Software

6

Heavy Load - Easy

7

Heavy Load - EasyFinding Functional Problems - Difficult

8

Latency Measurement

9

Latency Measurement

10

Testing Methods

1.

3.

Simulate real trading participants

2.

11

Diverse and Relevant Load / Validation

1. Random load “Pace Maker” 2. Variety of Passive Liquidity

3.  “Aggressors” try to interact with the market to increase the number of test cases happening in the environment

4. Analyze the situation in retrospect. Make sure that what happened is correct. Tick checkboxes against tests that were actually executed

Updated test

library

12

Failover Tests and Catching In-Flight Transactions

13

Have No Fear

14

Keep Calm

• Unrealistic Scenarios

• Annihilate the System Under Test

• Repeat the Same Tests Many Times

• Load Levels that Exceed Presumed Production

15

Multiple Failover Scenarios

16

• Test Tools• Confirmation that PROD is Configured as TEST• Max Load Figure• Load Scenarios for Existing Production System• All Test Scenarios in Advance• Constant Support

Do Not Beg For:

17

“Train Hard, Fight Easy” (A. Suvorov)

http://img-fotki.yandex.ru/get/31286/15397073.5e9/0_167068_902ac8e_XL

http://gsvu.mvd.ru/upload/site142/folder_page/temp/__1429820772_866/fb3fa781009c14b1cfb62847cf9f9485.jpg

Suvorov battling his way through the snow-capped Alps

18

Open Access Software Testing

www.exactpro.com

19

Questions & Answers

https://ru.linkedin.com/in/iosifitkinThe seventh EXTENT conference will take place in London, UK.

Find out more: www.extentconf.com

– LSEG Technology and Quality Assurance– Risk controls and FPGA– Market Surveillance systems– Blockchain and trading technology trends– Artificial intelligence