Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49

www.exactpro.com1

Disruptive Functional TestingThe next frontier in Post-Trade systems verification

Open Access Quality Assurance & Related Software Development for Financial Markets

Tel: +7 495 640 2460, +1 415 830 38 49

www.exactpro.com

16 October 2017, SIBOS ConferenceIosif Itkin, Exactpro CEO and co-founder, London Stock Exchange Group

• A specialist firm focused on functional and non-functional testing of systems that process

wholesale financial products, particularly market infrastructures

• A UK company with operations in the US and four QA & software development centres in

Russia

• Part of London Stock Exchange Group as of May 29, 2015

• Incorporated in 2009 with 10 people, our company has experienced significant growth as

satisfied clients require more services; now employing 536 specialists

Front office

Listed Derivatives CRM

Dark PoolT25 Clearing

FXFixed income

SEFReconciliation

Onboarding

Ticker PlantCommondities

Smart Order Router

Trading DesktopsIndex Dissemination

FPGA

EquitiesConnectivity

ExchangeMTF

Swaptions

Exactpro Timeline at a Glance

Delivering Quality to London Stock Exchange Group

Some of our projects (start date):

•Turquoise (2010)

•Borsa Italiana (2011)

•CCP.A (2011)

•FPGA-base Ticker Plant (2012)

•LSE (2012)

•Oslo Bors (2012)

•UnaVista (2013)

•FTSE (2013)

•EuroTLX (2014)

•Russell (2015)

•IDEM (2015)

•LSEDM (2015)

•ELITE (since 2015)

•CurveGlobal (2016)

•GATElab (2016)

•LCH (2016)

•TRADEcho (2016)

•MTS (2017)

•RNS (2017)

Open Access Quality Assurance – a Global Client Network

Faster, not Bigger

What is the main difference between incumbent and disruptive?

Incumbent means something you already know and expect.

Disruptive is when we encounter new things that will re-define what we already know.

Post Trade Systems

Software Testing Challenges

With incumbent testing, you are on the hook of pre-conceptions of how your system under test is supposed to work.

It's only when you do disruptive testing that you really gain new knowledge about the system and learn from it.

Get Off the Hook

Software Testing is Relentless Learning

Defect Mining

The goal of disruptive testing is to find defects that are unlikely to be found if testing is done with other methods.

We see ourselves as a mining operation extracting defects hidden in complex scalable technology platforms.

Test Harness

Agile Transformation

As in any other large organization in the financial services industry, technology used at the London Stock

Exchange Group is going through agile transformation.

Developing a Test Harness for Agile Process

What Agile Development Should Be… and Not

NOT LIKE THIS

LIKE THIS

Testing Challenges in Agile

Managing Complex Systems

The Law of Requisite Variety:

a control system must have at least

as many possible states as the

system it wants to control.

It means that Test Harness

development is a truly challenging

and continuous software

development process.

Test Harness Team

Testing Critical Infrastructures

Testing Critical Infrastructures

Safety Rule #1 with Submarines: don't open portholes when underwater!

Testing Critical Infrastructures

Safety Rule #1 with Submarines: don't open portholes when underwater!

Functional testing: iterate through a finite number of scenarios to prove that the porthole won't open

Non-Functional testing: iterate through a smaller number of scenarios to prove that it won't open by brute force

Testing Critical Infrastructures

Safety Rule #1 with Submarines: don't open portholes when underwater!

Functional testing: iterate through a finite number of scenarios to prove that the porthole won't open

Non-Functional testing: iterate through a smaller number of scenarios to prove that it won't open by brute force

Disruptive testing:

1) iterate through a huge number of random diverse scenarios under load to prove that it won't open

Testing Critical Infrastructures

Safety Rule #1 with Submarines: don't open portholes when underwater!

Functional testing: iterate through a finite number of scenarios to prove that the porthole won't open

Non-Functional testing: iterate through a smaller number of scenarios to prove that it won't open by brute force

Disruptive testing:

1) iterate through a huge number of random diverse scenarios under load to prove that it won't open

2) open the porthole

Thank you