© 2017 by© 2017 by .

Wolfgang Platz, Founder & Chief Strategy Officer

The Future of Testing

© 2017 by

locations

AustriaGermanySwitzerlandUnited KingdomThe NetherlandsPoland

USA

AustraliaSingaporeIndia

founded

2008

employees

390+

support

awards

LeaderGartner MQ

LeaderForrester Wave

customers Forbes Top 10 Global 2000 Companies

6

4

5

4

4

4

investment

computer services

major banks

Australian banks

life insurance

diversified insurance

ASP Award 2016 & 2017

© 2017 by 3

Back in 1999, this is how I felt

© 2017 by

Vision

To be the leader in software testing with the most innovative testing products that enable

our customers to deliver high quality software.

4

© 2017 by

© 2017 by

Jan 2011

Aug 2014

Dec 2015Nov 2017

© 2017 by

“It should be considered by enterpr ises that have struggled to make test autom ation work, and by those seeking to support agile continuous autom ation practices.”

© 2017 by 8

Testing is ready for Disruption

M arket M acros1

2

Testing the hype3

© 2017 by 9

81%of e n t e r pr is e s have adopted or plan to

a dopt Dev O ps .

http://www.rightscale.com/blog/cloud-industry-insights/new-devops-trends-2016-state-cloud-survey

© 2017 by

> 80%M a n u a l Te s t in g

< 20%Automated Testing

U I

A PI

Toda y

The reliance on m a n u a l t e s t in g is the # 1 t e ch n ica l ch a lle n ge in app development1.

1 … Source: World Quality Report 2015 - 2016

35%IT Budget Spend on QA & Testing1

$32BAnnual Spend on Testing Services

Legacy platforms require w e e k s to ex e cu t e t e s t in g.

10

© 2017 by

0

10

20

30

40

60

50

2017 2018 2019 2020 2021

Software Testing Market (total)

35.6

55.312%Annual Growth

(CAGR)

Source. www.technavio.com

Product TestingApplication Testing

Bi l l ionUSD

© 2017 by 12

Testing is ready for Disruption

Testing is still on the r ise

2

Testing the hype3

© 2017 by

Today’s reporting is based on num bers of test cases.

What does it tell us?

Version 7.5 42,278 10,086 910

Is 53,274 the right number of test cases?

Can I go pr odu ct iv e ?

13

© 2017 by 14

Front wingGear box

Tires

Telemetry Software

Steering Wheel

Cassis & MonocoqueEngineRear wing

BrakesExhaust

© 2017 by 15

© 2017 by

The law – (almost) 80:20

With 30% of test cases …

… we cover 80% of our

business risk.

16

© 2017 by

1 … Source: Customer projects, Tricentis

40%average r isk coverage achieved in

comprehensive enterprise test portfolios1

67%average level of redundancyin enterprise test portfolios1

2 out of 3 test cases never yield defects .

60% of risk is secured through hope & pray.

17

© 2017 by

We need to introduce a new currency in testing.

66% 9% 15% 10%Core Bank

Risk Coverage [%]

R isk Coverage

18

© 2017 by 19

18%(official) overall

test autom ationdegree

8%(official) enterpr ise

test autom ationdegree

Really ?

X?

© 2017 by

Continuous Testing requires Autom ation

20

80%Manual Testing

20%Automated Testing

U I

A PI

Today Future

Exploratory Tests

Automated UI Tests

API Tests

Orchestrated Service Virtualization

+85%

- key enabler for high automation rates -

© 2017 by

Comparing learning curves

21

100 pax

“Go train them on Selenium based scr ipting for a certain framework – you got 3 m onths!”

3 m onths

Head of QA Services

© 2017 by

Comparing learning curves

22

100 pax

“Go train them on test automation with Tr icentis Tosca – you got 3 weeks!”

3 m onths

Head of QA Services

100 pax

3 weeks

© 2017 by

Comparing learning curves

23

100 pax

How many made it?3 m onths

100 pax

3 weeks

15%

85%

© 2017 by

Delivering on the promise of ex t r e m e a u t om a t ionrequires a n ew pa r a digm .

24

© 2017 by

M odel-B asedB r ow s e r

Assert.AreEqual(“331,00",driver.FindElement(By.XPath("//table[@id='PriceList']/tbody/tr[10]/td[2]/b")).GetAttribute("value"));

B r ow s e rScr ipt-B ased

CodeSynchronization Behavior

CodeRecovery Handling

CodeData Driven Framework

CodeKeyword Driven Framework

1

2

3

4

25

© 2017 by

Continuous testing is ch a n geIt’s way more than just U I a u t om a t ion

© 2017 byThe three n igh t m a r e s of test automation

© 2017 by

Needs to be stateful & fully integrated Requires service virtualizationEscape it also for E2E systems & APIs

Test PortfolioM a in t e n a n ce

Te s t Da t a Management

Test SystemPr ov is ion in g

© 2017 by

Model-BasedTest Automation

35+ TechnologiesPackaged Applications

Robotics

B I &B ig Data

ServiceVirtualization

ExploratoryTesting

Risk-B asedTesting

TestManagement

ManualTesting

Analytics

IntegrationsTest Data

ManagementToscaFlood

ImpactAnalysis

DevDevelopment

O psOperations

© 2017 by 30

Testing is ready for Disruption

Testing is still on the r ise

Testing the hype3

© 2017 by

B lockchain

IoT Platform & Intelligent Things (Connected Home, etc.)

Innovation Trigger

Peak of Inflated

Expectations

Through of Disillusionment Slope of Enlightenment Plateau of

Productivity

Extracted from Gartner Hype-Cycle, As of July 2017

Plateau to be reached …2 – 5 years5 – 10 years> 10 years

(Deep) Machine Learning & AI

Autonomous Drive 2017Ex

pect

atio

ns

Time

© 2017 by

Tr u s t e d Transactions B lock ch a in will become a Se r v ice

No trusted 3r d pa r t y

B ot t om L in e . We need to test the a pplica t ion s using that s e r v ice

…

Bitcoin’s blockchain Ethereum

25+ significant Blockchain project s

Block chain

© 2017 by

M a ch in e -t o-M a ch in e communication through I n t e l l ige n t Th in gs

Smart h e a lt h , smart e n e r gy ,connected h om e s

B ot t om L in e . Testing needs to be ready for m in i- and m icr o-s e r v ice s

15Billion Devices, 2015

75Billion Devices, 2025

I n t e r n e t of Th in gs

© 2017 by

200,000 B.C.Fire

7,000 B.C.Wheel

Innovat ions

…that changed m ank ind

2007iPhone

© 2017 by

© 2017 by© 2017 by

M obile First Google-CEO Sundar Pichai, Google I/O, 2017

© 2017 by© 2017 by

A I First Google-CEO Sundar Pichai, Google I/O, 2017

© 2017 by© 2017 by

No need to test AI per se, but a pplica t ion s using the ou t com e s

of a r t i f ic ia l in t e l l ige n ce

© 2017 by© 2017 by

Today, there is a t r e m e n dou s a m ou n t of n ois e associated with

AI in the software industry.

© 2017 by© 2017 by

We need to become s m a r t e r intesting, even if there is n o A I involved.

© 2017 by 41

Testing is ready for Disruption

Testing is still on the r ise

Testing is ready for modern Technologies

© 2017 by

Thank You for your attention

Host: Wolfgang Platz

Email: w.platz@tricentis.comm

Follow us on Twitter: @tricentis

Stay in touch!

42