Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
1
Test SMART not HARDand deliver customer value fast
Beth Skurrie (DiUS)@bethesque
Copyright © DiUS 2017
Soundify
soundify
3Copyright © DiUS 2017
4
● Experience and API ○ Short iterations
● Core○ Design○ Build○ Test○ Release
Process
Copyright © DiUS 2017
5
Ideal customer value over time
Copyright © DiUS 2017
6
Core platform customer value over time
Copyright © DiUS 2017
7
Program customer value over time
Copyright © DiUS 2017
8
● Customers
● Business owner
● Delivery manager
● Teams
Problems
Copyright © DiUS 2017
9
Core platform team - iterate
Copyright © DiUS 2017
10
Improved customer value over time!
Copyright © DiUS 2017
Copyright © DiUS 2017
WHAT WE ARE KNOWN FOR
11
Core team iterations
Solved problems New problems
● Incremental customer value
● Reduced risk
● Overhead in releases● Need more testers● 4 week turnaround for
integration bugs
Copyright © DiUS 2017
12
Automated releases
Copyright © DiUS 2017
13
Automated e2e tests
Copyright © DiUS 2017
14
Much smug! So dream!
Copyright © DiUS 2017
Copyright © DiUS 2017
WHAT WE ARE KNOWN FOR
15
Automated e2e tests
Solved problems New problems
● Slow tests● Easy to break● Hard to fix● Scale badly● Lots of set up● Flakey tests ignored● Takes dev time away from
features
● Reduced risk
● Integration bug turnaround - days
● Confidence - when tests passed
Copyright © DiUS 2017
16
Is this worth it?
Copyright © DiUS 2017
17
Why do we do integration tests?
Copyright © DiUS 2017
18
We want to find bugs before they get to production
Copyright © DiUS 2017
19Copyright © DiUS 2017
20
To integration test everything or not to integration test everything?
Copyright © DiUS 2017
21
What if there was another way?
Copyright © DiUS 2017
22
A B
mock
Copyright © DiUS 2017
Copyright © DiUS 2017
WHAT WE ARE KNOWN FOR
23
Test symmetry
Solved problems New problems
● Hard to keep both sides in sync
● Fast feedback
● Few dependencies
● Reliable
● Easy to debug
● No extra infrastructure
Copyright © DiUS 2017
24
A B
mock
Copyright © DiUS 2017
25
A B
mock
Copyright © DiUS 2017
A B
mock
C P
contract
Copyright © DiUS 2017
Copyright © DiUS 2017
WHAT WE ARE KNOWN FOR
27
Contracts
Solved problems New problems
● ???● Keeping tests in sync
Copyright © DiUS 2017
Bug turnaround - minutes
Contracts tests
Copyright © DiUS 2017
Know before you commit
Contracts tests
Copyright © DiUS 2017
Make changes with speed and confidence
Contracts tests
Copyright © DiUS 2017
Deploy independently
Contracts tests
Copyright © DiUS 2017
Better API design
(Consumer) Contracts tests
Copyright © DiUS 2017
Are not functional tests
Contracts tests
Copyright © DiUS 2017
Are not load performance tests SLAs
Contracts tests
Copyright © DiUS 2017
Are not good for public APIs
(Consumer) Contracts tests
Copyright © DiUS 2017
Are not a silver bullet!
Contracts tests
Copyright © DiUS 2017
37Copyright © DiUS 2017
38
Do I still need integration tests?
Copyright © DiUS 2017
39
Integration test coverage
Consequences of bugx
Time to find bugx
Time to release fix
Copyright © DiUS 2017
Copyright © DiUS 2017
WHAT WE ARE KNOWN FOR
40
Speed up your releases
Do less Do more
● Aggregated logging● Metrics● Semantic logging● Alerting
● Integration testing
Copyright © DiUS 2017
Copyright © DiUS 2017
Another story This time a real one!
Copyright © DiUS 2017
42
pact - noun
A formal agreement between individuals or parties. "the country negotiated a trade pact with the US"
synonyms: agreement, protocol, deal, contract
pact.io
Copyright © DiUS 2017
43
● Open source● Multiple languages
○ JVM○ .NET○ Js○ Python○ + more
pact.io
● HTTP contracts● Message contracts
Copyright © DiUS 2017
Copyright © DiUS 2017
Your contract testing story
Copyright © DiUS 2017
45
A B
mock
C P
contract
Copyright © DiUS 2017
46Copyright © DiUS 2017
47
Pact Broker
Copyright © DiUS 2017
48
Automate the contract exchange
Copyright © DiUS 2017
49
WHEN the provider receives
<some request>THEN
it will return <some response>
Copyright © DiUS 2017
50
WHEN the provider receives
a GET request for /alligators/MaryTHEN
it will return a 200 OK response
with a JSON body {“name”: “Mary”}
Copyright © DiUS 2017
51
WHEN the provider receives
a GET request for /alligators/MaryTHEN
it will return
a 404 Not Foundresponse
a 200 OK response
Copyright © DiUS 2017
52
WHEN the provider receives
a GET request for /alligators/MaryTHEN
it will return a 404 Not Found
53
GIVEN<the provider is in a certain state>
WHEN the provider receives
<some request>THEN
it will return <some response>
Copyright © DiUS 2017
54
You still need to think about test data
Copyright © DiUS 2017
Your contract testing journey
● Automate the contract exchange
● You still need to think about test data
Copyright © DiUS 2017
56
Contracts should focus on the messages, not the technology
Copyright © DiUS 2017
Your contract testing journey
● Automate the contract exchange
● You still need to think about test data
● Contracts should focus on the messages, not the technology
Copyright © DiUS 2017
58
Contracts should be as flexible as possible - but no more
Copyright © DiUS 2017
Your contract testing journey
● Automate the contract exchange
● You still need to think about test data
● Contracts should focus on the messages, not the technology
● Contracts should be as flexible as possible - but no more
Copyright © DiUS 2017
60
The other service needs to know when a contract has changed
You need a way to introduce changes without breaking everything
Contracts are not a substitute for good communication between teams
Copyright © DiUS 2017
61
Pact Broker webhooks
Copyright © DiUS 2017
62
Pact Broker tags
Copyright © DiUS 2017
63
Contracts are STILL not a substitute for good communication between teams
Copyright © DiUS 2017
● Automate the contract exchange
● You still need to think about test data
● Contracts should focus on the messages, not the technology
● Contracts should be as flexible as possible - but no more
● The provider needs to know when a contract has changed
● You need a way to introduce changes without breaking everything
● Remember: contracts are not a substitute for good communication between teams
Your contract testing journey
Copyright © DiUS 2017
65Copyright © DiUS 2017
66
You need to share the verification results as well as the contracts
Copyright © DiUS 2017
67
Pact Broker verifications
Copyright © DiUS 2017
68Copyright © DiUS 2017
69
If you can’t deploy your services independently,
you don’t have microservices. You have a distributed monolith.
Copyright © DiUS 2017
70
Consumer version
Provider version
Verification result
11 31 success
12 31 failure
12 32 success
13 32 success
“The Matrix”
Copyright © DiUS 2017
71
Consumer version
Provider version
Verification result
11 31 success
12 31 failure
12 32 (prod) success
13 32 (prod) success
“The Matrix”
Copyright © DiUS 2017
72Copyright © DiUS 2017
73
Spring cloud contracts● Designed for JVM but also supports non-JVM
Copyright © DiUS 2017
74
What about Swagger/Open API?
Copyright © DiUS 2017
75
Provider contracts
Copyright © DiUS 2017
76
Consumer contracts
Copyright © DiUS 2017
77
A B
mock/write
C P
pact swagger
verify
verify
Pact+Swagger
Copyright © DiUS 2017
78
MinimumViableContracts
● Consumer○ mock○ Serialize
● Provider○ replay○ check
Copyright © DiUS 2017
79
● Fast, comprehensive contract tests
● Smoke tests● Semantic monitoring
in production● Alerts● Fast bug turnaround
Soundify
soundify
Copyright © DiUS 2017
80
Actual customer value over time
Copyright © DiUS 2017
81
�
��
�
Success
● Customers
● Business owner
● Delivery manager
● Teams
Copyright © DiUS 2017
Copyright © DiUS 2017
WHAT WE ARE KNOWN FOR
82
Contracts
Solved problems New problems
● ???● Deliver customer value faster
Copyright © DiUS 2017
83
dius.tech/contrca4apact.io
@pact_up
Test SMART not HARDand deliver customer value fast
Beth Skurrie (DiUS)@bethesque
Copyright © DiUS 2017
Copyright
1. The copyright to all content in this presentation including graphics, images, layouts and text belongs to DiUS or we have a licence to use those materials.
2. All trademarks, brands and logos generally identified either with the symbols TM or ® which are used in this presentation are either owned by us or we have a licence to use them. Your access to this presentation does not license you to use those marks in any commercial way without our prior written permission.