1

Lee Copeland Software Quality Engineering

lee@sqe.com

@grandpacopeland

Creating a Better

Testing Future

The World Is Changing and

Testing Must Change With It

2 2

Topics

• Our World is Changing

• The Curse of Past Successes

• Lee’s Don’ts and Do’s for Future Success

3 3

Technology Is Changing

Then

Now

4 4

Customers Are Changing

Less loyal to

brands

Expect more

capability Less tolerant

of low quality

5 5

Business Is Changing

• In the past, organizations controlled external

communication, both content and channels

• Today, with the rise of Social Media …

6 6

Communication is Changing

• The Hotel Story

7 7

Communication is Changing

8 8

Communication is Changing

• Organizational structure controlled tester

communication, both content and channels

• Today, social media can immediately connect

testers world-wide

9 9

Development Process Is Changing

Then

Now

10 10

Development Process Is Changing

• COBOL • CICS • VSAM

• C# • Java • HTML 4/5 • JavaScript • XML • SQL • noSQL

Then

Now

11 11

Testing Must Match Today’s Technology

IEEE 829 Testing

Standard

1983 1983 1987

12 12

Testing Must Match Development

13 13

Topics

• Our World is Changing

• The Curse of Past Successes

• Lee’s Don’ts and Do’s for Future Success

14 14

The Curse of Past Successes

15 15

The Curse of Past Successes

• Processes that made us successful

in the past may prevent us from

being successful in the future

16 16

The Curse of Past Successes

Camera Film

17 17

The Curse of Past Successes

Video Rental Stores

18 18

The Curse of Past Successes

Newspapers

19 19

The Curse of Past Successes

Brick and Mortar Stores

20 20

The Curse of Past Successes

Getting Bills in the Mail

21 21

Topics

• Our World is Changing

• The Curse of Past Successes

• Lee’s Don’ts and Do’s for Future Success

22 22

The Don’ts and Do’s

Focus

Organization

Test Process

Technology Planning

Automation

23 23

The Don’ts and Do’s

Organization

24 24

Organization

• DON’T

– Maintain teams of “developers” and “testers”,

throwing products over the wall

– This was supposed to foster “independence.”

Instead, it created “us vs. them” and “huh?”

Development Test

25 25

Organization

• DON’T

– Establish Testing Centers of Excellence

– “It’s where the disengaged gather to not

produce”

– David Hussman

26 26

Organization

• DO

– Integrate developers and testers together

on the same team

27 27

The Don’ts and Do’s

Test Process

28 28

Test Process

• DON’T

– Cling to a Waterfall style when your

organization is moving toward Agile

29 29

Test Process

• DON’T

– Try to test EVERYTHING

30 30

Test Process

• DON’T

– Confuse Checking with Testing

Michael Bolton

31 31

Test Process

• When we already believe something to be true,

we verify our belief by checking. Checking is a

process of confirmation, verification, and

validation

• Checking is a highly automatable process

32 32

Test Process

• Testing is something that we do with the

motivation of finding new information

• It is a process of exploration, discovery,

investigation, and learning

• Testing is not automatable. It’s using our brain

in real time, and it’s really fun

33 33

Test Process

• DO

– Adopt a Customer-focused approach to

testing

34 34

Test Process

• DO

– Focus on “What Value Does Testing Add?”

rather than “What Processes Does Testing

Execute?”

35 35

The Don’ts and Do’s

Technology

36 36

Technology

• DON’T

– Use Stone Age techniques, tools, and

technologies

37 37

Technology

• DO

– Embrace new technologies

38 38

Technology

• DO

– Consider open source test tools

39 39

The Don’ts and Do’s

Planning

40 40

Planning

• DON’T

– Adopt rigid, overly detailed

plans

– Restrict testing to explicit

requirements

– Remember: “No battle plan

survives contact with the enemy”

– Helmuth von Moltke

41 41

Planning

• DO

– Create plans that

Tell a story

Present essential data

Inspire and communicate

Describe a reasoned plan of action

42 42

The Don’ts and Do’s

Automation

43 43

Automation

• DON’T

– Rely on Capture/Replay

44 44

Automation

• DON’T

– Try to automate every test

45 45

Automation

• DON’T

– Try to replace humans in testing

46 46

Automation

• DO

– Use Test-driven Development

47 47

Automation

• DO

– Use Acceptance Test-driven Development

48 48

Automation

• DO

– Use Exploratory Testing

“Simultaneous

learning, test design,

and test execution.”

– James Bach

49 49

Automation

• DO

– and Session-based Exploratory Test

Management

Perform Exploratory

Testing in Sessions:

Charter

Timebox

Reviewable result

Debriefing Jon Bach

50 50

The Don’ts and Do’s

Focus

51 51

Focus on Prevention Over Finding

• DO

– Remember, while finding defects is

important

– We should be focused on defect prevention

activities

52 52

The Don’ts and Do’s

Focus

Organization

Test Process

Technology Planning

Automation

53 53

Thanks for Attending

Shameless

commercial

message

lee@sqe.com

54 54

My Thanks To

• Dorothy Graham

• David Hussman

• Jonathan Kohl

• Alon Linetzki

• Dan North

• Martin Pol

• Rob Sabourin

• Alan Shalloway

• Adrian Zwingli

For their ideas, review, and comments