When Testers Feel Left Out in the Cold

4/27/2015

1

© 2015 LogiGear Corporation. All Rights Reserved

Hans Buwalda

LogiGear

hans @ logigear.com

STAREAST 2015

Orlando, Florida

"Test Management" Track

Session W1

Wednesday, May 6, 11.30 – 12.30


Who is your speaker

� Software testing company, around since 1994

� Testing and test automation services:− consultancy, training

− test development and automation services

− "test integrated" development services

− focus on big and complex testing projects

� Products:− TestArchitect™, TestArchitect for Visual Studio™

− integrating test development with test management and automation

− based on modularized keyword-driven testing

www.logigear.comwww.testarchitect.com

� Dutch guy, in California since 2001

� Background in math, computer science, management

� Since 1994 focusing on automated testing− keywords, agile testing, big testing

Hans Buwalda

LogiGear Corporation

hans @ logigear.comwww.happytester.com

4/27/2015

2


Scope of this talk

� Introduction to the management aspects specific for a test

project

� Not a project management course

� A personal view, based on experiences with my own

method

� Collection of ideas and experiences, not necessarily a

complete picture

� Input and discussions welcome (also talk with each other!)

� The ideas are coming from mixed European, American and

Asian experiences, apply with care


The Challenges for a Test Process

� testing should be fun

� testing should be effective

� testing should be efficient

� testing should be under control

4/27/2015

3


Actions

4 actions, each

with an action

keyword and

arguments

read from top

to bottom

fragment from a test with actions

acc nr first last

open account 123123 John Doe

acc nr amount

deposit 123123 10.11

deposit 123123 20.22

acc nr expected

check balance 123123 30.33

• The test developer creates tests using actions with keywords and arguments

• Checks are, as much as possible, explicit (specified expected values)

• The automation task focuses on automating the keywords, each keyword is automated only once

• This technique can be very scalable. A similar approach is behavior based testing, which also works with human readable tests, but is more verbose


High Level Test Design - Test Development Plan

Objectives

Test Module 1

Test Cases

Test Module 2 Test Module N

Actions

. . .

AUTOMATION

Objectives Objectives

interaction test business test

Overview Action Based Testing

define the "chapters"

create the "chapters"

create the "words"

make the words work

Test Cases Test Cases

window control value

enter log in user name jdoe

enter log in password car guy

window control property expected

check property log in ok button enabled true

user password

log in jdoe car guy

first last brand model

enter rental Mary Renter Ford Escape

last total

check bill Renter 140.42

4/27/2015

4


Discussing costs and benefits


Examples of costs and benefits

� Involvement of users and

business specialists

� Need for investments

� Build up of a test organisation

� License costs

� Training, coaching

� Maintenance of (much) testware

� ...

� Preventation of incidents

� Shorter time to market

� Increased quality and certainty

� Lesser dependence on users

and experts

� Elimination of dull work

� Improved documentation

� ...

Costs? Benefits?

Business Benefits

• Time-to-Market

• Quality-to-Market

• Control

4/27/2015

5


What can make testing miserable . . .

� Discussions about costs and benefits

� Resistance

� Commitment

� Politics

� Dependence

� Unrealistic expectations

� Difficulty

� Motivation

� Practical issues and problems


Resistance is Normal

� “Let's reconsider”

� Now is not the time

� All the time new objections

� “Fine, but are we ready for this?”

� Saying nothing

� Saying yes, acting no

� The method is good, but in this specific case #.

� “I don't understand this”

� “I didn't expect all this”

� “This is going to cost me my job”

� I'm the star here, I don't need the competition

� “We can't achieve this”

� We will become too dependent on those guys

� Now they will find out how bad we are testing

WHEN THEY SAY : THEY COULD THINK :

4/27/2015

6


GETTING COMMITMENT...


Management Commitment

� Offer solutions, not additional problems

� Tell managers that a good tested system creates a positive

image (not only negative reasons for testing)

� Present/show what you're doing (glass box)

� Try to get clarity about policies and directions for testing

� Keep in mind: managers want things to be under control

− give clear and timely information about (1) progress and (2) results

� Use outsiders and/or books/articles to make your case

� Try to find some bugs . . .

Testing is often not popular . . .

Nobody wants an extra task (extra problem)

Bugs are bad

Testing is good

4/27/2015

7


Commitment, specifics

� “No time, no money, ...”− back to the problem

− you should not become the problem owner!

� “It is so expensive/it is so difficult”− testing is expensive and difficult

− test automation is difficult

� “The others should do the testing”− figure this out

− you can't deal with this yourself (strategic context mismatch)

− "doing things right" versus "doing the right things"

� General vagueness− hidden problems and conflicts


Politics . . .

� Notify the responsible managers early about what is going

to happen

� Make clear written down procedures, especially for the test

execution phase

� Not every bug is equally important (pick your battles)

� Make the test process transparent

� Maintain an atmosphere of cooperation and

communication with all

� Ask for help, sound the alarm, − don’t underestimate the “politics of failure”, it can hurt you

Who is to blame when things go wrong

Systems are sometimes released while not finished

Testers tend to get the bullet

4/27/2015

8


The Life Cycle of an IT Project

� Enthusiasm

� Disappointment

� ! ! ! PANIC ! ! !

� Looking for guilty ones ("blame-storming")

� Punishing the innocent

� Rewarding those who had nothing to do with it

An (Old) Joke, Still Valid


The 3 questions for a bug

1. Is it a problem?

2. What is the root cause?

3. Why did we not catch it?

Ask these question, in the given order

bug fixing…

4/27/2015

9


Dependencies . . .

� Make clear arrangements with everybody about everything,

as early as possible

− written down

� Keep in touch with the rest of the project

� Make the high level test products as early as possible

Testing and automation is dependent on many factors:

• Working systems

• Test environments (like virtual machines)

• Specifications, requirements, domain knowledge

• People

• ...


Dependencies, specifics

� “The system under test isn't available”− The automation will be the first in trouble

− Discuss this (early)

− Allocate resources when really needed

� “The system under test doesn't work”− The automation will be the first in trouble

− Create a “health check” (smoke test to see if all

functionalities that are relevant to the automation work)

� “There is no test environment”− Start planning this as early as you can

− Make clear how important this is

− “No test environment, no test”

4/27/2015

10


Expectations Around Automation

� Message: testing is difficult, automated testing is even

more difficult

� Make clear what is happening

� Communicate, manage expectations

� Don't put too low figures in a test plan− let somebody else lower them (in writing)

� Ask managers and other people involved what they expect

The impression can arise that automated testing (in

particular with keywords) is just a push of a button . . .

. . . even if nobody said so!


Difficult to Keep People Motivated

� Watch the signals

� Make sure that the right people are assigned− Automation engineers should have a programming background

− Testers should have a testing and/or business background

� Avoid “monks work” (Dutch expression for tedious repetitive work)

� Differentiate the work

� Avoid isolation of the test group

� Be prepared to switch roles

� Consider agile approaches

� Be ready for this problem in advance

� Create operational and professional communication structures− for example special interest groups to regularly discuss professional topics

Motivation of team members can erode when time progresses

Happens to both test developers and automation engineers

4/27/2015

11


Complexity: The Work is Difficult

� Don't be over-confident, use coaching from experienced

peers or outsiders

� Consider reference visits to other projects or sites

� Talk about the work, even create special interest groups

� Keep in touch with others doing the same work

� Organize interactions (meetings!), for example with:− users

− developers

− auditors

� Delegate tasks (better lazy then crazy...)

Making tests is supposed to be "difficult":

• Finding bugs of others

• Making concrete examples

Automating them is difficult as well


Practical Issues and Problems

� Be prepared for this, automated testing is a “cloud of

details”

� Put suitable team members in leading roles, and delegate

responsibilities to them

� For larger test projects test management is a full time job,

make sure you get this time

� Try to find easier ways, organize things better, or automate

more

− “If you have a difficult task, ask a lazy man. He will find an easier way”

Does the test tool work here ? ?

Do we have authorizations on the test environment ? ?

Where do we keep our test products ? ? Which tests do we have ? ?

Questions, chores, details, . . .

4/27/2015

12


Some final remarks

� Changing a process can mean:− changing of work processes

− a learning curve

− risks and benefits

� It is not the same as: buy a tool and all will be well

� Changing any process is more than anything an

organisational change and should be managed as

such


Thank you for listening

Pand...

keep cool, even in the cold

(or, in Florida, the heat . . .)