On Improving Quality (Rev.2)

8/9/2019 On Improving Quality (Rev.2)

1/39

Unit Testing

Functional Testing &

Test Driven Development

excellent.


2/39

May 6th 2010 automated stock trading softwaretriggers after a trade. Roughly $1Billion Dollars islost (and subsequently made back by others.)


3/39


4/39

Seriously,dont they

just piss you

off?

Raah


5/39

Industry Average of20 to 50 defects per

1000 Lines of Code.

We will all haveerrors in our code.

The trick is tominimize them.

i r hard

2 find


6/39

QA is a process bywhich we can

automate the hunt

for bugs.

Unit TestingFunctional TestingTest Driven

Development(TDD)


7/39

xUnit testing is a standardizedArchitecture automating the process ofbug hunting.

TheXrepresents the specificFramework(or language/implementation) youre testing in:JUnit (Java)CppUnit (C++)PHPUnit (PHP)BRUnit (Bromine/PHP)


8/39

Implementations of xUnit testingArchitecture will share common features:

A Setup method - variables are loaded intomemory to test against

At least one Assert - a true/false test to see ifsomething worked

A TearDown method - anything necessary tocleanup or reset the testing environment

A final report on success / failure of the tests Syntax might change but the patterns remain


9/39

The mostcommonly usedPHP Unit testing

frameworkCommand line Integrated into

the Zend IDE

QuickLocal


10/39

Developer has PHPUnit

tests stored in a folderin the SVN

Developer makeschanges to the system.Runs Testing on the

command line.

All Passed?

Commit to SVN

NO


11/39

A Java program designedto pretend to be aperson clicking througha web browser

Implemented acrossoperating systems andworks with (virtually)any browser

Selenium is another typeof xUnit specifically fortesting web applications.

Tests can be written orrecorded.


12/39

Broken up into a number of parts Selenium The object or scripting

language of selenium actions. Whencoding, youll tell your seleniumvariable to do things. (Open awebsite, Click on a button, etc)

Selenium IDE a Firefox addonused for recording tests. Writesselenium scripts in the seleniumlanguage. (a great way to learn!)

Selenium RC a Java based serverRemote Control will wait for aninstruction to start running seleniumcommands. Opens a browser and

executes clicks. Other stuff outside the scope of

our talk.


13/39

A Web server application / environmentfor running and storing the results of

Selenium Test Cases


14/39

Bromine Installed on a Web Server

Has list of RC nodesHas list web app to test

Bromine initiates a test sequence

Sends an instruction to a node toRun a single test

The Node opens a web browser,opens the URL of the web app ,executes tests and reports pass/fail

back to the server.


15/39

PHPUnit

Very good at quick tests Very good at testing objects

and classes in their Raw state

Very easy to distribute to thedeveloper team

A nice way to check yourwork before commits (instantfeedback)

Not necessarily ideal fortesting user interaction (but

can run selenium tests) Not restricted to web testing

Selenium

Can be much slower Can emulate a person using a

browser

Designed specifically to testweb applications.

Cross Platform (OS) Cross Platform (Browser) Cross Platform (Programming Language)


16/39

Tests must Improve QualityTests should help to understand the

System Under Test (SUT)

Tests should reduce (not introduce) riskTests should be easy to runTests should be easy to write and

maintain

Tests should require minimal maintenanceas the system revolves around them


17/39

Tests as Specification The heart of Test Driven Development (TDD) Understand what the system must do before writing

it.

Helps to identify ambiguous or contradictoryspecifications before writing. Tests as Bug Repellent

A test written will keep testing for failures Tests as Defect Localization

Unit tests are very small. With only a single pointbeing tested in each one. To that end, any bugintroduced will automatically be pinpointed by a passof the Unit tests.


18/39

Under the Tests as Documentationparadigm, tests tell us what they are

looking for while they run.


19/39

Unit Tests should beorganized in a waythat as is

immediately readableas possible.

The name of the testshould communicate:

What is being tested What the expected

result is


20/39

In legacy systems, changes to one sectioncan have an unexpected effect elsewhere

Like a really shitty domino effect

Automated testing acts as a defense against that Dont introduce risk.

Risk of too much faith in the system Beware of the temptation to add test logic to the system

though its OK to design for testability (eg: IDs)


21/39

Fully Automated- They must require noinput.

Self Checking The must detect andreport errors without manual inspection.

Repeatable they should give the sameresult every time, they should clean up

after themselves. Independent They must not require

any other tests in order to run.


22/39

Simple Tests Tests should verify oneconcern at a time. (Sometimes difficult in

functionality testing)

Separation of ConcernsTest code Separate from Production Code (ie:

dont have a method testme as part of a

production class)

Business Logic tested separately from UIlogic.


23/39

The system should evolve around thetesting.

Sometimes change is required. Tests willbreak.

Minimize impact of change by only testingone condition per test.

DRY Dont Repeat Yourself - Only TestOnce

The minimal number of tests will have to beupdated when change inevitably musthappen.


24/39

Dont spend more time writing tests thanthe code is worth.

Tests should not be significantly moredifficult to write than the systems theyre

testing (that is, after youve learned how to write tests)


25/39

The iPhone 4s early signal tests suggestno issues with the antenna. Youre

probably just holding the phone wrong.


26/39

Coding work does increase in the short term while youwrite the tests. In the longer term however there arenumerous places where youll find saved effort.

Find out right away if something breaks on a new release Find out what breaks & where... Find out if a dependent class breaks because of changes in logic. Find out BEFORE a customer tells you theres a problem


27/39

The risk of bad testing practices is that we writetests that have to be continually refactored, inaddition to the new tests we have to write.

Be aware that writing tests should not be donefor its own sake.

They are to save time and money.


28/39

Retrofitting testing toexisting applications ishard

like pushing water up ahill hard

like herding cats hard Hard.

Ideally applications arewritten to be testablefrom the start.


29/39

Test Driven Development Write the tests before you write the code. By writing the case correct outcome of your

code ahead of time, you are streamlined into

writing correct code. You gain confidence of knowing that your code is

complete. It provides a definition of success, what does

done look like? Youve got evidence that your code is complete.

(all tests pass) Youve got insurance against future muck ups.


30/39


31/39


32/39


33/39


34/39

$this->selenium->type("url_?-?", "http://custom.com");

$this->selenium->type([WHERE], [WHAT]);

[WHERE]

A locator resource. Element ID, CSS Selector id name Javascript DOM (eg: dom=document.forms[myForm].url_?-?;

dom=document.images[56])

XPATH (eg: table[@id='table1']//tr[4]/td[2];)[WHAT]

The data you would like to send


35/39

$X = $this->selenium->getText( "//table[@id='main_table']/thead/tr[1]/th[11]");

The variable $X can store the value on the page forlater use.

This is XPATH syntax //table Read all tables on the page [@id=main_table] that have the ID of main_table /thead/tr[1]/th[11] take the first row, 11th column of the THEAD

element.

Using the Element ID is best, but noteverything has an ID.


36/39

The Assert statement is the key to testingRequires TRUE/FALSE

assertText ( locator, pattern )assertText ( [TEXT HERE], [EQUALS THIS] )

Many commands http://release.seleniumhq.org/selenium-core/1.0/reference.html

But not always quite verbose enough$this->customCommand('waiting','Chk',"Database equals Page Display?", "$DB == $PG");

$this->assertTrue($DB == $PG);


37/39

$DOMAIN_ID_STRING = $this->selenium->getLocation();

$this->selenium->click("link=Domains");

$this->selenium->waitForPageToLoad("30000");

$this->selenium->open($DOMAIN_ID_STRING);

$this->assertTrue($this->selenium->isTextPresent(

"TEST-DOMAIN-$this->t_id.COM"));


38/39

Automated testing will suck in the shortterm.

It will save time in the long run.Unit testing = quick short tests of Classes

and Objects. Do public methods returnthe values that they should?

Functional Testing = longer tests ofapplication functionality. Do clicking thebuttons in the right order give theexpected result?


39/39

Questions?

Documents

On Improving Quality (Rev.2)