Testing Best Practices

Testing Best Practices

Kevin WhinneryPlatform EvangelistAppcelerator Inc.@kevinwhinnery

[email protected]

mailto:[email protected]

• The Challenge of Testing Rich Client Applications

• Levels of Testing• Titanium Testing Tools• The Future of Testing

Titanium Apps• Q&A

Testing Best Practices

The Challenge of Rich Client Testing

A Few Challenges:

• Relevant tests?• Tooling?• Automation?• Production

environment?• Human

interaction?

Rich Client Testing Challenges

Testing rich client applications

remains a challenge today due to

the visual and human aspects of

client applications.

• Model tier testing is highly doable, and test friendly

• UI testing is hard to do in a meaningful way:• Does code work in a long-

running scenario?• If I programmatically build a

UI structure, how close is that to the real UI structure at runtime?

• I can test object properties, but reliably ensuring visual integrity is hard.

How relevant are our tests?

• In JS, client side test tools are somewhat limited in functionality and scope

• What is available is more geared toward simple applications• …and typically tied to the

browser, in the case of JavaScript

What tools do we use?

• Scripting user interaction is difficult, especially in native mobile apps

• Requires multi-platform environment

• Requires some sort of connection to a running simulator or device, which must also be scripted

How do we automate tests?

• A simulator only goes so far• Need to test on a variety of

devices• Mobile is a hostile and volatile

environment:• Data connectivity?• Network speeds?• Software/hardware

combinations (iOS has this pain now too)

• Background processes?• Geographic locations?

How close are we to prod conditions?

• Does our app “look right” with different data sets?

• Are our buttons responsive enough, or do loading screens take too long?

• Are there operations which are unintuitive, or error prone?

• Are the properties of the UI we are changing and animating actually reflected in the UI how we expect?

How is our “look and feel”?

These problems can all be overcome, but testing can become a classic struggle between…

Time

Returns (Quality)

And Resources

Consider:

• How long will I support this app?

• How useful is the test automation I am adding – how true to life are my test results?

• Automation versus human testing

Diminishing Returns

Test automation can become

extremely expensive. Frameworks

need to be maintained, infrastructure

needs to be created, and time needs

to be spent writing, scripting, or

recording tests.

Make sure the investment is

justified.

Levels of Testing

Types of Testing:

• Ad hoc• Automated Unit

Testing• Continuous

Integration• Automated

Functional Testing• Crowdsourcing

Levels of Testing

Depending on the scope, shelf life,

and team size for every application,

only certain levels of testing may be

needed.

You probably already know these by

some name, but let’s establish a

common vocabulary.

Types of Testing:





Levels of Testing

Ad hoc testing by a single tester or

group of testers may be sufficient for

small applications with a short shelf

life.

Types of Testing:





Levels of Testing

Programmatic unit testing can

provide a baseline comfort level with

code in a project, giving the

developer knowledge that public

APIs have not regressed and are

working as expected.

Types of Testing:





Levels of Testing

Continuous integration can make

use of an existing suite of automated

unit tests, and ensure that all test

suites are passing as new code is

committed to the project.

Continuous integration becomes

especially useful as a team

increases in size and activity.

Types of Testing:





Levels of Testing

Automated functional testing tests

actual application interactions, and

inspects the state of an application

at known points.

This type of testing is sometimes

referred to as “record and play”

testing.

Types of Testing:





Levels of Testing

Crowdsourcing deploys an

application to real humans around

the world, who can test applications

on different data networks, devices,

and locations to provide real-world

feedback on application functionality.

Can also be employed to provide

feedback on look and feel and other

human elements.

Types of Testing:





Where We Are Today

Appcelerator (and our community)

have developed test automation

tools to provide a baseline level of

comfort with application functionality.

The testability of your application

can be greatly enhanced by

employing modular design.

Demo: Testable Titanium apps with behave.js

Types of Testing:





Where We’ll Be Tomorrow

Appcelerator, the community, and

our commercial partners are

collaborating on testing solutions

across the board.

Visit with SOASTA and uTest in the

hallways!

• Test automation is a great tool for ensuring quality

• For rich client apps, useful tests have always been a challenge

• Choose the level of testing appropriate for the project

• Join us in reducing barriers to testing and building awesome tools

To Sum Up

Downloads/Resources:

• http://bit.ly/• codestrong_src• ti_test_slides

To Sum Up

Kevin Whinnery@kevinwhinnery

[email protected]

mailto:[email protected]

Documents

Testing Best Practices