Automated Performance Testing for Desktop Applications by Ciprian Balea

Preview:

Citation preview

Automated Performance Testing for Desktop Applications by Ciprian Balea

2

THE SCARY QUESTION OF APPLICATION DEVELOPMENT

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

How does it perform in real conditions?Never as first expected!

3

BENCHMARKING• What impact does the app have on the system running it? (is the

system still reliable)• How does the app perform when running on the host system?

(does it keep all functionality)

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

Ideally,• consuming as little resources as possible without loosing

functionality• the app should not break /slow the system down

4

HOW TO PREDICT THE FUTURE?

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

Run it multiple times and average out.

A single run doesn’t accurately predict real life usage.

5

HOW TO GET A CREDIBLE RESULT?

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

Avoid inconsistencies induced by the outside world => no Internet connection, own

webserver & assets

6

WHAT’S OUR APPLICATION ABOUT?

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

A Custom Web Browser

7

HARDWARE SETUP

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

8

BIG PROBLEM #1: HOW TO KNOW HOW MUCH TIME THE PAGE TOOK TO LOAD

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

Based on the HTML content, loading time cannot be computed (element loading order is not consistent across

multiple runs).

Small solution:

9

BIG PROBLEM #2: HOW TO READ SYSTEM RESOURCE USAGE WITHOUT 3RD PARTY TOOLS

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

Small solution (a):

10

BIG PROBLEM #2: HOW TO READ SYSTEM RESOURCE USAGE WITHOUT 3RD PARTY TOOLS

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

Small solution (b):

11

BIG PROBLEM #3: HOW TO MANIPULATE THE APPLICATION’S GUI

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

Small solution:

12

WHAT DATA WE COLLECT?• Test 1

• Webpage load time• CPU usage• Memory usage• Disk I/O

• Test 2• Timing a “complex” operation

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

13

TEST 1: READING SYSTEM

STATISTICS

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

14

HOW IS THE DATA COLLECTED?

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

Not at the start of the game

Not at the end of the game

Throughout the entire

game

15

HOW IS THE DATA COLLECTED?

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

16

RUNNING THE TEST• One test sequence:

• clear cache• open the browser• start reading system data• load webpage• close the browser• save result in an Excel spreadsheet

• Full test:• Restore the “clean” snapshot of the test

environment• Automatically runs the test sequence for

1000 times on the native browser• Automatically runs the test sequence for

1000 times on the custom browser

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

17

RUNNING THE TEST

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

18

TEST 2: TIMING A

“COMPLEX” OPERATION

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

19

“COMPLEX” OPERATION

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

Open the browser

Load a webpage

Bookmark page

Go to homepage

Open bookmark

Wait for page to load

20

RESULTS

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

21

RESULTS

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

22

CONCLUSIONSPerformance testing is very important as it can reveal performance flaws.

It is very time and effort consuming to do it manually.

It can be automated with some initial effort, but the returned value is huge.

Offers very conclusive results on what needs to be improved in the application to boost performance.

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

23

Q&A

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

THANKS!Now let’s get to

work!

24© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential