Upload
4goptimizer
View
218
Download
0
Embed Size (px)
Citation preview
8/2/2019 uTest eBook Launching Killer Mobile Apps
1/43
Creating Killer
Mobile AppsThe keys to developing, testing
and launching mobile applications
that your users love
A Free Book from uTest
8/2/2019 uTest eBook Launching Killer Mobile Apps
2/43
8/2/2019 uTest eBook Launching Killer Mobile Apps
3/43
The Challenge of Launching Great Mobile Apps
Launching a killer app is like catching lightning in a bottle and this is especiallytrue of mobile applications. In an industry where there is fierce competition formedia attention, VC investment and/or consumer dollars, even the smallest flawcan doom a mobile app (and the business behind it). With so much at stake, howcan so many mobile app companies ignore the benefits of usability testing,functional testing and load testing? They want to launch apps that delighttheir users, right?
This eBook will walk you through the challenges of creating a high-quality mobileapplication, and will show you how to overcome them. We will demonstrate howcompanies of all sizes from startups to enterprises can overcome these
obstacles to build apps that their users love. In doing so, well also cover:
The Mobile Boom: Learn whats at stake and also whats driving theworldwide mobile application frenzy.
Mobile App Strategy: Well explain how building great apps begins longbefore code is written or tested.
Mobile Testing Types, Tools & Methods: Emulators vs. real testers; thedifferences between mobile development platforms and other critical,strategic decisions that will need to be made.
Key Testing Challenges: How to navigate through the peaks, valleys,twists and turns of testing your mobile application.
Crowdsourced Testing: Discover how tapping into the crowd can helpyou launch a killer app, while getting you to market faster and on-budget.
Key Players:If youre new to this mobile apps game, this appendix willhighlight the companies, technologies and players that you need to befamiliar with before getting started.
3Mobile App Challenges
8/2/2019 uTest eBook Launching Killer Mobile Apps
4/43
THE MOBILE APP BOOM
4
8/2/2019 uTest eBook Launching Killer Mobile Apps
5/43
The Mobile Boom
To say that the use of mobile applications is booming would be an obvious understatement. In 2010, mobile app stores will recorded anestimated $6.2 billionin overall sales, including 4.5 billion app downloads. By 2013, some analysts expect mobile app revenues to exceed$21 billion. With charts like this, we wouldnt bet against them. As they say in the investing world, the trend is your friend.
So if you think that mobile apps are for kids or that it wont affect your industry/company/team, think again!
5The Mobile Boom
8/2/2019 uTest eBook Launching Killer Mobile Apps
6/43
Why Is The Mobile Market Booming?
The accelerating shift towards mobile applications (and the smart phones they runon) is really quite simple, but frequently overlooked. Here are just a few of thereasons why users are flocking toward this technology:
Accessibility: With mobile devices, the web is always at your side. Thisincludes email, social media, video, games, news, stocks, sports, shopping andother actions. The ability to stay connected 24/7 has perhaps the mostwidespread appeal of all, and is the number one reason why the use of mobileapplications and devices show no signs of slowing down.
Adaptability: Mobile applications are sometimes thought of as being usedprimarily by kids and consumers only, but this couldnt be further from the truth.Business professionals were, in fact, the earliest adopters of this technology,and continue to make up a large segment of the overall user base. This can beseen in the widespread use of business & productivity apps for CRM, ERP,documents, presentations, email and messaging tools. Of course, you can alsoplay DoodleJump, too.
Efficiency: Mobile apps (and to a lesser extent, the mobile web) have greatlyenhanced the efficiency of our everyday online activities. As we will explainshortly, the goal of any mobile app should be to reduce each feature to its most
basic component. For example, users who want to see the latest market newswill generally download an app that focuses on this exclusively. They do notwant to enter a search term and click through five pages to see this information.Mobile applications, in other words, cut out the middleman and give users directaccess to the things they want most.
6The Mobile Boom
The Mobile Mantra
Much as we love freedom and
choice, we also love things that just
work, reliably and seamlessly. And
if we have to pay for what we love,
well, that increasingly seems OK.Have you looked at your cell phone
or cable bill lately?
- Chris Anderson, Wired
8/2/2019 uTest eBook Launching Killer Mobile Apps
7/43
Whats At Stake?
Still think that mobile apps are for games, music and entertainment purposesonly? Do you really think that mobile applications wont be relevant to yourcompany, your team and your job a year from now?
Think again. As the adoption rate increases, mobile applications will become evermore critical to the success of your business (and your career). Those who insiston waiting for further confirmation will have already been left behind. Companiesthat recognize this trend for what it is a technological revolution and takepreemptive action to improve the quality and usability of their mobile applications,will find themselves with a tremendous advantage.
Most companies will simply build a watered-down version of their website. Butthose who understand the implications of the mobile boom and who understandall of the feats this technology is capable of will build applications with enduringpopularity. Which side will YOU be on?
This mindset is essential. In order to take the steps required for developing andlaunching a quality mobile app, you will need to believe in what youre doing. Ifyou think mobile apps are simply a passing fad, it will be reflected in yourbusinesses and products.
Your users are counting on you. Dont let them down.
7Mobile Testing Challenges
The Mobile Future
According to a Morgan Stanley analyst,
the world is currently in the midst of the
fifth major technology cycle of the past
half a century. The previous four were
the mainframe era of the 1950s and 60s,
the mini-computer era of the 1970s and
the desktop Internet era of the 80s. The
current cycle is the era of the mobile
Internet, she says predicting that
within the next five years more users
will connect to the Internet over mobile
devices than desktop PCs.
- GigaOm
8/2/2019 uTest eBook Launching Killer Mobile Apps
8/43
STUFF YOU NEED TO KNOWFIRST
8
8/2/2019 uTest eBook Launching Killer Mobile Apps
9/43
Your Mobile App Strategy
One of the first and most critical decisions a mobile company will make is whether to develop a native application or spend theirresources optimizing for mobile web browsers.
9Mobile Web vs. Native Apps
vs.
Mobile Web Native Apps
8/2/2019 uTest eBook Launching Killer Mobile Apps
10/43
The Case For (and Against) the Mobile Web
Pros:
Single platform (the web)
One app to build, launch and maintain
All thats needed for some companies
Cons:
Less controlled user experience
Slower user experience
No app store distribution
Lack of standards across mobilebrowsers
10Mobile Web
8/2/2019 uTest eBook Launching Killer Mobile Apps
11/43
The Case For (and Against) Native Apps
Pros:
Rich media functionality
Controlled user experience
Faster experience for users
App store distribution
Cons:
Multi-platform
Must build, test & maintain multiple apps
Not necessary for some companies
11Mobile App Challenges
8/2/2019 uTest eBook Launching Killer Mobile Apps
12/43
Native Apps or Mobile Web? What the Users Say
There are particular native apps that I use on a regular basis, mainlydue to ease-of-use and simplicity. These include weather apps, travelplanning apps (kayak.com), music apps (Pandora). However, if Imaccessing news and searching for information, I use mobile sites thatautomatically render as mobile versions of their web sites (cnn.com,google.com)
I also foresee the future in which mobile web will be much easier
to use, and displace most (not all) of the need for native apps. Insteadof downloading the native app itself, there may be shortcuts on yourmobile device for easy access. Think about the recent progression awayfrom desktop apps and into purely web apps; mobile is simply a fewyears behind.
**********
I prefer native apps as they are more focused on your information
needs. I agree that the mobile web is more accessible and affordable,but it takes longer to wade through all the sites to get to what you want.Mobile web apps do solve the problem to certain extent, but theyre noton par with the native apps in my opinion.
12Mobile App Challenges
vs.
8/2/2019 uTest eBook Launching Killer Mobile Apps
13/43
Alternatives to Native Apps and Mobile Web
Of course, deciding what form of mobile app you plan to offer isnt aneither/or decision. There are alternatives, including:
WAC: The emergence of the Wholesale Applications Community (WAC)may end up providing software companies with their most f lexible solution.WAC is a consortium of mobile leaders, which has come to includecarriers like AT&T, Verizon, Sprint, China Mobile, Orange and others, aswell as manufacturers like Sony, Samsung and LG. This communityintends to enable developers to develop one mobile app that will workacross ALL carriers, devices and operating systems.
Hybrid Apps: Similar to the ideas behind developing for the mobile web,hybrid apps would allow developers to create mobile apps that share theirbackend with their web application. They would create a native app shellor wrapper which would utilize the same features, functionality, data andother characteristics.
*****
However promising in terms of simplicity and cost, these two options arenot without some major drawbacks, notably that of competing interests.Will these communities treat alldevices, carriers, etc. as equals? So dontrule out these alternatives immediately, but proceed with caution.
13Mobile Alternatives
Wholesale App Community
Though many doubted the long-term viability of the WAC, it looksas though the group is gaining themomentum it needs to make adifference.
Theres a long way to go, and theconcepts the WAC are working
towards are ambitious to say theleast, but the steps theyve takenso far seem to be the right ones.
- MobileMarketing.com
8/2/2019 uTest eBook Launching Killer Mobile Apps
14/43
Other Considerations
Mobile devices are evolving into increasingly sophisticated general purpose computers. This has led to the development of a variety ofplatforms and operating systems. Today, the operating system inside your mobile device is as equally important as the devicemanufacturer (OEM). The various features, GUI, processing speed, and most importantly, the applications available for your device willdepend greatly on the underlying OS, as well as the carrier and OEM.
14Mobile Decisions
CarriersOperating SystemsDevice Manufacturers
8/2/2019 uTest eBook Launching Killer Mobile Apps
15/43
Mobile OS: Past Present and Future
The mobile wars are heating up. This recent study by Quantcast shows that Android is gaining onApples web iOS, although the formerstill maintains a huge share of the overall market. Keep this trend in mind when deciding which operating systems you intend to build anapp for first.
15Mobile App Marketshare
8/2/2019 uTest eBook Launching Killer Mobile Apps
16/43
Mobile Constants
No matter which route you choose native apps or the mobile web certain tasksand backend functionality will remain a constant, including:
API / web services
Architecture
Database
The key lesson here is that app quality begins before coding or testing . You
should start by reviewing the makeup of your users. What features andfunctionality do they want? What types of information are they trying to access?And most importantly, how can you reach them on their terms?
Other key decisions that every mobile company will have to make include:
Static content vs. rich content
Commerce vs. functionality
16Mobile Constants
Mobile Fact
Consumer conversion (from surf topurchase) on iPhone or Android nativeapps is 3X - 10X times that of mobileweb sites
Why? Because users are more
comfortable with the device. In mostcases, the usability of the native app,including page load speed, is muchgreater than the mobile web.
8/2/2019 uTest eBook Launching Killer Mobile Apps
17/43
MOBILE TESTING TYPES AND METHODS
17
8/2/2019 uTest eBook Launching Killer Mobile Apps
18/43
Mobile Testing Types: Functional Testing
The most essential testing procedure will be to verify that the basic functionality of yourapplication. Early on, much of this attention should focus on the download activities ofthe prospective user (i.e. that the download process is seamless and intuitive). Fromthere, you can begin to verify more specific functionality. As well explain shortly, thiscan be done either via a mobile simulator (desktop browser) or by using real-worldtesters.
Finding device specific bugs: If, for example, you discover links that are notfunctional on certain devices, then you should first determine whether it works on thedesktop browser. If it is reproducible on the desktop browser, then it is obviously not a
device-specific bug, but rather a bug in the basic application. If a link is functional in thedesktop browser and it doesnt work in the device, then it should be identified as adevice specific bug.
Such functional testing should occur when the application is in the early developmentphase. At that point, testers should perform a quick test on the devices you deem asthe having the highest priority, in order to find any rendering or any functional issues assoon as possible. This way, development can have enough time to fix them or arrive ata proper decision as to whether to support these devices or not. If fixing the buginvolves high dev cost, they probably wont.
If this sort of functional testing doesnt happen at the an early stage (and most of thetime, it doesnt) there is the risk that the application might not work for certain devices.This can lead dev/PM to changes the design, which can further escalate costs.
18Testing Types: Functional Testing
Mobile Stats
51% of iPhone owners have five or
more games installed on their
devices
37% of iPhone owners play games
at least once per day puzzle games
are the most popular
46% of Blackberry owners have NO
games on their device at all
- Source: Mobile Entertainment
8/2/2019 uTest eBook Launching Killer Mobile Apps
19/43
Mobile Testing Types: Usability Testing
Usability testing plays an important role for launching a quality mobile application.Usability testing should occur as soon as the basic stable for the application isready. This is also true in order to test whether the application UI is user-friendlyor not, or whether some part/functionality of the application is easily discoverableor not. Also, usability studies should be done for the different types of devices,such as touch screen devices, since the user has different experience for thesame application on these devices.
A user may have a good experience on the iPhone, while another might have badexperience on a nontouch screen device. In this instance, changes to the UI
should be considered to improve the experience on the non-touch screen device.
For example, lets say there is a form with various fields and it is saved by clickingon the save button located at bottom of the screen. For this scenario, someoneusing iPhone will have no issues to scroll to the bottom to click on the save button,since the iPhone has a good scrolling mechanism. However, the same scenariowill give the bad user experience for the non-touch devices users, where scrollingto the bottom of the page is not as easy.
Needless to say, usability testing is an extremely critical step towards launching a
high-quality mobile application.
19Testing Types: Usability Testing
User Loyalty
While most will tolerate a minor
glitch, studies show that many
wont hang around if they are
disappointed with the
performance of a mobile app or
website. And the damage doesnt
stop there. Word-of-mouth will
ensure any bad experiences are
soon broadcast.
- Helen Leggatt, BizReport.com
8/2/2019 uTest eBook Launching Killer Mobile Apps
20/43
Mobile Testing Types: Performance Testing
Performance testing is another critical step in the successful launch of a mobileapplication. The key factor to be aware of here is mobile carriers, which can effectthe speed and ease-of-use of the application. If the application performance ispoor for the end user, they will be likely to divert to other apps and services. Theapplication must therefore be tested with different devices and carriers dependingon the county and region.
Most likely, you didnt build your application for automated users. You built it forreal users. Why then do so many companies rely solely on automated testingresults to verify functionality under stress? Until recently, it was because there
was no such mechanism by which a company could utilize real testers while asystem was under load.
But as well outline shortly, that obstacle has largely been overcome. Byleveraging real testers with automated tools, companies now have access to anentirely new layer of data.
Better yet, they are able to identify and resolve issues that would have beenreported later in the development process.
20Testing Types: Performance Testing
Mobile Testing Tip
If the application being developed will be
dynamic or interactive, then the browser
must support the following:
- JavaScript support: Mobile browsers must
support JavaScript in order build an
interactive application UI.
- XML HTTP Request Object: XML HTTP
request object support is required to
communicate with the back-end server and
to update the page with new data without
reloading the page. This will give the user a
smooth browsing experience for the site.
- CSS support: CSS defines how pageelements are to be displayed and enables
you to change appearance and layout of all
of the pages by editing a single file.
8/2/2019 uTest eBook Launching Killer Mobile Apps
21/43
Mobile Testing Methods: In-House Testing
Pros: In an ideal world, mobile testing could be performed under the watchful eyeof product managers, developers and QA managers, who could observe firsthandhow the application performs under every possible user scenario. The assembledtesting team consisting of perhaps a dozen or so of the most talented testers onthe market would have hundreds of handset models at their disposal: iPhone,Blackberry, Nokia, Samsung, Sony - you name it, they test it. It would be a testinglab that even the largest software company would envy.
With this type of control over the testing process, what could possibly go wrong?
Cons: The truth is, for mobile apps, building a comprehensive in-house testinglab is prohibitively time-consuming and expensive. This is true for almost everymobile app company, with very few exceptions.
Imagine the expense of building an in-house team and lab capable of assuring thefunctionality for iPhone, Blackberry and Android handsets (of all makes andmodels) across wireless carriers in the U.S., U.K, Spain and France. Now imaginethis with 3 apps 5 apps Will you send testers around the globe to assurequality? Of course not. Thus, its no surprise that mobile app companies fromstart-ups to large enterprisesdont rely solely on in-house testing resources.
To be fair, most firms dont intend to create an in-house team like the onedescribed above (although some have tried). Worse, they spend their moneytrying to get whatever they can from this method. That is, they have 3-5 in-housetesters, with a dozen devices (at most). But their testing coverage is confined totheir own office, and the whole point of comprehensive testing is lost.
21Testing Methods: In-House Teams
Just how important is global
testing coverage?
By the end of this year, the mobile
ad market is expected to grow to
$913.5 million. And by 2013,
Gartner expects mobile ad spending
to surpass $13 billion, with the
Asia-Pacific region bringing in the
most revenue, followed by North
America and Europe.
- Meghan Keane,
www.econsultancy.com
8/2/2019 uTest eBook Launching Killer Mobile Apps
22/43
Mobile Testing Methods: Outsourcing
Pros: Of course, when mobile application developers discovered that the in-houseroute couldnt provide the testing they needed, they logically said to themselves:Ill just outsource it. This was not an isolated decision; it was made by thousandsof software companies, and few could fault them for their logic at the time. Ifcomprehensive mobile testing was to be achieved, then surely an offshore firm(with their low-cost labor) would be the only entity capable of getting the job doneat a reasonable price.
Its this line of thinking that continues to define the way in which many mobilecompanies deal with their testing objectives.
Cons: While outsourcing has certainly proven to be less expensive than hiring aninternal team, developers who contracted with such firms quickly ran into thesame limits as that of their in-house teams while losing what little control theyhad over the QA process. And for what? With no insight into the day-to-day testingactivities, how could these developers expect to improve their application in atimely manner? Developers must remain close to their application at all times,which is antithetical to the outsourcing mentality.
And as it turns out, even the largest of traditional outsourcing firms dont have the
resources needed to provide adequate testing coverage required by todaysmobile environment. Just like their clients, they cannot easily (or affordably) testan app across dozens of handset makers, models, carriers, languages andlocations. Plus, the testers employed by these offshore firms are unlikely to beamong the target audience for a companys mobile apps.
22Testing Methods: Outsourcing
Common User Complaints:
Failed or stalled downloads
Credit card verification delays
No screenshots
App performance when there is
slow connectivityInconsistent buttons, fonts, etc.
Missing or broken links
App not supported by device
Outdated versions; bugs still in app
8/2/2019 uTest eBook Launching Killer Mobile Apps
23/43
Mobile Testing Methods: Simulators/Emulators
Pros: In many ways, the use of mobile emulators and simulators is essential fordeveloping and testing a quality application. Such technology enables developersto verify certain functionality that is not specific to any device, carrier or operatingsystem. They are also very useful in terms of usability, and especially design,including data input, screen size, button use, etc. all from the convenience oftheir own laptop.
Although many mobile app companies rely on these tools to identify bugs at anearly stage in development , they often fail to catch bugs at later stages. As weve
seen, these type of bugs are always the most expensive to correct.
Cons: One of the biggest challenges for mobile developers is that the testing ofapplications is occurring in an environment far removed from the real world, whereactual users run and interact with those applications ON their devices. Saiddifferently, the gap between in-the-lab simulation and in-the-wild usage is fartoo great to ignore.
Consider, for instance, an application that is data entry intensive. To assume thatit can be fully tested on a simulator (i.e. with full keyboard and mouse access) and
be ready for release is a mistake mobile app developers have continually madefor the past five years. The convenience of simulators and emulators has made iteasier than ever to be lured into a false sense of security. But the advantages ofsuch tools are limited in scope, and should never be considered a substitute forreal-world, on-device testing.
23Testing Methods: Simulators and Emulators
Mobile App Purgatory
Bugs in your mobile app cause
unintended consequences. Take
this story for example: A mobile
app developer is alerted to a
iPhone bug, and instead of being
able to fix it right away, he spends
weeks pleading with Apple to
allow him to proceed with the
update. If bugs in your app
appear, getting them fixed quickly
can be very problematic.
Of course, if the bug had beendiscovered BEFORE launch, this
would be a non-issue.
8/2/2019 uTest eBook Launching Killer Mobile Apps
24/43
Mobile Testing Methods: Beta Communities
Pros: Mobile companies that can attract a large beta community have a terrificadvantage over their peers, as thousands of users submit bugs and feedbackaround the clockfor free.
Needless to say, the allure of an active, engaged group of beta users is too greatfor many developers to pass up. Not only do they gauge your applicationsperformance before the big release, they act as advocates for your product,adopting a strong sense of ownership and helping you to establish a customerbase for your product. And so if youre at all uncomfortable with your appsperformance prior to launch - and dont want to go to great lengths running
professional usability tests beta programs seem like the easiest, safest route.
Cons: In practice, it is rare for a software company to attract such a large group ofbeta testers. After all, not every company can be Google or Mozilla, with itsvarious (and wildly popular) beta versions, where people line up to test drive theirsoftware. Chances are, your app either lacks enough mass appeal, or is relativelyunknown (for now), and is hence not that attractive to the beta testing types.
But even if you are able to assemble a sizeable beta group, the method still fallsshort of providing adequate testing coverage. And the reason for this is simple:
beta testers are more often like users, in that they will try to get your app tofunction properly, whereas a real tester would attempt to break your app.
In other words: Users use. Testers test.
24Testing Methods: Beta Communities
8/2/2019 uTest eBook Launching Killer Mobile Apps
25/43
KEY TESTING CHALLENGES
25
8/2/2019 uTest eBook Launching Killer Mobile Apps
26/43
The mobile app testing matrix is infinitely complex and getting worse
26Mobile App Challenges
Device Models
Wireless Carriers
Location
OS & Browser
Features
8/2/2019 uTest eBook Launching Killer Mobile Apps
27/43
The Challenges of Mobile Application Testing
The true cost of bugs: According to a study highlighted on TechCrunch, the average shelf life of an iPhone app is roughly 30 days. Itturns out that, for free apps, less than 20% of users return to an app even one day after downloading it. And by day 30, less than 5% ofusers are still utilizing the app. And for paid apps, the drop-off is even slightly steeper.
Why the drop-off? For starters: bugs - especially ones that appear within minutes of downloading - are likely to cause users to abandonthe application after a short period of use. With the variety of apps now available, users have an extremely low tolerance for buggyapplications. In a never-ending attempt to appease their user base, developers must therefore test their products across:
Handset Maker/Model
Operating System
Browser
Wireless Carrier
Location
Language
No matter what type of mobile applications you developwhether theyre chat tools, social networking, games, business apps orothers you face the same fundamental problems encountered by all mobile app developers.
Lets now take a quick look at how mobile app developers have attempted to deal with these problems
27Mobile Testing Challenges
0
20
40
60
80
100
Specification Design Coding Unit Test Integration
Test
Release Test Post-Release
(Relative cost of defect, by time of discovery)
http://www.techcrunch.com/2009/02/19/pinch-media-data-shows-the-average-shelf-life-of-an-iphone-app-is-less-than-30-days/http://www.techcrunch.com/2009/02/19/pinch-media-data-shows-the-average-shelf-life-of-an-iphone-app-is-less-than-30-days/8/2/2019 uTest eBook Launching Killer Mobile Apps
28/43
CROWDSOURCED TESTING FOR MOBILE
28
8/2/2019 uTest eBook Launching Killer Mobile Apps
29/43
Crowdsourced Mobile Testing
Combining the benefits of the aforementioned methods, the use of community-based testing or crowdsourcing has altered the way companies test theirmobile applications. This shift has empowered progressive mobile app companiesto make the best decisions, on a per-project basis, without sacrificing control,costs or accountability. By leveraging a global community of professional testers,QA teams are doing the impossible: maintaining app quality, achieving broadtesting coverage, meeting launch dates and staying within tighter budgets.
The net effect is that mobile app companies get instant access to a globalcommunity of testers via a platform that enables detailed management of the
entire test cycle (or even connects to their in-house bug tracking system).
This community can serve as a virtual testing team (for small companies andstart-ups), or as an on-demand extension for in-house QA departments.
While the thought of having a virtual QA team from around the globe might seemlike a complicated free-for-all, the reality is just the opposite. With the right onlineplatform, the process fits neatly into your existing teams and processes. Heres aquick look at how it works:
29Mobile App Challenges
What is Crowdsourcing?
"Crowdsourcing is the act of
taking a job traditionally
performed by a designated agent
(usually an employee) and
outsourcing it to a large group of
people in the form of an open call.
- Jeff Howe
Author ofCrowdsourcing
8/2/2019 uTest eBook Launching Killer Mobile Apps
30/43
Crowdsourced Testing: How It Works
30Crowdsourced Testing: How It Works
Customer Tester Community
Specify Testing Needs- Handset Maker, Model & OS
- Carrier, Location & Language
Selected testers invited
to project
Testers report in
real-timeCustomerapproves
Testers get paid
8/2/2019 uTest eBook Launching Killer Mobile Apps
31/43
Crowdsourcing Challenges: Quality and Control
Crowdsourcing has a reputation for being noisy and chaotic, without much actualproductive work. Thusits critical to choose a reputable community that can pointto its customers successes, and shares the past performance and reputation ofcommunity members. In short, make sure youre choosing a community and not amob. Other common challenges include:
Communication: As with any successful project, communication is key. Whilecrowdsourcing doesn't enable face-to-face conversations, it does enable decision-makers to converse with their virtual QA team in real-time through an easy-to-useonline platform. Also, make sure the testing company you choose provides adedicated project manager to help you manage the community and the process.
Intellectual Property: IP protection is sometimes a concern for customers whoare new to crowdsourcing. While it might seem like your IP could be exposed to acommunity of thousands, top crowdsourcing companies allow customers tocarefully select their testers and establish non-disclosure agreements (NDA) withtheir community members.
Quality: With crowdsourcing, its easy to believe that anyone can test yourapplication. Fortunately, the crowdsourcing model enables customers to be asselective as they want in terms of testers skill sets, experience, performance
ratings in determining who gets invited to test their app.
As with other methods, unexpected problems do arise, but what makescrowdsourcing different is its inherent ability to adapt to changes on-the-fly.
31Mobile App Challenges
Mobile ChallengesMobile applications come with
their own unique set of challenges.
Between Wi-Fi, 3G, Edge and all
their different behaviors, there are
so many angles you have to cover.
Until recently, testing for this
criteria would have beenimpossible, but crowdsourcing has
changed all that.
-Travis McElfresh,
-VP of Technology, 1Cast
8/2/2019 uTest eBook Launching Killer Mobile Apps
32/43
Benefits of Crowdsourced Mobile App Testing
Although the advantages of such an approach are numerous, heres a quick lookat what to expect when implementing crowdsourced QA for your mobile app:
Cost Control: Crowdsourcing allows managers to utilize lower-cost outsidesupport without being tied down by long-term commitments. The fundamentalstructure of online communities creates competition, making it more accountable,efficient and cost-effective than traditional outsourcing.
Dealing with Increased Complexity: Significant quality improvement is achievedwhen development and testing is done across a wider set of handsets, carriers,
locations and languages. Crowdsourced mobile app testers can be easilyrecruited based on a wide variety of criteria.
Faster Time-to-Market: Crowdsourcing enables you to ramp up or down to meetyour needs at any given time - eliminating the delays and staffing constraints thatoften stifle mobile app companies around peak release times.
Higher Quality: Customers expect their mobile apps to be fully functional andbug-free from day one. Companies that believe good enough is good enoughwhen it comes to quality and usability will learn a valuable lesson (the hard way).
Retaining Users: If your product has problems, or doesnt meet expectations, itseasier than ever for your customers (and prospective customers) to find yourcompetition.
32Benefits of Crowdsourcing
Why It Works
Because crowdsourcing is built
from the open source template,
any company looking to leverage
computer professionals has the
advantage of an audience already
familiar with the basic idea of
community production.
-Jeff Howe
Journalist, Wired.com
8/2/2019 uTest eBook Launching Killer Mobile Apps
33/43
Benefits of In the Wild Testing
HandsOn Device Interaction
On a simulator, you still use a mouse to touch the screen and simulate gestures.You also have a full-sized keyboard for data entry. Of course, this is very differentfrom using a mobile device.
First, a mobile device sits in your hand. Each of us likely has slightly differentways of holding and operating the device. For some, its done with one-hand usingyour thumb or a finger. For others, it might be two hands using both thumbs.
Second, theres the act of touching various screen elements like buttons andcontrols. This is much easier to do with a mouse pointer than a pudgy finger.
Based on the prior experience of many mobile testers, this difference is a criticalone, and the biggest, for testing application design and function. Using a mousewith the simulator, you do not get the full effect of having to scroll through a largelist view of items or having to play whack-a-mole on the screen with your thumbbecause button placement for navigating multiple screens is inconsistent.
Mobile developers are strongly encouraged to ensure that application testingbegins early, and happens often, on the mobile device itself rather than on asimulator. The same holds true for tablet devices.
33In-the-Wild-Testing
Mobile Testing Tip
Even on the same mobile
platform, screen sizes and
resolutions can vary based on
device type. For instance, the
screen size and resolution on the
HTC Incredible is different than
that on the HTC EVO 4G.Consequently, for an application to
have a consistent look and feel
across both devices and across a
variety of other devices, user
interface elements and graphics
need to be scalable.
-Web Trends, Mobile Analytics
8/2/2019 uTest eBook Launching Killer Mobile Apps
34/43
Benefits of In the Wild Testing
Screen Size and Viewing:
Even though the simulator application mimics the size of the iPhone screen, itsstill a different experience viewing an application on the device rather than amonitor. A screen that is readable on a simulator may not be and usually isntas readable on the device.
Data Entry:
No contest here. It would be a terrible mistake to develop an application thatsdata entry intensive and test it on a simulator, with full keyboard and mouseaccess, and believe that its ready to be released. Trying to type even a sentence
is a challenge on a touch screen keyboard.
Functionality:
In the world of mobile, there are some things you just cant test on the simulator.,including the use of accelerometers and location functions to name a few.
Todays mobile devices are pushing the boundaries of traditional applicationdesign and testing methodologies. While fundamentally still the same, thosemethodologies need to be applied in new ways, especially with regard to testingapplication usability.
34Screen Size, Data Entry and Functionality
Mobile Testing Tips
Interrupts: Test how the device
and application behaves when it is
interrupted by another action, e.g.
incoming/outgoing call, SMS,
MMS, video-call, low battery,
low/no connectivity.
Device Settings: Change thedevice settings: Date and Time,
Languages (Advanced regional
settings), Connections, Shortcuts
anything you have control over
and learn how each and combined
settings impact the device.
- Bernard Lelchuk
8/2/2019 uTest eBook Launching Killer Mobile Apps
35/43
Conclusion
Testing no longer has to be a neglected phase of mobile applicationdevelopment. There was a time when the testing matrix was indeed too complex,too burdensome to be completed using standard means. But thanks to the rapidevolution of crowdsourced testing, all that has now changed.
The companies that are now leveraging crowdsourced testing - particularly in themobile space - are gaining more of a competitive advantage than they perhapseven realize. As the mobile market doubles and triples in size over the nextdecade, those that have made testing coverage a priority will enjoy their ROI interms of increased market share, profitability and above all, user loyalty. Those
who continue to neglect the testing phase will struggle at best, and at worst, ceaseto exist. Its that simple.
Either way, the future of mobile applications remains bright. Once reserved for atech-savvy niche in the business world, the use of mobile applications is nowfirmly entrenched in the mainstream. Theres no turning back when it comes tomobile applications, and that includes methods for testing them.
For more on how crowdsourcing can complete your mobile testing efforts,chat with one of our QA gurus by clicking here or by calling 800.445.3914.
35Summary
Crowdsourced Capacity
Small development firms have
limited means today to beta test
and peer review their applications
with a cross section of
representative users.
Given the hundreds of thousands
of mobile apps, we believe that
efficient (crowd-sourced) testing
of apps in a global market of users
is considerably under-utilized.
- VisionMobile
http://www.utest.com/mobilehttp://www.utest.com/http://www.utest.com/contact-customershttp://www.utest.com/contact-customershttp://www.utest.com/contact-customershttp://www.utest.com/contact-customershttp://www.utest.com/http://www.utest.com/http://www.utest.com/http://www.utest.com/mobilehttp://www.utest.com/mobilehttp://www.utest.com/mobilehttp://www.utest.com/mobile8/2/2019 uTest eBook Launching Killer Mobile Apps
36/43
APPENDIX: MOBILE PLAYERS
36
8/2/2019 uTest eBook Launching Killer Mobile Apps
37/43
Mobile Operating Systems
iPhone OS from Apple
The iPhone uses its well known operating system, the iPhone OS, which isderived from Mac OS X. The iPhone OS has four abstraction layers: the Core OSlayer, the Core Services layer, the Media layer, and the Cocoa Touch layer. Theoperating system uses roughly 500 megabytes of the devices storage. TheiPhone app store is the online market for downloading these applications. Thirdparty applications were not officially supported until the release of iPhone OS 2.0on July 11th 2008. Presently, XCode3.1 is the development environment and theapplications are written in Objective C (a variant of language C). There is no doubtthat the iOs will continue to be among the preferred mobile operating systems
worldwide.
Android from Google
Developed by Google, Android is an open source, Linux-derived platform backed
by Google, along with major hardware and software developers (such as Intel,
HTC, ARM, and eBay, to name a few). Together, they form the Open Handset
Alliance. This OS, though very new, already has a cult following among
programmers eager to develop apps for its flexible, Open Source, back end.
Android promises to give developers access to every aspect of the phones
operation. This lends many to foresee the promise of further growth for the
Android platform. Like the iOS, Android figures to remain a prominent figure in the
future of mobile application development.
37Appendix
8/2/2019 uTest eBook Launching Killer Mobile Apps
38/43
Mobile Operating Systems
RIM Blackberry OSThe BlackBerry OS is a proprietary software platform, created by Research InMotion, for their BlackBerry phones. The OS is focused on easy operation and wasoriginally designed for business professionals. The Blackberry operating systemprovides multi-tasking and supports specialized input devices that have beenadopted by RIM for use in its handhelds, particularly the trackwheel, trackball, andmost recently, the trackpad and touchscreen.
The BlackBerry platform is perhaps best known for its native support for corporateemail, through MIDP 1.0 and, more recently, a subset of MIDP 2.0, which and allows
complete wireless activation and synchronization with Microsoft Exchange, LotusDomino, or Novell GroupWise email, calendar, tasks, notes, and contacts, whenused in conjunction with BlackBerry Enterprise Server. The operating system alsosupports WAP 1.2. Recently it has seen a surge in third-party applications and hasbeen improved to offer full multimedia support.
Linux Operating System
The Linux operating system is strongest in China, where it is used by Motorola, and
in Japan, where it is used by DoCoMo. Rather than being a platform in its own right,
Linux is used as a basis for a number of different platforms developed by several
vendors, including Android, LiMo, Maemo, Openmoko and Qt Extended, which are
mostly incompatible. PalmSource (now Access) is moving towards an interface
running on Linux. Another platform based on Linux is being developed by Motorola,
NEC, NTT DoCoMo, Panasonic, Samsung, and Vodafone.
38Appendix
8/2/2019 uTest eBook Launching Killer Mobile Apps
39/43
Mobile Operating Systems
HP/Palm webOS
The HP/Palm webOS is Palms next generation operating system. PalmSourcetraditionally used its own platform developed by Palm Inc. Underneath the graphicaluser interface sits a software stack that has much in common with mainstream Linuxdistributions for desktop computers. webOS uses, among other components, theLinux kernel, Upstart, GStreamer as multimedia framework, libpurple for instantmessaging, Simple DirectMedia Layer (SDL) for game programming, PulseAudio foraudio management, and WebKit Applications for webOS are written in JavaScript,HTML, AJAX, and some other common web technologies. It supports streamingvideo in RTSP, H.263, and H.264 formats.
Bada from Samsung
The Bada operating system has a kernel configurable architecture, which allows theuse of either the Linux kernel, or another Real-time operating system (RTOS) kernel.The Linux kernel is the preferred choice for smartphones, while RTOS is used forcheaper feature-phones, due to its smaller memory footprint. It has a web browserbased on the open-source WebKit, and also features Adobe Flash. Both WebKit andFlash can be embedded inside native Bada applications. Bada offers interactive
mapping with Point of interest (POI) features, which can also be embedded insidenative applications.
Bada supports various sensors, such as motion sensing, vibration control, facedetection, accelerometer, magnetometer, tilt, and GPS, which can be incorporatedinto applications.
39Appendix
8/2/2019 uTest eBook Launching Killer Mobile Apps
40/43
Mobile Operating Systems
Windows Phone 7 from MicrosoftOn February 15th, 2010 Microsoft unveiled its next-generation mobile OS,Windows Phone 7. The new mobile OS includes a completely new over-hauled UIcalled Metro. It includes full integration of Microsoft services such as Zune, XboxLive and Bing.
Windows mobile is based on the Windows CE 5.2 kernel, and features a suite ofbasic applications developed using the Microsoft Windows API. It is designed tobe somewhat similar to desktop versions of Windows, feature-wise andaesthetically.
Third-party software development is available for this OS, with several options fordevelopers to use when deploying a mobile application. This includes writingnative code with Visual C++, writing managed code that works with the .NETCompact Framework, or Server-side code that can be deployed using InternetExplorer Mobile or a mobile client on the users device. The .NET CompactFramework is actually a subset of the .NET Framework and hence shares manycomponents with software development on desktop clients, application servers,and web servers which have the .NET Framework installed, thus integratingnetworked computing space called the cloud.
It has been criticized for having a user interface which is not optimized for touchinput by fingers; instead, it is more usable with a stylus. However, unlike iPhoneOS, it does support both touch screen and physical keyboard configurations.
40Appendix
8/2/2019 uTest eBook Launching Killer Mobile Apps
41/43
Mobile Development Platforms
BREW (Binary Runtime Environment for Wireless) from Qualcomm
Brew is an application development platform that was originally developed forCDMA mobile phones, but it also supports GSM for some devices. It debuted inSeptember 2001 as a software platform. It can download and run small programsfor playing games, sending messages, sharing photos, and the like. The mainadvantage of BREW platforms is that the application developers can easily porttheir applications between all Qualcomm devices. BREW acts between theapplication and the wireless device on-chip operating system in order to allowprogrammers to develop applications without needing to code for system interfaceor understand wireless applications.
J2me by Sun
Java 2 Micro Edition (J2ME) is Suns version of Java aimed at machines withlimited hardware resources such as PDAs, cell phones, and other consumerelectronic and embedded devices. J2ME is aimed at machines with as little as128KB of RAM and with processors a lot less powerful than those used on typicaldesktop and server machines. J2ME actually consists of a set of profiles. Eachprofile is defined for a particular type of device cell phones, PDAs, microwave
ovens, etc. and consists of a minimum set of class libraries required for theparticular type of device and a specification of a Java virtual machine required tosupport the device.
There are now more than 2 billion Java ME enabled mobile phones and PDAs.
41Appendix
8/2/2019 uTest eBook Launching Killer Mobile Apps
42/43
Mobile Development Platforms
LiMoThe LiMo Platform is developed by the LiMo Foundation (founded by a group ofcellular handset makers and network operators) for handheld devices, and it usesLinux as its operating system. It has a modular plug-in architecture, and supportsDRM. LiMo application developers will be able to use SDKs to write managed coderunning in a Java virtual machine, browser apps for WebKit, and native code.
Qtopia
Qtopia is a platform for embedded Linux devices that provides the end-user with aset of core applications, such as PIM applications (Address Book, Calendar and
Todo) and a Media Player. Qtopia provides the developer with a clean object-oriented API to write applications for an embedded device. Most of the APIpresented to the developer is the same API offered by Trolltech for cross-platformdevelopment between Windows, Mac and the UNIX/Linux. The same code can beshared and reused on all of these desktop platforms and now Qtopia is just anotherplatform to run Qt applications.
The Qtopia platform encompasses both the software that runs on an embeddeddevice and the software that runs on an end-user desktop. The software that runs onthe desktop, called Qtopia Desktop, allows the user to synchronize their data
between the embedded device and the desktop. Qtopia Desktop has built-inversions of the PIM application found in Qtopia. Qtopia Desktop allows developersto write plug-ins to synchronize their information.
42Appendix
8/2/2019 uTest eBook Launching Killer Mobile Apps
43/43
Acknowledgements: Special thanks to Bernard Lelchuk, Brad Sellick, JigarPatel and Anuj Gupta for their contributions to this eBook.
About uTestuTest provides in-the-wild testing services that span the entire softwaredevelopment lifecycle including functional, security, load, localization andusability testing. The companys community of 45,000+ professional testers from180 countries put web, mobile and desktop applications through their paces bytesting on real devices under real-world conditions.
Thousands of companies -- from startups to industry-leading brands rely onuTest as a critical component of their testing processes for fast, reliable, and cost-effective testing results.
More info is available at www.utest.com or blog.utest.com, or you can watch abrief online demo at www.utest.com/demo.
uTest, Inc.
153 Cordaville Road
Southborough, MA 01772
p: 1.800.445.3914
w:www.utest.com
http://www.utest.com/http://blog.utest.com/http://www.utest.com/demomailto:[email protected]://www.utest.com/http://www.utest.com/mailto:[email protected]://www.utest.com/demohttp://www.utest.com/demohttp://www.utest.com/demohttp://www.utest.com/demohttp://www.utest.com/demohttp://blog.utest.com/http://blog.utest.com/http://blog.utest.com/http://blog.utest.com/http://blog.utest.com/http://www.utest.com/http://www.utest.com/http://www.utest.com/http://www.utest.com/http://www.utest.com/