Upload
satya-kaliki
View
2.975
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Cloud Computing and Mobile platforms (e.g. Android, iPhone) have emerged as compelling choices for a large number of software systems and Apps that are built today. While these new paradigms present opportunities for suppliers to provide innovative services, they also present significant challenges to quality engineers. This case study presents practical solutions to overcome those challenges.
Citation preview
Challenges and Solutions in Testing Modern Software Paradigms like
Mobile and Cloud Computing Satya Kaliki
Chief Technology Officer
Who am I?
CTO and Co-founder of ZANEC Technologies
2
Interim CTO at an Insurance Company owned by Harvard Medical Institutions
Passionate about Mobile and Cloud Computing, Semantic Web, SOA
13 Years of industry experience
Agenda
Mobile Computing
Challenges
Factors to consider in your test strategy
Case Study – Search App for iPhone, Android, Windows Phone 7
Cloud Computing
Challenges
Factors to consider in your test strategy
Case study - Air Cargo Supply Chain
Q&A
3
CHALLENGES
Features
5
Locations D
ev
ice
s
Carriers
Test case complexity
Mobile Apps vs. Traditional Apps
6
vs.
Native Apps vs. Mobile Browser App
7
Courtesy: Global Intelligence Alliance
Location and Localization
8
Field Testing – Signal strength variations
9
THINGS TO CONSIDER IN YOUR TEST STRATEGY
11
Use Emulators in the Lab
Use Real Devices in the Lab
Use Real Devices in the Field
Adopt a crowdsourcing strategy
App Maturity and Testing Progress
Lowest Common Denominator
and choose based on market share
12
Usa
bil
ity
13
Navigation
14
Varied end user expectations
15
Field Testing
16
Ba
ckg
rou
nd
no
ise
Field Testing - Multitasking
17
Use Emulators to simulate network variations
18
Android Emulator Commands
emulator – netdelay gprs [Emulates a GPRS, means min of 150 max of 550 ms]
emulator – netspeed gprs [Transfer rates of a gprs network]
emulator – netspeed 14.4 14.4 [Exact transfer rate for up and down]
CASE STUDY Search App for iPhone, Android,
Windows 7 Mobile
App Overview
The Search App provides secure, real-time access to documents, structured data and messages - all from a single click Can connect to wide variety of sources including PC’s local hard
drives, Popular social Networking sites, Internet Search engines, Internet Email Providers, MS SharePoint, MS Exchange Server
Can distinguish variety of file types including MS Office
Currently the App is targeting two broad categories of Users Personal Use - Social Networking, SMS, Contacts, Media, Photos Business Users - Email, Exchange, Contacts etc
20
Test Strategy
iPhone as first Platform and iOS Versions 4.1 (3G, 3GS and 4)
Extended to Android in Phase 2
Windows Phone 7 is Phase 3
21
Test Approach
Emulators in the Lab
Real Devices in the Lab
Real Devices in the Field
Customer’s priority focus is UK
Test Strategy
Scope of Testing
Usability
Navigation
Intuitiveness
User Personas
Varied demographics
Mobile Literacy not considered for this App
Performance
Multi-tasking
Varying signal and network parameters
22
Challenges in Cloud Computing
Testing
Introduction
Cloud Computing presents significant benefits to organizations that adopt this paradigm:
Reduced Cost
Improved Flexibility
Significant Mobility
Increased Service Focus
Testing in a Cloud Computing environment must address key risks associated with migrating to Cloud
In the absence of a comprehensive Test Strategy realization of the promise remains a dream
24
Cloud Computing
Risks
Security
Reliability
Manageability
SLAs
Multi-tenancy
Resilience
&
Self-healing
Governance
Location Independence
25
Test teams must understand these models
Consumers
Networks & Firewalls
Data Centers and Power
Physical Servers
Virutalisation
Application Platforms
Operating Systems
Software
Infr
astr
uctu
re a
s a
Serv
ice (
IaaS
)
Pla
tform
as a
Serv
ice (
PaaS
)
Soft
ware
as a
Serv
ice (
SaaS
)
THINGS TO CONSIDER IN YOUR TEST STRATEGY
Choose a Test Environment
On-Premise vs. On-Cloud
28
vs.
Must include tests for each of the Cloud Computing risks
29
Don’t forget to include the App Functional Testing!
Assess Data Security
30
Evaluate App specific security best practices V
uln
era
bility
A
na
lysis
Au
dit
, Co
mp
lia
nce
a
nd
Go
ve
rna
nce
31
Performance Testing
32
Multi-tenancy and
data security
33
Test for Reliability
34
Test for Maintainability and
Manageability
35
Estimate optimal Service Levels and test whether you are able to
meet them
36
Checklist for assessing Go Live
Readiness
CASE STUDY Air Cargo Supply Chain
App Overview
The App provides secure, real-time access to shipment documents and role based view of consignment location One-Time Data Entry End to end, role based visibility to all stakeholders
(Ground handlers, General Sales Agents, Airlines, Customs and Freight Forwarders)
38
Offers great benefits to stakeholders Increased reliability on consignment’s current status Reduced costs Improved Accuracy and Quality Enormous Scalability
Test Strategy
Infrastructure as a Service model
On Cloud Test Environment
Security model best practices
Multi-tenant
Multi-tiered (Same document, different view based on organisation, role and level within)
Need to perform well at 1 million consignments being shipped concurrently
39
Test Strategy
Scope of Testing
Functional
Usability
Various levels of Security
Performance
Manageability
Recovery
Operational readiness
40
Questions?