Software Testing in the Cloud

Software Testing in the Cloud

Leah Riungu-Kalliosaari

Contents

− Cloud computing

− Cloud Computing use and distribution− Nordic countries− Europe− USA

− STX Research: Software Testing in the Cloud

Cloud Computing

− Cloud Computing represents a collective term for pay-per-use IT services that are delivered over the internet.

− On-demand access to services on a pay-per-use model.

Source: Nordic Public Sector Cloud Computing – a discussion paper, Nordic Council of Ministers

Nordic Public Sector − There are challenges

− Scarce resources− Increased pressure on budgets

− Need for more efficiency and innovativeness

− Deliver more with less

− Increased access to broadband and mobile devices

− National IT strategies− Focus on cost efficient IT− Deliver value to the end users


Benefits of cloud computing− Cost effectiveness

− Reduced capital and maintenance costs

− On-demand self-service− Reduced (unnecessary) interaction with service providers

− Scalability− Flexibility and pay-per-use

− Quality of Service− Possible to monitor, control and report resource usage

− Innovation − Cloud architecture supports services in different systems and organizational

barriers.

− Quick time to market, reliability and business continuity, efficiency, green savings, cheaper security tools


Cloud Computing Use and Distribution

− Nordic countries are generally ranked highly in e-readiness indices.

− Sweden established an eGovernment strategy in 2009

− The Finnish government has a digital agenda for 2011-2020 which includes cloud computing as one of the initiatives

− Iceland has got an e-strategy called “Iceland the e-Nation”


Cloud Computing Use and Distribution

− Denmark has a strategy focused on renewing digital services especially in the public sector.

− Norway is working on a national digital agenda

− In Europe, cloud services are expected to generate about EUR 35billion by 2014

− USA has a Cloud First Policy− Every federal agency will identify three “must- move” services within

three months and move one of those services to the cloud within 12 months and the remaining two within 18 months.


Nordic Cooperation on Public SectorCloud Computing

− Knowledge sharing can help in overcoming non-technical barriers e.g. addressing legal and regulatory issues

− Develop a common view on security and legal issues

− Improve the buying power in the Nordic region

− Improve procurement processes for cloud-based services

− Define common demands and standards

− Attract data centres

− Encourage public innovation


Our research contribution

Software Testing in the Cloud

Overview

The study looks at an intersection of cloud computing and software testing

Applications are tested as services by use of cloud- based resources.

Daily operation, maintenance, and testing support through web-based browsers, testing frameworks and servers

Testing is seen as an arena for piloting cloud computing adoption

Objective

To understand how organizations can successfully use the cloud for testing.

Observe the adoption of cloud computing in different organizational contexts

Impact of cloud computing on testing; testing as a service

Use of empirical observations, with qualitative research methods

Testing in the Cloud

Testing in the cloud affects

The acquisition model (cloud based testing emphasizes services The business model (cloud based testing emphasizes pay per use instead of license fees The access model (services are accessed over the internet The technical models of testing (e.g. scalability)

Testing in the Cloud

1a. SaaS software

3. Testing the cloud

2. Testing environments in the cloud

1b. Non-SaaS software

1. The system or application under test is available online

2. Testing infrastructure and platforms are hosted in the cloud (Including crowdsourcing/Human as a Service-(Haas))

3. Testing of the cloud itself

Facets of testing in the cloud

Source: L.M. Riungu, O. Taipale, K. Smolander, “Research Issues for Software Testing the Cloud,”2nd International Conference on Cloud Computing Technology and Science, 2010.

Roadmap towards testing in the Cloud

Develop an understanding of cloud computing

Understand the risks and prepare to address them.

Carry out pilot projects. Explore the viability of testing in the cloud and the potential benefits.

Come up with elaborate strategies For example, criteria for the selection of applications suitable for cloud-based testing;

criteria for the selection of potential cloud vendors.

Enhance team interaction and prepare for complexities

Organizations need to be prepared for additional testing brought about by the complexities and new requirements for cloud-based applications and systems.

Enhance co-operation between research and industry

Focus on addressing cloud related issues that are relevant for the software industry (including testing)

Source: L. Riungu-Kalliosaari, O. Taipale, K. Smolander, “Testing in the Cloud: Exploring the Practice,

Accepted, ”Special issue on Software Engineering for Cloud Computing, IEEE Software, March/April 2012.

Security

− Security is seen as a requirement for testing in the cloud− Data security across networks, confidentiality of

customer data

− Security is seen as an obstacle− Where is the data stored?− Who owns the data?− Who handles the data?− What happens to the data in case of service failure?

Aspects of Security (1) Trust An entity A is considered to trust another entity B when entity A believes that entity B will behave exactly as expected and required (Artz, et al., 2011) Level of certainty to the customer that the cloud provider is capable of providing the subscribed service properly and accurately

Governance Management and control over policies, defining roles and responsibilities, standards for application development and special attention for managing security risks/threats (CSA 2009)

Design, identification and implementation of organizational structures along with monitoring, control and testing of deployed services in the public cloud (Jansen, et

al., 2011)

Compliance Compliance is the process of ensuring adherence to policies derived from internal directives, procedures and requirements, or from external laws, regulations, standards and agreements (Proctor, 2011).

Involves measuring the effectiveness and adherence of the rules and understanding the followed process. Cloud providers need to provide assurance and proof to the subscribers that they have control over security. Customers need to verify their own internal security measures with their own auditors.

Identity and Access Management Provision of privacy and protection of data sensitivity

Who has access to the data?

Aspects of Security (2)

Availability Service interruption e.g. g-mail had one-day outage in 2008. Distributed denial of services (DDOS) - servers and networks are brought down by the flood of network traffic and prevent users to access the internet based services Incompatibility between the cloud provider’s storage services and applications that need to be tested

Data Security Data protection and confidentiality especially in shared multi-tenant environments Change management e.g. skills development

Instance Isolation and its Failure Ensure that different instances running on the same physical machine are detached from each other (Ertaul, et al., 2009).

Architecture A public cloud may enable one vendor’s SaaS to be hosted within some other vendor’s PaaS or IaaS service. Nested hosting platform and network risks lack of transparency between the customers and actual point of operations even during testing (Lumley, 2010).

Security Approach (1)

Security Approach (2) Define a strategic cloud security roadmap

Mainly guided by the requirements of the organization Evaluate the cloud provider’s risks and various types of risk assessment methods

Define the business and IT strategy Evaluate the information: what can be public/private? The organization type

Identify the risks Point out the risks, threats and vulnerabilities Design some initial control mechanisms to deal with the risks

Document the plan Detail all the important aspects in a plan that can be disseminated and effectively communicated

Assess the cloud security requirements Map the customer’s security needs to the provider’s ability to meet them Identify the gaps and how to resolve them

Comparing security strategies of popular cloud providers

CSP/Security Issues

Amazon Google

App Engine

Microsoft

Azure

Rackspace

Trust Very strong Strong Strong Very strong Very strong

Governance Strong Satisfied Strong Satisfied Satisfied

Compliance Strong Satisfied Strong Strong Strong

Identity and Access Management

Strong Strong Strong Strong Strong

Availability Strong Strong Strong Strong Strong

Data Security Strong Strong Strong Strong Very strong

Instance isolation and its failure

Strong Strong Strong Strong Strong

Architecture Strong Strong Strong Strong Strong

Documents

Software Testing in the Cloud