Software Quality Sourcebook

8/12/2019 Software Quality Sourcebook

1/20
http://www.sdtimes.com/


2/20

Platinum Sponsor Gold Sponsor

Silver Sponsors

April 8-10, 2013

Boston, MA

www.BigDataTechCon.com

Discover how to master Big Data from real-world practitioners instructors

who work in the trenches and can teach you from real-world experience!

A BZ Media Event Big Data TechCon is a trademark of BZ Media LLC.

Big Data gets real

at Big Data TechCon!

Come to Big Data TechConto learn the best ways to: Collect, sort and store massive quantities of structured

and unstructured data

Process real-time data pouring into your organization

Master Big Data tools and technologies like Hadoop,

Map/Reduce, NoSQL databases, and more

Learn HOW TO integrate data-collection technologies

with analysis and business-analysis tools to produce

the kind of workable information and reports your

organization needs

Understand HOW TO leverage Big Data to help

your organization today

Untangling the Relationship Hairball with a Graph Database

Beyond MapReduce

In-Database Predictive Analytics

Microsofts Big Data Story

Hands-on NoSQL for the DBA

MapReduce Tips and Tricks

Apache Cassandra A Deep Dive

HBase Schema and Table Design Principles

A Survey of Probabilistic Data Structures

High-Speed Data Ingestion with Sharded NewSQL Databases

Implementing a Real-Time Data Platform Using HBase

Big Data Science: Extracting Truth from Large,Multi-structured Data, Parts I & II

Analytics Maturity Model

How to Fit a Petabyte in Apache HBase

Building an Impenetrable ZooKeeper

Mastering Sqoop for Data Transfer for Big Data

Building High-Volume Web Applications

Building Successful Data Science Teams ...and LOTS more!

Download the full Course Catalog TODAY!

Over50how-to

practicalclass

es

andtutorials

tochoose

from!

Check out this list of classes!

The HOW-TO conference for Big Data and IT Professionals

Register Early and SAVE!

Hyatt Regency Cambridge

Adam LaiacanoData Scientistand Engineerat Tumblr

Improve, Dont Abuse:

Use Big Data for

Your Customers,

not Against Them

Oscar BoykinStaff SoftwareEngineer atTwitter

Addition in the Large:

Simple Counts and

Not-So-Simple Counts

Dr. MichaelStonebrakerDatabase Pioneerand AdjunctProfessor at MIT

Big Data Is (at Least)

Four Different Problems

Check out these keynotes!
http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/


3/20

THE 2013 SD TIMES SOFTWARE QUALITY SOURCEBOOK MARCH 3

Quality as Art,

Science, NecessityQ

uality is essential in every software development project. Yet while com-

puter science programs teach developers about software architecture,

object-oriented design, algorithms and programming, theres little formal

emphasis on testing and quality assurance. Its left to managers and employers,

in many cases, to both teach the fundamentals of quality assurance and inspire

a culture of quality.

Every day, managers implore their programmers to code faster. But do they

provide them with the tools and knowledge needed to write better code? Do they

equip their test teams with the best resources to stamp out defects and vulnera-

bilities? All too often, testing is neglected at every stage. Thats not only true inenterprise development, of course, but also in small-business environments,

contract work, open-source projects and more. We

preach quality as an ethic... and then end up having to

enforce it with tools and policies.

Quality assurance is an art. A developer or tester

needs to understand where quality assurance fits into

the application development life cycle, and why theres

more to quality than passing a test suite or meeting some

arbitrary metric.

Quality assurance is a science. Over the past 30 years,

a significant body of knowledge has produced best prac-

tices for functional testing, unit testing and performance

testing, but now those classic techniques are being aug-

mented by recent innovations in the field of defect man-

agement, test automation, software-configuration management, metrics, test

development, and security/vulnerability testing.

Quality assurance is a necessity. As you will read in Software Quality: More

Than Testing, software errors cost real money, not only in wasted productivity

and product delays, but also in real business losses and even legal fines. Thats

not what you want.

This Special Supplement to SD Times explores solutions providers focused on

helping you improve software quality. All of the companies profiled in this Sup-

plement offer test and quality-assurance tools that can make a real difference.

Be sure to consider these companies when evaluating quality solutions for

your team, or for your entire organization.We hope you enjoy this special Supplement to SD Times.

ContentsFeatureSoftware Quality: More Than Testing ........................4

By Lisa Morgan

Software Quality LeadersParasoft Eliminates Trade-Offs BetweenInnovation Speed and Quality...................................7Parasoft Platinum Sponsor

Design Quality with TechExcel...............................11TechExcel Platinum Sponsor

Test/QA AutomationIncreasing Testing Productivity with IDTs

Automated Test and Re-Test (ATRT)......................15Innovative Defense Technologies Gold Sponsor

Test ManagementReal-Time Test Management

from Zephyr Drives Instant Visibility.....................17Zephyr Gold Sponsor

Mobile Test/QAMaster Software Quality with Orasi ......................19Orasi Silver Sponsor

Alan Zeichick

Editorial Director

Alan Zeichick

Senior Art Director

Mara Leonardi

Copy Editor

Adam LoBelia

Lead Writer

Lisa L. Morgan

Customer Service/

SD Times Subscriptions

+1-847-763-9692

[email protected]

BZ Media

President

Ted Bahr

Executive

Vice President

Alan Zeichick

BZ Media LLC

7 High Street, Suite 407

Huntington, NY 11743

+1-631-421-4158

FAX +1-631-421-4130

www.bzmedia.com

[email protected]

A Supplement to SD Times March 2013 2013 BZ Media LLC, All Rights Reserved


4/20

Software development has rapidly evolved to meet the

needs of todays fast-paced business environment, but

testing and quality-assurance practices have not neces-

sarily kept pace as evidenced by high-profile failures and

less-than optimal user experiences. Although solution

providers have been addressing the development/QA gap for

a while, their approaches are becoming more sophisticated

to align with the modern requirements of a world that runs

on software.

Software has permeated almost every aspect of our lives,and it really does drive business innovation, said Wayne Ario-

la, chief strategy officer at test-solution maker Parasoft. Yet,

we as an industry have not truly addressed the cost of quality

as a business issue.

Software failures cost companies billions of dollars each

year. Real money. Last year, in a matter of minutes, Knight

Capital lost $440 million, causing its stock to plummet 75%,

which in turn triggered a class-action suit on behalf of share-

holders. That failure also caused the United States Securities

and Exchange Commission to start drafting new automated-

trading rules. Software glitches also were to blame for the

delay of Facebooks IPO on its first tradingday, as well as for canceled United Air-

lines flights. Whether the problem is

caused by an errant algorithm or the

merging of two companies mission-criti-

cal applications, inadequate testing is

usually blamed.

Failures often stem from a lack of

established standards for software release

quality, said Tieren Zhou, founder and

CEO of TechExcel, a test-solutions

provider. Many times, quality standards

are not established because it is difficult toobjectively quantify quality.

While metrics such as the number of

bugs and the amount of test coverage have

historically served as indicators of soft-

ware quality, now one must consider the

complex operating environment, the user

experience and the risks to the business.

As an example, California sued Delta air-

lines for failing to include a privacy policy

in its Fly Delta mobile app.

There were probably some very exu-

berant developers working on the Deltamobile app who pushed it out to the marketplace as

fast as possible without understanding the business implica-

tions of doing so, said Parasofts Ariola.

Speed KillsMany organizations have embraced agile practices with the

goal of releasing smaller pieces of software faster. At the same

time, the nature of software and the environments in which it

runs have become more complex. Balancing the needs for

both speed and quality can be challenging as software ratings

and reviews indicate.The world is operating from a quality perspective the same

way its always operated: It essentially views tests as something

that happens at the end of a cycle, and so errors are discovered

really late, said Ariola.

While testing earlier in the software development life cycle

isnt a new concept, the notion of what should be tested earli-

er is changing. Parasoft and TechExcel are good examples.

Parasoft has a virtual test environment developers can use

to more effectively prevent errors from appearing in actual test

environments. It allows developers to do functional testing

and performance testing in addition to the usual unit testing

and static analysis. Parasoft also has a development-test-ing platform that enforces quality processes, including

static analysis, unit testing, coverage analysis, code

reviews and runtime testing, so software teams can ensure that

quality-assurance standards are being met consistently.

TechExcels DevTest test-management and quality-assur-

ance solution also enforces quality standards. It integrates

with defect-tracking tools and allows test cases

to be tied to requirements, so quality can be

built into products earlier in the software life

cycle.

You need to quantify quality in a stan-

dard way that can be applied to both agile andnon-agile teams, said TechExcels Zhou.

Quality control should begin as soon as the

first requirement is committed. Once you

know what features youre going to build, you

should be thinking about how they should be

tested so they can deliver a high-quality expe-

rience to the end users.

Better Ways to AutomateEnterprises and ISVs have been trying to address

the need for speed and more comprehensive test-

ing using test-automation solutions. Automatedtesting allows more tests and more types of tests to

Software Quality:

More than TestingBy Lisa L. Morgan

4 MARCH THE 2013 SD TIMES SOFTWARE QUALITY SOURCEBOOK


5/20

be executed faster than would be possible to do manually,

although testing more doesnt always mean testing smarter.

Developers get all this time to write the code, then theywant to shove it into a pipe where automation happens and

then clean code comes out, said Parasofts Ariola. Automa-

tion is exposing just how brittle the testing infrastructure is.

Parasoft and TechExcel are addressing the higher-level prob-

lem of having enforceable quality standards in place that apply

to both automated and manual testing. Innovative Defense

Technologies provides consulting services that help organiza-

tions apply automated testing practices in

ways that align with predefined outcomes.

IDT also offers military-grade test-manage-

ment and analysis solutions that are used by

the U.S. Department of Defense, DoD con-tractors and enterprises that are doing life-

cycle software testing in highly complex

environments.

While automated testing can accomplish

more in less time, fast release cycles can

nevertheless be negatively impacted by time-intensive status

meetings and traditional reporting practices. Zephyr is

addressing this issue with real-time test-management solu-

tions that provide 24x7 visibility into the status of tests via

dashboards.

Complex Environments Complicate TestingThe growing number of platforms, protocols, systems,

devices, networks, apps, software components and software

versions continues to complicate software testing. TechEx-

cels Zhou said that to ensure quality, test cases have to be

designed for each requirement, and that the permutations

for all the possible runtime variables need to be specified for

each test case.

Parasoft is addressing environmental complexity with sim-

ulated test-environment and cloud/SOA/API testing solutions.

Were seeing major disruptions in the SDLC today because

clouds are challenging the concepts of where software lives

and how it should behave. We also have mobile devices that areforcing us to rethink the user experience and whats impor-

tant, said Ariola. Then we have the concept of APIs, which are

driving composite applications, so testing is a more complex

endeavor.

Clouds represent three problems. First, they are forcing

software teams to expand testing outside their own walls. Sec-

ond, when services and software run in the cloud, they are sub-

ject to cloud disruptions. One high-profile example was the

Netflix outage that occurred on Christmas Eve 2012, which, as

reported by TechCrunch, was caused by errors caused by Ama-

zon Web Services Elastic Load Balancing API calls. Amazon

Web Services disruptions have also negatively affected suchhigh-traffic sites as Foursquare, Instagram, Pinterest and Red-

dit. When the failures occur, users blame the brand, not the

platform. Clouds are also lowering software-switching costs.

Many software publishers have replaced or complementedtheir on-premise solutions with SaaS-based alternatives.

Monthly SaaS subscriptions are currently popular, although

some users are clamoring for pay-per-use and shorter-term

models. The trend is extending to software testing, which is

why Orasi offers a cloud-based version of HP LoadRunner that

is available in 24-hour increments.

The lower out-of-pocket costs and easy onboarding of SaaS

solutions and mobile apps versus tradition-

al software translate to lower costs of

switching, underscoring the need for com-

prehensive software testing. Parasofts Ario-

la replaced an SaaS-based marketing-automation solution because it failed to

load critical data in a timely fashion. The

final straw was an executive presentation

in which the graphics failed to load initially,

followed by graphics that displayed stale

data. In three days, he migrated all his data assets to a new

platform and integrated the new solution with Salesforce.com.

When mobile app users get frustrated, they simply delete the

app. In the case of on-premise software, customers may simply

decide against upgrading if the switching costs are too high.

Customers lose confidence when your software quality is

poor, said TechExcels Zhou. You can try to make up for itwith a fancy marketing campaign, but if youve lost your cus-

tomers trust, theyll stick with the older versions. This cost of

maintaining old versions is a huge expense for ISVs, and it can

also hurt your future growth.

Bottom LineQuality cant wait for late-stage testing if software teams want

to ensure it. While developers wont be replacing QA engineers

anytime soon, there is merit to testing more aspects of quality

earlier in the software life cycle in a manner that doesnt derail

software development. While agile development teams often

include developers, testing/QA, and maybe end users, testingby developers is not a substitute for QA, and QA is not a substi-

tute for designing quality in to begin with.

Zhou suggested assigning QA floaters to development

teams so that more comprehensive tests can be created while

software is being developed. The alliance gives developers bet-

ter insight into quality, and QA teams get better insight into

what needs to be tested and why.

We have to remove the constraints associated with waiting

until a project is complete in order to test, said Parasofts Ari-

ola. If you really want to reduce risks, you have to [test earlier

in the cycle], compress your defect-discovery curve, and pro-

mote methods that prevent errors from getting into the soft-ware development life cycle in the first place.


While developers wont be

replacing QA engineers

anytime soon, there is

merit to testing more

aspects of quality earlier.


6/20
http://sdtimes.com/pdf/TakeControl/1867


7/20

Software quality stinks! Defect-free software is difficult to

achieve with short release cycles and the ever-increasing com-

plexity of environments. With Parasoft Service Virtualization,

Development Testing and API Testing solutions, organizations can

replace incomplete testing practices with software-quality process-

es that deliver defect-free software faster and more efficiently.

Software is the primary interface to the business. Defects or fail-

ure to satisfy end-user expectations have much greater impact as

application-switching costs are at an all-time low. Yet the approach

to software quality is the same as it was 20 years ago, said Wayne

Ariola, chief strategy officer at Parasoft. As an industry, we need to

realize that the cost-schedule-quality triangle has shifted, and wecan no longer compromise quality. Gone are the days of end users

being QA testers. Software quality must be achieved with a repeat-

able process for the business to survive.

Parasoft helps you achieve an unprecedented level of

defect prevention and process visibility with the indus-

trys first Development Testing Platform. This is

complemented by test-environment simula-

tion technology, which lets an organization test

anytime, anywhere. Using Parasofts solutions

lets teams deliver thoroughly tested software

knowing it is secure, reliable and compliant.

Test Anytime, AnywhereParasofts Service Virtualization simulates a

complete test environment so that more comprehensive testing can

be accomplished earlier in the software development life cycle.

Rather than waiting for defects to appear in actual test environ-

mentsor worse: in the fielddevelopers can simulate any envi-

ronment necessary for accurate functional testing and performance

testing, as well as agile and parallel development.

If you really want to improve software quality, you need to be

able to access a realistic, high-fidelity test environment early in the

process, said Ariola. If youre waiting until the project is complete

to test, youre discovering errors really late. Delays are rampant in

release cycles; for example, new features are added, or unexpected

complexity in the implementation. Since the release date wont

necessarily change, the first thing that gets cut is testing. Service

Virtualization gives you a simulated test environment so you can

begin testing much sooner and catch defects much earlier.

With an intuitive interface, team members can model complex,

simulated test environments, and then run large-scale, high-

throughput load and functional tests without impacting existing

systems or operations. Since a simulated test environment is creat-

ed, there is tremendous flexibility to model scenarios that would be

impossible to recreate in a staged test environment.

Reduce RisksDelivering more software features in less time means trade-offsbetween coding and testing. With Parasofts Development Testing

platform, software teams can implement and enforce processes

that reduce the risk of software failures. The platform ensures more

comprehensive static analysis, unit testing, coverage analysis, code

reviews and runtime testing, as well as more accurate traceability.

When you shift quality left, developers become inundated

with fixing defects at the same time that they are responsible for

building new features. Because a developer only has so much time,

you must promote methods that prevent errors from getting into

the software development life cycle in the first place, said Ariola.

Service Virtualization and Development Testing are now converg-

ing in a way that not only helps organizations start testing earlier,

but also reduces the amount of defects that are injected into thesoftware. This means that you can not only test earlier, you can

also actually test less and go to market significantly fasterwith

more innovative, better-quality deliverables.

Notably, Parasofts Develop-

ment Testing platform enables

organizations to implement

and enforce a defect-preven-

tion strategy that ensures best

practices. Developers can build

security into their applications,

prevent costly defects, and get

the traceability and auditability they need to comply with industry

standards. When software teams embrace policy-driven develop-ment, they can assure the reliability and performance of their

applications.

We introduced the software policy concept eight years ago in

the industrys first development testing platform. Developers

always say that theyre going to do static analysis or unit testing, but

there was no way of measuring the effectiveness of it or determin-

ing if it was being completed, said Ariola. Using our Development

Testing platform, you can declare how the code should be delivered

to satisfy business expectations. Then you can automate adherence

to the policy, aligning development with business goals.

Ensure Quality in Complex EnvironmentsParasofts API Testing solution simplifies the testing of critical APIs,

SOA, or other composite or cloud-based applications by automat-

ing complex test scenarios across Web UIs, messaging layers, enter-

prise service buses, databases, and mainframes. It enables accurate

event tracing and multi-layer validation while providing the flexi-

bility needed to easily extend, update, and reuse test artifacts.

APIs are driving composite applications, requiring organiza-

tions to orchestrate multiple endpoints. Its a complex endeavor

thats fraught with challenges, said Ariola. Our API-testing solu-

tion ensures API integrity so you can be confident that you are

releasing secure, reliable and compliant business applications.

By integrating Service Virtualization, Development Testing and

API/Cloud/SOA Testing, Parasoft reduces the time, effort and costof delivering secure, reliable and compliant software.

Learn more at www.parasoft.com.

Parasoft Eliminates Trade-Offs Between

Innovation Speed and Quality


Gone are the days of end usersbeing QA testers. Softwarequality must be achievedwith a repeatable process.

Wayne Ariola

SOFTWARE QUALITY LEADERSPlatinum Sponsor


8/20


9/20


10/20
http://sdtimes.com/pdf/DevSuite9.0/1868


11/20

Achieving consistent quality levels across all software projects is

difficult given the typical variations in software team practicesand runtime variables. With TechExcels DevTest, organiza-

tions can improve quality levels using limited QA resources, as well

as provide a means for both agile and traditional teams to integrate

QA processes into their development processes.

If you want to establish an objective standard for quality, you

have to be able to attach a quantified numeric value to it. Without

an accurate measurement of quality, it becomes harder to plan

quality at earlier stages of the software development life cycle, said

Tieren Zhou, founder and CEO of TechExcel. DevTest allows you to

repeatedly deliver quality releases.

Even though the length of software development cycles is con-

tinuing to shrink, many organizations are still using word proces-sors and spreadsheets to track software quality and test cases. But

regardless of whether testing is manual or automated, the test cas-

es often fail to cover the full range of the target softwares behavior,

resulting in errors, rework and delays.

The complexity of runtime variations makes it very

difficult to cover all the aspects of software behavior that

determine quality, said Zhou. We have

enabled quality quantification and reporting

for any release, sub-release or iteration. For

the past 12 years, we have been dedicated to

making software quality measurable.

DevTest Enforces Quality StandardsDevTest is a complete quality-assurance-

management solution that enforces testing

standards and improves efficiency across all test cycles. It allows

users to execute test assignments and submit defects using the

same interface. Users also have the ability to create a central repos-

itory for test cases, knowledge items and test automation scripts, so

quality can be achieved consistently.

TechExcel allows you to integrate quality-assurance practices

with a development project regardless of the methodology being

used, said Zhou. You can define test cases as well as the imple-

mentation process to ensure that specifications and requirements

are tested and verified during every stage of development.The DevTest system integrates with popular automated testing

tools using TechExcels TestLink API.

While testing early and often is a well-accepted practice, qual-

ity is not always contemplated at the development-task level.

Using DevTest, software teams can account for all test phases

using detailed histories of test cases, data and test results. The cen-

tralized test library also improves test standardization, reuse and

revision control.

DevTest allows you to comprehensively represent all aspects of

quality assurance by utilizing test cases that are predefined in a

template, said Zhou. The template defines the functional testing

procedures and the expected result.

DevTest allows easy scheduling of automated and manualtests, and also provides real-time test-result reporting, tracking

and analysis, so the exact state of testing is always known. And it

provides out-of-the box integration with JIRA, Mantis, Bugzillaand DevTrack, allowing easy integration with existing tracking

tools.

Compliance-management solution provider Gael Quality had

been using spreadsheets to document tests and record results

before adopting DevTest. The company was looking for a solution

that could handle more than 15,000 test cases, and could integrate

results from external sources. In just three days, the company was

able to implement DevTest and begin testing at scale. Before long,

Gael Quality was able to manage 15,000 master test cases and well

over total 100,000 test cases with ease, which was previously impos-

sible using manual methods.

Build Quality into DevelopmentMany TechExcel customers use a combination of DevTest and Dev-

Track defect tracking to improve the results of testing in develop-

ment and overall quality levels. As tests are executed, defects are

automatically flagged so issues

can be resolved efficiently.

DevTrack automatically routes

issues to the appropriate user

when an issue has been creat-

ed or forwarded. If the issue is

not resolved within the user-

specified time frame, it is auto-

matically escalated to notify the appropriate engineers or issueowners.

DevTrack provides a quality-feedback mechanism, and it helps

build quality into software during software development, said

Zhou. As developers start to work on development tasks, test cases

are presented as a special tab. Meaning, for every single task, a test

case can be defined within the developers process.

DevTrack supports sophisticated workflow rules for controlling

issue states and status transitions. It also allows subtasks to be cre-

ated and assigned to team members so they can help track and

resolve parent work items and issues. If the status changes on a par-

ent issue, pre-defined sub-issues can be triggered automatically.

In addition to providing full access control, DevTrack can be

integrated with popular SCM tools, so source-code files and

check-in/check-out operations can be tied to DevTrack work

items and defects.

Ensure Complete TraceabilityDevTest and DevTrack seamlessly combine with DevSpec, TechEx-

cels requirements-management solution, enabling complete, end-

to-end traceability. The products are available individually or

together as DevSuite, a complete ALM solution.

If you want to ensure quality, test cases should be linked with the

requirements so you have total traceability driving quality, said Zhou.

When the test cases associated with every single development task

are verified, you can dramatically improve software quality.For more information visit www.techexcel.com.

Design Quality with TechExcel


Without an accurate measurementof quality, it becomes harderto plan quality at earlierstages of the dev cycle.

Tieren Zhou

SOFTWARE QUALITY LEADERSPlatinum Sponsor


12/20


13/20


14/20
http://sdtimes.com/pdf/Authority/1869


15/20

Test organizations are increasingly challenged with needing to

test larger, more complex software baselines in less time, while

also being able to ensure a high-quality software product is

released. Innovative Defense Technologies (IDT) has developed an

automated testing suite of tools called Automated Test and Re-Test

(ATRT) to address these testing challenges. With the use of IDTs

ATRT suite of tools, enterprises with mission-critical systems, such

as the U.S. Department of Defense (DoD), can increase the volume,

test coverage, and complexity of integration and accept-

ance tests while dramatically reducing test cycle times.

In todays environment, software teams want to get

to market fast at the lowest cost without compro-

mising software quality, said Bernie Gauf, presi-

dent and CEO of IDT. ATRT is an innovative tech-

nology that we have consistently seen improve

testing productivity by more than 75%.

Since 2006, IDT has been pioneering new

ways of designing, developing and implementing

best-in-class automated software testing solu-

tions for the DoDs complex systems. The compa-

ny also offers automated test-strategy and design services that

improve the effectiveness of testing and QA programs.

Automate Tests with Confidence

IDT offers two robust DoD-approved ATRT solutions that supportthe entire testing life cycle: ATRT Test Manager (TM) and ATRT

Analysis Manager (AM).

ATRT TM is designed to support the testing of large, complex,

heterogeneous multicomputer systems. ATRT TM is non-intrusive

to the system under test, and provides a model-based approach to

GUI automation, which does not require the development of any

test scripts.

Another thing that differentiates ATRT TM is its ability to evalu-

ate all messages at the message-traffic level. Its message-based

interface testing capability spans multiple protocols, including

SOAP over HTTP, TCP/IP, UDP and Multicast.

As software products get more complex and depend on interac-

tion with other software products, the interfaces between compo-

nents become critical, said Gauf. You have to be able to validate

the performance of software products at the interface-specification

level to have high confidence that once systems are deployed, they

will work as expected.

ATRT TM also allows you to combine GUI-based automation

with message-based automation in one test. In addition, ATRT TM

provides a central repository for testing artifacts from test-case

development and documentation, to requirements traceability and

test-case results.

Accurately Analyze Massive Data Volumes

Complex, high-reliability systems often generate more test datathan can be analyzed using manual approaches. IDTs ATRT AM

automatically analyzes all test data using a model-based approach

that characterizes the systems behavior and evaluates the test

results against requirements and performance specifications.

Large complex systems can generate gigabytes or terabytes of

data that are not possible for people to evaluate. They have the

information needed, but as their systems grow, they discover they

cant evaluate everything because its too time-consuming and

labor intensive, said Gauf. Because ATRT AM is able to analyze

against requirements and performance specifications in an auto-

mated fashion, you can increase test coverage and reduce the time

it takes by an order of magni-

tude compared to manual

methods.

Testing and QA teams oftenonly have time to evaluate a

small subset of the data col-

lected, and as a result have the

potential to miss errors. ATRT AM is capable of analyzing all the test

data, pinpointing errors, and explaining how the test results differ

from the way the system is expected to perform.

Improve Test OutcomesAutomation speeds up testing, but by itself may not help organiza-

tions achieve critical-path goals. According to Gauf, technical

teams often start automating processes without considering what

theyre automating and why theyre automating them. Becausetheyre approaching automated testing incorrectly (e.g., not being

selective about the tests theyre automating, etc.), they mistakenly

conclude that automation is not making the type of impact the

organization expected.

IDT helps customers define an Automated Testing Strategy

thats tailored to their unique requirements and test programs.

With an Automated Testing Strategy in place, its possible to iden-

tify the critical-path tests that will yield the highest ROI based on

objectives.

You can automate anything, but you should be automating

what makes sense, said Gauf. If you want to release software

faster, well focus on those tests that are consistently in the critical

path for each software release, and require the most time and peo-ple. That way, you can dramatically reduce your testing cycle, and

you have the ability to repeat the tests in future releases.

IDT designs and builds automated tests in partnership with cus-

tomers internal teams. IDT also builds custom turnkey automat-

ed-testing solutions that customers can simply execute. The solu-

tions are built using ATRT technology, so they are capable of

supporting the entire testing life cycle of a system, including phas-

es such as component testing, system integration, system testing

and test result analysis, among others.

In addition to its products and services, IDT offers on-site and

remote training for customers who want to learn how to implement

ATRT effectively. The training programs range from introductory to

expert levels.

For more information about IDTs ATRT products and services,

visit www.idtus.com.

Increasing Testing Productivity with IDTsAutomated Test and Re-Test (ATRT)


If you want to release softwarefaster, well focus on those

tests that are consistently inthe critical path.

Bernie Gauf

TEST/QA AUTOMATIONGold Sponsor


16/20
http://sdtimes.com/pdf/Ditch/1870


17/20

Keeping pace with agile release cycles is challenging when test-

ing and test management cant be done in real time. Zephyrs

real-time test-management solutions provide instant visibility

into the actual status of tests 24x7, driving teams to improve work-

flows, processes and collaboration.

Testing teams used to be in one building or on one floor, but

now theyre spread out all over the world as a result of globally dis-

tributed teams and outsourcing, said Samir Shah, CEO of Zephyr.

When youre working in different time zones, traditional methods

of waiting for reports and facilitating status calls are inefficient.

Project teams need instant information to stay on the same page at

all times.All Zephyr productsZephyr Enterprise Edition, Zephyr for

JIRA and Zephyr Community Editionmeasurably improve pro-

ductivity and software quality.

Zephyr Enterprise Edition Improves Complex,Global Test ManagementZephyr Enterprise Edition manages all

aspects of software quality, and is suited for

departments and enterprises that run multi-

ple projects in parallel, have structured testing

cycles, and do automated testing or a combi-

nation of automated and manual testing. It

offers full two-way integration to JIRA.

Project teams still behave as if they work

in the same location, said Shah. They have

long conference calls that substitute for meetings in which they ask

for status reports; we automate these types of manual processes.

Using Zephyr Enterprise Edition, data-driven decisions can be

made while the information is still relevant. And, when anyone

anywhere makes a change to a requirement, test case, assignment,

defect or document, the data is automatically pushed out to dash-

boards so stakeholders can view the current status.

Updated information is automatically displayed on your dash-

board, so you dont have to press the refresh button or ask someone

for it; the data finds you, said Shah. Traditionally, testers have hadto send their daily reports to managers who package and send the

reports to project managers. PMs then extract pieces of data and

send it up to management. The whole process is fraught with laten-

cy and inefficiency.

With Zephyr Enterprise Edition, there are no reports to run or

metrics to calculate. It all happens automatically. Whats more, the

software includes a knowledgebase that stores apps, release ver-

sions, features, functions and tests so test assets can be reused. It

also offers built-in APIs for tight integrations to the users preferred

automation frameworks, providing manual and automated test

metrics in a single view.

People dont think about the knowledge that walks out the

door, said Shah. If you do not have a single repository that every-

one is accessing, you lose a lot of efficiency, especially if youre

working in a globally distributed fashion.

E-commerce platform provider GSI Commerce leveraged both

JIRA and Zephyr Enterprise Edition to manage testing across 30

projects, 90+ releases, and 12,000 test cases spread across a team of

280 people in three global locations. The company can now easily

access and reuse testing assets globally; create custom reports; and

customize the workflow in one system without losing the cus-

tomization in the other.

Zephyr Enterprise Edition is available both as a SaaS solution

and as an on-premise solution.

Zephyr for JIRA Provides Seamless Test ManagementZephyr for JIRA is a native add-on built exclusively for the JIRA 5platform. It completes end-to-end project management in JIRA by

adding testing to the overall planning, development, bug-tracking

and reporting processes. Zephyr for JIRA is best suited to project

teams that want to seamlessly

integrate testing into their

workflows.

Zephyr for JIRA gives you

sophisticated testing capabili-

ties inside JIRA, said Shah.

Project teams that have stan-

dardized on JIRA want to incorporate testing within the exact same

interface. We released it a year ago and, since the day it waslaunched, it continues to remain as either the No. 1 or No. 2 Top

Grossing add-on each week on the Atlassian Marketplace.

Transportation and language service company ProCare used

Zephyr for JIRA to add more structure to its testing process and

to create a repeatable process. Before using Zephyr for JIRA, Pro-

Care had been using JIRA, Confluence and spreadsheets. Because

a lot of the work was manual, bugs would surface after releases.

Now, instead of guessing when test cycles might be completed,

ProCare can instantly view the top defects affecting tests, and use

burndown charts to predict when test cycles will be completed

right within the JIRA interface. The company is benefitting from

more detailed analyses of bugs, easy navigation between test cas-es, and the traceability that comes from linking test cases to

issues.

Get Zephyr Community Edition FreeInstead of foregoing test-management tools because the out-of-

pocket costs are too high, thousands of users have turned to Zephyr

Community Edition, which is free for project teams of up to 10

users who want the benefits of real-time updates and the ability to

build a knowledgebase. The community edition can be easily con-

verted to Zephyr Enterprise Edition with a new license key.

We decided to offer a free version for small teams that needed the

functionality, flexibility and maintenance of an enterprise-class solu-

tion but were only working with an open-source budget, said Shah.Learn more at www.getzephyr.com.

Real-Time Test Management from

Zephyr Drives Instant Visibility


If you do not have a singlerepository that everyone isaccessing, you lose a lot ofefficiency.

Samir Shah

TEST MANAGEMENTGold Sponsor


18/20

@SPTechConA BZ Media Event www.sptechcon.com

August 11-14, 2013Sheraton Boston

Choose from more than 85 classes

and tutorials that will let youput together your own customSharePoint training experience.

Whether you need help migrating to SharePoint 2013

or are still making the most out of SharePoint 2010,

you will find the SharePoint training you need at

SPTechCon. Many of our speakers are Microsoft MVPs,

and all are SharePoint experts. A rigorous review

of the classes after each conference allows us to weed

out the weaker classes and speakers.

Classes at SPTechCon average almost 9 on a 10-point

scale, and classes scoring below an 8 are not invited

back. No wonder SPTechCon is the largest independent

SharePoint training conference in the world.

The BestSharePoint Training

in the World!

SPTechCon is a trademark of BZ Media LLC. SharePoint is a registered trademark of Microsoft.

At SPTechCon, all the experts get together in one place and

present fantastic, current and relevant content.Keith Budurka, IT Director, Infrastructure, Benco Dental

If your goal is to learn something from a trusted leader in

the industry and get an opportunity to network, you must go

to SPTechCon.Brian Perryman, SharePoint Developer, Health South Corporation

SPTechCon is a great event and well worth the admission fee.Chris Kauke, System Consultant, MassMutual
http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/


19/20

T

odays businesses run on software, and yet errors still cause

service disruptions, cost overruns and customer-support

headaches. With Orasis help, enterprises can ensure theirapplications run predictably at scale.

Since 2002, Orasi has helped enterprise customers create soft-

ware-testing environments that reduce the cost and risk of software

failures. The company resells HPs Application Lifecycle Manage-

ment product line and provides support for ALM solutions as an HP

Support Partner. Orasi is an HP Specialist Enhanced Partner, and

was named HP Support Partner of the Year in 2009, 2011 and 2012,

and Partner of the Year (US) in 2011. Orasi also maintains a dedicat-

ed SAP practice aimed at enterprises that are deploying or upgrad-

ing SAP solutions, and has achieved SAP Gold Level Partner status.

Orasis testing-service capabilities span the entire application

life cycle, so clients can get expert assistance from initial imple-

mentations to enterprise rollouts. To help customers get the most

from their investments, Orasi developed its Rapid Return on Invest-

ment (R2OI) approach that translates customers business objec-

tives into a flexible set of services that enable faster time to market,

reduced risks and lower costs.

Organizations realize they have to make software quality a pri-

ority because software is the lifeblood of their business, said Nick

Kavadellas, president and CEO of Orasi. As a leading software

reseller and professional services company, we have deep knowl-

edge across functional, performance, mobile device, security, and

specific application testing.

Orasi Solutions Mirror Enterprise RequirementsCustomer-requested Orasi testing solutions suit the modern

demands of enterprise teams. Its JIRA Bridge for HP Quality Centersynchronizes defect information between Atlassians JIRA and HP

Quality Center to improve communication between QA and devel-

opment teams. The tool creates issues when test cases fail, and pro-

vides metrics that help improve the reliability of release cycles.

Software teams want to link test cases back to development

artifacts, said Kavadellas. Because JIRA has been so popular, cus-

tomers asked us to integrate JIRA and HP Quality Center. Now,

more than 100 customers are using it.

Orasi also offers a cloud-based version of HP LoadRunner called

Orasi CloudPerform, which can be purchased in 24-hour incre-

ments. The tool allows customers to capture, run and analyze bottle-

necks in cloud-based applications. It supports Silverlight and Flex;

Packages are also available for Citrix and Microsoft RDP testing.

Take Advantage of Affordable Onshore TestingOrasi established a rural testing center in Greensboro, N.C., five

years ago. The test center now employs more than 120 engineers

who assist clients throughout the U.S.

Software development organizations are realizing how impor-

tant it is to have an effective methodology and process in place,

said Kavadellas. Four years ago, mobile apps were considered fun.

Now, you have to make sure theyre mission-critical.

Learn more at www.orasi.com.

MOBILE TEST/QA

Master Software Quality with Orasi

Silver Sponsor
http://sdtimes.com/pdf/CloudCurrent/1871


20/20

Get the best real-world Androiddeveloper training anywhere!

Choose from more than 75 classesand tutorials

Network with speakers and other

Android developers

Check out more than

40 exhibiting companies

BOSTON May 28-31,2013The Westin Boston Waterfront

AnDevCon is a trademark of BZ Media LLC Android is a trademark of Google Inc Googles Android Robot is used under terms of the Creative Commons 3 0 Attribution LicenseRegister NOW at www.AnDevCon.com

AnDevCon is one of the best

networking and information hubs

available to Android developers.Nate Vogt, Android Developer,

Willow Tree Apps

KEYNOTE!
http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/

Documents

Software Quality Sourcebook