Upload
sarah-mccurdy
View
218
Download
0
Tags:
Embed Size (px)
Citation preview
Software DevelopmentSoftware DevelopmentQA Best PracticesQA Best Practices
May 20, 2010
Suzette Hackl, CSMSenior Project Manager
Skyline Technologies, Inc.
Agenda
• Definition of “Best Practices”Definition of “Best Practices”• Software Quality Assurance as it relates to:
– Process– People– Technology
• Application to SCRUM• Questions?
“Best Practices”• A collection of Software Quality “What Works Now” practices. • Suzette’s collection of practices come from many sources and span across industries,
sectors, and experiences.• Recognizable, Obvious, Debatable…..• The list of practices in the QA space is large. A large laundry list is hard to conceptualize
and translate into implementation. I have grouped my culled down list by People, Process, and Technology and categorized them by:
– Basic: The basic are exactly that. They are the training wheels you need to get started and when you take them off, it is evident that you know how to ride. When you eventually take off your training wheels and ride that does not mean that you always remember how to ride. Each one of these basic practices is “needed” for successful quality assurance. Removing or discontinuing one of these basic practices will degrade quality of your systems and potentially put business at risk.
– Foundational: The foundational practices are the rock and mortar that will hold your “Quality” program together. They have significant value adds to an organization. Unlike the basic, they are probably not as well known and therefore need implementation help.
– Incremental: The incremental practices are to be judiciously applied in special conditions and bring many returns in terms of value. Individually they do not provide broad gains across the board of testing because of their specialized nature.
Agenda
• Definition of “Best Practices”• Software Quality Assurance as it relates to:Software Quality Assurance as it relates to:
– ProcessProcess– People– Technology
• Application to SCRUM• Questions?
QA ProcessesStandard & Consistent Requirements for Test Planning
One of the roles of software testing is to ensure that the product meets the requirements of the client. Capturing the requirements therefore becomes an essential part not only to help develop but to create test plans that can be used to gauge if the developed application is likely to meet the customers’ needs. Therefore, requirement management and its translation to produce test plans is an important step. The premise of this practice is that requirements are consistently and clearly documented so as to be interpreted correctly by the development and QA team members.
Standard & Consistent Functional Specifications or Solution Design Documentation Functional Specifications are a key part of many development processes. While it is a development process
aspect, it is critically necessary for software functional testing. The advantage of having a functional specification is the test generation activities could happen in parallel with the development of the code. This is ideal from several dimensions. Firstly, it gains parallelism in execution, removing a serious serialization bottleneck in the development process. By the time the software code is ready, the test cases are also ready to be ran against the code. Secondly, if forces a degree of clarity from the perspective of a designer and an architect so essential for the overall efficiencies of development. Thirdly, the functional specifications become documentation that can be shared with the business to gain an additional perspective on what is being developed.
Common Glossary of QA Terminology This may seem to be the most basic of all concepts but it is one of the most important. Having clearly defined
terms communicated across the organization will go a long way in the acceptance and engagement of Quality Assurance
QA Processes - continuedReviews & Inspection
This best practice includes code reviews, Internal and external QA Audits, and Project Audits.
Test Monitoring & Governance The premise of test monitoring is to ensure quality key performance indicators are being met. This is done thru
verification across project phases and deliverables. By test monitoring, QA assures software completeness and readiness for delivery.
QA Processes - continuedFormal Entry and Exit Criteria The idea is that every process step has a precise entry and precise exit criteria. These are defined by the development
process and are watched by management to gate the movement from one stage to another. The preciseness of these criteria is determined by each individual project leadership. However, this practice allows much more careful management of the software development process.
Functional Test - Variations Most functional tests are written as black box tests working off a functional specification. The numbers of test cases that are generated usually are variations on the input space coupled with visiting the output conditions. Functional testing involves writing down different variations to cover as much of the state space as one deems necessary for a program. The best practice involves understanding how to write variations and gain coverage which is adequate enough to thoroughly test the function. Given that there is no measure of coverage for functional tests, the practice of writing variations does involve an element of art
User Validation and Acceptance The idea is to release an application to a limited number of users and get feedback to fix problems before implemented to the entire user community. This best practice has everything to do with getting functionality in the hands of users as early in the process so as to reduce cost / business interruptions and post implementation issues
QA Processes - continued
Automated Test Execution The use of automated testing to support rapid testing, capturing issues earlier in the development lifecycle, and providing detailed logs of frequent test results (sometimes through nightly builds). The automated test environment must be carefully setup to ensure accurate and consistent testing outcomes. Automatic testing should be run at code check-in to minimize changes of disrupting a working build. The goal of automated test execution is to minimize the amount of manual work involved in test execution and gain higher coverage with a larger number of test cases. The automated test execution has a significant impact on both the tool sets for test execution and also the way tests are designed. Integral to automated testing is the verification of current operations and logging of test failures with diagnosis information
“Nightly” Builds & Continuous Testing The concept captures frequent builds from changes that are being promoted into the change control
system. The advantage is firstly, that if a major regression occurs because of errors recently generated, they are captured quickly. Secondly, regression tests can be run in the background and cycled when it makes sense. Thirdly, the newer releases of software are available to developers and testers sooner. Continuous testing uses excess cycles on a developer's workstation to continuously run regression tests in the background, providing rapid feedback about test failures as source code is edited. It reduces the time and energy required to keep code well-tested, and prevents regression errors from persisting uncaught for long periods of time.
QA Processes - continuedIntegration Testing with User Scenarios As we integrate multiple software products and create end user applications that invoke one or multiple
systems, the task of testing the end user features gets complicated. One of the viable methods of testing is to develop user scenarios that exercise the functionality of the applications. These are called user scenarios. The advantage of the user scenario is that it tests systems in the ways that most likely reflect a user’s everyday usage
Software Development Lifecycle & Incorporating QA Quality Assurance encompasses the entire software development process, which includes processes such as software design, coding, source code control, code reviews, change Management, configuration management, and release management. There are specific phase activities that should be conducted during the software life cycle. At the conclusion of each phase there is a key gate check and management decision that the existing phase has been completed and approval to proceed to the next phase. Below is an example of how this applies to SCRUM.
QA Processes - continuedQuality Assurance Benchmarks and Performance Indicators
Establish and collect useful metrics that will facilitate decision making. Benchmarks and Performance Indicators (PI´s) cannot be meaningfully formulated unless there is a direct connection between them and the related policies and goals.
Usability Testing Usability testing means a way to measure how people (intended/end user) find it (easy, moderate or
hard) to interact with and use the system keeping its purpose in mind. It is a standard statement that "Usability testing measures the usability of the system". Some would argue that application usability perceived by the user community is the utmost factor that determines success. Usability testing needs to not only assess how usable a system is but also provide feedback on methods to improve the user experience and thereby gain a positive quality image.
Test Case Quality and Re-usability This practice is two-fold. Firstly, the valuation of a test case’s quality. Secondly, the value of re-using
test cases where applicable. Secondly, the re-usability of test cases.
Pairing There are various methods of pairing skilled individuals of a project team to produce higher quality of
work more efficiently; pairing developers, teaming testers with developers, etc. It’s been recognized for a long time that the close coupling of testers with developers improves both the test cases and the code that is developed
QA Processes - continued
Risk Management and Confidence The notion of this practice is to focus testing on risky areas and to find the important bugs; you have to practice some kind of Risk Management. There is a factor of confidence used here. The practice of accessing a risk factor to tests can also help with resource modeling. For example; testing of high risk, low confidence functionalities make more sense to be staffed by those QA Team members with high domain knowledge while low risk, high confidence functionalities could be staff by lower priced resources.
QA Processes - continuedBug-Tracking and Resolution with Root Cause Analysis
Estimating
Traceability The linkages between test cases, their parent design elements, and their parent
requirements are maintained through a technique termed Requirements Traceability. In essence, it is necessary to be able to trace the linkage from a test case all the way through to a grandparent goal.
INPUTS OUTPUT Requirements WBS Project Charter /Other Project Resource Plan Documents Schedule System Docs
Budget (total project ETC)
User/Customer Interviews Architecture Diagrams
Estimating Approaches:
1 – Top Down
Start at the system level and assess the overall system
functionality and how this is delivered through sub-systems
2 – Bottom Up
Start at the component level and estimate the effort
required for each component. Add these efforts to reach a
final estimate
Agenda
• Definition of “Best Practices”• Software Quality Assurance as it relates to:Software Quality Assurance as it relates to:
– Process– People– Technology
• Application to SCRUM• Questions?
PeopleSoftware Quality Ownership The premise of this practice is that business owns the task of determining the quality expectations
and KPIs of an application. Quality Control owns the task of assuring that software meets those quality guidelines.
Clearly Defined QA Roles & Expectations QA Professional/Senior Lead (QA Analyst III) QA Lead (QA Analyst II ) QA Analyst (QA Analyst I) SCRUM Team Member
QA Analyst Domain Knowledge Testers require domain knowledge and have to be at least as skilled as an expert user/power user.
Good Writing Skills are Mandatory Writing understandable and repeatable test scripts, which also contains information about the idea
and the intention of the test, the expected results and technical background, isn’t easy. There's also a fine line between too detailed test scripts, which are difficult to maintain, and too superficial ones. This is truly an “art” and not a science and takes experience. A good test case is one which brings you closer to your goal. There’s no simple formula or prescription for generating “good” test cases.
People - continued
Software Delivery & Capacity Planning A unit of work represents both the functionality being delivered and the quality of the functionality. We see that each work unit has both functionality and a quality component. We don’t want to intentionally plan to deliver functionality without quality
QA Members have broad and comprehensive knowledge in IT
The premise of this best practice is that QA folks mustn’t be experts or gurus, but they should be able to warm up fast to many technologies. For example different operating systems, networks, Internet, databases, program languages, etc. Quality Assurance is not only closely related to development but also support, technical writing and field service. Therefore QA engineers should have some knowledge and experience in at least one of these areas. There shouldn't also be any borders or constraints that prevent or discourage QA people to switch to development, technical writing, field service or support - and vice versa.
People - continuedQA people need to be good communicators and team players They
must be diplomatic (but sometimes tough), convincing and able to present.
QA Job Satisfaction - Paradigm shift Awareness of the value of Quality Assurance to an organization greatly impacts job satisfaction for a
QA professional. This includes the understanding and acceptance of testing limitations.
Agenda
• Definition of “Best Practices”• Software Quality Assurance as it relates to:Software Quality Assurance as it relates to:
– Process– People– TechnologyTechnology
• Application to SCRUM• Questions?
Technology
System Intellectual Intelligence This practice is related to the system knowledge held internally by an organization and its owning
group. This knowledge ranges from architectural design to code practices and standards used in the application. This is a control measure for protection. Augmenting development teams with consultants is a common practice as a means of meeting surges and recessions in business demands. Nevertheless, retaining an underlying knowledge level is important for continuity and risk mitigation. The level of intellectual intelligence retained is unique to the specific software application, what business functions, and the value of these functions.
Application Ownership We have already discussed the concept of quality ownership. The practice is the answer to the
question “Who is the person that has the ultimate ownership of the application?” It is the one single ring able neck and product owner as described in SCRUM. IT has the responsibility for ensuring the system is maintained, sustained, and operational.
Automation Testing Tools Unit test automation, System or Functional test automation, Performance test automation, Regression test automation
Technical Infrastructure and Support Test Environments & Supporting applications and job scheduling tools
TechnologyUtilization of single platform to support QA Processes The concept of a single platform to support multiple QA processes is not new. There are studies that
show how quality attributes can be lost between systems resulting in delivery of a product which is not what the client expected. The use of a single platform for work/project requests, requirements, test plans, test cases, design, development, test execution is very useful. It reduces the potential for loss of data and redundancy as well as provides one source for metric reporting.
Refactoring Refactoring is the concept of cleaning up, improving the inner working of a system. All system
functionality and outcomes are to remain unaffected given the same inputs. The amount of refactoring is dependent on the “swamp” or “ball of spaghetti” inside the applications. Lack of architecture direction and strategic planning of application growth can often result in this situation. Refactoring neither fixes bugs nor adds new functionality, though it might precede either activity. Rather, it improves the understandability of the code, changes its internal structure and design, and removes dead code. The amount of refactoring is based on the technical team’s willingness and speed to technical debt repayment plan.
Code Coverage Analysis/Tool The process of finding areas of a program not exercised by a set of test cases, creating additional test
cases to increase coverage, and determining a quantitative measure of code coverage, which is an indirect measure of quality. An optional aspect of code coverage analysis is identifying redundant test cases that do not increase coverage. Analysis of code coverage (or test coverage analysis) is a technique whereby test cases are created to exercise all parts of the software.
Agenda
• Definition of “Best Practices”• Software Quality Assurance as it relates to:
– Process– People– Technology
• Application to SCRUMApplication to SCRUM• Questions?
The Scrum Framework
Potential Deployment
Sprint Review
Product & Team Backlog Formation
Sprint Planning
2 Parts: Selection and Decomp
Daily Scrum
Sprint 2-4 Weeks
Team Retrospective
Agenda
• Definition of “Best Practices”• Software Quality Assurance as it relates to:
– Process– People– Technology
• Application to SCRUM• Questions?Questions?
Questions?
Thank You
For questions or more information, please contact:
Suzette Hackl, Senior Project Manager