Embed Size (px)
Citation preview
System Testing and Quality: How Real Partnerships Deliver Accelerated Time-to-Market & Increased Revenue
David GehringerVP Marketing, Fanfare
About Fanfare• Fanfare provides software solutions to 70 global high-tech
equipment manufacturers and service providers that simplify and accelerate system and device testing.
• Our technology enables QA teams and developers to build and run automated tests, measure pass-fail criteria and automatically generate documentation that can be easily shared.
We help accelerate time to market while increasing product quality.
Quality Costs
• Quality may not be job 1, but it should be• Poor quality….
– Costs in development cycles– Lengthens testing – Delays releases– Create customer support costs– Deteriorates brand equity– Cost 100x to fix once in the field
Defect Case Study• Global leading Carrier and Manufacturer
• Realize these costs are for each company• Deep breath…
– None of these costs resolve the defects
Action Time
Communication about defect(Time Zones / Language barriers)
2 Days
Average # of defects discussed(mostly bugs but some are not)
400
Time reporting defects 800 Man Days (~4 man years)
Cost to discuss defects / release ~$500k
Product Release Schedule
Device Testing Device Testing
Service ProviderDevice Testing & Feature Validation
Service ProviderDevice Testing & Feature Validation
Interoperability and System Testing
Interoperability and System Testing
Dep
loym
ent
Dep
loym
ent
Months to Market from first production prototype
Quality is Really Simple
Developers
CustomerAdvocacy
Support
Automation
ServiceProvider
•Feature testing •Regression Testing•Abstraction•Response map ownership•Testbed maintenance•Only Legacy on tcl/perl
Testers
•System Testing
Personal Regression
LegendTC – Test caseTB – TestbedTR – Test ReportRM – Response MapCR – Capture ReportProc - Procedures
•System Testing•Interoperability testing•Device testing•Regression Systems
•Unit Testing•Build Regression
Quality Workflow
• Testing assets should flow from the manufacturer to the service provider
– The tests should have already passed– Assets reduce the work of the service provider– Speeds identification of defects– Triage defect causes quickly
What Assets to Share
• Test Cases• Test Reports• Setup & tear down procedures• Test bed / topology files• Abstraction
– Output, such as device response, GUI elements, Data sets, etc..
– Device centric variables
What is Failing
• In-house systems– Information and assets hard to move and share – Sapping the best talent to maintain– Creates a center of the universe problem
• Scripting– Similar to coding and is hard to understand– Small talent pool – Brittle and costly to maintain
Testing Relationship
Value Chain of Quality
Quality is Simple
Developers
CustomerAdvocacy
Support
Automation
ServiceProvider
•Feature testing •Regression Testing•Abstraction•Response map ownership•Testbed maintenance•Only Legacy on tcl/perl
Testers
•Version Control•Response Maps•Procedures
•System Testing
TC
RMProcs
RM, Procs RM, Procs
TC TR
CR TRTC
TC TR
TC TR
TC TRTBProcs
Personal Regression
LocalRM, TB, Procs
LegendTC – Test caseTB – TestbedTR – Test ReportRM – Response MapCR – Capture ReportProc - Procedures
•System Testing•Interoperability testing•Device testing•Regression Systems
•Unit Testing•Build Regression
Asset Repository
Asset Repository
Quality Business Equation
It is in the financial best interest of both manufactures and service provider to work as partners for quality improvement.
• Quality Issues– Delay sales, delay deployment, and raise support– Reduce both the top line and bottom line – Customer loss and brand erosion
Service Provider Trends
– Business advantages to NEMs who provide assets
– Better relationships drive ongoing costs down
– RFPs that require test assets from NEMs
The Good Samaritan
It is not greening the planet, but high quality makes good business sense.
Those who figure this out will have significant market advantage.
