The Six Pillars of a Successful Testing Program Six Pillars . of a . Successful Testing Program. This

  • View

  • Download

Embed Size (px)

Text of The Six Pillars of a Successful Testing Program Six Pillars . of a . Successful Testing Program....

  • Page 1

    The Six Pillars of a Successful Testing Program

    This year, Cyber Monday became the largest

    shopping day of the year in the United States.

    According to Internet Retailer, total revenues

    closed above $3.45 billion, an increase of over 12

    percent from 2015, and beat out the 2016 single-

    day revenue on Black Friday by 3.3 percent. For

    years, digital sales have been considered as much

    a brand play as an actual driver of revenue. As the

    total share of sales has shifted from traditional

    channels, however, digital has transformed from an

    opportunity to assist consumer decision-making to

    a critical point of conversion.

    Online retailers are not the only businesses feeling

    the impact of this transition. In a 2013 industry

    survey conducted by Adobe, 76 percent of the

    digital marketers questioned responded that they

    felt “marketing had changed more in the past

    two years than in the previous 50.” Since then,

    the dramatic pace of change has only become

    more acute. Beyond the sizable and still-growing

    share of business that digital marketing leaders

    are now responsible for, they must also manage

    an increasingly complex portfolio of tools and

    technologies, understand emerging research

    explaining the ever-evolving behavior of consumers

    in an omnichannel world, and drive toward lofty

    goals that can be measured, modeled, and projected

    quarters into the future. At the same time, the very

    nature of many organizations is changing. Reorgs

    are regular occurrences and the underlying culture

    of business has shifted.

  • Page 2Page 2

    Where once experience and assigned authority served as

    evidence in favor of opinions, data is now the coin of the

    realm. What that data means—not to mention, how we

    acquire and compile it—remains a persistent challenge. In the

    2016 Econsultancy Adobe Digital Trends Report, 77 percent

    of respondents identified data-driven marketing as a first

    or second priority for their organizations. This same survey,

    however, found that most businesses still struggle to make

    sense of the data available to them: 46 percent reported just

    getting access to relevant data was “difficult” or “very difficult,”

    42 percent said the same about their process for ensuring a

    data-driven strategy was carried out effectively, and 41 percent

    felt technology related to data-driven marketing presented

    the same challenge. In 2011, McKinsey declared Big Data as

    “the next frontier for innovation, competition, and productivity.”

    Today, many businesses are still wondering when this big data

    will yield big profits.

    Applying data to business strategy and decisions is surprisingly

    difficult. Sometimes, it seems, people, process, and technology

    all conspire to preserve a status quo based on intuition,

    opinion, and gut instinct. At Brooks Bell, we have found that

    implementing a performance marketing program based on A/B

    or multivariate testing is the most practical, effective means

    of building a culture that embraces data as a tool for decision

    making. At its core, testing is about asking challenging business

    questions and seeking answers that are based on observed

    results and validated buy actual in-market consumer behavior

    data. It is an experiment-based method that allows marketing

    leaders to understand, expand, and activate data from multiple

    databases and across digital channels. Testing is a powerful

    tool for actualizing the promise of “Big Data.” But, as useful and

    practical as testing is, there are still many challenges inherent

    in the adoption and success of a comprehensive testing

    program. These challenges combine to create a “utilization

    gap”—a discrepancy between the opportunity presented by

  • Page 3

    testing at an organization and the realized return of

    the testing program, if one exists.

    We have had the opportunity to work directly with

    dozens of A/B testing and optimization programs

    and, thanks to events like Click Summit, have

    built relationships with many more. This means

    we have gained insight into hundreds of testing

    programs spanning businesses of all sizes, across

    all industries. From this data, we have identified

    six key factors that predict or determine the

    success of a testing program. These six pillars—

    culture, team, process, strategy, performance, and

    technology—combine with one another in different

    ways in different organizations, but the effective

    implementation and management of each is critical

    if a testing program is to be successful.

    These six pillars fall into three groups:

    People Forming a high-performing, experienced team is often the first challenge a testing program faces; building a

    culture that fully embraces experimentation and data-informed decision-making is a perennial hurdle.

    Process To ensure testing is executed in a consistent, reliable way, a formal, standardized process must be designed,

    implemented, and enforced; generating ideas and refining them into test strategies requires a systematic

    approach; a plan for measuring, reporting, and improving performance over time is necessary not only for

    managing a testing program but also for illustrating its contribution to the business.

    Technology The tools and systems used for testing must be implemented and integrated effectively, address the

    immediate needs of a business, produce valid, reliable results, and provide an opportunity for growth.

  • Page 4

    People When it comes to building a successful testing

    program, one of the immediate challenges is

    that of resources. Securing enough support to

    design, develop, QA, and analyze a test can seem

    impossible, especially when there is little support

    for or buy-in to the essential idea of testing.

    Communicating the potential value of the process is

    critical at this stage and, though a bit of a chicken-

    egg paradox, the program must produce tests to

    support the argument for more testing.

    Whether the testing program is completely new or

    struggling to grow, there are two important factors

    that must be managed: team and culture.

    Team Most testing programs will rely on resources drawn

    from across various teams in an organization

    including those that may not be direct reports. The

    complex, matrixed nature of such an organization

    is one of the challenges of building a team that

    will be able to successfully develop and execute

    testing plans. A group of individuals, after all, cannot

    become a high-performing team, with a cluster of

    priorities, goals, and responsibilities competing and

    conflicting with one another.

    To address this dilemma, identify people

    within the organization who show a desire and

    enthusiasm for testing. These members can help to

    communicate testing strategies and methodologies

    across organizations. Help them to share the

    understanding that excellence in testing leads to

    an excellent customer experience and this requires

    all teams to participate. These inter-departmental

    resources will also help you to scale the testing

    program from a single team to a company-wide

    priority. The speed of implementation and the

    velocity of the testing processes will improve.

  • Page 5

    Developing this team within the organization doesn’t have to be

    a complex challenge. Training that explores the basics of testing

    can help educate individuals unfamiliar with testing strategies

    while also helping to identify resources who may be advocates

    of your testing ecosystem. A basic training program is essential,

    of course, for building competencies and capabilities across

    the organization. Training is also critical for building engaged

    partners and advocates of testing, which directly supports the