Running a Successful B2B Beta Test

1

Running a Successful B2B Beta Test

Meet the Speaker

Mario Sancho

Sales Director @ Centercode

mario@centercode.com

/mariosancho

Agenda

1. Beta Challenges

2. The Beta Process

3. Adjusting the Beta Process for B2B Success

4. Q&A

Beta Testing =Customer Validation

Targeted customers using your product in real environments

Increases quality, improves user experience, validates performance

B2B Beta is when customers are using your product in a business environment

Who is Centercode?

Services Fully managed tests and programs

Software Complete beta program management platform

Testers Access to 150k+ deeply profiled global candidates

Content Free tools and best practices

Customer validation is our core competency

We've been collecting customer feedback for tech companies since 2001

Beta Challenges

Typical Beta Challenges

Project design and scoping

Allocating qualified resources

Recruiting ideal testers

Persistent user engagement

Collecting useful feedback

Discovering and delivering insight

The Beta Process

Beta Closure

● Final Survey

● Incentive Management

● Product Collection

● Wrap-up ➤ Closure Report

Beta Preparation

● Scoping & Design ➤ Beta Test Plan

● Platform Workspace Setup

● Tester Recruitment & Selection

● Product Distribution

Test Period● Onboarding and Support

● Feedback Collection (Bugs, Suggestions, Journals, Discussions)

● Directed Activities (Surveys, Tasks, Interviews)

● Engagement Monitoring + Management

● Team Support

● Weekly Reports

BetaProcess

Meeting

Deliverable

Adjusting the Process for B2B Beta Testing

More often than not, testers are your current customers

Tests are usually smaller in size

Instead of testers, you have sites and sites might have multiple testers

Product being tested is typically highly integrated and of high value

Two major challenges:

1. Finding testers2. Participation

What Makes B2B Betas Unique

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Starts with clearly defined objectives for the product and testers

Testers are first and foremost your customers

If multiple testers come from one company go with “Test Sites”

There are often two Beta Plans:

● The overall plan that outlines feedback objectives and target market

● Individual site beta plans to balance what features will be tested

Scope & Design

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Confidentiality

● Individual vs company NDA

● Can testers collaboration between each other?

Use Segmentations

● Segment testers by sites or individually● Create a segmentation for your sales team or SEs

○ They will often submit feedback on behalf of the tester/site

Will the testers/site be able to call for support or will they use the

provided workspace?

Platform Workspace

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Candidate sources

● Marketing● Sales● Support

Recruit through your sales team

● Send them an invite to refer their customers

Recruit a “primary tester” then recruit additional testers within their company

Even if testers have been preselected create an application to allow the testers the opportunity to opt-in

Consider “why” they are testing when selecting testers

Recruitment & Selection

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Install and test the product internally

Ship the product yourself (not through corporate logistics)

Create a welcome letter/package

Verify shipping address prior to shipping (probably not the same as the address in the CRM system)

Product Distribution

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Start with tester readiness - are they still ready, willing, and able?

Follow a proactive support strategy, especially in week one

● Do not assume they will start as soon as the product is received● Remind them of their commitment to participate

Lean on your task lists

● Creates an action item in the mind of the testers● Is perceived as the minimum participation requirement

Onboarding & Support

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Types of Ongoing Feedback Reports

1. Bug Reports

2. Feature Suggestions

3. Journals

4. Support Requests

5. Test/Site Reports

6. User Forums

Ongoing Feedback Management

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Site Beta Plans

● Should differ from site to site

● Have been agreed to by your customer prior to posting

Tasks Lists

● Created/posted weekly

● Keep them short

● Assigned by site if using sites

Directed Activities

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Surveys

● Created/posted weekly

● Minimize the use of open ended questions

Interviews

● Standard set of questions asked of every tester/site

● Plan for 15 minutes in length

Directed Activities

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Requires more nurturing than most betas

Will typically only participate if:

1. Experiencing a bug2. Require support

Think tactically - If you are using a tool like Centercode, post your surveys before giving the the opportunity to submit a support request

If a site is unresponsive:

● Check with their sales contact● Determine the strategic value to the company● If client serves a strategic value, move them to a VIP team and

stop engaging them

Engagement Management

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

If there is a product meeting ensure that beta is represented

Standard reports should be on stated objectives

Report on some basic KPIs:

● Installs● Issues reported/fixed● Test Plan results (possibly by sites)● Weekly surveys

Team Support & Reports

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Participation reports

● Create a weekly participation report ● If the test is small or not all testers are engaging consider

reporting on:○ % of Test Plan objectives completed

Stick to the original test plan unless a major issue causes you to pivot

Avoid taking request from the team for additional information

Team Support & Reports

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Consider it a “mini” project

Can cover all the objectives of the beta

Remember to use “N/A”

Advertise and focus on “Final” as the last thing they must do

Hold a wrap-up meeting with the tester/site

Final Survey & NPS

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

Consider giving two types of incentives:

● The company incentive● The individual incentive

Merit based incentives can be used but not announced because all the testers are customers

Verify they do not violate any corporate gift giving policy

Incentive Management

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

If product needs to be retrieved, withhold the individual incentive until the product is received

Send all instructions when requesting the product

● Include the “When, Where and Why”● Should include a box and a pre paid shipping label

What if they are unresponsive?

● Create a series of letters each escalating in seriousness● At least one of the letters should sound legal in nature without

threatening any legal action● Engage sales for help? Can be a double edge sword

Product Collection

Scope & Design → Test Plan

Platform Workspace Setup

Tester Recruitment & Selection

Product Distribution

Final Survey & NPS

Incentive Management

Product Collection

Wrap-up → Closure Report

Tester Onboarding & Support

Ongoing Feedback Management

Directed Activities

Engagement Management

Team Support & Reports

Preparation

Test Period

Closure

B2B Beta Process

If possible hold a wrap-up meeting with the internal stakeholders

At a minimum, create a closure report which should cover:

● Objectives from the beta plan● Results of the objectives● Total feedback collected● Verified issues and issues addressed● Overall tester sentiment and insights

Consider sending each site a “mini” version of the report focusing on:

● All issues addressed● How their feedback was used or will be used

Wrap -up

27

Final Thoughts

Create and understand your objectives from the start

Individual “Site objectives” should be balanced by the strategic value of your company. The higher the value, the lower the participation.

Ongoing reporting should be as much about the product as the client/site

During Beta, support is proactive and not reactive

28

Q&A

Mario Sancho

Sales Director @ Centercode

mario@centercode.com

Sign up for our newsletter and our upcoming webinars

For more resources, visit our library

29

Thank You!

30

Stop you went too far!