A Test of B2B Sales Forecasting MethodsJuly 2012

A test of B2B sales forecasting methods

©2012 Nimble Apps LimitedShare this white paper!

Table of contents

1 Introduction

2 Scope and methodology2.1. Scope2.2. Methodology

3 Analysis of sample3.1. Closing dates are always optimistic3.2. Losing takes longer than winning

4 Forecasting methods4.1. Description4.2. Results

5 Conclusion

A test of B2B sales forecasting methods

....................................................................................................................................

.................................................................................................

...........................................................................................................................................................

.............................................................................................................................................

.................................................................................................................

..............................................................................................

...................................................................................................

...........................................................................................................

...............................................................................................................................................

........................................................................................................................................................

........................................................................................................................................

1

222

223

334

6

Share this white paper!

1 Introduction

1

Sales forecasting is a major issue for B2B companies. On one hand, B2B companies often lack the

thousands of data points that statistical forecasting techniques require. But on the other hand, recent

research by Aberdeen Group shows a clear link between forecasting best practices and sales

performance.

The implication is obvious: robust, B2B-specific forecasting methods would change the life of sales

managers.

This white paper describes the test of common and not-so-common B2B forecasting techniques we

recently performed on a sample of SalesClic client data. Our research yields a number of confirmations and

a few surprises.

A test of B2B sales forecasting methods

Share this white paper!

2 Scope and methodology2.1. Scope

Our sample data - representative of B2B compa-

nies selling complex products and services -

was made up as follows:

• 12 sales teams

• The teams were located in the US, UK and Asia

• The teams operated in the software, electronic

equipment and financial services industries

• The teams managed “structured” sales pipelines

(i.e. following stage-by-stage sales processes)

• Over the research period, we totaled 144,817

closed opportunities

• The sales cycles were from 75 to 250 days

2.2. Methodology

Training and test periods

We divided the historical data of these 12 teams

into training and test periods for the selected

algorithms. The training period is always twice

as long as the test period, with a minimum of 1.5

years, an average of 4.5 years and a maximum of

9 years. Training periods contained at least 500

sales opportunities.

Measure of forecasting error

We measured forecasting errors over the test pe-

riods using their root mean square, normalized by

the average amount of opportunities in the sample.

A test of B2B sales forecasting methods

3 Analysis of sampleBefore discussing the accuracy of the forecasting

techniques included in the test, it is worth noting

2 interesting patterns in the sample data.

3.1. Closing dates are always optimistic

Initial closing dates are optimistic for 10 teams out

of 12 in our sample. On average, it takes 22%

longer than initially expected to win an oppor-

tunity for the sample teams.

That is worrying in a B2B context, where sales

forecasts are very sensitive to closing dates. For

sales managers and sales operations managers,

monitoring closing dates is a clear priority.

2Share this white paper!

A test of B2B sales forecasting methods

The bias increases with the length of the sales

cycle, as illustrated by figure 1 above. In our

sample, we find a closing date error of 16% for the

team with the shortest sales cycle, and of 109%

for the team with the longest sales cycle.

50 100 150 200 250 300

-20%

0%

20%

40%

60%

80%

100%

120% Closing date error

Sales cycle length (days)

Figure 1 - Closing date error x Sales cycle length

3.2. Losing takes longer than winning

In our sample, two-thirds of closed-lost oppor-

tunities are lost after the closing date initially

expected, and losing an opportunity takes an

average 1.7 times longer than winning one.

Stagnation in the pipeline does not bode well for

pending opportunities, and B2B companies have

much to gain from detecting “stuck” opportu-

nities as early as possible.

4.1. Description

The following describes the 8 forecasting

techniques we tested.

• Weighted pipeline #1 is a simple weighted pipe-

line using declared opportunity amounts, closing

dates and closing probabilities

• Weighted pipeline #2 uses declared opportunity

amounts and closing dates but historical closing

probabilities

• Weighted pipeline #3 uses declared opportunity

amounts, historical opportunity time-to-wins1 and

historical closing probabilities

• Weighted pipeline #4 is a variation of weighted

pipeline #3, using declared opportunity amounts,

historical stage durations2,3 and historical closing

probabilities

• We also tested most combinations of the 4

weighted pipelines methods mixing declared

and historical inputs

• The “linear predictors” assume a linear rela-

tionship between stage amounts on day d and

closed-won amount on day d+n

• The “decision tree predictors” are sophisticated

algorithms using the decision tree technique. Our

trees are grown and pruned on stage amounts

• The “daily closing rate” method assumes that,

until the end of the forecast period, a team will

close the same daily amount as during the last n

days (see example p. 5)

1. The “time to win” of an opportunity is the average time required to win opportunities that have reached the corresponding pipeline stage.

2. The “duration” of a pipeline stage is the average time that opportunities spend in that stage. 3. Time to win for pipeline stage n and the sum of stage durations for stages n to i usually differ

because of early losses, stage jumps and back-and-forth opportunity movements.

4 Forecasting methods

3

A test of B2B sales forecasting methods

We classified these techniques according to 2 criteria that are very relevant to CRM software users:

• Whether they use declared or calculated closing dates and closing probabilities

• How computer intensive they are

Figure 2 - Classification of forecasting methods

Closing date Closing probability Computer intensive

Weighted pipeline #1 Declared Declared Low

Weighted pipeline #2 Declared Calculated Medium

Weighted pipeline #3 Calculated Calculated Medium

Weighted pipeline #4 Calculated Calculated Medium

WP combinations Both Both Medium

Linear predictors Both None High

Decision tree predictors None None High

Daily closing rate None None Low

4.2. Results

Judgmental forecasts are not reliable

The simple weighted pipeline forecasting tech-

nique (declared amounts, declared closing dates

and declared closing probabilities) is the second

worst performing in the sample.

This research thus confirms what most sales ma-

nagers already know: simple weighted pipelines

cannot be trusted.

Leveraging historical data helps

Traditional CRM software is unable to leverage

properly the historical data of sales teams for

optimization purposes.

This means that sales teams are sitting on a

huge amount of forecasting information they

could be using to inform their judgments.

Our research shows that replacing sales rep

and manager judgment on closing dates and

closing probabilities with historical averages

increases forecast accuracy.

• Weighted pipeline #3 is 7% more accurate than

weighted pipeline #1

• Weighted pipeline #4 is 35% more accurate than

weighted pipeline #1

4Share this white paper!

Sophistication pays... up to a point

Averaging time-to-wins, durations and closing

probabilities is a straightforward way to leverage

historical data. How do the more sophisticated

techniques tested here perform?

• Linear predictors don’t work very well.

In particular, they are outperformed by weighted

pipeline #4. This is disappointing but not

surprising since sales pipelines can have widely

different shapes, and linear equations are ill

equipped to deal with such irregularities.

• Decision trees perform well. Compared to the

simple weighted pipeline, they increase forecast

accuracy by an average 46%. However, they are

quite hard to implement.

A test of B2B sales forecasting methods

Figure 3 - Performance of forecasting methods vs. simple weighted pipeline

And the winner is...

As shown by figure 3 (below left), the best fore-

casting technique on our sample data is a simple

but nimble one: the “daily closing rate”. Here is an

example of how it works:

• Suppose your team has closed €300K

over the past rolling 3 months

• That is a daily average of €3.3K

• Suppose that you are 30 days

into the current quarter (1/3 of the quarter)...

• ...and that you have closed €50K so far

• Your “historical daily closings” forecast for the

quarter is: €50K + €3.3K x 60 days = €250K

This method improves simple weighted pipeline

forecasts by 53%. Forecast accuracy also in-

creases by 20% compared to weighted pipeline #4.Weighted pipeline #1 Reference point

Daily closing rate 53%

Decision tree predictors4 46%

Weighted pipeline #4 42%

WP combinations 20%

Linear predictors5 11%

Weighted pipeline #3 7%

Weighted pipeline #2 -8%

Accuracy improvement

4. This is the average of 3 decision tree predictors – all 3 tightly grouped around this average. 5. This is the average of 6 linear predictors. The best one improves forecast accuracy by 21% over weighted pipeline #1.

5

This research suggests immediate ways for B2B companies to improve sales forecast accuracy.

• Measure “pipeline dynamics” (opportunity time-to-wins, pipeline stage durations,

closing probabilities by pipeline stage) and use that information for forecasting purposes.

• Calculate your “daily closing rate” and use that information for forecasting purposes.

Implementing forecasts based on decision trees is also a good idea, although potentially complicated.

For additional progress, we believe that moving from the analysis of a pipeline’s “macro structure” (pipeline

stages essentially) to a pipeline’s “micro structure” (the behavior of individual opportunities) is required.

Nimble Apps will continue to study and share insights on these topics.

Nimble Apps is the publisher of SalesClic, a simple and powerful solution for visualizing,

analyzing and forecasting your sales pipeline.

SalesClic integrates with Google Apps, Highrise and Salesforce.

A test of B2B sales forecasting methods

5 Conclusion

About Nimble Apps Limited

6

Follow SalesClic on