Embed Size (px)
Citation preview
Journal of Forecasting, Vol. 3 , 139-159 (1984)
Accuracy in Forecasting: a Survey
ESSAM MAHMOUD Concordia University, Quantitative Methods Department, Montreal, Canada
ABSTRACT
In this study, the author provides a brief and concise summary of empirical investigations pertaining to forecasting with special reference to the accuracy of different forecasting techniques. The study mainly focuses on comparisons of the accuracy of these techniques. Thecomparisonscover both quantitative and qualitative methods. In addition the summary includes studies seeking to test or improve accuracy by combining forecasting techniques.
KEY WORDS Forecasting Time-series Accuracy survey Comparison Evaluation studies
Accuracy plays an important part in selecting and testing a given forecasting technique. Forecasters and managers have a wide choice of ways to forecast, ranging from purely intuitive or judgemental approaches to highly structured and complex quantitative methods. During the past thirty-five years there have been many studies reported in the area of forecasting and, in particular, the accuracy of forecasting techniques. The major purpose of this paper is to summarize the research findings on the accuracy performance of different forecasting techniques. The paper emphasizes the most important studies in the area of accuracy. It discusses the significance of accuracy and briefly reviews accuracy measures. Subsequently, a summary of the research studies in the area of accuracy is presented in Exhibits 1-5. Exhibit 6 reports the investigations relating to combining forecasting techniques. Practical implications of the research findings are suggested. Finally, ideas for future research in the area of forecasting accuracy are proposed.
THE IMPORTANCE OF ACCURACY
‘Accuracy in sales forecasts must be considered in the light of both the purpose of the sales forecast and the special difficulties inherent in sales forecasting’ (Sartorius and Mohn, 1976, p. 10). When defining the purpose of a sales forecast, we are really, to some extent, defining the degree of accuracy needed. ‘For some decision makers, anywhere between plus or minus 10% may be sufficient for their purposes, but in other cases a variation of as much as 5 % could spell disaster for the company’ (Wheelwright and Makridakis, 1980, p. 9). Furthermore, Pan et al. (1977) surveyed 251 companies selected from the Fortune list of the 500 largest industrial companies for 1973. The 0277-6693 f84fO20139-2lSO2.10 Received January 1982 0 1984 by John Wiley & Sons, Ltd. Revised September I983
140 Journal of Forecasting Vol. 3, Iss. No. 2
survey results indicated that most companies aimed for accuracy of within plus or minus 10 per cent, but only about two-thirds of the companies achieved this goal.
Effective sales forecasting has become a prerequisite for the successful management of a company, and the necessity for accurate predictions of both unit and dollar values is increasing. Rothe (1978) found that most marketing executives feel efforts to improve forecasting are worth the required costs. His study also showed that producing good forecasting is becoming more difficult and that actual errors in forecasting are growing. Furthermore, a bad forecast can kill the best of plans, and accurate forecasts can be offset through poor strategies, planning or implementation (Makridakis, 1981, p. 11).
Improvements in forecasting results will be achieved when the systems are designed and managed to produce meaningful data (Rothe, 1978). The special difficulties inherent in achieving accuracy in sales forecasting arise partly from the fact that a forecaster must start with the data and information actually available for a particular forecasting problem, not what a theorist would like to have (Sartorius and Mohn, 1976), because the assumption of constancy or structural ability in the data might not be realistic in real-life forecasting situations. The forecaster should also consider judgemental methods, especially when systematic changes from established pattern/ relationships occur (Hogarth and Makridakis, 198 1). This requires, from the forecaster or practitioner, before applying any judgemental adjustment, that he must understand the nature of the system to which the forecasts refer. It may be argued that forecasting should not be judged on the simple accuracy criterion, but its role should be concerned with its ability to improve decision making within organizations (Hogarth and Makridakis, 1981).
Unless an analytical approach is used, it is very difficult to obtain a measure of the forecasting error or the accuracy of the forecast. If the manager is not supplied with a measure of the forecast accuracy, he will impute some degree of accuracy, depending on his confidence in the way the forecast was derived.
ACCURACY MEASURES
A problem is that, although accuracy represents an important factor in selecting a forecasting method, ‘one of the difficulties in dealing with the criterion of accuracy in forecasting situations is the absence of a single universally accepted measure of accuracy’ (Makridakis et al., 1983b). ‘Generally the greater the investment in developing and testing a forecasting model, the greater will be its accuracy and reliability.’ (Doyle and Fenwick, 1976). Clearly the forecaster or the practitioner is faced with a trade-off between the cost of applying a forecasting technique or an opportunity loss from basing decisions upon an inaccurate forecast and the value of increased accuracy in the selection of a technique.
A survey of the relevant literature reveals the description, development and empirical testing of many accuracy measures. Makridakis and Wheelwright (1978b) discussed in detail a variety of the most commonly used accuracy measures and their advantages and disadvantages. These measures were the mean squared error (MSE), the mean percentage error (MPE), the mean absolute percentage error (MAPE), Theil’s U-statistic and the root mean squared error (RMSE). Bretschneider and Carbone (1979) evaluated some of these accuracy measures in terms of opportunity cost expressing the advantages and the disadvantages. In addition Armstrong (1978) discussed eleven different accuracy measures: mean error (ME), mean absolute deviation (MAD), root-mean-squared error (RMSE), mean absolute percentage error (MAPE), adjusted mean absolute percentage error (MAPE), Theil’s measure, the coefficient of variation, the coefficient of determination R 2 , an accuracy ratio, turning points (TP’s) and hits and misses.
b i! s s.
Are
a of
app
licat
ion
Mai
n re
sults
L
itera
ture
sou
rces
in
Win
ters
' met
hod
vers
us
judg
emen
tal
fore
cast
s Sa
les
opin
ions
and
cor
pora
te
exec
utiv
es v
ersu
s ex
pone
ntia
l sm
ooth
ing,
har
mon
ic
smoo
thin
g an
d B
ox-J
enki
ns
Qua
ntita
tive
met
hods
ver
sus
judg
emen
tal
met
hods
Fina
nce
area
Inab
ility
of
qual
itativ
e m
etho
ds
Win
ters
' met
hod
prod
uced
for
ecas
ts w
hich
wer
e m
ore
accu
rate
than
thos
e of
hum
an
fore
cast
ers.
T
he s
tudy
indi
cate
d th
at fo
reca
sts
base
d on
opi
nion
s of
the
sale
s for
ce a
nd c
orpo
rate
ex
ecut
ives
gave
less
acc
urat
e re
sults
than
did
the
othe
r met
hods
. It w
as a
lso
foun
d th
at
quan
titat
ive
tech
niqu
es c
ost
less
and
took
less
tim
e.
Thes
e st
udie
s fo
und
that
qua
ntita
tive
met
hods
pro
vide
d be
tter
fore
cast
s th
an
judg
emen
tal
met
hods
. Sp
ecifi
cally
, Mee
hl (
1975
) ha
d th
e sa
me
findi
ngs,
exc
ept
he
foun
d on
ly o
ne c
ase
in w
hich
clin
ical
judg
emen
t was
sup
erio
r to
a s
tatis
tical
mod
el.
Thes
e st
udie
s com
pare
d th
e fo
reca
sts
of e
arni
ngs p
er s
hare
mad
e by
ana
lyst
s an
d th
e re
sults
obt
aine
d fr
om q
ualit
ativ
e m
etho
ds.
It w
as c
oncl
uded
tha
t an
alys
ts d
o no
t pe
rfor
m a
s w
ell a
s do
qua
ntita
tive
tech
niqu
es.
Inve
stig
ated
the
cau
ses
behi
nd t
he i
nabi
lity
of c
linic
al ju
dgem
ent
to o
utpe
rfor
m
quan
titat
ive
met
hods
. It
was
fou
nd t
hat
the
lack
of
appl
icat
ion
of v
alid
prin
cipl
es,
anch
orin
g ef
fect
s, re
gres
sion
bia
ses,
lack
of r
elia
bilit
y an
d th
e ba
sing
of p
redi
ctio
ns o
n irr
elev
ant i
nfor
mat
ion
cont
ribu
ted
tow
ards
poo
rer p
erfo
rman
ce o
f clin
ical
judg
emen
t.
Ada
m a
nd E
bert
(197
6)
a M
aber
t (I
975)
Sarb
in (
1943
), M
eehl
(196
5),
Saw
yer (
1966
), G
oldb
erg
(l97
0), A
rmst
rong
and
G
rohm
an (l
972)
, Sl
ovic
(1
972)
, Hog
arth
(197
9,
Dal
rym
ple
(197
5), C
erul
lo
and
Avi
la (
1975
), Li
bby
(l97
6), C
leve
land
and
Tia
o (1
976)
, Lor
ek e
t al
. (19
76),
Daw
es (1
977
), A
rmst
rong
(1
978)
, Fild
es a
nd F
itzge
rald
(1
981)
G
reen
and
Seg
all (
1967
), G
ragg
and
Mal
kiel
(l96
8),
Elto
n an
d G
rube
r (1
972)
, N
iede
rhof
fer
and
Reg
an
0
(197
2)
Slov
ic (l
972)
, K
ahne
man
Q
and
Tver
sky
(l97
3), T
vers
ky
Kah
nem
an (
1974
), D
awes
2 5
(1 97
4), T
vers
ky a
nd
3'
(197
7)
2 R 9'
Exhi
bit
I.
Stud
ies
indi
catin
g th
at q
uant
itativ
e m
etho
ds o
utpe
rfor
m q
ualit
ativ
e m
etho
ds
s 00
I
P
Are
a of
app
licat
ion
Mai
n re
sults
Li
tera
ture
sou
rces
t4
Perf
orm
ance
of
anal
ysts
C
laim
ed th
at a
naly
sts c
an d
o be
tter t
han
quan
titat
ive
met
hods
pro
vide
d th
at th
ey h
ave
accu
rate
eco
nom
ic a
nd in
dust
rial
inf
orm
atio
n.
Fina
ncia
l app
licat
ion
Con
clud
ed th
at an
ticip
ator
y su
rvey
s of i
nves
tmen
t spe
ndin
g in
the
U.S
. wer
e at
leas
t as
succ
essf
ul a
s ec
onom
etri
c m
odel
s an
d so
met
imes
did
bet
ter.
(a)
Rul
and
(197
6) w
as c
once
rned
with
the
eva
luat
ion of a
ccur
acy
and
info
rmat
ion
cont
ent o
f one
year
dur
atio
n ea
rnin
gs fo
reca
sts
by fi
rms.
The
stud
y in
dica
ted
that
ju
dgem
enta
l for
ecas
ts w
ere
supe
rior
to th
ose
of th
e qua
ntita
tive
met
hods
. For
ecas
t re
port
s of
larg
er f
irm
s te
nded
to
be m
ore
accu
rate
tha
n th
ose
of s
mal
ler
firm
s.
(b)
Bro
wn
and
Roz
eff (
1 979
) rea
naly
sed
data
from
Gre
en a
nd S
egal
l ( 19
67) a
nd fo
und
that
jud
gem
enta
l m
etho
ds u
sing
in
terim
re
port
s,
do b
ette
r th
an q
uant
itive
m
etho
ds.
John
ston
and
Sch
mitt
3
(l97
4), C
ritch
field
er
a/.
(1
978)
, Bra
ndon
and
Jar
rett
EL 5
( 197
9)
Lieb
ling
and
Rus
sell
(l96
9),
Jorg
enso
n er
al.
(l97
0), G
ray
9
Arm
stro
ng (1
978)
. Bro
wn
E an
d R
ozef
f (1
979)
S’
09
(197
4), R
ulan
d (1
976)
, 2
Thes
e ex
hibi
ts a
re b
ased
on
an id
ea f
rom
Hog
arth
and
Mak
rida
kis,
(198
1).
Exhi
bit
2.
Stud
ies
indi
catin
g th
at q
ualit
ativ
e m
etho
ds o
utpe
rfor
m q
uant
itativ
e m
etho
ds
Are
a of
app
licat
ion
Mai
n re
sults
Li
tera
ture
sou
rces
Fina
nce
and
acco
untin
g St
udie
d the
dis
aggr
egat
ed ju
dgem
enta
l eco
nom
ic fo
reca
sts i
n th
e acc
ount
ing
liter
atur
e.
Giv
en t
he c
onfli
ctin
g re
sults
rep
orte
d in
the
stu
dies
, it
is di
ffic
ult
to a
rgue
tha
t on
e pa
rtic
ular
fo
reca
stin
g m
odel
or
type
con
sist
ently
out
perf
orm
s its
com
petit
ors.
Sp
ecifi
cally
, Fild
es a
nd F
itzge
rald
(19
81) m
ade
a co
mpa
riso
n be
twee
n ju
dgem
enta
l fo
reca
sts
and
Box
-Jen
kins
an
d B
ayes
ian
time
serie
s. T
he s
tudy
exa
min
ed t
hree
ju
dgem
enta
l fo
reca
ster
s w
orki
ng f
or t
he C
ity o
f L
ondo
n in
stitu
tions
who
pro
duce
m
onth
ly b
alan
ce o
f pay
men
ts fo
reca
sts.
The
y fo
und
that
judg
emen
tal f
orec
asts
wer
e at
le
ast
as g
ood
in t
erm
s of
for
ecas
ting
erro
r as
the
Box
-Jen
kins
mod
els.
Abd
el-K
halik
and
Th
omps
on (
1977
), B
row
n an
d R
ozef
f (1
979)
, Rul
and
(197
8), R
icha
rds
and
Fras
er
(197
8), F
ildes
and
Fitz
gera
ld
(198
1)
5 ~ 2
5 2 d
Exhi
bit 3
. St
udie
s in
dica
ting
that
the
re a
re n
o di
ffer
ence
s in
acc
urac
y be
twee
n qu
antit
ativ
e an
d qu
alita
tive
met
hods
h,
Type
of
stud
y T
he s
tudy
and
mai
n fin
ding
s
Rev
iew
of j
udge
men
tal
met
hods
Man
agem
ent j
udge
men
tal
vs. a
naly
sts
judg
emen
tal
fore
cast
s B
ox-J
enki
ns v
s. he
uris
tics
by
the
corp
orat
e co
ntro
ller
Box
-Jen
kins
vs.
eco
nom
etric
m
odel
s
Box
-Jen
kins
vs.
regr
essi
on
Box
-Jen
kins
vs.
expo
nent
ial
smoo
thin
g
In a
revi
ew o
f ev
iden
ce o
n fo
reca
stin
g m
etho
ds f
or th
e so
cial
scie
nces
, the
fol
low
ing
conc
lusi
ons
for
fore
cast
ing
in s
ituat
ions
inv
olvi
ng la
rge
chan
ges
wer
e:
1.
Cau
sal
judg
emen
tal
met
hods
pro
vide
d m
ore
accu
rate
for
ecas
ts t
han
naiv
e ju
dgem
enta
l m
etho
ds.
2.
Obj
ectiv
e ju
dgem
enta
l m
etho
ds
prov
ided
m
ore
accu
rate
fo
reca
sts
than
su
bjec
tive j
udge
men
tal
met
hods
. 3.
In
depe
nden
t ex
perts
are
mor
e ac
cura
te th
an t
hose
invo
lved
in t
he s
ituat
ion.
C
oncl
uded
that
the
accu
racy
of m
anag
emen
t jud
gem
enta
l for
ecas
ting
was
bet
ter
than
an
alys
ts’ j
udge
men
tal
fore
cast
s. A
cros
s al
l st
udie
s, t
he m
ean
abso
lute
per
cent
age
erro
rs (M
APE
) for
man
agem
ent
and
anal
ysts
wer
e 15
.9 an
d 17
.7, r
espe
ctiv
ely.
T
he s
tudy
indi
cate
d th
at f
orec
asts
usi
ng th
e B
ox-J
enki
ns t
ime
serie
s an
alys
is o
f th
e qu
arte
rly
earn
ings
of
Stan
dard
Oil
Com
pany
of
Indi
ana
wer
e 20
per
cen
t m
ore
accu
rate
than
tho
se e
mpl
oyin
g he
uris
tics
by t
he c
orpo
rate
con
trol
ler.
A c
ompa
riso
n w
as m
ade
betw
een
Box
-Jen
kins
mod
els
and
stru
ctur
al e
cono
met
ric
mod
els.
Res
ults
sho
wed
tha
t th
e B
ox-J
enki
ns
mod
els
wer
e m
ore
robu
st
than
ec
onom
etric
mod
els.
R
eid
conc
lude
d th
at th
e B
ox-J
enki
ns m
odel
s did
at l
east
as w
ell a
s lar
ge e
cono
met
ric
mod
els.
C
ompa
red
Box
-Jen
kins
with
reg
ress
ion
met
hods
. T
he r
esul
ts i
ndic
ated
tha
t th
e B
ox-J
enki
ns p
roce
dure
is s
uper
ior
to th
e us
e of
reg
ress
ion.
D
alry
mpl
e in
dica
ted
that
Box
-Jen
kins
had
low
er av
erag
e for
ecas
ting
erro
rs th
an tr
end
regr
essi
on, b
ut t
hey
do p
rese
nt s
ome
prob
lem
s of
impl
emen
tatio
n.
Kin
ney
(197
8) in
his
stud
y of
com
pari
ng B
ox-J
enki
ns m
odel
s with
regr
essi
on in
dica
ted
that
bot
h m
etho
ds p
erfo
rmed
abo
ut t
he s
ame.
T
he st
udy
indi
cate
d th
at th
e B
ox-J
enki
ns f
orec
asts
wer
e bet
ter t
han
thos
e de
rived
from
H
olt
and
Win
ters
met
hods
. T
he s
tudy
indi
cate
d th
at:
(a)
The
for
ecas
ting
erro
rs b
y bo
th m
etho
ds w
ere
the
sam
e.
(b)
All
expo
nent
ial s
moo
thed
sea
sona
l mod
els y
ield
ed s
mal
ler a
vera
ge e
rror
s tha
n
The
stu
dy e
xam
ined
one
ser
ies
and
they
fou
nd
that
B
row
n’s
mod
el a
nd t
he
Box
-Jen
kins
met
hod
perf
orm
ed e
qual
ly w
ell.
In a
com
preh
ensi
ve st
udy
of I
1 1 ti
me-
serie
s for
det
erm
inin
g th
e ac
cura
cy o
f diff
eren
t tim
e-se
ries f
orec
astin
g te
chni
ques
, the
resu
lts in
dica
ted
that
Box
-Jen
kins
mod
els w
ere
infe
rior
to p
roje
ctio
ns m
ade
usin
g si
mpl
e na
ive,
mov
ing
aver
age,
and
exp
onen
tial
smoo
thin
g.
the
Box
-Jen
kins
mod
els.
i;‘
Lite
ratu
re s
ourc
es
LY
Arm
stro
ng (
1975
)
0
k
=L
Arm
stro
ng (
1984
)
Nel
son
( 198
1)
( Nel
son
1979
) (1
972)
, Sch
mid
t
Rei
d (1
971,
197
5)
Nel
son
(l97
2),
Nay
lor
and
Seak
s (1
972)
, Nar
ashi
mha
n c’t
al. (
1974
). M
akrid
akis
and
H
ibon
(19
79),
Dal
rym
ple
(l97
8), B
ox a
nd T
iao
(197
9).
Zobe
r (1
981)
K
inne
y (1
978)
New
bold
and
Gra
nger
b 2 T R
( 197
4)
Gro
ff (1
973)
k 3
CPJ
Geu
rts
and
Ibra
him
(19
75)
0 E 3’ 2
Mak
ridak
is a
nd H
ibon
09
- ( 1
979)
- P
Ex
hlbl
t 4
(ron
rmur
d)
w
- Ty
pe o
f st
udy
The
stu
dy a
nd m
ain
findi
ngs
Lite
ratu
re s
ourc
es
%
Box
-Jen
kins
and
ada
ptiv
e m
etho
ds
Mov
ing
aver
age,
exp
onen
tial
smoo
thin
g an
d re
gres
sion
Econ
omet
ric
vs.
time
serie
s m
odel
s (r
ando
m w
alk,
si
mpl
e ex
pone
ntia
l sm
ooth
ing,
sec
ond-
orde
r ex
pone
ntia
l sm
ooth
ing
and
Box
-Jen
kins
) D
oubl
e ad
aptiv
e sm
ooth
ing
vs. S
tate
spa
ce
Mul
tiple
tim
e-se
ries
mod
el
and
the
stat
e sp
ace
prog
ram
me
Expo
nent
ial s
moo
thin
g vs
. re
gres
sion
Smoo
thin
g m
odel
s
A s
tudy
of
com
pari
sons
bet
wee
n ad
aptiv
e fil
terin
g (a
dapt
ive
estim
atio
n pr
oced
ure
(AE
P),
Wid
row
-Hof
f ad
aptiv
e al
gori
thm
), th
e B
ox-J
enki
ns
met
hodo
logy
an
d m
ultip
le r
egre
ssio
n an
alys
is.
Bot
h ad
aptiv
e pr
oced
ures
pro
duce
d m
ore
accu
rate
fo
reca
sts
than
the
Box
-Jen
kins
app
roac
h.
The
stud
ies f
ound
that
exp
onen
tial s
moo
thin
g of
fere
d th
e be
st p
oten
tial a
ccur
acy
for
shor
t-te
rm f
orec
astin
g.
Kir
by f
ound
in
term
s of
mon
th-t
o-m
onth
for
ecas
ting
accu
racy
tha
t ex
pone
ntia
l sm
ooth
ing
did
best
, but
with
a ti
me
hori
zon
of o
ne to
six
mon
ths m
ovin
g av
erag
e an
d ex
pone
ntia
l sm
ooth
ing
gave
bet
ter r
esul
ts. T
he re
gres
sion
mod
el w
as th
e be
st m
etho
d fo
r lo
nger
-ter
m f
orec
asts
of
one
year
or
mor
e.
Dal
rym
ple
and
Kin
g st
udie
d th
e ac
cura
cy o
f th
ree
fore
cast
ing
met
hods
(m
ovin
g av
erag
es, t
rend
regr
essi
on a
nd H
olt’s
exp
onen
tial s
moo
thin
g). T
he re
sults
show
ed th
at
the
regr
essi
on m
etho
d w
as t
he m
ost
accu
rate
tech
niqu
e us
ed i
n th
e st
udy.
Th
e per
form
ance
of e
cono
met
ric m
odel
s was
com
pare
d w
ith th
at of
tim
e ser
ies m
odel
s.
Res
ults
ind
icat
e th
at th
e tim
e se
ries
mod
els
dom
inat
e th
e ec
onom
etri
c m
odel
s.
The
stud
y su
ppor
ted
the
conc
lusi
on o
f be
tter
resu
lts b
y us
ing
the
met
hod
of d
oubl
e ad
aptiv
e sm
ooth
ing
whi
ch o
utpe
rfor
ms
the
met
hod
of e
xpon
entia
l sm
ooth
ing
usin
g th
e sm
ooth
ing
fact
ors
of T
rigg
and
Lea
ch (
1967
) by
usin
g a
stat
e sp
ace
prog
ram
me.
In
a s
tudy
of
the
stat
e sp
ace
top
dow
n in
vent
ory
cont
rol
stra
tegy
for
a U
.S.
man
ufac
turi
ng co
mpa
ny it
was
indi
cate
d th
at th
e acc
urac
y of
ship
men
t for
ecas
ts co
uld
be i
mpr
oved
by
usin
g a
mul
tiple
tim
e-se
ries
mod
el.
Acc
urat
e fo
reca
sts
of s
ales
, sh
ipm
ents
, in
vent
ory
and
prod
uctio
n ca
n be
obt
aine
d by
usi
ng t
he s
tate
spa
ce
fore
cast
ing
prog
ram
me.
Th
e st
udy
incl
uded
dru
g an
d of
fice a
nd e
quip
men
t ind
ustr
ies a
nd fo
od in
dust
ries
. The
re
sults
indi
cate
d th
at ex
pone
ntia
l sm
ooth
ing
was
the m
ost a
ccur
ate b
asic
tech
niqu
e. In
th
e fo
od i
ndus
try,
the
mov
ing
aver
age
perf
orm
ed s
light
ly b
ette
r th
an e
xpon
entia
l sm
ooth
ing.
The
ove
rall
mea
n er
ror w
as 1
2.21
per
cen
t in
the
drug
indu
stry
whe
reas
it
was
13.
52 pe
r cen
t in
the o
ffice
and
equi
pmen
t ind
ustr
y. T
his i
ndic
ated
that
the
mod
els
used
for
bot
h in
dust
ries
wer
e no
t w
ithin
the
lim
it su
gges
ted
by t
he S
ecur
ities
and
Ex
chan
ge C
omm
issi
on (
SEC
): 5-
10 p
er c
ent.
In a
com
pari
son
of th
e pe
rfor
man
ce o
f nin
e sm
ooth
ing
mod
els
for e
stab
lishi
ng s
ome
guid
elin
es f
or m
odel
sel
ectio
n, t
he s
tudy
indi
cate
d th
at a
dapt
ive
smoo
thin
g m
odel
s ha
ve a
pro
noun
ced
tend
ency
to
gene
rate
uns
tabl
e fo
reca
sts.
The
adv
anta
ge o
f th
e ad
aptiv
e m
odel
s’ a
bilit
y to
reac
t to
sudd
en s
hift
s in
mea
n de
man
d w
as o
ffse
t by
thei
r te
nden
cy t
o ov
er-r
eact
to p
urel
y ra
ndom
fluc
tuat
ions
in d
eman
d.
3 3’ 9 s 2 2 9’
Bre
tsch
neid
er e
t al.
(1 97
9)
E
Gro
ss a
nd R
ay (I
965
). K
irby
( I
966)
, Le
vine
( 19
67).
Rai
ne
(197
1), K
ram
pf (1
972)
%
n
00
Dal
rym
ple
and
Kin
g (1
98 I)
Nar
ashi
mha
n (1
975)
Lars
on (
I98 1
)
Cam
eron
( 19
8 1)
Alfo
rd (
I 978
)
7
0
!-
bJ
John
son
and
Mon
tgom
ery
2
(1 97
9)
tr, 2
?
Ada
ptiv
e. B
ox-J
enki
ns, t
he
stat
e sp
ace
and
com
bini
ng
fore
cast
s
Com
pari
son
of a
dapt
ive
fore
cast
ing
met
hods
Fore
cast
per
form
ance
of
Box
-Jen
kins
usi
ng c
ensu
s X-
1 I f
or s
easo
nal a
djus
tmen
t O
rdin
ary
leas
t squ
ares
vs.
Mon
te C
arlo
R
educ
ing
fore
cast
ing
erro
rs
Dat
a di
sagg
rega
tion
Com
pare
d fiv
e fo
reca
stin
g m
etho
ds (
adap
tive
filte
ring
(AF)
, ad
aptiv
e es
timat
ion
proc
edur
e (A
EP)
, the
Box
-Jen
kins
mod
els,
the
stat
e sp
ace a
ppro
ach
and
a m
etho
d of
co
mbi
ning
for
ecas
ts).
The
res
ults
ind
icat
ed t
hat
the
Box
-Jen
kins
m
odel
s w
ere
cons
iste
ntly
out
-per
form
ed b
y th
e ot
her m
etho
ds. T
he s
tate
spac
e ap
proa
ch w
as t
he
best
indi
vidu
al fo
reca
stin
g te
chni
que,
bec
ause
it a
llow
ed p
aram
eter
val
ues
to c
hang
e th
roug
h th
e sa
mpl
e pe
riod
. N
ine
adap
tive
met
hods
wer
e co
mpa
red:
1.
2.
3.
4.
5.
6.
7.
8.
9.
Sing
le E
xpon
entia
l Sm
ooth
ing
(SE
S).
Trig
g an
d Le
ach'
s m
etho
d (1
967)
(T &
L).
Shon
e's
met
hod
(196
7) (S
HN
).
Emer
genc
y R
espo
nse
Rat
e (E
RR
).
Hol
t's m
etho
d (1
957)
(HL
T).
Hol
t's m
etho
d m
odifi
ed t
o us
e Tr
igg
and
Leac
h's
met
hod
(HL
T/T
& L
). H
olt's
met
hod
mod
ified
to
use
Shon
e's
met
hod
(HL
T/S
HN
).
Hol
t's m
etho
d as
an
optim
um p
aram
eter
sea
rch
rout
ine
(HL
TjO
PT).
A
dapt
ive
filte
ring
(Mak
rida
kis
and
Whe
elw
right
, 197
7) (A
F).
T
he r
esul
ts in
dica
ted
that
ada
ptiv
e fil
terin
g w
ith t
wo
wei
ghts
was
out
stan
ding
. The
y in
dica
ted
the
perf
orm
ance
of
ad
aptiv
e fil
terin
g (A
F) m
etho
d in
the
fol
low
ing
quot
atio
n:
'Its
appl
icat
ion
to c
omm
erci
al e
nviro
nmen
ts is
bes
t sui
ted
to a
reas
whe
re a
hig
h de
gree
of f
orec
ast a
ccur
acy
is re
quire
d an
d w
here
com
putin
g tim
e is
not
crit
ical
' (P
. 98)
. T
he s
tudy
ran
ked
the
met
hods
by
aver
agin
g th
e re
lativ
e po
sitio
ns o
f th
e m
etho
ds
obta
ined
with
the
diff
eren
t dat
a ty
pe. T
he re
sults
wer
e as
follo
ws:
ER
R, H
LT
/SH
N,
SHN
, AF,
HL
T/O
PT, T
& L
, HL
T/T
& L
, SE
S. H
owev
er, t
hey
stat
ed t
hat t
his l
ist is
no
t in
tend
ed t
o be
an
orde
r of
mer
it.
The
stud
y su
gges
ted
that
app
lyin
g a
seas
onal
adj
ustm
ent p
rogr
amm
e us
ing
cens
us X-
II p
rodu
ced
fore
cast
s w
hich
wer
e le
ss a
ccur
ate
than
th
ose
obta
ined
usi
ng t
he
unad
just
ed d
ata.
Th
ey fo
und
that
ord
inar
y le
ast s
quar
es s
tatis
tics p
erfo
rmed
rea
sona
bly
wel
l ove
r the
M
onte
Car
lo m
etho
d.
The
stud
y st
ated
that
err
ors c
an b
e re
duce
d by
add
ing
reso
urce
s to
the
fore
cast
ing
task
. W
hen
com
pani
es in
volv
e mor
e pe
ople
, hire
cons
ulta
nts a
nd u
se co
mpu
ters
, for
ecas
ting
erro
rs d
eclin
e. R
educ
ing
fore
cast
ing
erro
rs ca
n be
ach
ieve
d by
ada
ptin
g ne
w o
r mor
e ac
cura
te m
etho
ds.
The
obj
ectiv
e of
the
stu
dy w
as t
o de
rive
the
cond
ition
s un
der
whi
ch d
isag
greg
ated
ac
coun
ting
data
con
trib
ute
to m
ore
accu
rate
fore
cast
s of c
orpo
rate
per
form
ance
. The
re
sults
indi
cate
d th
at d
isag
greg
ated
dat
a do
not
nec
essa
rily
prod
uce
bette
r for
ecas
ts of
co
rpor
ate
perf
orm
ance
than
do
aggr
egat
ed d
ata.
Hor
rell
ef a
t. (1
981)
Hol
lier ef
a/. (
1981
)
Plos
ser (
1979
)
b 2 G 3
Fulle
r an
d H
asza
(198
0)
Dal
rym
ple
( 197
5)
3 S'
8 B
avne
a an
d La
koni
shok
n
3'
@Q
(1 98
0)
s % - P VI
Exhi
bit 4
. St
udie
s com
pari
ng th
e ac
cura
cy o
f di
ffer
ent f
orec
astin
g m
etho
ds
Are
a of
app
licat
ion
The
stud
y an
d m
ain
findi
ngs
Diff
eren
t fo
reca
stin
g te
chni
ques
and
cos
t/acc
urac
y tra
deof
f
Perf
orm
ance
of
time-
serie
s
Sim
ple
vs. s
ophi
stic
ated
m
etho
ds
Tran
sfor
mat
ions
and
the
Bo
x Je
nkin
s m
etho
d
Acc
urac
y of
tim
e-se
ries
The
stu
dy e
mph
asiz
ed t
he c
ompa
riso
n of
the
rel
ativ
e ac
cura
cy o
f se
vera
l sa
les
fore
cast
ing
met
hods
app
ropr
iate
for
sea
sona
l dat
a an
d ex
amin
ed t
he c
ost/a
ccur
acy
trade
-off
of
thes
e m
etho
ds. T
he s
tudy
was
app
lied
to m
onth
ly o
ccup
ancy
rat
es fo
r a
dow
ntow
n C
hica
go h
otel
fro
m J
anua
ry 1
977
to J
une
1980
. The
res
ults
of
seas
onal
m
etho
ds s
how
ed t
hat t
he le
ast c
ostly
met
hod
had
the
low
est m
ean
squa
red
erro
rs.
In t
he s
tudy
of
I1 1
time-
serie
s fo
r ex
amin
ing
the
accu
racy
of
vari
ous
fore
cast
ing
met
hods
, par
ticul
arly
tim
e-se
ries
met
hods
, the
stu
dy in
dica
ted
that
sim
pler
met
hods
pe
rfor
m w
ell i
n co
mpa
riso
n to
the m
ore c
ompl
ex an
d st
atis
tical
ly s
ophi
stic
ated
AR
MA
m
odel
of
Box
-Jen
kins
tec
hniq
ues.
The
stu
dy a
lso
foun
d th
at e
xpon
entia
l sm
ooth
ing
had
the
low
est e
rror
s w
hen
fore
cast
ing
up to
fou
r per
iods
in t
he f
utur
e, a
nd th
at th
e si
mpl
e mov
ing
aver
ages
met
hod
had
the
low
est c
umul
ativ
e err
ors f
or al
l tw
elve
fore
cast
pe
riod
s.
Sim
pler
met
hods
suc
h as
exp
onen
tial s
moo
thin
g or
nai
’ve f
orec
astin
g, d
id b
ette
r tha
n m
ore
soph
istic
ated
met
hods
.
They
conc
lude
d th
at th
ere w
ere
no d
iffer
ence
s whe
n th
e B
ox -J
enki
ns m
etho
d w
as u
sed
with
no
tr
ansf
orm
atio
ns
to
achi
eve
stat
iona
rity
an
d w
ith
thos
e in
volv
ing
tran
sfor
mat
ion.
T
he re
sear
cher
s ad
dres
sed
them
selv
es t
o th
e qu
estio
n of
whi
ch f
orec
astin
g m
etho
ds
wer
e be
tter
and
unde
r wha
t cir
cum
stan
ces.
The
y ap
plie
d di
ffer
ent
tech
niqu
es o
f tim
e se
ries f
or 1
001 t
ime-
serie
s. Ex
pert
s in
each
fiel
d an
alys
ed a
nd fo
reca
sted
the t
ime s
erie
s. T
he s
tudy
was
an
exte
nsio
n of
the
pre
viou
s st
udy
by M
akri
daki
s an
d H
ibon
(19
79).
The
resu
lts s
how
ed t
hat:
I,
Sin
gle
expo
nent
ial s
moo
thin
g pe
rfor
med
ext
rem
ely
wel
l fo
r m
onth
ly d
ata
but
it di
d ba
dly
for
year
ly d
ata
beca
use
the
year
ly d
ata
usua
lly i
nclu
ded
tren
ds.
2. F
or y
early
and
qua
rter
ly d
ata,
Lew
ando
wsk
i’s m
etho
d w
as th
e be
st, a
nd b
oth
Hol
t an
d H
olt-W
inte
rs’
met
hods
wer
e eq
uiva
lent
. 3.
The
Bay
esia
n fo
reca
stin
g m
etho
d an
d th
e B
ox-J
enki
ns
met
hod
wer
e ab
out
the
sam
e as
sin
gle
expo
nent
ial s
moo
thin
g.
4.
On
the
mic
ro
leve
l da
ta,
the
sim
ple
met
hods
wer
e m
uch
bette
r th
an t
he
soph
istic
ated
met
hods
. For
exa
mpl
e, L
ewan
dow
ski’s
ove
rall
MA
PE w
as 1
3.7
per
cent
for
mic
ro d
ata
and
18.2
per
cen
t for
mac
ro d
ata.
5.
On
the
mac
ro le
vel d
ata,
the
soph
istic
ated
met
hods
wer
e th
e be
st.
For
exam
ple,
Pa
rzen
’s M
APE
was
1 1.
2 per
cen
t for
mac
ro d
ata
and
18.4
per c
ent f
or m
icro
dat
a.
6. F
or s
easo
nal
data
, si
ngle
and
ada
ptiv
e re
spon
se r
ate,
exp
onen
tial
smoo
thin
g,
dese
ason
aliz
ed re
gres
sion
, B
ayes
ian
fore
cast
ing
and
Parz
en’s
met
hod
wer
e ab
out
the
sam
e as
far
as
over
all M
APE
was
con
cern
ed.
Lite
ratu
re s
ourc
es
L
-$
Pe
tto (1
981)
2 r 5 !? M
akri
daki
s an
d H
ibon
%
( 1
979)
%
9
Bau
man
(1 9
65).
Gro
ff
(197
3). G
eurt
s an
d lb
rahi
m
(197
5).
McC
oubr
ey a
nd
McK
enzi
e (1
976)
, New
bold
(1
974)
. Mak
rida
kis
and
Hib
on
(197
9). M
ahm
oud
(198
2)
Mak
rida
kis
and
Hib
on
( 197
9), N
elso
n (1
974)
Mak
rida
kis ef
a/.
(198
2)
2
?
For
non-
seas
onal
dat
a, s
ophi
stic
ated
met
hods
wer
e re
lativ
ely
bette
r. A
dapt
ive
estim
atio
n pr
oced
ures
(A
EP)
out
perf
orm
ed m
ost m
etho
ds f
or n
on-s
easo
nal d
ata.
W
hen
the
Box
-Jen
kins
m
etho
d w
as u
sed
with
no
tran
sfor
mat
ions
to
achi
eve
stat
iona
rity
in th
e va
rian
ce, i
t pro
duce
d re
sults
whi
ch w
ere
no d
iffer
ent
than
thos
e in
volv
ing
tran
sfor
mat
ion.
The
se fi
ndin
gs w
ere
cons
iste
nt w
ith th
ose
of M
akri
daki
s an
d H
ibon
(197
9).
Des
easo
naliz
ing
the
data
by
a si
mpl
e de
com
posi
tion
proc
edur
e w
as e
noug
h to
m
ake
the
maj
ority
of m
etho
ds (
both
sim
ple
and
soph
istic
ated
) per
form
the
sam
e.
The
rese
arch
er d
evel
oped
a fo
reca
stin
g pr
oced
ure w
hich
hel
ps fo
reca
ster
s or m
anag
ers
to fo
reca
st. T
he re
sear
ch in
vest
igat
ion,
invo
lvin
g an
em
piri
cal t
est o
f th
e de
velo
ped
proc
edur
e co
vere
d sh
ort-
term
for
ecas
ting
tech
niqu
es o
nly
and
used
I4
time-
serie
s. Pa
rt o
f th
e st
udy
conc
entr
ated
on
the
mea
sure
men
t of
the
acc
urac
y of
diff
eren
t fo
reca
stin
g te
chni
ques
. All
fore
cast
ing
tech
niqu
es a
vaila
ble
in t
he S
IBY
L-R
UN
NE
R
fore
cast
ing
pack
age
(Whe
elw
righ
t and
Mak
rida
kis,
197
8) w
ere
appl
ied
to th
e 14
tim
e-
serie
s. M
oreo
ver
som
e qu
antit
ativ
e te
chni
ques
and
qua
litat
ive
tech
niqu
es (m
anag
er’s
ad
just
men
t) w
ere
com
bine
d an
d te
sted
. Man
y di
ffer
ent a
ccur
acy
mea
sure
men
ts w
ere
used
: M
SE, M
APE
, MPE
, U
-sta
tistic
, G
arde
nfor
’s I
val
ue a
nd a
los
s co
st fu
nctio
n.
The
stu
dy in
dica
ted
that
:
Rel
ativ
e ac
cura
cy o
f tim
e-
serie
s an
d qu
alita
tive
met
hods
Mah
mou
d (1
982)
1.
2.
3.
4.
5.
6.
7.
8 9. H
arri
son’
s ha
rmon
ic w
as b
est
over
all f
or 2
8 pe
r ce
nt o
f al
l ser
ies.
Gen
eral
ada
ptiv
e an
d de
com
posi
tion
met
hods
eac
h w
ere
best
for
14
per
cent
of
the
serie
s. T
he n
a’iv
e met
hod,
sin
gle
expo
nent
ial s
moo
thin
g, li
near
and
sea
sona
l exp
onen
tial
smoo
thin
g an
d tim
e-se
ries
mul
tiple
reg
ress
ion
each
pro
vide
d th
e lo
wes
t co
st o
f fo
reca
stin
g er
ror
of 7
per
cen
t out
of
the
tota
l se
ries.
The
Box
-Jen
kins
met
hod
prov
ided
the
low
est c
ost o
f for
ecas
ting
erro
r for
onl
y on
e se
ries.
Com
bini
ng d
iffer
ent
quan
titat
ive
fore
cast
ing
tech
niqu
es to
geth
er a
nd c
ombi
ning
qu
antit
ativ
e and
qua
litat
ive
met
hods
, sho
wed
a re
duct
ion
of th
e cos
t of f
orec
astin
g er
ror
as w
ell a
s re
duci
ng M
SE.
In g
ener
al t
he r
esul
ts i
ndic
ated
tha
t si
mpl
e m
etho
ds o
utpe
rfor
m s
ophi
stic
ated
m
etho
ds s
uch
as B
ox-J
enki
ns.
This
was
als
o fo
und
by M
akri
daki
s an
d H
ibon
(1
979)
and
Mak
rida
kis
et a
l. (1
982)
. T
he st
udy
sugg
este
d th
at th
e com
pany
unde
r inv
estig
atio
n ne
eds t
o us
e at
leas
t nin
e fo
reca
stin
g m
etho
ds i
n or
der
to f
orec
ast
diff
eren
t ta
sks.
The
refo
re f
orec
astin
g pa
ckag
es w
ith m
ultip
le p
rogr
ams a
re d
esir
ed fo
r bus
ines
s ap
plic
atio
n, s
uch
as th
e SI
BY
L-R
UN
NE
R p
acka
ge.
Man
ager
s sp
ecifi
cally
can
rely
on
MSE
as a
mea
sure
men
t of a
ccur
acy
rath
er th
an
usin
g G
arde
nfor
’s I
val
ue i
n de
term
inin
g th
e se
lect
ion
of t
he m
etho
d(s)
. T
he s
tudy
fur
ther
sug
gest
ed t
hat i
t is
impo
rtan
t th
at th
e fo
reca
ster
s or
man
ager
s sp
end
suff
icie
nt ti
me
in a
pply
ing
diff
eren
t at
tem
pts
(run
s) w
ith d
iffer
ent
valu
es o
f pa
ram
eter
s (i.
e. a
, fi,
6..
. etc
.) to
cho
ose
the
estim
ated
par
amet
ers
whi
ch p
rovi
de
the
bette
r fo
reca
sts.
Alte
rnat
ivel
y, c
ompu
ter
prog
ram
s w
hich
sea
rch
for
optim
al
para
met
ers
may
be
used
.
b
n n Q T 2
3”’
Exhi
bit
5 (c
onri
nurd
)
Are
a of
app
licat
ion
The
stud
y an
d m
ain
findi
ngs
Lite
ratu
re s
ourc
es
Expe
rts
vs. n
on-e
xper
ts
usin
g th
ree
time-
serie
s m
et ho
ds
The
stud
y w
as c
ondu
cted
to te
st t
hree
fac
tors
(tec
hnic
al e
xper
tise,
hum
an ju
dgem
ent,
and
the
time
spen
t by
an a
naly
st) i
n de
term
inin
g th
e ac
cura
cy o
f fo
reca
sts
obta
ined
w
ith t
he u
se o
f a ti
me
serie
s for
ecas
ting
met
hod.
A c
ontr
ol e
xper
imen
t was
des
igne
d to
em
piric
ally
test
thes
e fa
ctor
s. It
invo
lved
the
part
icip
atio
n of
exp
erts
and
per
sons
with
lim
ited
trai
ning
(firs
t yea
r M
BA
stu
dent
s). F
orec
asts
wer
e ge
nera
ted
for 2
5 tim
e se
ries
usin
g th
e B
ox-J
enki
ns
and
Hol
t-W
inte
rs
(Win
ters
, 19
60)
tech
niqu
es
and
Car
bone
- Lon
gini
AE
P fi
lterin
g (C
arbo
ne a
nd L
ongi
ni.
1977
). A
n av
erag
e of
14
0 ho
urs
was
spe
nt b
y ea
ch t
eam
in
gene
ratin
g th
e fo
reca
sts.
A
ppro
xim
atel
y 60
per
cen
t of
the
tim
e w
as d
evot
ed t
o an
alys
ing
the
serie
s w
ith t
he
Box
-Jen
kins
tech
niqu
e, 25
per
cent
with
the
Hol
t-W
inte
rsm
etho
d an
d I5
perc
ent w
ith
the
Car
bone
-Lon
gini
met
hod.
The
re w
as v
ery
little
var
iatio
n in
the
se p
erce
ntag
es
acro
ss th
e te
ams.
Th
e fo
llow
ing
conc
lusi
ons
wer
e dr
awn:
I.
A si
gnifi
cant
diff
eren
ce in
fore
cast
ing a
ccur
acy
exis
ted
betw
een
met
hods
whe
n th
ese
wer
e ap
plie
d by
nov
ices
. Si
mpl
er m
etho
ds p
rovi
ded
the
mos
t ac
cura
te f
orec
asts
. 2.
Jud
gem
enta
l adj
ustm
ent b
y th
e st
uden
ts d
id n
ot im
prov
e ac
cura
cy.
3. I
ndiv
idua
lized
ana
lysi
s by
nov
ices
did
not
im
prov
e th
e ac
cura
cy o
f fo
reca
sts
4. V
ery
little
exp
ertis
e w
as r
equi
red
whe
n us
ing
a so
phis
ticat
ed fo
reca
stin
g m
etho
d
The
sru
dy e
valu
ated
for
ecas
ts w
hich
wer
e do
ne b
y a
varie
ty o
f ec
onom
ists
usi
ng
Livi
ngst
on’s
sam
ple
for
the
peri
od (
1947
-197
8).
The
resu
lts
indi
cate
d th
at t
he
econ
omis
ts in
the
sam
pled
id n
ot p
rodu
ce fo
reca
sts t
hat w
erea
ble
to o
utpe
rfor
m s
impl
e m
odel
s.
Car
bone
or u
l. (I 9
83)
obta
ined
thr
ough
aut
omat
ic p
roce
dure
s.
such
as
the
Box
-Jen
kins
tec
hniq
ue.
Econ
omis
ts’
fore
cast
ac
cura
cy
Ahl
ers
and
Lako
nish
ok (
1983
)
s
Exhi
bit 5
. St
udie
s in
dica
ting
the
accu
racy
of
time
serie
s
Essam Mahmoud Accuracy in Forecasting 149
Gardenfors (1980) introduced a relative accuracy measure (the I value) to determine the accuracy of the forecasted model relative to the naive model. Mahmoud (1982,1984)~in evaluating the I value, discussed the similarity of using either the Ivalue or the mean squared error; hence a forecaster can just as easily rely on the simply formulated MSE as on the more complex I value. Mahmoud (1982) and Mahmoud and Goyal(l983) developed and illustrated the use of loss cost functions as accuracy measures in terms of dollars. The first study (Mahmoud, 1982) showed the development and testing of a loss cost function as a special case for a company which faced a continuous backlog. The second study (Mahmoud and Goyal, 1983) developed loss cost functions for different general cases. Makridakis and Hibon (1979) introduced the adjusted U-statistic and Smyth (1983) introduced the standardized root-mean-squared error, which facilitates comparison of the forecasting accuracy for different variables or different sets of data.
Finally, there are many studies which have illustrated the use of accuracy measures. Examples are Makridakis and Hibon (1979), Makridakis et al. (1982, 1983b), Mahmoud (1982) and Smyth (1983).
In Exhibits 1-5, a summary of all major studies in the area of accuracy is presented in which the studies are grouped according to the similarity of their findings. In particular, Exhibits 1-3 show comparisons between quantitative and qualitative methods. Furthermore, Exhibits 4 and 5 summarize studies comparing the accuracy of different forecasting methods as well as other studies related to accuracy.
The indications of Exhibits 1-5, that is the findings regarding the accuracy record of forecasting methods, are disturbing for both academics and practitioners when confronted with choosing between alternative forecasting techniques. In general, forecasters should consider the following statement: ‘Accuracy varies from one set of data to another and from one time period to the next’ as discussed by McNees (1976) and Makridakis and Wheelwright (1978b).
COMBINING FORECASTING TECHNIQUES
‘By combining forecasting techniques, we retain more insight than is obtainable from the use of any simple technique’ (Doyle and Fenwick, 1976, p. 63). In the case of the absence of one of the assumptions or erroneous data or other violations of the standard assumptions, it is important to consider the possibility of combining forecasting techniques (Krasker, 1980, p. 1334). Cooper and Nelson (1975) suggested that robustness could be improved by combining different types of forecasting models, thus using a combining approach to provide better forecasts. ‘It is often better to combine sales forecasting methods than to select between them’ (Doyle and Fenwick, 1976, p. 60). Thus, combinations of forecasts are frequently more accurate because they retain more information about the potential market. In today’s increasingly volatile markets, the combining of forecasting methods is particularly important.
The literature illustrates some factors supporting the combining techniques, its importance and its advantages (Doyle and Fenwick, 1976; Krasker, 1980).
Different approaches for combining two or more forecasts into a composite forecast have been applied by Bates and Granger (1969), Reid (1969), Newbold and Granger (1974), Doyle and Fenwick (1976), Makridakis et al. (1982), Makridakis and Winkler (1983a), Winkler and Makridakis (1983), Granger and Ramanathan (1984) and Zarnowitz (1984). Doyle and Fenwick (1976) introduced the following three different approaches:
1. Aueraging. They showed the possibility of taking a simple average of two forecasts for each period (also see Makridakis et al., 1982).
Are
a of
app
licat
ion
Des
crip
tion
Mai
n fi
ndin
gs
L
Lite
ratu
re s
ourc
es
Box
-Jen
kins
with
oth
er
tech
niqu
es
Com
bine
d B
ox-J
enki
ns p
roje
ctio
n w
ith
thos
e ge
nera
ted
by o
ther
for
ecas
ting
tech
niqu
es. T
hey
also
intr
oduc
ed t
he
wei
ghte
d av
erag
e of
for
ecas
ts th
at
min
imiz
es t
he v
aria
nce
of t
he c
ombi
ned
fore
cast
err
or.
Box
-Jen
kins
an
d C
ombi
ned
a re
gres
sion
mod
el
regr
essi
on
with
a u
niva
riat
e B
ox-J
enki
ns m
odel
. B
ox-J
enki
ns a
nd e
xpon
entia
l C
ombi
ned
fore
cast
s fr
om t
he a
pplic
atio
n sm
ooth
ing
of e
xpon
entia
l sm
ooth
ing
and
Box
-Jen
kins
ty
pe m
odel
s to
air
line
traff
ic
data
.
Bay
esia
n ad
just
men
t to
give
n B
ox-J
enki
ns u
niva
riat
e te
chni
que
Expo
nent
ial s
moo
thin
g w
ith
regr
essi
on
Hol
t's s
moo
thin
g w
ith
adap
tive
resp
onse
rat
e of
si
ngle
exp
onen
tial s
moo
thin
g Q
uant
itativ
e m
etho
ds a
nd
qual
itativ
e m
etho
ds
Gue
ssw
ork
and
stat
istic
s (g
uess
tics)
Form
al B
ayes
ian
adju
stm
ents
wer
e m
ade
to a
giv
en B
ox-J
enki
ns
univ
aria
te
tech
niqu
e.
Mab
ert (
1978
) com
bine
d si
mpl
e ad
aptiv
e re
spon
se e
xpon
entia
l sm
ooth
ing
with
a
regr
essi
on m
odel
to
trac
k hi
stor
ical
che
ck
volu
me.
D
evel
oped
the
met
hodo
logy
of
mer
ging
ex
pone
ntia
l sm
ooth
ing
mod
els
and
mul
tiple
reg
ress
ion.
The
dev
elop
men
t re
sults
in s
impl
e fo
rmul
ae th
at a
llow
the
us
er t
o up
date
the
regr
essi
on c
oeff
icie
nts
in a
n ad
aptiv
e fa
shio
n.
The
stu
dy u
sed
a si
mpl
e av
erag
e fo
r co
mbi
ning
for
ecas
ting
tech
niqu
es.
Fore
cast
ing
tech
niqu
es w
ere
com
bine
d us
ing
the
sim
ple
aver
age
met
hod.
In
volv
ed a
com
bina
tion
of g
uess
wor
k an
d st
atis
tics.
The
sal
es tr
end
grap
h pl
ays
an im
port
ant r
ole
in th
is t
echn
ique
(usi
ng
the
mov
ing
aver
age
met
hod)
.
a'
The
com
bini
ng p
roce
dure
pro
vide
d th
e D
alry
mpl
e (1
978)
, B
inro
th e
t 5
best
ove
rall
fore
cast
s. B
ox-J
enki
ns
fore
cast
s ca
n fr
eque
ntly
be
impr
oved
up
on b
y co
mbi
natio
ns w
ith e
ither
H
olt-W
inte
rs
or s
tepw
ise
auto
regr
essi
ve
is w
ell w
orth
try
ing
and
requ
ires
ver
y
9 s
al. (
1979
), N
ewbo
ld a
nd
Gra
nger
(l97
4), M
ahm
oud
( 198
2)
crl : 5
fore
cast
s. T
hey
indi
cate
d th
at c
ombi
ning
little
eff
ort.
S'
The
com
bine
d fo
reca
sts
redu
ced
the
mea
n Pi
ndyc
k an
d R
ubin
feld
09
n
squa
red
erro
rs b
y a
larg
e fa
ctor
. Fo
reca
stin
g ac
cura
cy im
prov
ed b
y co
mbi
ning
the
fore
cast
s of
bot
h m
etho
ds.
They
als
o em
phas
ized
the
pos
sibi
lity
of
the
rele
vant
wei
ghts
cha
ngin
g th
roug
h tim
e.
The
exp
ecte
d re
sults
wer
e in
fav
our
of t
he
Gre
gg (1
980)
co
mbi
ning
app
roac
h.
The
com
bine
d ap
proa
ch p
rovi
ded
bette
r fo
reca
sts.
(197
6), A
dam
s (1
978)
B
ates
and
Gra
nger
(196
9)
Mab
ert
(197
8)
The
fore
cast
ed re
sults
wer
e ac
cept
able
. C
rane
and
Cro
tty
(196
7).
Bon
ini
and
Free
land
(19
79)
The
com
bine
d ap
proa
ch p
rovi
ded
bette
r M
SE a
nd a
red
uctio
n of
the
cos
t of
fore
cast
ing
erro
r.
The
fore
cast
ed re
sults
sho
wed
im
prov
emen
t in
accu
racy
. T
he a
ppro
ach
show
ed i
mpr
ovem
ent
in
fore
cast
s.
Mah
mou
d (1
982)
Mah
mou
d (1
982)
Gol
d (1
979)
Mul
tidet
erm
inan
t ap
proa
ch
Com
bine
d fo
reca
sts
usin
g co
mpu
teri
zed
deci
sion
su
ppor
t sys
tem
(D
SS)
Com
bini
ng fo
reca
sts
usin
g si
mpl
e or
wei
ghte
d av
erag
e
Rei
d (1
969)
sho
wed
diff
eren
t ap
proa
ches
fo
r com
bini
ng tw
o or
mor
e fo
reca
sts.
Rei
nmut
h an
d G
eurt
s (19
79) u
sed
a re
curs
ive
regr
essi
on a
lgor
ithm
for
two
or
mor
e fo
reca
sts
and
crea
ting
a m
ultid
eter
min
istic
for
ecas
t m
odel
.
They
dev
elop
ed a
mod
el t
o he
lp d
ecis
ion
mak
ers
thro
ugh
an e
xpan
ded
deci
sion
su
ppor
t sys
tem
. The
met
hodo
logy
use
d w
as m
ultip
le o
bjec
tive
linea
r pr
ogra
mm
ing (MOLP). T
hey
used
thr
ee
diff
eren
t for
ecas
ting
met
hods
(ex
pone
ntia
l sm
ooth
ing,
har
mon
ic s
moo
thin
g an
d m
ultip
le r
egre
ssio
n).
Mak
rida
kis e
t al.
(l98
2,19
83a,
c) ad
dres
sed
the
issu
e of
com
bini
ng f
orec
astin
g te
chni
ques
in o
rder
to im
prov
e ac
cura
cy.
They
use
d ‘C
ombi
ning
A m
etho
d w
hich
co
nsis
ted
of a
sim
ple
aver
age
of s
ix
met
hods
and
‘Com
bini
ng B
met
hod
usin
g a
wei
ghte
d av
erag
e of
the
six
met
hods
ba
sed
on th
e sa
mpl
e co
vari
ance
mat
rix
of
fitti
ng e
rror
s.
Ten
for
ecas
ting
met
hods
wer
e ap
plie
d to
10
01 ti
me
serie
s. A
com
bina
tion
rule
was
de
velo
ped
to fi
nd c
ombi
ned
fore
cast
s fo
r se
vera
l pe
riod
s ah
ead.
Fiv
e pr
oced
ures
w
ere
used
for
est
imat
ing
wei
ghts
whe
n co
mbi
ning
met
hods
.
Som
e ap
plic
atio
ns o
f co
mbi
natio
ns o
f m
etho
ds d
emon
stra
ted
impr
ovem
ents
of
fore
cast
s. Em
piric
al r
esul
ts in
volv
ing
thre
e da
ta s
ets
sugg
este
d th
at th
e re
gres
sion
com
bina
tion
proc
edur
e pr
ovid
ed s
ubst
antia
l im
prov
emen
t ove
r th
ose
obta
ined
by
usin
g a
unid
eter
min
istic
for
ecas
t mod
el.
The
con
clus
ion
was
tha
t com
bine
d fo
reca
sts
wou
ld b
e pr
efer
red
to in
divi
dual
fo
reca
sts
in a
wid
e va
riety
of
deci
sion
en
viro
nmen
ts.
Bot
h co
mbi
ning
met
hods
indi
cate
d th
at
the
resu
lting
fore
cast
s pe
rfor
m v
ery
wel
l ov
eral
l an
d be
tter
than
the
in
divi
dual
met
hods
incl
uded
in t
he
aver
age.
Com
bini
ng A
per
form
ed b
ette
r th
an C
ombi
ning
B.
Of
the
five
wei
ghtin
g pr
oced
ures
use
d,
two
outp
erfo
rmed
the
oth
ers.
The
co
mbi
ned
fore
cast
s w
ere
mor
e ac
cura
te
than
for
ecas
ts f
rom
indi
vidu
al m
etho
ds
unde
r m
ost c
ondi
tions
with
larg
e tim
e ho
rizo
ns p
rovi
ding
s so
me
exce
ptio
ns. T
he
accu
racy
of w
eigh
ted
aver
ages
ou
tper
form
ed t
hat o
f th
e si
mpl
e av
erag
e.
It a
ppea
rs t
hat d
iffe
rent
ial w
eigh
ting
can
lead
to
impr
oved
for
ecas
ts.
Rei
d (1
969)
Rei
nmut
h an
d G
eurt
s (19
79)
Ree
ves
and
Law
renc
e (1
982)
Mak
rida
kis
et af. (
1982
, 19
83a,
c)
Win
kler
and
Mak
rida
kis
(198
3)
Are
a of
app
licat
ion
Des
crip
tion
~~
~
A s
impl
e av
erag
e w
as u
sed
to c
ombi
ne
the
fore
cast
s of
the
var
ious
met
hods
co
nsid
ered
in
the
stud
y. F
ourte
en
fore
cast
ing
met
hods
and
11 1
tim
e se
ries
wer
e us
ed.
Form
ing
and
inte
rpre
ting
the
com
bine
d fo
reca
sts
Dis
cuss
ion
of h
ow l
inea
r com
bina
tions
of
fore
cast
s m
ay b
e fo
rmed
, the
pro
perti
es
of t
he c
ombi
ned
fore
cast
s an
d ho
w t
he
resu
lts m
ay b
e in
terp
rete
d. A
n ap
plic
atio
n us
ing
thre
e di
ffer
ent c
ombi
ned
met
hods
w
as p
erfo
rmed
.
Com
bini
ng c
orre
spon
ding
se
ts o
f in
divi
dual
fo
reca
sts
The
stu
dy in
vest
igat
ed th
e ac
cura
cy o
f co
mbi
ning
corr
espo
ndin
g pr
edic
tions
from
di
ffer
ent s
ourc
es a
.rd th
e co
rres
pond
ing
sets
of
indi
vidu
al fo
reca
sts.
~~
Mai
n fin
ding
s Li
tera
ture
sou
rces
The
accu
racy
of
com
bine
d fo
reca
sts
was
lit
tle in
fluen
ced
by t
he s
peci
fic m
etho
ds
incl
uded
in t
he c
ombi
natio
n. A
ccur
acy
impr
oved
with
incr
ease
s in
the
num
ber
of
met
hods
bei
ng c
ombi
ned,
alth
ough
a
degr
ee o
f sa
tura
tion
was
rea
ched
afte
r ab
out f
our
or fi
ve m
etho
ds.
The
varia
bilit
y of
acc
urac
y am
ong
diff
eren
t co
mbi
natio
ns d
ecre
ased
as
the
num
ber
of
met
hods
inc
lude
d in
the
com
bina
tion
incr
ease
d.
The
supe
riorit
y of
one
of
the
thre
e co
mbi
ned
met
hods
was
dem
onst
rate
d. I
t pr
ovid
es th
e sm
alle
st m
em s
quar
ed e
rror
an
d an
unb
iase
d co
mbi
ned
fore
cast
eve
n if
indi
vidu
al fo
reca
sts
are
bias
ed. T
here
are
cl
ear
adva
ntag
es in
com
bini
ng f
orec
asts
bu
t a
num
ber
of p
oten
tial
met
hodo
logi
cal
diff
icul
ties r
emai
n.
The
resu
lts s
how
ed th
at th
ere
are
gain
s to
the
fore
cast
s use
rs f
rom
com
bini
ng
pred
ictio
ns fr
om d
iffer
ent s
ourc
es.
Mak
ridak
is a
nd W
inkl
er
(198
3)
Gra
nger
and
Ram
anat
han
(198
4)
Zarn
owitz
(19
84)
Exhi
bit 6
. St
udie
s of
com
bini
ng f
orec
astin
g te
chni
ques
.
Essam Mahmoud Accuracy in Forecasting 153
2. Historical weightings. Only alternative was to base weights on the accuracy of past forecasting performance of best fit to past data. A specific approach was that each forecast should be weighted by the ratio of one minus its mean squared error to the total mean square for all the forecasts. Subjective weightings. This approach is known as the Bayesian approach to forming composite forecasts. Management may prefer to weight the forecasts based upon their personal judgements on which methods more closely reflect the changing reality.
Another method was developed by Bates and Granger (1969), Newbold and Granger (1974) and Makridakis et al. (1982) and applied by Makridakis and Winkler (1983a) and Winkler and Makridakis (1983). This method was a weighted average based on the sample covariance matrix of fitting errors (in terms of percentage errors). The idea behind this approach is that the combined forecasts can be improved by taking into account the accuracy of each method and the covariance between the methods. Makridakis and Winkler (1983a) investigated empirically the impact of the number of forecasting methods from which to choose when combining forecasts, on the accuracy of simple averages. They concluded that the forecasting accuracy improves, and that the variability of accuracy among different combinations decreases as the number of methods on the average increases. Furthermore, Winkler and Makridakis (1983) tested some propositions put forward by Bates and Granger (1969) and Newbold and Granger (1974). They used a large sample of 1001 series to test a combination of ten forecasting methods. The results favoured the combining approach rather than the individual method.
3.
Exhibit 6 summarizes the most important studies of combining forecasting techniques.
IMPLICATIONS OF THE ACCURACY FINDINGS
This paper has emphasized the accuracy criterion as an important factor in the selection of a forecasting model. In this section a discussion of the implications of the research findings in Exhibits 14 is included.
From the foregoing survey of research in forecasting accuracy, the following conclusions and associated implications may be presented. On the whole, past research suggests that quantitative methods outperform qualitative methods. This is of obvious significance to practitioners wishing to improve their forecasting accuracy. Forecasters must, however, be aware of the particular circumstances of the empirical tests in which the superiority of quantitative methods was demonstrated. Only where the circumstances are similar in practice can more accurate forecasts using quantitative techniques be expected. In some cases, the forecaster may face a situation of limited past data availability, for example, where the applicability of qualitative techniques has been illustrated. Here, studies such as those by Armstrong (1975, 1981) suggest that it is also advantageous to experiment with more than one qualitative method as some are more accurate than others.
An interesting conclusion from the point of view of the practitioner is that simple forecasting methods perform more accurately than, or at least as accurately as, sophisticated methods. This has been shown by a leading study of Makridakis and Hibon (1979) which used 1 1 1 time series and was confirmed by the more comprehensive study of Makridakis et al. (1982). In addition, an in- depth examination of this issue using 14 series and a real business application (Mahmoud, 1982) found that for only one series did the Box-Jenkins technique perform more accurately than simple methods. The implication of these findings is to encourage practitioners to view forecasting methodologies more positively. This may be especially so in the case of managers who wish to be able to predict and cope with future uncertainties but do not have the training or expertise to deal
154 Journal of Forecasting Vol. 3, Iss. No. 2
with very complex forecasting techniques. Furthermore, simpler techniques are less costly to apply in terms of computer time and manpower. For theorists, the implications are to concentrate their efforts on the development and refining of simpler forecasting models, and on the simplification of more complex techniques.
One of the important issues that the forecaster faces is the improvement of accuracy. Concerning the factors associated with accuracy, Carbone et al. (1983) found that technical expertise, judgemental adjustment and individualized analyses were of little value in improving forecast accuracy as compared to black box approaches. In addition, simpler methods were found to provide significantly more accurate forecasts than the Box-Jenkins method when applied by persons with limited training (see Exhibit 5). Accuracy can be improved either by spending sufficient time and following the correct procedures in choosing and determining the correct estimated parameters (see Mahmoud, 1982; Exhibit 5) or by combining forecasting techniques (see Exhibit 6). Various studies (for example; Bates and Granger, 1969; Dalrymple, 1978; Makridakis et al., 1982; Mahmoud, 1982; Makridakis and Winkler, 1983) have indicated the advantages of combining forecasting techniques. The implication of these results is that forecasters can improve the quality of their predictions by relying on more than one forecasting method. Additionally, the results have implications for acquiring computer packages for forecasting. Comprehensive packages which offer possibilities for combining techniques will be preferred.
FUTURE DIRECTIONS FOR RESEARCH
The survey of accuracy-related research suggests that several issues need further investigation. This section discusses these issues.
Replication of many of the studies reported here is encouraged to provide confirmation of the results. In particular, there is a need for more in-depth empirical studies concentrating on different business sectors to confirm the superiority of simple forecasting methods over complex techniques. A case study approach evaluating business applications of forecasting and the accuracy achieved may be useful. This approach could examine accuracy in the context of the strategic planning and decision-making process of which forecasting is a part. Studies are required to determine the factors which affect accuracy. For example, there is a need to replicate the study of Carbone et al. (1983). The results of such studies would be valuable in selecting the most accurate forecasting methods for specific situations. What is needed is an understanding of when and under what circumstances one method is to be preferred over the others. Perhaps a procedure could be developed to be used by the forecaster to estimate the relative accuracy of different techniques. One problem in determining forecasting accuracy is the proliferation of different measures for this purpose, each with its assumptions, advantages and disadvantages. More work on the evaluation of accuracy measures along the lines of that conducted by Makridakis and Wheelwright (1978b), Armstrong (1978) and Mahmoud (1984) should be done. This would provide information to help forecasters apply the criterion of accuracy more effectively by narrowing the range of essential measures to be considered in comparing and selecting forecasting methods.
Finally, much more theoretical and empirical research is required to determine the best approach for combining forecasting techniques and which techniques should be combined. This may lead to a general theory of combining forecasting techniques.
CONCLUSION
To summarize, three valuable conclusions can be drawn from this survey of accuracy in forecasting. First it appears that simple forecasting methods perform reasonably well in
Essam Mahmoud Accuracy in Forecasting 155
comparison to the sophisticated forecasting methods. The question, however, is whether the extra cost of the sophisticated method is justified by increases in accuracy, since differences among methods are small. Secondly, most of the studies indicate that quantitative methods are more accurate than qualitative methods. Some studies have, however, demonstrated the superiority of the latter. Thirdly, forecasting accuracy can be improved by combining techniques.
This survey should be useful in assisting forecasters to select more accurate techniques or combinations of techniques, given that various constraints such as data, computer facilities and understanding of the techniques are taken into account. For researchers, the survey reviews the state of the art in forecasting accuracy. Undoubtedly, further study of several issues is necessary. It is time we learned from the results of empirical studies and tried to understand what is wrong with our existing forecasting methods and how existing methods can be improved.
ACKNOWLEDGEMENTS
The author would like to thank Professors Arun Jain and Gillian Rice for their advice.
REFERENCES
Abdel-Khalik, A. R. and Thompson, R. B., ‘Research on earnings forecasts: the state of the art’, The Accountinx Journal, March (1977).
Adam, E. E.-and Ebert, R. J., ‘A comparison of human and statistical forecasting’, AIITE Trans., 8 (1976), 1 20- 1 27.
Adams, A. J., ‘Modeling and forecasting seasonal time series of product sales across territories: a
Ahlers, D. and Lakonishok, J . ‘A study of economists’ consensus forecasts’, Management Science, 29
Alford, R. M., ‘Sales forecasting: an analysis of the laccuracy of various time series forecasting techniques in
Armstrong, J. S., ‘Monetary incentives in mail surveys’, Public Opinion Quarterly, 39 (1975), Spring,
Armstrong, J. S . , Long Range Forecasting from Crystal Ball to Computer, New York: Wiley, 1978. Armstrong, J. S., ‘Relative accuracy of judgemental and extrapolation methods in forecasting’, Journal of
Armstrong, J. S. and Grohman, M. C., ‘A comparative study for long-range market forecasting‘,
Bass, F. M., ‘A new product growth model for consumer durables’, Management Science, 15 (1969), January
Bass, F. M., ‘An integration of a new product growth model with the experience cost function and optimal
Basu, S. and Schroeder, R. G., ‘Incorporating judgments in sales forecasts: applications of the Delphi method
Bates, J. M. and Granger, C. J., ‘The combination of forecasts’, Operational Research Quarterly, 20 (1969),
Batty, M., ‘Monitoring and exponential smoothing forecasting system’, Operational Research Quarterly, 20
Bauman, W . S., ‘The less popular stocks versus the most popular stocks’, Financial Analysts Journal, 21 (1965), 61-69.
Bavnea, A. and Lakonishok, J., ‘An analysis of the usefulness of disaggregated accounting data for forecasts of corporate performance’, Decision Sciences, I1 (1980), (l), January, 17-26.
Binroth, W., Burshtein, I., Haboush, R. K. and Hartz, J. R., ‘A comparison of commodity price forecasting by Box-Jenkins and regression techniques’, Technological Forecasting and Social Change, No. 14 (1979),
comparative study’, Ph.D. Thesis, The University of Iowa, 1978, p. 12.
(1983), 1113-1125.
selected consumer products’, Ph.D. Thesis, Texas A 8t M University, 1978.
11 1-1 16.
Forecasting, 3 (1984), in press.
Management Science, 19 (1972), 21 1-227.
215-227.
pricing’, presented at the ORSAITIMS National Meeting, New York City, May 1978.
at American Hoist and Derrick’, Interfaces, May (1 977, 18-23.
451-461.
(1969), 319-323.
169-1 80.
156 Journal of Forecasting Vol. 3, Iss. No. 2
Bonini, C. P. and Freeland, J. R., ‘Forecasting by smoothed regression: development and application to predicting customer utility bills’, in Makridakis, S. and Wheelwright, S . C. (eds), TIMS Studies in the Management Sciences, 12 (l979), 279-296.
Box, G. E. P. and Tiao, G. C., ‘Intervention analysis with applications to economic and environmental problems’, J. Amer. Stat. Assoc., 70 (1975), 7C79.
Brandon, C. H. and Jarrett, H. E., ‘Accuracy of externally prepared forecasts’, Review of Business and Economic Research, 13 (1979), Fall, 3547.
Bretschneider, S. and Carbone, R., ‘On the evaluation of forecasting performance from a managerial perspective’, Working Paper, The Ohio State University, Columbus, Ohio 43210, U.S.A. 1979.
Bretschneider, S., Carbone, R. and Longini, R. L., ‘An adaptive approach to time-series’, Dec. Sciences, 10 (1979). ( I ) , January.
Brown, L. and Rozeff, M. S., ‘The predictive value of interim reports for improving forecasts of future quarterly earnings’, Accounting Review, 54 (l979), 585-591.
Cameron, A. V.,‘Applications of state space forecasting’, presented at the First International Symposium on Forecasting, Quebec, Canada, May, 198 1.
Carbone, R., Andersen, A,, Corriveau, Y. and Corson, P. P., ‘Comparing for different time series methods the value of technical expertise, individualized analysis, and judgmental adjustment’, Management Science, 29 (1983). (5), 559-566.
Carbone, R. and Longini, R. L., ‘A feedback model for automated real estate assessment’, Management Science, 24 (1977), 241-248.
Cerullo, M. J. and Avila, A,, ‘Sales forecasting practices-a survey’, Managerial Planning, V24 (1975), (2) September/October, 33-39.
Cleveland, W. P. and Tiao, G. C., ‘Decomposition of seasonal time series: a model for the census X-I1 program’, J. Amer. Stat. Assoc., 71 (1976), 581-587.
Cooper, J. P. and Nelson, C. R., ‘The ex-ante prediction performance of the St. Louis and FRB-MIT-PENN econometric models, and some results on composite predictions’, Journal ofMoney, Credit and Banking, 7
Crane, D. B. and Crotty, J. R., ‘A two-stage forecasting model: exponential smoothing and multiple
Critchfield, T., Dyckman, T. and Lakonishok, J., ‘An evaluation of security analysts’ forecasts’, Accounting
Dalrymple, D. J., ‘Sales forecasting methods and accuracy’, Business Horizons, (1975), December, 69-74.
(1975), 1-31.
regression’, Management Science, 13 (1967), 501-507.
Reciew, 53 (1978), 651458.
Dalrymple, D. J., ‘Using Box-Jenkins techniques in sales forecasting’, Journal ofBusiness Research, (1978), 1 33- 145.
Dalrymple, D. J. and King, B. E., ‘Selecting parameters for short-term forecasting techniques’, presented at
Dawes, R. M., in Carroll, J. S. and Payne, J. W. (eds), Shallow Psychology. Hillsdale, N. J.: Laurence
Doyle, P. and Fenwick, I . A,, ‘Sales forecasting-using a combination of approaches’, Long-Range Planning,
Elton, E. J. and Gruber, M. J., ‘Earnings estimates and the accuracy of expectational data’, Management
Fildes, R. and Fitzgerald, D., ‘The use of information in balance of payments forecasting’, presented at the
Fuller, W. and Hasza, David, ‘Predictors for the first-order autoregressive process’, Journal of Econometrics,
Gardenfors, P., ‘On the information provided by forecasting models’, Technological Forecasting and Social
Geurts, M. D. and Ibrahim, I. B., ‘Comparing the Box-Jenkins approach with the exponentially smoothed
Gold, J. G., ‘Guesswork and statistics in sales forecasting’, Marketing (London), (1979), November, 88-91. Goldberg, L. R., ‘Man versus model of man: a rationale, plus some evidence, for a method of improving
Gragg, J. and Malkiel, B., ‘The consensus and accuracy of some predictions of the growth in corporate
Granger, C. W. J. and Ramanathan, R., ‘Improved methods of combining forecasts’, Journal of Forecasting,
the First International Symposium on Forecasting, Quebec, Canada, May 1981.
Erlbaunn, 1977.
9 (l976), (3), June, 6 M 9 .
Science B, 18 (1972), 409-424.
First International Symposium on Forecasting, Quebec, Canada, May 1981.
13 (1980), 139-157.
Change, 16 (l980), 351-361.
forecasting model application to Hawaii tourists’, J . Marketing Research, 12 (1979, 182-188.
clinical inferences’, Psychol. Bull., 73 (l970), 422432.
earnings’, J . Finance, (1 968), March, 67-84.
3 ( 1 984), 197-204.
Essam Mahmoud Accuracy in Forecasting 157
Gray, W. S. III., ‘The role of forecast information in investment decisions’, in Parakash, P. and Rappaport, A., (eds.) Public Reporting of Corporate Financial Forecasts, New York: Commerce Clearing House, 1974,
Green, D. and Segall, J., ‘The predictive power of first quarter earnings reports’, Journal of Business, 40
Gregg, D. P., ‘Business forecasting: the good news and the bad‘, OMEGA the International Journal of
Groff, G. K., ‘Empirical comparison of models for short range forecasting’, Management Science, 20 (1973),
Gross, D. and Ray, J. L., ‘A general purpose forecasting simulator’, Management Science B, 11 (1965), January, 119-135.
Hogarth, R. M., ‘Cognitive processes and the assessment of subjective probability distributions’, J . Amer. Stat. Assoc., 70 (1975), 271-290.
Hogarth, R. M. and Makridakis, S., ‘Forecasting and planning: an evaluation’, Management Science, 27 (1981), (2), February, 115-138.
Hollier, R. H., Khir, M. and Storey, R. R., ‘A comparison of short-term adaptive forecasting methods’, OMEGA the International Journal of Management Science, 9 (1981), (l), 9698.
Horrell, J. F., McLeod, G. and Papadopoulos, P. P., ‘A forecasting tournament revisited’, presented at the First International Symposium on Forecasting, Quebec, Canada, May, 198 1.
Johnson, L. A. and Montgomery, D. C., ‘Forecasting with exponential smoothing and related methods’, in Makridakis, S. and Wheelwright, S. C. (eds), TIMSStudies in the Management Sciences, 12 (1979), 3 1 4 .
Johnston, T. E. and Schmitt, T. G., ‘Effectiveness of earnings per share forecasts’, Financial Management, 3 (1974), Summer, 64-72.
Jorgenson, D., Hunter, J. and Nadiri, M., ‘A comparison of alternative econometric models of quarterly investment behaviour’, Econometrica, 38 (1970), 187-21 3.
Kahneman, D. and Tversky, A., ‘On the psychology of prediction’, Psychological Review, No. 80, (1973),
Kinney, W. R., Jr., ‘ARIMA and regression in analytical review: an empirical test, The Accounting Review, 53
Kirby, R. M., ‘A comparison of short and medium range statistical forecasting methods’, Management
Krampf, R. F., ‘The turning point problem in smoothing models’, Ph.D. Thesis, University of Cincinnati,
Krasker, W. S., ‘Estimation in linear regression models with disparate data points’, Econometrica, 48 (1980),
Larson, L. K., ‘Comparison of double adaptive smoothing and state space’, presented at the First
Levine, A. H., ‘Forecasting techniques’, Management Accounting, (1967), January, 8695. Libby, R., ‘Man versus model of man: some conflicting evidence’, Organiz. Beh. and Human Perf., 16 (1976),
1-12. Liebling, H. I. and Russell, J. M., ‘Forecasting business investment by anticipation surveys and econometric
models in 196849’, 1969 Proceedings of Business and Economic Statistics Section, American Statistical Association, 1969, 250-255.
Lorek, K. S., McDonald, C. L. and Patz, D. H., ‘A comparative examination of management forecasts and Box-Jenkins forecasts of earnings’, Accounting Review, 51 (1976), 321-330.
Mabert, V. A., ‘Forecast modification based upon residual analysis: a case study of check volume estimation’, Decision Sciences, 9 (1978), (2), April, 285-296.
Mahmoud, E., ‘Short-term forecasting: matching techniques to tasks. An integrated framework and empirical investigation’, Ph.D. Thesis, State University of New York at Buffalo, June, 1982.
Mahmoud, E., ‘Gardenfors “I” value: a comment on the measurement of the relative accuracy of sales forecasting models’, Technological Forecasting and Social Change, (1984), in press.
Mahmoud, E. and Goyal, S. K., ‘Loss-cost functions for measuring the accuracy of sales forecasting methods’, Working Paper, Concordia University, Montreal, Quebec H3G 1 M8, 1983.
Makridakis, S., ‘If we cannot forecast, how can we plan? Long Range Planning, (England), 14 (1981), (3), 1c20.
Makridakis, S. and Hibon, M., ‘Accuracy of forecasting: an empirical investigation’, J.R. Statist. SOC., 142 (1979), Part 2, 97-145.
47-80.
(1963, January, 44-45.
Management Science, 8 (1980), (3), 361-374.
(l), 22-31.
237-251.
(1978), January, 4840.
Science, 13 (1966), (4), December, 202-210.
1972.
(6), September, 1333-1346.
International Symposium on Forecasting, Quebec, Canada, May 1981.
158 Journal of Forecasting Vol. 3, Iss. No. 2
Makridakis, S. and Wheelwright, S. C., Interactive Forecasting: Univariate and Multivariate Methods, San Francisco: Holden Day, 1978a.
Makridakis, S. and Wheelwright, S. C., Forecasting: Methods and Application, New York: Wiley, 1978b.
Makridakis, S. and Wheelwright, S. C., ‘Adaptive filtering: an integrated autoregressive/moving average filter for time series forecasting’, Operational Research Quarterly, 28 (1977), 1425-437.
Makridakis, S. and Winkler, R. L., ‘Averages of forecasts: some empirical results’, Management Science, 29 (1983a), 987-996.
Makridakis, S., Wheelwright, S. C. and McGee, V. E. Forecasting: Methods and Applications, 2nd edn, New York: Wiley, 1983b.
Makridakis, S.,Andersen,A.,Carbone, R., Fildes, R., Hibon, M., Lewandowski, R., Newton, J., Panen, E. and Winkler, R., ‘The accuracy of extrapolation (time series) methods: results of a forecasting competition’, Journal of Forecasting, 1 (1982), 11 1-1 53.
Makridakis, S.,Andersen, A.,Carbone, R., Fildes, R., Hibon, M., Lewandowski, R., Newton, J., Parzen, E. and Winkler, R. The Accuracy of Major Extrapolation (Time Series) Methods. London: Wiley, in press (1983~).
McCombrey, C. A. and McKenzie, W. Jr., ‘Forecasting collections using expert opinions in deterministic model’, presented at ORSAITIMS Cotlference, Miami, Fla., November 1976.
McNees, S. K., ‘The forecasting performance in the early 1970s’, New England Econ. Reo., July-August, (1976), 29-40.
Meehl, P. E., ‘Clinical versus statistical prediction’, Journal of Experimental Research in Personality, 1, 27-32.
Narashimhan, G. V. L., ‘A comparison of predictive performance of alternative forecasting techniques: time series models vs. an econometric model’, Proceedings of American Statistical Association, (1979, August, 459-464.
Narashimhan, G. V. L., Castellino, V. F. and Singpurwalla, N. D., ‘On the predictive performance of the BEA quarterly econometric model and a Box-Jenkins type of ARIMA model’, Proceedings of the American Statistical Association, Business and Economic Section, (1974), 560-564.
Naylor, T. H. and Seaks, T. G., ‘Box-Jenkins methods: an alternative to econometric models’, International Statistical Review, 40 (1972), (2), 1 13-1 37.
Nelson, C. A,, ‘Box-Jenkins time series analysis of the quarterly earnings of Standard Oil Company of Indiana’, presented at the First International Symposium on Forecasting, Quebec, Canada, May 1981.
Nelson, C. R., ‘The prediction performance of the FRB-MIT PENN model of the U.S. Economy’, American Economic Review, 62 (1972), December, 902-917.
Nelson, C. R., ‘The first order moving average process: identification, estimation and prediction, Journal of’ Econometrics, 2 (1974), 121-141.
Newbold, P., ‘The exact likelihood function for a mixed autoregressive-moving average process’, Biometrika, 61 (1974), 423-426.
Newbold, P. and Granger, C. W. J., ‘Experience with forecasting univariate time series and the combination of forecasts’, J.R. Statist. SOC., Series A , 137 (1974), 131-165.
Niederhoffer, V. and Regan, D., Summarized in Barron’s Magazine, 18 December, (1972). Pan, J., Nicholas, D. R. and Joy, 0. M., ‘Sales forecasting practices of large US industrial firms’, Financial
Management, 6 (1977), (3), Fall, 72-77. Petto, A. C., ‘Comparing the relative accuracy of seasonal sales forecasting methods-a case study of an
urban hotel’, presented at the First InternationalSymposium on Forecasting, Quebec, Canada, 27-29 May, 1981.
Pindyck, R. S. and Rubinfeld, D. L., Econometric Models and Economic Forecasts, New York: McGraw- Hill, 1976.
Plosser, C. I., ‘Short-term forecasting and seasonal adjustment. J. Amer. Stat. Assoc., 74 (1979), (365), March, 15-23.
Raine, J. E., ‘Self-adaptive forecasting considered’, Decision Science, 2 (1971), (2), 264-279. Reeves, G. R. and Lawrence, K. D., ‘Combining multiple forecasts given multiple objectives’, 1 (1982), (3),
Reid, D. J., ‘A comparative study of time series prediction .techniques on economic data’, Ph.D. Thesis,
Reid, D. J., ‘Forecasting in action: comparison of forecasting techniques in economic time series’,
271-279.
University of Nottingham, 1969.
Proceedings of the Joint Conference of Operations Research Society, 1971.
Essam Mahmoud Accuracy in Forecasting 159
Reid, D. J., ‘A review of short-term projection techniques’, in Gordon, H. A., (ed.), Practical Aspects of
Reinmuth, J . E. and Geurts, M. D., ‘A multideterministic approach to forecasting’, in Makridakis, S. and
Richards, R. M. and Fraser, D. R., ‘The predictability of corporate earnings’, Atlanta Economic Review, 28
Rothe, J . T. ‘Effectiveness of sales forecasting methods’, Industrial Marketing Management, Nethertands, 7
Ruland, W., ‘The accuracy and information content of earnings per share: forecast reports by firms’, Ph.D.
Forecasting, London: Operational Research Society, 1975, 8-25.
Wheelwright, S. C. (eds), TIMS Studies in the Management Sciences, 12 (1979), 203-21 1.
(1978), March-April, 43-47.
(1978), (2), April, 114-1 18.
Thesis, The State University of New York at Buffalo. 1976. Ruland, W., ‘The accuracy offorecasts by management.and by financial analysts’, Accounting Review, 53
(1978). 439-447. Saibin, T . R., ‘Contributions to the study of actuarial and individual methods of prediction’, American
Sartorius, L. C. and Mohn, N. C., ‘Sales forecasting models: a diagnostic approach’, Research Monograph
Sawyer, J., ‘Measurement and prediction, clinical and statistical’, Psychological Bulletin, 66 (1966), 178-200. Schmidt, J. R., ‘Forecasting state retail sales: econometric vs. time series models’, The Annals of Regional
Slovic, P. , ‘Psychological study of human judgement: implications for investment decision making’, Journal
Smyth, D. J., ‘Short-run macroeconomic forecasting: the OECD performance’, Journal of Forecasting, 2
Tversky, A,, ‘Assessing uncertainty (with Discussion)’, J.R. SOC. B, No. 36, (1974), 148-159. Tversky, A. and Kahneman, D., ‘Judgment under uncertainty: heuristics and biases’, Science, 185 (1974),
Wheelwright, S. C. and Makridakis, S . , Forecasting Methods for Management, 3rd edn, New York: Wiley,
Winkler, R. L. and Makridakis, S., ‘The combination of forecasts’, JRSS Series A , (1984), in press. Winters, P. R., ‘Forecasting sales by exponentially weighted moving averages’, Management Science, 6
Zarnowitz, V., ‘The accuracy of individual and group forecasts from business outlook surveys’, Journal of
Zober, M., ‘A comparison of Box-Jenkins methods of forecasting with regression’, presented at the First
Journal of Sociology, 48 (1 943), 593402.
No. 69, Georgia State University, Atlanta, Georgia, 1976.
Science, XI11 (1979), (3), November, 91-101.
of Finance, 27 (1972), 779-799.
(1983), 37-49.
1124-1 13 1.
1980.
(1960), (3). 324342.
Forecasting, 3 (1 984), 10-27.
International Symposium on Forecasting, Quebec, Canada, May 198 1,
Author’s biography: h a m Mahmoud, B.A., MBA, Ph.D., is Assistant Professor of Quantitative Methods at Concordia University. He received his MBA and Ph.D. from the State University of New York at Buffalo (SUNY). He has held teaching and research appointments at the University of Technology at Cairo and at SUNY. His research interests are applied forecasting, forecasting accuracy, opportunity cost as an accuracy measure and the evaluation and selection of forecasting software.
Author’s address: Essam Mahmoud, Concordia University, Quantitative Methods Department, 1455 de Maisonneuve Boulevard West, Montreal, Quebec H3G 1M8, Canada.
