Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
ARROWS EVALUATION
WA.RD 608.1
Research ReportFebruary 2005
^-rrf-utt Waehington StatcDopartmcnt of llansportation
Washington State Transportation CommissionPlanning and Capital Program Managementin cooperation with:U.S. DOT - Federal Highway Administration
Final Report
ARROWS EVALUATION
by
Larry Senn Ed BosellyRWIS Engineer Road Weather Program Manager
AdvancedTechnologyBranch MaintenanceandOperationsWashington State Department of Transportation
Washington State Department of TransportationTransportation Building
Olympia, Washington 98504
Prepared for
Washington State Transportation CommissionDepartment of Transportation
and in cooperation withU.S. Department of Transportation
Federal Highway Administration
February2005
TECHNICAL REPORT STANDARD TITLE PAGE 1. REPORT NO. 2. GOVERNMENT ACCESSION NO. 3. RECIPIENT'S CATALOG NO.
WA-RD 608.1
4. TITLE AND SUBTITLE
ARROWS EVALUATION
7, AUTHOR(S)
Larry Senn and Ed Boselly
9, PERFORMING ORGANIZATION NAME AND ADDRESS
Washington State Transportation Center (TRAC) University of Washington, Box 354802
S. REPORT DATE
February 2005 6. PERFORMING OROANIZI\TION com;
8. PERFORMING ORGANIZATION REPORT NO.
IO. WORK UNIT NO.
11. CONTRACT OR GRANT NO.
University District Building; 1107 NE 45th Street, Suite 535 Seattle, Washington 98105-4631 12. SPONSORING AGENCY NAME AND ADDRESS
Research OfficeWashington State Department of TransportationTransportation Building, MS 47372Olympia, Washington 98504-7372Doug Brodin, Project Manager, 360-705-797215. SUPPLEMENTARY NOTES
13. TYPE OF REPORT AND PERIOD COVERED
Research Report
14, SPONSORING AGENCY CODE
This study was conducted in cooperation with the U.S. Department of Transportation, Federal Highway Administration.
I6. ABSTRACT
In a June 2002 meeting, winter maintenance managers at the Washington State Department of
Transportation (WSDOT) said that more accurate forecasts, that told not only where but when inclement
weather was due, would be a valuable tool to aid them in keeping Washington State highways safe and
passable during winter months. In response, the WSDOT ITS Office teamed with the University of
Washington Department of Atmospheric Sciences to build weather information and forecasting Web
pages specifically tailored to the managers' desires and needs.
The result was the development of the ARROWS (Automated Real-time Road Weather System)
Web pages, made available to all WSDOT winter maintenance managers for the 2003-2004 winter
season. WSDOT subsequently conducted an inhouse survey to determine what ARROWS users thought
about the usefulness and presentation of information. This report presents the results of that survey,
which have been used as input to the continuing development of ARROWS.
17. KEY WORDS
Roadway maintenance, winter maintenance, road weather information systems, weather forecasts
18. DISTRIBUTION STATEMENT
No restrictions. This document is available to the public through the National Technical Information Service, Springfield, VA 22616
19, SECURITY CL ASS!F. (of this report) 20. SECURITY CLASS1P. (of this page) 21. NO, OF PAGES 22. PRICE
None None
DISCLAIMER
The contents of this report reflect the views of the authors, who are responsible
for the facts and the accuracy of the data presented herein. The contents do not
necessarily reflect the official views or policies of the Washington State Department of
Transportation. This report does not constitute a standard, specification, or regulation.
lll
IV
CONTENTS
The ARROWS Web Pages .......
Sources of ARROWS Forecasts
User Acceptance ........
Sources of Weather Information.....
Summary
I
7
8
ARROWS Features Most Liked
ARROWS Features Most Disliked.............
ARROWS Features Needing Change
Rankings of ARROWS Output Parameters
ARROWS' Usefulness and Continuation...
New Features for Development
Additional Comments
10
12
11
13
t4
15
t6
t7
18
V
Figure
F'IGURES
Sources of weather information other than ARROWS..
Features most liked by ARROWS users..,...
Features of ARROWS disliked by users....
Items users want changed in ARROWS...............
Rankings of ARROWS output parameters
Page
11
l2
13
t4
15
t6
2
J
4
5
6
9
J
4
I Sample ARROWS page showing the statewide precipitation andpavement temp erature forecast
2 State view of "Air Temperature/Wind" page.......
State view of Warnings page
Close up of Org.code 425220 showing areas that have warnings......
5 Doppler radar image .....
6 Number of survey responsos by WSDOT region
7 Number of survey rosponses by job title 10
8
9
10
11
t2
13 Overall ARROWS evaluation from user perspective....
V1
ARROWS EVALUATION
In a June 2002 meeting, winter maintenance managers at the Washington State
Department of Transportation (WSDOT) said that more accurate forecasts, that told not
only where but when inclement weather was due, would be a valuable tool to aid them in
keeping Washington State highways safe and passable during winter months. In response,
the WSDOT ITS Office teamed with the University of Washington Department of
Atmospheric Sciences to build weather information and forecasting Web pages
specifically tailored to the managers' desires and needs.
The result was the development of the ARROWS Web pages, made available to
all WSDOT winter maintenance managers for the 2003-2004 winter season. WSDOT
subsequently conducted an in-house survey to determine what ARROWS users thought
about the usefulness and presentation of information. The results of that survey, which
have been used as input to the continuing development of ARROWS, are presented here.
THE WEB PAGES
ARROWS is a Web-based graphical presentation of weather conditions (see
Figure 1). It also provides links to other useful sites to aid winter maintenance managers
in anticipating weather conditions.
I
Alert Bu fm :Washhaton State
Tfr'- Tfi*jg-sg.tg-
nt0LF.&:gjll!ilI0484:la&!nx
:U!A
Lodel Folffil.
SlcTemo/Recio
AirTeftblwind
Warninos
Curst Data
Obs6ryaiions
Red6r
g.t6llits
Camerds
Olhq Ptoviderr
Nll Wsalh6r Svc
NWAC
N\dWaalherNot
Air Ou;litv
Orqco(rt:Im[: avo rit(]s
@lWashinrlton Slat.l
Wrshinr,{or St.rle D0I;A =R =iR
C -.ilt S
I ore{:asloal {}rdl0re ard
lrir,rr r-.n
r:l.i- I',:-: r lL I
Figure L: sample ARRows page showing the statewide precipitation andpavement temperature forecast
Forecasts are provided in 4-hour increments, and three views of each forecast are
available for each period. The preferred view is selected from the left hand menu, while
the time period is selected from the top menu.
The "surface temperature and precipitation" view indicates the forecast pavement
temperature with color coding, in which green indicates forecast temperatures over 37oF,
yellow between 37oF and 33oF, and red under 33'F. Forecast precipitation is also
indicated by color: various shades of green indicate rain and shades of gray indicate
snow. Deeper shades indicate more intense precipitation. Red is also used to show areas
of pos s ible fr eezing r ain.
2
Similar to the first page, the air temperature and wind page (see Figure 2) reports
forecast air temperatures by color: yellow indicates temperatures between 37'F and 33"F
and red indicates temperatures below 33'F. The standard meteorological symbols are
used to indicate forecast wind direction and speed.
Figure 2: State view of "Air TemperatureAVind" page
The "Warnings" screen (see Figure 3) relays statewide weather information, along
with specific warning notices addressed to the local maintenance unit regarding
conditions that may affect its work.
In winter these warnings are for snow, frost, freezing fog, or other weather that
will adversely affect the highways. In the example in Figure 3, the summer warnings are
3
= SW t5mh
eFe:llflaslsese:$3!tls&Io&:{1!Aslsesdr:{1lgsl2eFo&:{1g1g
:WA
:{A@snffftro&:{5110st?04
e[o{.:gillq
l{odel Fo,ec6li
SfcTemo/Precio
Air TemolWind
Warninos
Culetrt Dda
Obee*alions
Badar
Saiellite
Cameras
Othsr Providor*
Ntl \y'eather Svc
NWAC
NWWeathsrNet
Air Qualitv
;A. n :l?1, C -_,1, SWnihinqton Slntc DOT
lorecdsted Air I re and lrvitrd
Orqcodo
[@I dvoritesrfiffidi@t
Washinqton Stdte
l,i r,r r- | r rji:rrrril,r, :trl:FiL I :i'il lj iir rlr, rilr l,r Fnt t :t'L_,a ! r,n r r ,,t1
for high temperatures, which is important for some paving and pesticide application
tasks.
llodel Forecast8
9fc Teroo/Frecio
Air Temotu/ind
Wouq:
Curent Data
ObseNations
Fadar
gatellite
Other Providerr
tllllWedllrer Svc
NV/AC
Nll WealherNet
Figure 3: State view of Warnings page
Users can narrow the display to their immediate area by entering their WSDOT
organrzation code ("org.code"), which identifies a maintenance group responsible for a
particular area (see Figure 4). The use of the WSDOT organization code for navigation
was desired by the attendees of the 2002 meeting. However, this method of navigation
would make the use of ARROWS difficult for non-WSDOT personnel.
4
rUfol?Prd 12P[tIo4FM 4PMIpgl!4 8ffdTo12ru t?STo4An 4StToBStn rfi [r\lr I t. I I.l
Alert Bu for :Washington State
eFo& :{ts!on28e$.&:ltstDsl?5A 28
s90s2sst 181
5tI0rNTEI
ks..e:t@JRITSRT{
sn?6
sR90
sf, 170
snl?l5\262
ftFo& :!A&l
Orgcode
[@favorites
@WashinUlon State
Univor6ily of Woshington
A. =?:ll. c'-'ll s
S3rnrng: lnf rr'Sjhrndr,n itilrAlE i I g DLr li Ati Tn a!9 '1 I 1l Lr[ in Ftl
Gorrgle-
::. r,lr. La.nr,r_
Qe-r' ,*) i1]
Ilmpdew-ptpaE_t
I precip
=o
IauHa
4n
20
AM8nt
Lr&tlg
oa
!i
IPl\,1
8H
1
Texl Forecast
E1
Forecast for I90 MP 169
,1
rlI
Mlltlort Emdor
168.71 11i 1
7 8 S1011121 2 3 4 5 6 7 IAMsfl
Niu Ey W.rdig.Augll mr00 AItlTotustl l2{0 mI
SRMP Wsn$C
!!9 Hj$rq
Rodc
90
1!! Hidrrqj.C.I tliA]qE! Hjdr rq!!9 lsd|rq.110 HiarqU.! IfiA rry
L*rE re:!AagsR Itsf,n3Rmtl t3tnm5n&rR8ete:@5B It$nilsnrtDn3ntil5R ltt
it5520,-!
:lallS
'!l tabular2 Microsoft lnlerlret txptorera,sldtowarningFrane il
Orqcode
Vali.l tor:Io Au( 31 l2::16:J6
llodel Fqccorlr
Slc Temo/Precio
Air Te m olwi n d
Wanihos
Currcrl Dala
ObseNdlions
Radar
Sat6llit6
Cameras
Olher P.ovilerJ
Ntl W€.lh6r Svc
NU/AC
N\tWeatherNet
Figure 4: Close up of Org.code 425220 showing areas that have warnings (red dots)
Clicking on a location brings up a graphical forecast for air temperature, road
temperature, and dewpoint. The forecast is also available in tabular form.
These tools are intended to give maintenance managers a way to anticipate the
occurrence of freezing pavement or air temperatures, snow or rain, frost, and transitions
between rain and snow. These are all considerations for managers deciding on a course
of treatment. Accurate forecasts have the potential to improve the effectiveness of snow
and ice treatments, which will result in a safer roadway while reducing labor and
materials costs. Reducing material usage also reduces the amount of pollutants released
into the environment.
5
Cument observations from WSDOT weather sites are available, as well as a
composite weather doppler radar showing the entire state (see Figure 5). The composite
image is generated at the UW Department of Atmospheric Sciences. A loop of radar
images covering the last 3 hours is also available, allowing users to see the track of a
storm front or precipitation as it moves across the state.
Figure 5: Doppler radar image
In addition to the forecasts, ARROWS provides useful links to outside sources for
additional information. A link to the local National Weather Service (NWS) gives direct
access to several satellite images from visible light, infrared, and water vapor images.
Several of these products also can be viewed as a loop showing movement over the last
6
l?dMTr4PV 4FliTogPM sFtolrol?ru8tstsAlert Bu for :Washington State
lN*lr:al.-sEeieheaJlblhYEd
hr,ii'iriiit]upruiuiriorus
lPmurlmnt
lN;;Ri.lil*]meucxre roumrr 8:00 AM
A:55 AM
a:OO AM
S:55 AM
8:63 AM
8:{ AM
Si35 AM
8:OO AM
.a:OO AM
S:OO AM i
!,oo eu i
.8:4
A:31
8:15
.8:00
8:00
lpaurqor
lerr"!
lB.inbidebr.nd
]t*je!ruii pms
irt"n&"
Hodel Forecasls
Slc Temo/Precio
ArrTemoAVind
Warninos
Current Ddld
Observaiions
E{d-al
Satellile
Camerds
Olher ProYiders
Nll Wealher Svc
NWAC
!\qld!!ltE4lc!
Air Oualiv
Ws!hington Stato DOT;A=R=?CWS
Currcnt (}bieryatr.'n. I n.
Washrnalon Stak:Washinqton State
Favorites
@
Orqcode
il@
few hours. Other links connect to the Northwest Avalanche Center and to weather
consultants under contract with WSDOT.
SOURCES OF' ARROWS FORECASTS
ARROWS is driven by a group, or ensemble, of MM5 (Mesoscale Model 5)
forecasts, which are run twice a day. Seventeen different forecast runs make up the
ensemble. Each forecast uses initial and boundary conditions from a large-scale global
model. Eight of the forecasts use different large-scale models for initial conditions; one
forecast uses the average of the eight initial conditions (called the "centroid"); and eight
other forecasts use the same eight initial conditions but each.with a different set of
physical parameters within in the MM5 model.
There is a trade-off between running a high resolution MM5 forecast and running
many different forecasts in an ensemble at lower resolution. Insufficient computer power
was available to do both, so the team chose to use the ensemble for ARROWS. This
ensemble of forecasts has several advantages over using one high resolution model. First
of all, it can continue running even if one or more of the global models fails to provide
initial conditions on a given day. Second, the ensemble gives the ability to identify how
likely a forecast is to be accurate. If all the ensemble members predict the same thing,
that forecast is more likely to be true than if the ensemble forecasts vary widely. The
measure of how close the ensemble members are to one another is called "spread." A
high spread indicates low confidence in the forecast, and a low spread equates to greater
confidence. While this assumption does not always prove correct, there is a definite
correlation between spread and forecast skill.
7
USER ACCEPTAI\CE
In general, field personnel reported that they like ARROWS, want continued
access to ARROWS, and would like the development process to continue.
As people began to use ARROWS they identified a few deficiencies in the data
displays. The researchers at the University of Washington quickly installed fixes. These
included adding road detail and geographic references to the map displays for ease in
location registration, adding aradar data intensity scale, and adding date time groups to
surface observation tables.
A training course was developed to help potential users understand ARROWS,
how it was created, and how to use it. The training was conducted in group sessions or as
a hands-on session in a maintenance facility. Although the course was originally an
informal training session, it is now a formal course in WSDOT's training system
ARROWS performed well during the first portion of the 2003-2004 winter. It
correctly forecast an early snowfall in the Cascades Mountains in September. In late
October, ARROWS also correctly forecast a no-snow situation in the Puget Sound area,
which became the basis for a television interview with a WSDOT highway maintenance
crew. The crew touted the benefits of having an accurate forecast to avoid unnecessary
personnel callouts and associated costs. Shortly thereafter the Road Weather Program
Manager issued a mild alert to ARROWS users to not expect the same sort of success all
of the time and reminded them that ARROWS continues to be only one of the "tools in
the toolbox."
Unfortunately, in early January 2004 ARROWS suffered the same problems that
all other meteorological models faced at that time. They are all unable to predict the
8
location and influence of a layer of cold air near the surface. Consequently, the models
did not forecast the extended freezing rain and subsequent ice storms that occurred in
southwest Washington the week of January 5th. Similar problems arose on the east side
of the mountains during mid-January. As a result of these problems, some users reverted
to their previous methods of gathering or obtaining weather information. Of these, a
limited number never returned to using ARROWS.
At the end of the winter season, the Road Weather Program Manager sent a
questionnaire to ARROWS users in all of WSDOT's regions to determine whether
ARROWS was meeting their needs. The rate of return was somewhat underwhelming;
with approximately 150 surveys distributed, 38 responses were returned. Figure 6 shows
the number of responses by region (Northwest, Olympic, Southwest, North Central,
Eastem, and South Central). Figure 7 shows that the largest number of responses came
from supervisors and technicians. Other respondents included superintendents and lead
technicians.
Number of Responses
1B
16
14
12
10B
642
0
NW Oly SW NC E scRegion
Figure 6: Number of survey responses by WSDOT region
9
Number of Responses
14
12
10
I6
4
2
0
Supt Supv Lead Tech
Job Title
Tech Other
Figure 7: Number of survey responses by job title
Below are some of the results from the respondents. Note that all of the extremely
negative comments came from two individuals located at the same maintenance facility.
Sources of Weather Information
Figure 8 shows the various sources of weather information used by the
respondents, including the Weather Channel, the University of Washington Department
of Atmospheric Sciences web site (http://www.atmos.washington.eduldatal), National
Weather Service (NWS), Northwest Weathemet, newspapers, television newscasts, DTN,
radio, NOAA Weather Radio and others. The graph highlights the apparent importance
of television weathercasters, which ranked higher than the NWS in terms of use.
10
25
Wx Chan U Dub NWS
ARROWS Survey Responses
Newspaper TV
Other Weather Souroces
DTN Lmd Radio Wx Radio
20
15
opEI
10
0
Other
Figure 8: Sources of weather information other than ARROWS.
ARROWS Features Most Liked
Figure 9 shows the ARROWS features that users said they liked. The most
popular features were the main distinctive features of ARROWS: the road maps, weather
warnings, and pavement temperature information. Most maintenance people understand
the importance of pavement temperature for their winter maintenance activities. [n
conversations, this was the key forecast parameter. Other features identified by users
were the forecast graphs, icons, ease of use, links, ease of access, and "other," the
composite weather radar images.
11
ARROWS Evaluation
Road maps Fcst Grph lcons Ease Use Warnings Links Access Pvmnt T Other
Figure 9: Features most liked by ARROWS users
ARROWS Features Most Disliked
Figure 10, on the other hand, shows items or features disliked by users. The one
item identified most, albeit by only five respondents, was "ease of use." In talking with
users, most of the people who found ARROWS difficult to use did not have ARROWS
training. The forecast graphs presented some problems because users didn't always
understand the graph process. Training has helped users understand that text forecasts
are available. The two individuals who selected "Links" wanted to see additional links.
The "Other" category was a catch-all for comments such as "not always up," "inaccurate"
(2), "don't like the Vo confidence going to zero with the precipitation forecast,"
"reliability," "too hard to decipher," and "not as good as TV/intemet."
35
30
25
o
oEo
E tcEtz
10
T2
7
iooE
z
4
0
ARROWS Survey Responses
Ease Use Warnings
Lsst Liked
Access Pvmnt T Other
3
Road maps Fcst Grph lcons Llnks
tr'igure 1.0: Features of ARROWS disliked by users
ARROWS Features Needine Chanse
Figure 11 shows what people said they would like to see changed in ARROWS.
The most requested change was to increase the forecast length. Currently ARROWS
forecasts are in six 4-hour time blocks. Most users wanted a minimum of 36 hours; some
at least 48 hours. The issue of map sizes also generated discussion. Some users said they
would like to see the map views larger on the screen. The screens appear to provide
some unused space that could be used for larger maps. Also, users said they wanted the
ability to click on a map to zoom in or out rather than enteringorganization codes to
navigate.
A few others commented on adding or deleting links, changing the wamings, and
pavement temperature. The "Other" category comments included "increase the
13
reliability" [accuracy] (2), "need forecasts out to seven days," "make more user friendly,"
and "make at least as good as TV Web sites"
ARROWS Survey Responses
b
5
4
3
2
2.I5z
0
Add llnks Delete llnks Map slzes Road maps Fcst lenglh Wamlngs Pvmnl T Otherlcons
Want Changed
Figure 11: Items users want changed in ARROWS
Rankines of ARROWS Output Parameters
Figure 12 presents the rankings for various ARROWS output parameters,
including pavement and air temperatures; precipitation amount and time; wind speed and
direction; and snow, frost, and fueezingfog warnings. Not all respondents ranked each
parameter. Most features were subjectively rated 6'OK" or better. Few users rated
parameters "Very Bad." The most dissatisfaction was indicated for snow warnings.
Much of that dissatisfaction derived from users' experiences in the first week of January,
when the computer models had difficulty dealing with cold air trapped near the Earth's
surface west of the Cascade Mountains. Given some computer model adjustments that
t4
have been made since last winter, this forecast feature should improve, and satisfaction
should increase.
Few comments were provided regarding the output parameters. In the "Other"
category, the two Very Bad and the ten Very Good responses were accompanied by no
comments. One verbal comment was, "Freezingrain good."
ARROWS Survey Responses
45
40
35
930oEo
frruEob20oEtz 'tB
lVery Good
lGoodloktr Bad
lVery Bad
10
0
""dis^a.(\'
e" .-""-a.c'
_d.a
e\e-C .u..-
"."." ..J -."" do"'^s
"'-'Parameter Ralings
Figure 12: Rankings of ARROWS output parameters
ARROWS' Usefulness and Conti{ruation
One of the most important components of the evaluation was a question about
whether people were better able to do their jobs because of ARROWS. Figure 13 shows
the results: 2l said "yes" and 9 said "no." A similar question asked whether WSDOT
15
should continue with development of ARROWS. In a resoundingly positive reply, 30
said "yes" and 3 said "no" (see Figure l3).
ARROWS Evaluallon
40
35
30
6
EruEoec20o
3E5i5z
10
5
0
tNoI Yes
Work belter? Contlnue?
Overall Evaluatlon
New Features for Development
Users were also asked for input on what they would like to see developed in
ARROWS. Verbatim responses folloW:
a ARROWS + 5 days, in6-12 hr incr. OK
Waming notification capability, esp when below freezing
Better warnings of freezelsnow events
Snow Techs need info in a hurry, not searching thru graphics ... they can call
NW WeatherNet (a local meteorological forecasting company)
Figure 13: Overall ARROWS evaluation from user perspective
a
o
a
t6
More forecast links, more site estimates
Add more specific sites
a More sites and make it more current
a Links to cameras.
Additional Comments
Respondents were also was asked to provide additional comments. Below is a list
of the comments as written:
o Need RWIS sensors to work
o Not easy to use, not fast enuf
r Little opportunity to use 'cuz of connectivity probs
o Comments based on limited time
o Temps OK early, but from mid Dec on was off, showing snowing when not
o Little or no use, inaccurate, questionnaire just as bad
o As long as ARROWS is another tool and not a tool for scheduling/committing
resources, we'll be fine
o I get better forecasts from TV Wx sites
1 vory useful ... please continue to provide and improve
. only access 3 times. Don't think it was any more useful than the local forecast
o Pavement temp are not very accurate
o Other sources proved much more accurate
o Good tool to use. Only as good as raw data
o Need better data to improve the end product.
t7
Some of the comments showed an understanding of the process; some did not. Some of
the issues mentioned are being resolved. For example, maintenance shed connectivity is
being improved. Also, more computers are being acquired to give more people access.
SUMMARY
One issue buried within the evaluation and the success of ARROWS is training.
Many of the comments about user friendliness related to a lack of proper training. A
formal course now exists. The hope is that more maintenance locations will request the
training and that the training will be conducted at a level below Maintenance Area so that
hands-on training will become the norm
It is apparent from the questionnaire responses, and from conversations with
users, that the initial ARROWS test proved positive and that development and
improvement should continue.
18