Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
1
Validation of a driving simulator study on driver behaviour at passive 1
rail level crossings 2
Grégoire S. Larue ab
*, Christian Wullems ac
, Michelle Sheldrake a, Andry 3
Rakotonirainy a 4
aCentre for Accident Research and Road Safety - Queensland, Queensland University of 5
Technology (QUT), Brisbane, Australia; bAustralasian Centre for Rail Innovation, 6
Canberra, Australia; cCRC for Rail Innovation, Brisbane, Australia 7
*corresponding author: 8
130 Victoria Park road, Kelvin Grove 4059, Australia 10
+61 7 3138 4644 11
12
Abstract 13
Objective: The behavioural validation of an advanced driving simulator for its use 14
in evaluating passive level crossing countermeasures was performed for stopping 15
compliance and speed profile. 16
Background: despite the fact that most research on emerging interventions for 17
improving level crossing safety is conducted in driving simulator, no study has validated 18
the use of simulator for this type of research. 19
Method: We monitored driver behaviour at a selected passive level crossing in the 20
Brisbane region in Australia for three months (N=916). The level crossing was then 21
replicated in an advanced driving simulator, and we familiarised participant drivers 22
(N=54) with traversing this crossing, characterised by low road and rail traffic. 23
Results: We established relative validity for the stopping compliance and the 24
approach speed. 25
Conclusion: This validation study suggests that driving simulators are an 26
appropriate tool to study the effects of interventions at passive level crossing with low 27
road and rail traffic, which are prone to reduced compliance due to familiarity. 28
2
Application: This study also provides support for the findings of previous driving 29
simulator studies conducted to evaluate compliance and approach speeds of passive level 30
crossings. 31
32
Précis 33
Railway level crossings with stop signs require interventions to reduce road users’ 34
complacency, which can result in catastrophic crashes with trains. This study validated 35
the use of driver simulators to evaluate changes in driver behaviour when designing 36
countermeasures for such crossings. 37
38
Keywords: driving simulation; validation; railway crossing; on-road study 39
Introduction 40
Driving simulators are valuable tools in road safety and human factors research and 41
have been used to assess a variety of driving performance (Mullen, Charlton, Devlin, & 42
Bédard, 2011), as they provide a safe environment for investigating driver behaviour 43
and new interventions ethically and effectively, both in terms of cost and time. It is 44
recognised however, that a simulator will never provide an accurate reproduction of 45
reality and each simulator produces specific compromises of a real-life situation (Espié, 46
Gauriat, & Duraz, 2005). In particular, participants may not drive as they normally 47
would in a simulator for reasons such as perceiving the driving task as a game, feeling 48
motion sickness and finding the driving task not realistic. Therefore, the validation of a 49
simulator is an important component of any research study and is required for individual 50
simulators (Godley, Triggs, & Fildes, 2002). Without validation, findings from 51
simulator studies may not translate to real roads, and this could result in limited funds 52
being wasted on ineffective interventions, or worse on interventions increasing risks on 53
the road. It has been argued that simulator validity is dependent on the particular 54
simulator due to differences in equipment (low, medium, high fidelity) (Hoskins & El-55
Gindy, 2006; Nilsson, 1993), the specific driving task and the realism of its 56
3
implementation (Kaptein, Theeuwes, & Van Der Horst, 1996), participant 57
characteristics (Blana, 1996), the variables selected to measure performance, and the 58
data collection measures (Blana, 1996; Kaptein et al., 1996). These issues do have 59
implications for the generalisability of findings from one simulator to another, and even 60
from one simulator task to another. Godley et al. (2002, p. 590) suggest that 61
“accumulated evidence from different driving simulators and a range of driving tasks 62
does add weight to the validity of simulator research”. 63
An area of road safety research where simulators are now being frequently used 64
is in the assessment of driver behaviour at railway level crossings. However, to date we 65
are not aware of any published simulator validation studies in this context. 66
Background 67
Use of simulators for rail level crossing research 68
Railway level crossings represent a complex road safety control environment, where the 69
potential for road vehicle collisions that can result in multiple fatalities means they carry 70
high social and economic costs, making them a priority for safety authorities. Level 71
crossings are divided into crossings with (i) passive and (ii) active controls (Standards 72
Australia, 2015). Passive crossings are equipped with a stop or give way sign, and 73
require road users to ensure no trains are approaching before traversing the crossing. 74
Active crossings have flashing lights and sometimes boom gates, and are activated by 75
the approach of a train. Road users are not allowed to enter the level crossing while 76
flashing lights are activated. 77
Analysis of crash history data adjusted for road and rail traffic indicates that passive 78
crossings are the least safe with the greatest proportion of fatal crashes occurring at such 79
crossings (Independent Transport Safety Regulator, 2011), human factors being the 80
4
major cause of these collisions (Australian Transport Safety Bureau, 2002; European 81
Level Crossing, 2011). Despite this, the majority of Australian level crossings (80%) are 82
equipped with passive controls, mainly due to their large number (~20,000 in Australia), 83
and the cost associated with upgrading passive crossings to active protection. 84
Studies that assess changes to driver behaviour in response to a new or modified 85
level crossing warning system, can pose significant safety risks if conducted at a live 86
level crossing. For this reason, researchers often use simulation to assess driver 87
response to the proposed road user interface. Several researchers have conducted studies 88
of rail level crossings using driving simulators to assess and compare driver response 89
with different warning systems including: traffic lights versus boom gate barriers 90
(Rudin-Brown, Lenné, Edquist, & Navarro, 2012); stop signs, flashing lights and traffic 91
lights (Lenné, Rudin-Brown, Navarro, Edquist, & Trotter, 2011; Tey, Wallis, Ferreira, 92
& Tavassoli Hojati, 2011); a range of different Intelligent Transport Systems (ITS) 93
(Kim, Larue, Ferreira, Tavassoli, & Rakotonirainy, 2013; Larue, Kim, Rakotonirainy, 94
Haworth, & Ferreira, 2015); and a number of innovative warning systems (Tey, Wallis, 95
Cloete, Ferreira, & Zhu, 2012). 96
The study by Tey, Wallis, et al. (2011) is the only one to include both a field 97
study using video observations at a range of level crossings, and a replication study in a 98
driving simulator. However, they did not take advantage of the data they collected to 99
provide a quantitative validation of their simulator study, and only focused on a high 100
level discussion of the high degree of consistency between their field and simulator 101
results: high compliance for active crossings, and low compliance for passive crossings. 102
Rudin-Brown et al. (2012) assessed driver response to different types of safety signals 103
at level crossings using an advanced simulator previously validated against real-world 104
driving measures (Godley et al., 2002), but point out it is not known if and how driver 105
5
responses to simulated level crossings might differ from their responses in the real 106
world. 107
This study aims therefore to investigate the validity of advanced driving 108
simulators to assess driver behaviour, particularly compliance, at railway level crossings 109
with passive controls. 110
Establishing simulator validity 111
Two types of validity have been identified as components of simulator validation: 112
physical and behavioural (Blaauw, 1982; Blana, 1996; Jamson, 1999; Rouzikhah, King, 113
& Rakotonirainy, 2010), also described as validation of vehicle response or driver 114
response (Hoskins & El-Gindy, 2006). Physical or vehicle response validity refers to the 115
extent to which the physical components of the simulator vehicle such as the layout, 116
visual displays, and dynamics (e.g., feel of braking and steering) correspond to an on-117
road vehicle (Blaauw, 1982; Hoskins & El-Gindy, 2006; Mullen et al., 2011). 118
Behavioural or driver response validity, also known as external validity (Kaptein et al., 119
1996; Reimer, D’Ambrosio, Coughlin, Kafrissen, & Biederman, 2006), refers to the 120
degree of correspondence between driving behaviours in the simulator and on real roads 121
(Blaauw, 1982; Jamson, 1999; Mullen et al., 2011). Blaauw (1982) states that 122
behavioural validity can be established in one of two ways: either through absolute 123
validity where the numerical values from the two environments are the same; or through 124
relative validity where the differences between the two environments are in the same 125
direction, and of the same or similar size (Godley et al., 2002). 126
The majority of validated measures to date only show relative validity however; 127
Törnros (1998) argues that relative validity is satisfactory when using a driving 128
simulator as a research tool, since research questions deal with matters relating to 129
effects of various independent variables. Overall it seems to be accepted that simulators 130
6
offer a safe alternative to on-road studies and most studies support the use of simulators 131
given that they approximate (relative validity) but not exactly replicate (absolute driving 132
behaviour) on-road driving behaviour. However, validation of individual simulators for 133
each specific driving task or scenario is also important. 134
Method 135
Field study 136
Observation site 137
An observation site was selected with the following criteria: the level crossing was 138
selected with a stop sign, low road and rail traffic (250 road vehicles and 27 trains per 139
day) and with long sighting distances (~3 km) to favour familiarity and complacency. 140
Data were collected for 3 months; during that time, less than ten vehicles would have 141
approached the crossing at the same time as a train. 142
Protocol 143
Four sets of pneumatic tubes were installed at the selected site (see Figure 1). Three 144
were installed prior to the crossing (North) and one after the crossing (South), as only 145
the southbound traffic was considered in this study. Prior to the crossings, tubed were 146
installed 150 meters, 40 meters and 20 meters in advance of the crossing. The south set 147
of tubes was installed 20 meters after the crossing. The set of tubes 150 meters to the 148
crossing was installed to increase the reliability of the classification of vehicles, as well 149
as estimating the cruising speed of vehicles on the road before approaching the crossing. 150
151
152
153
154
Measuring point to classify vehicles Measuring points for speed profile and compliance
7
155
156
Figure 1: Aerial view of the Lanefield level crossing site (adapted from GoogleMaps) 157
Dependent measures 158
Speed profile 159
Speed was measured with the pneumatic tubes 150, 40 and 20 meters away from the rail 160
track. 161
Stopping compliance 162
Speed measurements from the tubes were used to estimate the stopping 163
compliance at the crossing following the methodology developed by Larue and 164
Wullems (2017). A vehicle is considered to have not complied with the stop sign at the 165
crossing if its speed did not reach zero at that point. To evaluate this incompliance, we 166
compared the time taken by the vehicle to travel between the last set of tubes before the 167
crossing and the first set of tubes after the crossing, to a critical time . The 168
critical time, computed for each vehicle, is the minimum time necessary to travel 169
between these two points at the measured speeds, while stopping at the crossing. This 170
critical time is modelled and evaluated using linear motion equations (for more details 171
see Larue and Wullems (2017)) in conditions of a typical stopping brake to come to a 172
stop (decelerating at 3.4m/s2), followed by the typical acceleration that a car is capable 173
of (2.5m/s2) as in the American Association of State Highway and Transportation 174
Officials (2011). Despite the simplicity of the approach and the strong assumptions on 175
the constancy of acceleration/deceleration, compliance rates as computed with this 176
approach have been shown to be largely insensitive to small variations of the critical 177
time (Larue & Wullems, 2017). 178
8
Driving simulator study 179
Experiment design 180
During the driving simulator session, participants drove a 6km long route six times 181
(referred to as one scenario). Each scenario took five minutes to complete. The route 182
was a typical rural Australian road with three level crossings with passive controls, and 183
two intersections (one with a stop sign, one with a give-way sign). The first level 184
crossing encountered was the replication of the observation site, at the exception of the 185
T intersection. 186
During their first drive, participants encountered a train at one of the three level 187
crossings, the selected level crossing being counterbalanced between participants. This 188
drive was used to ensure that participants would be aware that trains could travel 189
through the replicated level crossings. All five following drives did not include any 190
trains, resulting in a very low probability to encounter a train (5% chance), in line with 191
the field trial. This allowed complacency to develop without any short term increased 192
attention at level crossings shortly after seeing a train, and in a consistent manner 193
between participants. 194
Instrument 195
An advanced driving simulator composed of a complete Holden Commodore vehicle 196
with automatic transmission and working controls and instruments was used in this 197
study (see Figure 2). The simulator used SCANeRstudio software version 1.4 from 198
Oktal with eight computers, projectors and a six degree of freedom (6DOF) motion 199
platform that can move and rotate in three dimensions. When seated in the simulator 200
vehicle, the driver is immersed in a virtual environment which includes a 180 degree 201
front field of view. 202
9
203
Figure 2 Advanced driving simulator used in this study 204
Geo-specific modelling in the simulator 205
The road and environment were developed to respect Australian Standards at rail level 206
crossings. The observation site was replicated in the simulator in terms of road/rail 207
geometry and visual environment, with exception of the T intersection south of the 208
crossing. While the T intersection was removed, a set of trees on the side of the road, a 209
curve and a hill in the far distance were added in the simulated environment. This was 210
used to reduce the visibility in the distance, as the actual site had a line of trees visible at 211
the intersection (see Figure 3), and hence limit any potential influence of this change in 212
the environment on the results. The road geometry was recreated by using a map of the 213
site when the road network was created in the simulator. The road had no grade at this 214
site, except for a small grade at the level crossing (less than half a meter) and hence 215
road elevation was not replicated. Important features of the observation site were 216
replicated in the driving simulator: number and size of lanes (two lanes of 3.25 meters 217
each), road width (7.10 meters), road markings, level crossing signage, and other type 218
10
of objects in the environment (trees, few farms, fences, bush). Figure 3 shows the 219
observation site and the site as it was replicated in the simulator. 220
221
222
Figure 3 Observation site and simulated geo-specific model 223
Participants 224
Sixty participants with a valid driving license and at least one year driving experience 225
were recruited for this study. They were recruited from the general public in the 226
Brisbane region of Queensland, Australia (location of the driving simulator). Power 227
calculation demonstrated that this sample size was required to attain a power of .9 at 228
level alpha .05 with medium size effects .25 with a correlation among repeated 229
measures of .5. 230
Assuming medium effect sizes for speeds (d=.5), a .80 power and .05 type I error, and a 231
ratio of 20 between the field and simulator samples, the required sample size was 232
calculated to be 60 for the simulator group and 1,000 for the field group. 233
Six participants were not able to complete the study due to motion sickness, resulting in 234
a final sample size of 54 participants (35 males, 19 females) with a mean age of 33 ± 11 235
years (range: 19-62). Each participant was paid AUD$100 for running the experiment 236
11
(they received this incentive at the end of a second driving session which was beyond 237
the scope of this study). 238
This research complied with the American Psychological Association Code of Ethics 239
and was approved by the Ethics Committee at the Queensland University for 240
Technology QUT. Informed consent was obtained from each participant. 241
Procedure 242
Upon arrival, participants read information about the study and provided their consent 243
to participate in the study. They were then introduced to the simulator and drove a 244
familiarisation drive. 245
Once the familiarisation phase was complete, participants were instructed to 246
drive to a given house as they normally would, located six kilometres away, 247
representing an everyday life scenario. Participants drove this route six times. The only 248
difference between the scenarios was a train was approaching the crossing as the driver 249
approached the level crossing in the first scenario. Data were analysed for the last 250
scenario only, once participants were familiarised with approaching level crossings with 251
low train traffic. 252
Dependent measures 253
Speed profile 254
The approach speed was extracted at the same three points of interest as the ones used 255
in the on-road study. 256
Stopping compliance 257
The driving behaviour at the crossing was divided in three categories: drivers who 258
stopped at the crossing, drivers who slowed down but did not stop completely, and 259
12
drivers who did not attempt to stop. Compliant drivers were the ones who stopped at the 260
crossing. 261
Driver behaviour was also statistically modelled using only data that were available in 262
the field data study in order to have an objective validation of the behaviour observed in 263
the simulator: speed at the tube locations, time taken to travel between tubes, and 264
modelled critical time. 265
Validation analysis 266
The validation analysis started with evaluating whether the simulator study resulted in 267
complacency. This was done by assessing whether speed increased the more 268
participants approached level crossings. The validation analysis included then a 269
validation of the stopping behaviour at the level crossing as well as a validation of the 270
approach speed. 271
Statistical analyses were performed on the driving simulator data to identify factors 272
which could predict the behaviour of the driver at the crossing. One factor was 273
identified as predicting the different types of behaviour observed in the simulator. The 274
distribution of this factor was modelled using both the simulator and field data, and 275
allowed to calibrate the model for the field study. Behaviour rates were then compared 276
using Generalised Linear Models (GLM). 277
The statistical analyses performed to compare the speed between the road and simulated 278
environment are Generalised Linear Mixed Models (GLMM) from the Gaussian family 279
to take into account the repeated measures (at the different points of interest). 280
All statistical analyses were performed with the R software environment 281
(version 3.4.1). 282
13
Results 283
The field data collection resulted in the observation of 1,016 car traversals of the level 284
crossing. Out of these traversals, 100 occurred during periods of the day with reduced 285
luminosity (after dusk and before dawn), and were removed from the dataset. This 286
filtering was done since the simulated environment only recreated day-time driving 287
conditions. This resulted in a sample size of 916 level crossing traversals at the 288
Lanefield site. 289
Approach speeds 290
The approach speed 150 meters away from the crossing was 76.0 km/h 291
(sd=11.5) in the simulator for participants who stopped at the crossing, and 73.4 km/h 292
(sd=11.6) for drivers who did not attempt to stop (see Figure 4). On the road, these 293
speeds were 62.8 km/h (sd=11.3) and 71.3 km/h respectively (sd=9.9). 294
In the approach section where drivers decelerate to stop, the speed for drivers 40 295
meters to the crossing was 50.5 km/h (sd=11.9) in the simulator, and 39.4 km/h (sd=6.8) 296
in the field when drivers were about to stop or almost stop. Twenty 20 meters to the 297
crossing, these speeds were 40.4 km/h (sd=11.5) and 30.4 km/h (sd=6.1) respectively. 298
For drivers who did not stop at the crossing, the speed 40 meters to the crossing was 299
57.8 km/h (sd=15.8) in the simulator, and 50.7 km/h (sd=9.2) in the field. Twenty 20 300
meters to the crossing, these speeds were 58.6 km/h (sd=16.3) and 48.8 km/h (sd=12.9) 301
respectively. For complying drivers, the resulting effect size was 1.17, 1.55 and 1.53 at 302
150, 40 and 20 meters respectively. The effect size was .21, .73 and .75 at these 303
distances in the case of drivers not stopping at the crossing. Large effects were found, 304
except for the approach speed 150 meters to the crossing for uncompliant drivers, for 305
which the effect size was small. 306
14
In the driving simulation, twenty meters to the crossing, drivers who did not 307
comply with the stop sign drove at 45.1 km/h (sd=21.1) for their first run without trains, 308
and at 57.3 km/h (sd=17.1) during their last drive. A Generalised Linear Mixed Model 309
(GLMM) was fitted and it showed that they drove 28.2 km/h faster than drivers who 310
stopped at level crossings (t=9.0, df=268, p<0.001). It also showed that their speed 311
increased by 2.0 km/h for each successive drive (t=3.0, df=1935, p=0.003), resulting in 312
an overall increase of 8 km/h between the second drive (first drive without any train), 313
and the last drive, showing a development of complacency. This increase in speed was 314
not found for participants who complied at the level crossing. 315
316
Figure 4: Comparison of speed measured on site by pneumatic tubes and in the driving simulator during the 317 approach of the crossing (150, 40 and 20 meters) with standard deviations 318
319
A GLMM was fitted to evaluate the effect of the environment, the speed 320
measurement location, stopping behaviour and their interaction on speed. All factors 321
had a statistically significant effect on speed, directly or indirectly through interactions 322
(see Table 1). 323
Before slowing down for the level crossing (150 meters to the crossing), speed 324
was 71.4 km/h. Getting closer to the crossing, speed reduced by 20.5 km/h 40 meters to 325
15
the crossing (t=-57.8, df=1935, p<0.001), and by a further 22.1 km/h 20 meters to the 326
crossing (t=-62.4, df=1935, p<0.001). For drivers who stopped or almost stopped at the 327
crossing, their speed was 8.4 km/h slower than drivers who did not attempt to stop (t=-328
12.9, df=1935, p<0.001), but it was 5.5 km/h higher in the simulator (t=4.5, df=1935, 329
p<0.001). Compared to field data, speed of complying participants was 5.0 km/h higher 330
(+11%) in the simulator 40 meters to the crossing (t=2.5, df=1935, p<0.001), and 1.8 331
km/h higher (4%) 20 meters to the crossing (t=3.8, df=1935, p<0.001). 332
16
333
Table 1: Statistical model of distance and environment on speed during the approach of the crossing (150, 40 and 334 20 meters) 335
Factor/Interaction Speed
(km/h)
Standard
error
df t-value p-value
Intercept 71.4 .43 1935 164.6 p<.001
Stopping at level crossing
or almost stopping (comply) -8.4 .65
1935 -12.9 p<.001
40 meters -20.5 .35 1935
-57.8 p<.001
20 meters -22.1 .35 1935
-62.4 p<.001
Simulator:20 meters 8.9 2.74 1935
3.2 0.001
Simulator:Comply 13.9 3.1 1935
4.5 p<.001
Comply:40meters -2.9 .53 1935
-5.4 p<.001
Comply:20meters -10.2 .53 1935
-19.3 p<.001
Simulator:40meters:Comply -22.5 2.75 1935
-8.2 p<.001
Simulator:20meters:Comply -41.1 2.75 1935
-14.9 p<.001
Stopping compliance 336
In the driving simulator study, 57.4% of drivers completely stopped at the crossing, 337
20.4% almost stopped, and 22.2% did not attempt to stop to proceed through the 338
crossing (see Table 2). 339
Using linear motion equations, we found that drivers who traversed the crossing in a 340
shorter amount of time than the critical time were the drivers who did not stop at the 341
crossing. For the remaining participants, a GLMM showed that the difference between 342
the time to traverse the crossing and the critical time (referred to as amount of time over 343
17
the critical time in the reminder of the paper) was statistically related to completely 344
stopping at the crossing or not (t=4.99, df=42, p<0.001), while the following other 345
factors did not: speed before or after the crossing, deceleration before the crossing or 346
acceleration after the crossing. This difference was apparent in the distribution of the 347
amount of time over the critical time, characterised by two distinct peaks. This 348
distribution was modelled as a Gaussian mixture model using the EM algorithm. We 349
found that drivers who completely stopped at the crossing represented 76% of such 350
drivers, and took on average 7.3 seconds longer than the critical time to traverse the 351
crossing (sd=2.3), while drivers who only slowed down at the crossing took 2.4 seconds 352
more than the critical time (sd=1.4). 353
We applied this approach to the data collected in the field, and we looked at the 354
distribution of the time over the critical time for drivers who could not have stopped at 355
the crossing. We found that this distribution also had two distinct peaks and we fitted a 356
Gaussian mixture model, showing that 68% of drivers (drivers completely stopping) 357
took 5.2 seconds longer than the critical time to proceed through the crossing (sd=2.3), 358
while the remaining 32% (drivers almost stopping) took only 1.3 seconds longer than 359
the critical time (sd=1.1), resulting in the behaviour rates for the field study presented in 360
Table 2. 361
GLM shows that there was a statistically significant difference between the road and the 362
simulation rates of drivers who did not attempt to stop at the crossing (z=-4.42,p <.001), 363
with field drivers being 2.5 times more likely to traverse the crossing without 364
attempting to stop than drivers in the simulated environment. A GLM also showed that 365
there was no statistically significant difference between the road and simulator studies 366
once the drivers who did not attempt to stop were not considered (z=.01,p =.992).367
18
368
Table 2: Behaviour rates at the level crossing 369
Behaviour Rate
Environment Percentage 95% confidence interval
Simulator Stopped 57.4 ±14.8
Almost stopped 20.4 ±12.1
Did not attempt to
stop
22.2 ±11.1
Road Stopped 30.0 ±4.2
Almost stopped 14.4 ±3.2
Did not attempt to
stop
55.6 ±3.2
Discussion 370
Validity of simulator for passive level crossing behaviour evaluation 371
The aim of the current study was to validate the use of advanced driving simulators for 372
research on rail level crossings with passive controls. This validation focused on 373
approach speeds and stopping compliance. Relative validation was considered the most 374
important for validating the results of past railway crossing experiments on the 375
simulator (Kim et al., 2013; Larue et al., 2015; Larue, Rakotonirainy, & Haworth, 2016; 376
Larue, Wullems, & Naweed, 2016; Lenné et al., 2011; Tey, Wallis, Cloete, & Ferreira, 377
2013), as well as future experiments of new interventions. 378
Speed was different for each measured location, which shows that the selected 379
points of interest accurately represent the zone where drivers slow down to stop at level 380
19
crossings, as shown by previous research (Luoma & Poutanen, 2011; Moon & Coleman, 381
1999; Ng & Saccomanno, 2010; Ward & Wilde, 1996). Speeds measured in the braking 382
zone of the level crossing were statistically significantly different and absolute validity 383
could not be found in this study. However, the approach speed profile, demonstrated 384
relative validity with speed changes replicated in the simulator at similar locations as 385
on-road this study, following the same decreasing trend. Speeds measured in the 386
simulator were however consistently higher than on the real road, consistent with other 387
research on speed behaviour in driving simulators (Bella, 2005; Bittner, Simsek, 388
Levison, & Campbell, 2002; Blaauw, 1982; Blana & Golias, 2002; Godley et al., 2002; 389
Klee, Bauer, Radwan, & Al-Deek, 1999; Törnros, 1998), which is one of the most 390
commonly studied measures of behavioural validity of driving simulators. 391
We designed the study to induce complacency by repeatedly driving through a 392
level crossing with a low chance to encounter a train, and it resulted in complacency, as 393
shown by the significant proportion of participants in the simulated drive who did not 394
attempt to stop at the crossing, and their increasing approach speeds the more they were 395
traversing the crossing, showing a deliberate decision to proceed through the crossing 396
disregarding the controls in place. This behaviour is in line with both the literature and 397
the observations conducted in the present road study: the observation study by Tey, 398
Ferreira, and Wallace (2011) shows that 59% of drivers did not comply at a passive 399
level crossing with a stop sign, with 41% of drivers only slowing down, while 18% did 400
not even slow down. Compliance was more pronounced at the level crossing selected 401
for this study, as indicated by Queensland Rail, which monitored this particular crossing 402
and reported that more than half of the drivers did not stop at that particular crossing. 403
This was also observed by the research team on the multiple occasions they visited the 404
site. This is in line with the literature, which shows that drivers tend to underestimate 405
20
speeds of large objects such as trains (Clark, Perrone, & Isler, 2013; Larue et al., 2018), 406
and report that they would enter level crossings even when train are very close to level 407
crossings when sighting distances are very long (Larue et al., 2018), as at the trial site 408
considered in this study. 409
While we selected participants in Brisbane city, located 50 km to the city of 410
Ipswich where the field site was selected, the proportion of drivers who did behave this 411
way was almost half in the simulated when compared to the road study, suggesting that 412
it is difficult to obtain absolute validity. This could be due to the fact that participants in 413
a driving simulator study know that their behaviour is observed. It could also be due to 414
the fact that the sample of participants used in the study was not representative of the 415
population of drivers at the actual site: users of the local level crossing are likely to be 416
regular users of the crossing, given the low traffic at that crossing and the fact that the 417
road was going to a specific location only. However this study highlighted that the 418
simulator was very effective at inducing the non-compliance characterised by not 419
completely stopping at the crossing, suggesting that a significant proportion of 420
participants became ‘regular drivers’ of the replicated level crossing. This behaviour 421
results in drivers being more likely to assess the situation at the crossing in conditions 422
that do not give them much margin of action in case something they do not expect 423
occurs (such as the arrival of a train). 424
Our results are consistent with the only other experiment which attempted to 425
validate compliance at level crossings: Tey, Wallis, et al. (2011)’s study also resulted in 426
relative validity and managed to recreate in the simulated environment the non-427
compliance characterised by not attempting to stop. Similar to our experiment, this 428
behaviour was two times more likely on the road than in the simulator. That study was 429
however unable to recreate the difference between drivers who stopped completely, and 430
21
the participants who only slowed down: most participants in their simulator study 431
completely stopped, while their field observations showed the same proportions for both 432
behaviours. Multiple reasons could play part in this result, including the lack of 433
replication of an actual level crossing in the simulator and the lower fidelity of the 434
simulator used (static simulator with one screen), or the lack of repetition of driving 435
through the level crossing to familiarise the participants with the crossing and trigger 436
complacent behaviour. 437
Our study has implications for the evaluation of interventions in driving 438
simulators. We have shown that the complacent driver behaviour at passive level 439
crossing can be recreated in the simulator, whether this behaviour is deliberate (driving 440
through without slowing down), or more as an error, failing to completely stop. This 441
shows that a range of driver behaviour issues can be evaluated using simulators, from 442
errors to actual violations. Deliberate violations are however less likely in a simulator as 443
compared to an actual level crossing, and this should be considered when designing the 444
sample size of a simulator study. 445
Strengths and limitations 446
The present study used a unique validation process for the use of driving 447
simulators to evaluate driver behaviour at passive level crossings through the replication 448
of a level crossing and naturalistic data collection in the field. This approach 449
complements the literature on the validity of simulators to a new driving task, and 450
provides supporting evidence to the research conducted in simulator for level crossings 451
interventions. A strength of this study is the use of a large sample size of participants in 452
the simulator study (N=54). Further, we used a methodology (pneumatic tubes) to 453
observe behaviour in the field which is unlikely to have had an effect on driver 454
compliance (Larue & Wullems, 2017), as this technology cannot be used for 455
22
enforcement and is largely perceived as a traffic monitoring device. This allowed us to 456
have an estimation of on-road compliance over a long period of time. 457
There were, however, some limitations of the present study that should be 458
considered when interpreting the results. First, we could not obtain demographics 459
information about the drivers in the field study. This does not allow confirming whether 460
our sample of simulator participants is representative of the drivers using that particular 461
level crossing. However, we used a sample of participants representing the general 462
driving population both in terms of gender and age. Further, participants in the 463
simulator were accustomed to the level crossing by driving multiple times through it, 464
resulting in complacency and driving behaviour similar to the one observed from 465
regular level crossing users. This study was also unable to validate the behaviour of 466
participants as a train approaches a level crossing. However, errors and violations when 467
trains are approaching are related to the complacency at level crossings (Edquist, 468
Stephan, & Wigglesworth, 2009), and therefore validating a simulator when no train is 469
approaching can be sufficient to determine whether interventions are likely to improve 470
behaviour at passive level crossings. The estimations of on-road compliance and speed 471
profile was also coarse with the approach used in this study. However, we collected 472
speed data at locations known to be where speed changes, and our results are in line 473
with other field studies at level crossings, with speed reductions around 50 meters to the 474
crossing (Luoma & Poutanen, 2011; Moon & Coleman, 1999; Ng & Saccomanno, 475
2010; Ward & Wilde, 1996). The estimation of compliance with pneumatic tubes, while 476
relying on a number of coarse assumptions, is still robust to variations of timing 477
measurements, as shown by sensitivity analysis (Larue & Wullems, 2017). 478
23
Future directions 479
This study has shown that simulators can be effective at inducing complacency, with a 480
significant number of participants disregarding controls in place. Further research is 481
needed to understand how the likelihood of seeing a train at level crossing impacts 482
complacency in the field, and how this translates in simulated driving. Future work 483
should also investigate how complacency develops when repeating the same action 484
repeatedly, and what amount of repetition is required in the simulator to result in a 485
similar behaviour, given that simulation is conducted over a much shorter period of time 486
compared to habituation in the field. Qualitative research is also warranted in order to 487
further understand the mental models of drivers both in the field and in the simulator, 488
which may explain partly the differences observed between field and simulated driving 489
in this study. 490
Conclusion 491
This study provides evidence that compliance and approach speeds obtained from an 492
advanced driving simulator are valid measures of driver behaviour at passive level 493
crossings. Such measures were found to be similar to the ones measured in-situ at the 494
level crossing replicated in the simulated environment. This validation study suggests 495
that driving through replicated level crossings with low chance of encounter with a train 496
in an advanced driving simulator is an adapted approach to study the effects of 497
interventions at level crossing with passive controls with low road and rail traffic, which 498
are prone to reduced compliance due to familiarity. This study also provides good 499
support for the previous studies conducted in this field with driving simulators. 500
Key points 501
Most research on emerging interventions for improving level crossing safety is 502
conducted in driving simulator 503
24
No study has validated the use of simulator for this type of research 504
We measured speed and compliance at a level crossing and its replica in a simulator 505
Our study shows relative validity for approach speed and compliance and validates the 506
use of simulators for passive level crossings 507
Acknowledgements 508
The authors gratefully acknowledge the CRC for Rail Innovation (established and supported 509
under the Australian Government’s Cooperative Research Centres program) and Queensland 510
Rail for the funding of this research. Project No. R2.195. 511
References 512
American Association of State Highway and Transportation Officials. (2011). A Policy 513
on Geometric Design of Highways and Streets (6th ed.) (6th ed.). Washington 514
D.C.: American Association of State Highway and Transportation Officials. 515
Australian Transport Safety Bureau. (2002). Monograph 10. level crossing accidents. 516
ACT: Commonwealth Department of Transport and Regional Services. 517
Bella, F. (2005). Validation of a Driving Simulator for Work Zone Design. 518
Transportation Research Record: Journal of the Transportation Research 519
Board, 1937, 136-144. doi:10.3141/1937-19 520
Bittner, A., Simsek, O., Levison, W., & Campbell, J. (2002). On-Road Versus 521
Simulator Data in Driver Model Development Driver Performance Model 522
Experience. Transportation Research Record: Journal of the Transportation 523
Research Board, 1803, 38-44. doi:10.3141/1803-06 524
Blaauw, G. J. (1982). Driving experience and task demands in simulator and 525
instrumented car: a validation study. Human Factors, 24(4), 473-486. 526
Blana, E. (1996). Driving Simulator Validation Studies: A Literature Review. Working 527
Paper. Institute of Transport Studies, University of Leeds, Leeds, UK. 528
Blana, E., & Golias, J. (2002). Differences between Vehicle Lateral Displacement on 529
the Road and in a Fixed-Base Simulator. Human Factors, 44(2), 303-313. 530
doi:10.1518/0018720024497899 531
Clark, H. E., Perrone, J. A., & Isler, R. B. (2013). An illusory size–speed bias and 532
railway crossing collisions. Accident Analysis & Prevention, 55(0), 226-231. 533
doi:http://dx.doi.org/10.1016/j.aap.2013.02.037 534
Edquist, J., Stephan, K., & Wigglesworth, L. M. (2009). A literature review of human 535
factors safety issues at Australian level crossings. Retrieved from Melbourne: 536
Espié, S., Gauriat, P., & Duraz, M. (2005). Driving simulators validation: The issue of 537
transferability of results acquired on simulator. Paper presented at the Driving 538
Simulator Conference, Orlando, US. 539
European Level Crossing, F. (2011). Background Information on Level Crossings. 540
About Level Crossings, 2011(28th April 2011). 541
Godley, S. T., Triggs, T. J., & Fildes, B. N. (2002). Driving simulator validation for 542
speed research. Accident Analysis & Prevention, 34(5), 589-600. 543
Hoskins, A. H., & El-Gindy, M. (2006). Technical report: Literature survey on driving 544
simulator validation studies. International Journal of Heavy Vehicle Systems 545
(IJHVS), 13(3). doi:10.1504/IJHVS.2006.010020 546
25
Independent Transport Safety Regulator. (2011). Transport Safety Bulletin: Level 547
crossing accidents in Australia. 548
Jamson, H. (1999). Curve negotiation in the Leeds driving simulator : a validation 549
study. Engineering psychology and cognitive ergonomics; 3; 351-358, 3. 550
Kaptein, N., Theeuwes, J., & Van Der Horst, R. (1996). Driving simulator validity: 551
Some considerations. Transportation Research Record: Journal of the 552
Transportation Research Board, 1550, 30-36. doi:10.3141/1550-05 553
Kim, I., Larue, G. S., Ferreira, L., Tavassoli, A., & Rakotonirainy, A. (2013, 2 - 4 554
October 2013). Evaluating ITS interventions at railway level crossings using a 555
driving simulator. Paper presented at the Australasian Transport Research 556
Forum, Brisbane, Australia. 557
Klee, H., Bauer, C., Radwan, E., & Al-Deek, H. (1999). Preliminary Validation of 558
Driving Simulator Based on Forward Speed. Transportation Research Record: 559
Journal of the Transportation Research Board, 1689, 33-39. 560
Larue, G. S., Filtness, A. J., Wood, J. M., Demmel, S., Watling, C. N., Naweed, A., & 561
Rakotonirainy, A. (2018). Is it safe to cross? Identification of trains and their 562
approach speed at level crossings. Safety Science, 103, 33-42. 563
doi:https://doi.org/10.1016/j.ssci.2017.11.009 564
Larue, G. S., Kim, I., Rakotonirainy, A., Haworth, N. L., & Ferreira, L. (2015). Driver’s 565
behavioural changes with new intelligent transport system interventions at 566
railway level crossings—A driving simulator study. Accident Analysis and 567
Prevention, 81, 74-85. doi:10.1016/j.aap.2015.04.026 568
Larue, G. S., Rakotonirainy, A., & Haworth, N. L. (2016). A simulator evaluation of 569
effects of assistive technologies on driver cognitive load at railway level 570
crossings. Journal of Transportation Safety & Security, 8(Supplement 1), 56-69. 571
doi:10.1080/19439962.2015.1055413 572
Larue, G. S., & Wullems, C. (2017). A new method for evaluating driver behaviour and 573
interventions for passive railway level crossings with pneumatic tubes. Journal 574
of Transportation Safety & Security, 0-0. doi:10.1080/19439962.2017.1365316 575
Larue, G. S., Wullems, C., & Naweed, A. (2016). Evaluation of a new level crossing 576
warning concept to improve safety of level crossings in remote locations. Paper 577
presented at the 11th World Congress on Railway Research, Milan, Italy. 578
http://eprints.qut.edu.au/95767/ 579
Lenné, M. G., Rudin-Brown, C. M., Navarro, J., Edquist, J., & Trotter, M. (2011). 580
Driver Behaviour at Rail Level Crossings: Responses to Flashing Lights, Traffic 581
signals and Stop Signs in Simulated Rural Driving. Applied Ergonomics, 42, 582
548-554. 583
Luoma, J., & Poutanen, M. (2011). How Drivers Understand Safe Behaviour and 584
Perceive Risks at Passive Railway-Road Level Crossings. The Open 585
Transportation Journal, 5, 88-91. doi:10.2174/1874447801105010088 586
Moon, Y., & Coleman, F. (1999). Driver's Speed Reduction Behavior at Highway-Rail 587
Intersections. Transportation Research Record: Journal of the Transportation 588
Research Board, 1692(1), 94-105. doi:10.3141/1692-11 589
Mullen, N., Charlton, J., Devlin, A., & Bédard, M. (2011). Simulator Validity 590
Handbook of Driving Simulation for Engineering, Medicine, and Psychology: 591
CRC Press. 592
Ng, O. K., & Saccomanno, F. F. (2010). Speed Reduction Profiles Affecting Vehicle 593
Interactions at Level Crossings with No Trains. Transportation Research 594
Record: Journal of the Transportation Research Board, 2149(1), 108-114. 595
doi:10.3141/2149-13 596
26
Nilsson, L. (1993). Behavioural research in an advanced driving simulator-experiences 597
of the VTI system. Paper presented at the Human Factors and Ergonomics 598
Society Annual Meeting, Seattle, US. 599
Reimer, B., D’Ambrosio, L. A., Coughlin, J. F., Kafrissen, M. E., & Biederman, J. 600
(2006). Using self-reported data to assess the validity of driving simulation data. 601
Behavior Research Methods, 38(2), 314-324. doi:10.3758/bf03192783 602
Rouzikhah, H., King, M., & Rakotonirainy, A. (2010). The validity of simulators in 603
studying driving behaviours. Paper presented at the 2010 Australasian Road 604
Safety Research, Policing and Education Conference, Canberra. 605
http://eprints.qut.edu.au/38011/ 606
Rudin-Brown, C. M., Lenné, M. G., Edquist, J., & Navarro, J. (2012). Effectiveness of 607
traffic light vs. boom barrier controls at road–rail level crossings: A simulator 608
study. Accident Analysis & Prevention, 45, 187-194. 609
doi:http://dx.doi.org/10.1016/j.aap.2011.06.019 610
Standards Australia. (2015). Manual of Uniform Traffic Control Devices, Part 7: 611
Railway Crossings. (AS 1742.7-2015). Sydney, Australia: Standards Australia. 612
Tey, L.-S., Ferreira, L., & Wallace, A. (2011). Measuring driver responses at railway 613
level crossings. Accident Analysis & Prevention, 43(6), 2134-2141. 614
Tey, L.-S., Wallis, G., Cloete, S., & Ferreira, L. (2013). Modelling driver behaviour 615
towards innovative warning devices at railway level crossings. Accident 616
Analysis & Prevention, 51(0), 104-111. 617
doi:http://dx.doi.org/10.1016/j.aap.2012.11.002 618
Tey, L.-S., Wallis, G., Cloete, S., Ferreira, L., & Zhu, S. (2012). Evaluating Driver 619
Behavior Toward Innovative Warning Devices at Railway Level Crossings 620
Using a Driving Simulator. Journal of Transportation Safety & Security, 5(2), 621
118-130. doi:10.1080/19439962.2012.731028 622
Tey, L.-S., Wallis, G., Ferreira, L., & Tavassoli Hojati, A. (2011, 28-30 September 623
2011). Using a driving simulator to assess driver compliance at railway level 624
crossings. Paper presented at the 34th Australasian Transport Research Forum 625
2011, Adelaide, SA, Australia. 626
Törnros, J. (1998). Driving behaviour in a real and a simulated road tunnel—a 627
validation study. Accident Analysis & Prevention, 30(4), 497-503. 628
doi:http://dx.doi.org/10.1016/S0001-4575(97)00099-7 629
Ward, N. J., & Wilde, G. J. S. (1996). Driver approach behaviour at an unprotected 630
railway crossing before and after enhancement of lateral sight distances: An 631
experimental investigation of a risk perception and behavioural compensation 632
hypothesis. Safety Science, 22(1-3), 63-75. doi:10.1016/0925-7535(96)00006-9 633
634
Biographies 635
Gregoire S. Larue, senior research fellow at CARRS-Q, PhD in mathematics from QUT 636
(Australia) obtaind in 2010 637
Christian Wullems, research fellow at CARRS-Q, PhD in computer science from QUT 638
(Australia) obtained in 2005 639
Michelle Sheldrake, senior research officer at CARRS-Q, MSc in psychology from 640
University of Otago in 1992
641
27
Andry Rakotonirainy, professor at CARRS-Q, PhD in computer science obtained from 642
Universite de Paris VI (France) in 1995 643