Driver Lane Change Intention Recognition by Using Entropy-Based Fusion

Techniques and Support Vector Machine Learning Strategy

A Thesis Presented

by

Xianyi Huang

to

The Department of Mechanical and Industrial Engineering

In partial fulfillment of the requirements

for the degree of

Master of Science

in

Computer Systems Engineering

in the field of

Engineering Software Design

Northeastern University

Boston, Massachusetts

December 2012

i

ABSTRACT

In this Thesis, we focus on the analysis of driver lane-changing behavior based on

the fact that lane changing is a ubiquitous driving maneuver in common driving

environments and regarded as the most critical driving intention. Therefore, lane

changing as a case study for driving intention recognition is introduced in this study. Our

methodology is to employ machine learning method i.e., support vector machine, to the

classification of driving intentions using vehicle performance data and driver eye gaze

data from the measurement of the driving tasks (i.e., lane following and lane changing) in

the well-designed simulation environment. To improve recognition performance, this

thesis illustrates the use of entropy based fusion method to discard those largely negative

dependent data to decrease the redundancy level of input data. Based on entropy

correlation coefficient analysis, heading angle, as one kind of performance data, is

harmful for recognition result. Incorporation of eye gaze data enhances the recognition

performance. Introduced by three-stage nested design, experiments are executed to screen

out the best “supplies” for pattern learning. Final results show that feature set fused by

steering angle, gas pressure, velocity and acceleration performance data as well as eye

gaze data achieves 88.78% accuracy with a time length of 0.6 seconds at 5% false alarm

level.

ii

Table of Contents ABSTRACT .......................................................................................................................................... i

Table Index ...................................................................................................................................... iv

Figure Index ..................................................................................................................................... iv

1. INTRODUCTION ....................................................................................................................... 1

1.1 Motivation ............................................................................................................................ 1

1.2 Research Questions ............................................................................................................. 3

1.3 Thesis Outline ...................................................................................................................... 4

2. RESEARCH APPROACHES AND RELATED WORKS .................................................................... 5

2.1 Driving Intention ................................................................................................................. 5

2.1.1 Overview ......................................................................................................................... 5

2.1.2 Research on Lane Change Intent .................................................................................... 7

2.2 Information Fusion ............................................................................................................ 10

2.2.1 Overview ....................................................................................................................... 10

2.2.2 Fusion Method ............................................................................................................. 12

2.2.3 Summary ....................................................................................................................... 13

3. METHODOLOGY ..................................................................................................................... 15

3.1 Pattern Learning by Support Vector Machine ................................................................. 15

3.1.1 Overview of SVM .......................................................................................................... 15

3.1.2 Toolkit for SVM ............................................................................................................. 17

3.2 Uncertainty Reduction by Entropy ................................................................................... 17

3.2.1 Overview of Entropy ..................................................................................................... 17

3.2.2 Uncertainty Reduction.................................................................................................. 19

4. EXPERIMENTS AND RESULTS ................................................................................................. 22

4.1 Experiment Design ............................................................................................................ 22

4.1.1 Definition of Lane Change ............................................................................................ 22

4.1.2 Experiment Set-Up ....................................................................................................... 23

4.1.3 Experiment Data ........................................................................................................... 24

4.2 Three-Stage Nested Design ............................................................................................... 27

4.2.1 Data Fusion/Combination (C) ....................................................................................... 27

4.2.2 Data Representation (S) ............................................................................................... 29

4.2.3 Time Window (T) .......................................................................................................... 31

iii

4.3 Results Evaluation ............................................................................................................. 32

4.3.1 Comparison of fused multiple feature and single feature ........................................... 35

4.3.2 Comparison of data combination (C) ........................................................................... 35

4.3.3 Comparison of time window (T) ................................................................................... 36

4.3.4 Comparison of data structure (S) ................................................................................. 37

5. CONCLUSIONS AND DISCUSSIONS ......................................................................................... 41

5.1 Conclusions ........................................................................................................................ 41

5.2 Discussions ......................................................................................................................... 43

Appendix ........................................................................................................................................ 44

Reference ...................................................................................................................................... 51

ACKNOWLEDGEMENTS ................................................................................................................. 55

iv

Table Index Table 1 Samples of eye gaze and performance data ......................................................... 26

Table 2 Entropy correlation coefficients ........................................................................... 29 Table 3 Summary of AUC for each model after training ................................................. 34

Figure Index Figure 1 Driving interactions .............................................................................................. 5 Figure 2 A canonical architecture for information fusion ................................................ 11 Figure 3 Fusion level ........................................................................................................ 12

Figure 4 An example of two categories of points separated by a hyperplane .................. 16 Figure 5 Two normal distribution illustrating the higher value of entropy for broader

distribution ........................................................................................................................ 18 Figure 6 Sketch map of lane changing process ................................................................. 23

Figure 7 Experiment set-up ............................................................................................... 24 Figure 8 A three-stage nested design ................................................................................ 27

Figure 9 Two data representation structure ...................................................................... 30 Figure 10 Sample labeling ................................................................................................ 31 Figure 11 ROC curves of different combinations ............................................................. 32

Figure 12 Comparison of combined features and single feature under ROC ................... 35 Figure 13 Comparison of data combination...................................................................... 37

Figure 14 Comparison of eye gaze effect on performance ............................................... 38 Figure 15 Comparison of time window under non-overlapping data structure (S1) ........ 38

Figure 16 Comparison of time window under overlapping data structure (S2) ............... 39 Figure 17 Comparison of data representation under the same time window (T1) ............ 39

Figure 18 Comparison of data representation under the same time window (T2) ............ 40 Figure 19 Comparison of true positive rate of three-stage nested design at 5% false

positive rate ....................................................................................................................... 40

1

1. INTRODUCTION

1.1 Motivation

A recent statistical report released by the U.S. National Highway Traffic Safety

Administration (NHTSA) shows that although the fatality rate of traffic accidents in 2009

declines to the lowest on record since 1954, there are still more than 33,000 people died

in vehicle crashes, while in 2005 this figure even climbs to 43,510 [NHTSA, 2010].

Annually, traffic accidents result in approximately 1.2 million fatalities and over 50

million serious injuries worldwide released by World Health Organization [WHO, 2004].

In addition, according to NHTSA and the U.S. Department of Transportation, more than

40 percent of U.S. vehicle-related fatalities are associated with rear-end, roadway

departure and lane-change/merge, and the economic costs of vehicle crashes in the

United States results in hundreds of billions of dollars loss each year [Bljncoe et al.,

2002]. Accidents often arise from misbehavior of one or several drivers when inducing a

driver maneuver. From the analysis of research institution of Benz, the reason of traffic

accidents can be classified into five columns: Road departure (account for 19% of all

traffic accidents), lane change and merge (account for 4%), intersection (account for

29%), rear end (account for 26%), and other reasons account for about 22% [An et al.,

2006]. Today, the solution to reduce accidents caused by driver misbehavior is to

introduce laws to persuade drivers to obey traffic rules by force, e.g., that prohibit using

cell phone and buckle up seat belt while driving. However, these restrictions cannot

fundamentally prevent drivers from performing secondary tasks, e.g., that using GPS for

locating destination route, using iPod for listening to music or even chatting with

2

passengers. With the help of sensor technology and intelligent vehicle engineering, an

alternative to instituting additional traffic laws is the development of intelligent driver

assistance system (IDAS), which has the potential to reduce crashes, especially when the

driver cannot concentrate on driving or the intended maneuver is not properly adjusted to

the current traffic situation. Any well-designed IDAS has the ability to gather information

about a driver/vehicle and predict the driver intention at the right time with the right kind

of assistance, and then allow neighboring drivers and/or pedestrians to obtain timely pre-

warning.

Systematic efforts to understand and characterize driver behavior within the task

of driving are essential in the development of IDAS. In this Thesis, we focus on the

analysis of driver lane-changing behavior based on the fact that lane changing is a

ubiquitous driving maneuver in common driving environments and regarded as the most

critical driving intention. Therefore, lane changing as a case study for driving intention

recognition is introduced in this study. By doing so, we hope to provide the basis for

appropriate design of lane change warning or collision avoidance system, as well as a

foundation for IDAS in general. Our methodology for achieving this goal is to employ

machine learning methods in the classification of driving intentions using vehicle-related

performance data and driver eye gaze data from the measurement of the driving tasks (i.e.,

lane following and lane changing) in the well-designed simulation environment. Thus,

once the resultant relationship is built up, we can derive the intention of changing lane

based on the recognition of current driving pattern using the knowledge learnt from

previous patterns.

3

1.2 Research Questions

Theoretically, from driver’s viewpoint, IDAS assists driver in avoiding collision

by making drivers aware of impeding hazards from crossing pedestrians or adjacent

vehicles. As an important subsystem of IDAS, lane charge warming system extends the

ability of the driver by monitoring the surrounding vehicles and allows the driver to react

as needed before making a lane change. However, for any such system to be effective, i.e.,

at the right time with the right kind of assistance, it is imperative that IDAS knows what

the driver is doing or trying to do, in other words, driver’s intention. A consequent

problem is that driving intention is an internally cognitive mental state, difficult to be

directly measured. People will ask, in the end, how to read driver’s intended action?

Undoubtedly, robustness or reliability is the key concern when people (i.e., driver)

are willing to adopt IDAS. Robustness is of paramount importance, however, the

robustness of IDAS is not easy to achieve, given e.g., the noise generated by limited

experimental conditions, the error of the instrument itself, improper operations, as well as

the effects from surrounding natural environment, etc. How best to improve the

robustness of driver intention recognition, which is the second question people have to

ask.

Seeking answers to the above questions has long been an endeavor of researchers

yet is still a challenging task. Although driver intention is hard to be perceived, driver’s

driving patterns leave us some clues. By identifying and learning someone’s driving

pattern, we can build the model of the current situation relevant for driver current intent.

From this model, and by using the patterns of driving history, we may predict driver

4

intended maneuver. Now back to the second question, if we consider driving as a whole

system, not a single isolated event, the system encompasses three major parts (i.e., driver,

vehicle and road) and their mutual relations. If we can obtain the relevant information

about these three parts and clearly integrate the interdependent relationship between them,

this will help to improve the robustness of recognition system.

1.3 Thesis Outline

The primary goal of this research is to find answers to the above mentioned

questions. In this thesis, we formulate the problems as the pattern classification problems,

with emphasis on the lane change intention recognition.

The rest of the thesis is organized as follows. Existing studies on driving intention

and fusion methods are summarized in Section 2. The description of entropy-based fusion

method and support vector machine learning method is introduced in Section 3. Details of

experimental design and data analysis for lane change are done in Section 4. Section 5

shows conclusion and discussion.

5

2. RESEARCH APPROACHES AND RELATED WORKS

2.1 Driving Intention

2.1.1 Overview

For many humans, driving is a simple activity, perceived as a natural extension of

their ordinary capabilities, but driving is a complex decision-making process as a result of

multidimensional construct of the relationships between the major entities involved in

driving (i.e., environment, vehicle and driver) and the dynamic nature of these entities.

We understand that driving situation consists of three entities, vehicle, driver and

environment. It is useful to view the driving from the viewpoint of interactions among the

three entities [Park et al., 2002] as shown in Figure 1.

Figure 1 Driving interactions

The driver-vehicle interaction includes driver’s action to the vehicle and the

vehicle’s reaction to the driver. It includes steering wheel operation, pedal shifting, etc. as

driver’s action, and heading angle, speed meter etc. as the reaction of the vehicle.

The driver-environment interaction includes the influence of environment on

driver and the driver’s reaction to the surroundings. Influence of surroundings includes

weather change (rainy, cloudy, sunny, and foggy), day time/night time, etc. The driver’s

6

reaction includes attention distraction, action of controlling instrument (close/open

sunroof, turn on/off headlights), etc.

The vehicle-environment interaction includes influence of environment on the

vehicle and the vehicle’s reaction. Influence of environment includes traffic road

conditions (one-way/two-way, road curvature/slope, etc), and adjacent vehicle distance.

The vehicle’s reaction includes acceleration/deceleration, lane change, etc.

Driver’s driving intention refers to what the driver wants to do in the next time

with respect to the current time. On the other hand, driving intention is an internally

cognitive state and is hardly to be directly measured, but it can be inferred through

indirect ways. Salient features of intention are summarized as follows [Geddes, 1989]: (1)

Intention is not directly observable; (2) Intention may be inferred by a series of

observations; (3) Intention is associated with both a desired result (e.g., lane change) and

a means of achieving that result (e.g., a sequence of actions); (4) Intention has a

beginning and ending condition; (5) individual differences have an influence on intent

formation.

7

2.1.2 Research on Lane Change Intent

2.1.2.1 Mathematical Model Based Approach

Regardless of the motivation (overtaking a slower car, taking the correct position

for the next turn, avoiding an obstacle, etc.), the behavior of a vehicle that does a “lane

changing” consists of moving from one lane to another in a few seconds, and with the

constraints of the vehicles in the target and source lane. This definition opens a door to

apply mathematical models to describe driver’s intention, along with a variety of

computational models. For instance, normal lane change maneuvers can be modeled as a

sin function of time for lateral acceleration in Ref. [Chovan et al., 1994].

A widely used mathematical model in judging whether a lane change take place in

reality is gap acceptance model [Ahmed, et al., 1996]. That is, once a driver has chosen a

target lane to change to, he/she may evaluate if the vehicle under control has the place to

perform the change, normally computing the gap between the ahead and behind vehicles

in the target lane; that change is performed only if the gap is greater than a minimum gap.

Though the gap acceptance model is simple in theory, the pitfall of this model is that the

minimum gap can hardly be fixed due to the effects of many factors, e.g., the skill level

of driver, the age of driver, the type of vehicle, the road traffic, etc.

Another well known indicator of lane changing/keeping is the time to line

crossing (TLC), which is defined as the time duration before the vehicle cross the lane

boundary assuming a constant motion of the vehicle [Mammar, et al., 2006].

Unfortunately, exact real-time TLC computation is not an easy task due to several

limitations concerning the prior knowledge of both vehicle trajectory and the lane

8

geometry. Besides this, another major restriction factor is the complexity of its

computation in real-time.

2.1.2.2 Driver-Vehicle-Environment Oriented Approach

Modeling driver’s actions in terms of interactions with the vehicle and with the

traffic environment has been exploited by many researchers and a lot of research work

has been published. Ref. in [Fazio, et al., 1990] constructed a desired speed change lanes

(SCL) model to predict the driver behavior of freeway exiting by applying steering

control length, the deceleration in gear length, and the braking distance. Authors in Ref.

[Pentland & Liu, 1999] reported that a number of subjects drove through the simulated

world in which they had to execute a vehicle maneuver as a result of the text command

presented on a simulator screen and the human driving behavior was modeled as a

Markov chain of control states, but their recognizer offered only discrete recognition. Ref.

in [Kuge, et al., 2000] reported the similar experiment with a continuous recognition

system, but their system only used steering-based features without considering the

environment information. Another similar research was from Drexel University, where

they proposed a framework of “mind tracking architecture” implemented in ACT-R

cognitive architecture [Salvucci, 2004], which simulated a set of possible driver intention

and their resulting behaviors. Their system can detect a driver’s intention to change lanes,

achieving an accuracy of 85% with a false alarm rate of 4%.

Many vehicle-related performance technologies have been reported in the

literature for detecting the behavior of the driver by monitoring the transportation

9

hardware systems under the control of the driver. Previous research [Salvucci, 2004] has

concentrated on the analysis of driver’s steering input, and confirmed that steering wheel

angle can be a valid predictor. Driving behavior is a multidimensional construct, which

means that no single driving performance measure will capture all the effects, varieties of

data are combined to provide potentially more accurate in inference according to

information fusion [Coue, et al., 2002]. Apart from steering data, other vehicle-related

performance data such as speed of the vehicle [Wei, et al., 2000], acceleration [Salvucci

& Liu, 2002], gas pedal pressure [McCall, et al., 2005] also have some indicative power

about the intention to change lanes.

2.1.2.3 Visual Cue Based Approach

In the study of driving, researchers have extensively utilized eye movements or

gaze, as a window into how driver execute driving tasks. It was found that specific eye

glance patterns did take place before lane change initiation, for example, prior to making

a lane change to the left, the likelihood is highest of glancing at the forward view (the

probability of 1.0), followed by a glance to the left mirror (0.52) and so the rear view

mirror (0.52) [Tijerina et al., 1997]. Also, there had been several recent studies that

include eye gaze measurements as a cue for driver fatigue [Ji, et al., 2006].

Ref. [McCall, et al., 2005] presented a robust computer vision method for

identifying and tracking freeway lanes and driver head motion and utilized these

information as well as vehicle parameters for lane change intent analysis using a sparse

Bayesian learning methodology.

10

2.2 Information Fusion

2.2.1 Overview

There is a universally perceptual phenomenon that whenever the brain has

multiple pieces of information it must decide if they relate to one other or are

independent [Beierholm, et al. 2007]. The integration of data, recorded from a multiple

sensor system, together with knowledge, is knows as information fusion. Information

fusion first appeared in the literature is in the 1960s, when the Department of Defense

(DoD) developed a mathematical model and applied it for location, characterization and

identification of weapon systems and military units. There are various types of

architectures for information fusion in the literature, e.g. waterfall fusion process model

[Markin et al., 1997], JDL fusion model [Hall & Llinas, 1998], LAAS Architecture

[Alami et al., 1998], Omnibus model [Bedworth & O’Brien, 1999], and Time-

Triggered Sensor Fusion Model [Elmenreich, 2002], etc.

A canonical architecture for information fusion is illustrated in Figure 2 that a

phenomenon is observed and processed by a number of individual techniques, and

information from the individual technique is combined at a fusion center. The individual

techniques might be sensors or different classifiers. In general, fusion center includes

modeling of belief, observation combination and decision rule [Chung and Shen, 2000].

Modeling of belief concerns how the degree of belief in the observations and output are

modeled. Observation combination defines the way to aggregate the beliefs in multiple

11

observations for an output. Decision rule chooses the consensus output in order to

maximize the occurrence probability for the classification problem.

Figure 2 A canonical architecture for information fusion

Based on where the combining operation takes place in the information extraction

process, information fusion may be performed at different stages: data level, feature level

and decision level [Gunatilaka and Baertlein, 2001], as shown in Figure 3.

12

Figure 3 Fusion level

2.2.2 Fusion Method

A wide variety of fusion methods have been proposed and implemented in the

literature including committee methods, clustering algorithms, weighted average, fuzzy

set theory, and Dempster-Shafer evidence theory. The next section gives an overview of

these popular approaches.

Committee methods are vote based fusion methods. It includes two aspects: voter

(committee), and voting rules (hypothesis). The major vote rule, i.e. the hypothesis with

the most voting, is finally chosen. Although expert committees are selected out and

expected to choose the correct hypothesis, the higher number of hypothesis increases, the

likelihood of an incorrect choice is also increased. Moreover, the voting process is full of

intuitive and vulnerable to the impact of surroundings.

13

Clustering fusion methods group N individual observations or classifiers into M

clusters, where M is the number of hypotheses. For an observation X, the fused decision

falls into the hypothesis associated with the cluster to which the observation is allocated.

The principle behind this grouping process is that objects (i.e., classifiers or observations)

within a cluster are “similar” to one other and “dissimilar” to objects in other cluster.

Clustering algorithms are suitable for static offline data processing, while driving-related

data is a time-series observations.

Applying Fuzzy Set Theory to a problem requires the fuzzy representation,

operator and defuzzification be defined. Dempster-Shafer Evidence Theory is no

exception, too. In the literature, their choice is often subjective and application dependent.

2.2.3 Summary

Clearly, besides the statistical advantages, information from various sources are

blended to continually present the most complete and accurate situational assessment.

However, at times, different streams may carry similar information, and at other times,

different streams may carry complimentary information. It is absolutely discouraging if

the combination of multiple observations brings about a consensus output of greater

uncertainty. Reducing the uncertainty associated with the consensus output is the main

goal of information fusion. A robust system should be capable of dealing with various

unreliable observations. These observations stem from the improper use of sensor

equipment, error design of algorithm, as well as the absence of some data or the addition

of noise to data processing, and so on. When a sensor gives contaminated observation, its

14

observation will become noisy and unreliable. A good fusion technique should try to get

rid of these “outliers”. This research will demonstrate a way to overcome the problem of

unreliable observation.

The methods described above can be attributed to two categories: discrete and

continuous. Committee and clustering method belong to the former, while Fuzzy Set and

Dempster-Shafer theory belong to the latter. In general, discrete methods are more

suitable for static data processing; continuous methods commonly used for the analysis of

time series data. Sometimes, time-series data are not all continuous. In this case, the usual

practice is to divide the original data into a series of time periods (i.e. time window); each

time window is considered as continuous. Studies have shown that time window selection

has a great impact on the final results. In this thesis, the appropriate time window length

will be decided in experiment.

15

3. METHODOLOGY

3.1 Pattern Learning by Support Vector Machine

3.1.1 Overview of SVM

Most recent researchers follow the machine learning approach based on these two

facts that: (1) human’s capability of driving is widely based on the experience and the

possibility to learn from experience; (2) data we collect from vehicle driver or external

environment are definitely non-linear and human behavior is non-deterministic. Since a

key motivation in this thesis is to provide driver assistance with performance measure

(recognition confidence), a learning strategy for driver intention inferring should have

some mechanism for representing uncertainty and should work well within time-series

valued domain. For these reasons, this thesis adopts a learning strategy based on support

vector machine (SVM).

Support Vector Machine (SVM), presented by Vapnik in 1990s [Cortes and

Vapnik, 1995], has been widely used in pattern recognition. Based on statistical learning

theory, SVM is similar to the root of multilayer neural networks. Given some sets of data

classified, when a new data added, SVM can predict which set it should belong to. The

following is a binary classification example. With a series of training data,

*( ) ( ) ( )+ , , the associated labels are for

Class 1 and for Class 2. If the training data is linearly separable, there exists a

series of separating planes called hyperplanes, represented by a plane equation

, ( ) where is an m-dimensional vector of weights, and is known as

bias. When , the separating hyperplane is in the middle of two hyperplanes with

16

and . Those points along the hyperplanes ( ) are called support

vectors. If the margin, defined as the distance between the hyperplanes with support

vectors, is maximized, the middle hyperplane within such margin is called the optimal

hyperplane, as shown in Figure 4. In training a support vector machine, it is necessary to

select an appropriate kernel function and its parameters if a classification problem is not

linearly separable in the input space. Kernel function performs as mapping the original

input space into a higher dimensional feature space. Typically kernel function used for

SVM is Radial Basis Function (RBF), which is well suitable for dealing with the

nonlinear relationship between attributes and class labels [Huang & Lin, 2008].

Figure 4 An example of two categories of points separated by a hyperplane

17

3.1.2 Toolkit for SVM

The toolkit used for the training Support vector machine in this report is by

LIBSVM. The LIBSVM is an integrated software for multi-class support vector

classification [Chang & Lin, 2001]. The software is available at

http://www.csie.ntu.edu.tw/~cjlin/libsvm. For more information about LIBSVM and the

details on how to use it are described in the mentioned website.

3.2 Uncertainty Reduction by Entropy

3.2.1 Overview of Entropy

Entropy is frequently useful in machine learning for things like feature selection.

In this thesis, we want to apply it in information fusion to achieve the purpose of

uncertainty reduction. The concept of entropy can be traced back to Shannon’s

information theory published in the year of 1948; however, not until 1957 had the

Maxent (maximum entropy) gained extensive attention after Jaynes raised the maximum

entropy principles [Jaynes, 1988]. Since then, Maxent has been successfully applied to

many fields. )(XEntropy is defined as follows:

b

adxxpxpXEntropy )(log)()( 2

where, )(xp is the p.d.f. of a continuous random variable X, subject to the

constraints 0)( xp and b

adxxp 1)( . Distributions )(xp that are spread more evenly

18

across many values will have higher entropy, whereas those that are sharply peaked

around a few values will have relatively low entropy, as illustrated in Figure 5.

Figure 5 Two normal distribution illustrating the higher value of entropy for broader

distribution

The maximum entropy can be found by maximizing H using the method of

Lagrange multiplier for optimization in the presence of constraints. Thus, the objective

function can be constructed as

b

a

b

adxxpdxxpxpXJ 1)()(log)()( 2 ,

and differentiate with respect to )(xp to obtain

1)(log

)(2 xp

xp

J.

19

Setting the above equation to zero and solving for )(xp gives 12)( xp .

Choosing to satisfy the constraint gives )(log1 2 ab , yielding

.,0

,1

)(

otherwise

bxaabxp .

Since the sign of the second partial derivative is negative for all x :

)(

1

)( 2

2

xpxp

J

,

and the solution spontaneously satisfies 0)( xp , that the solution is a maximum

rather than a minimum. This also verifies the disclamation that uniform distribution has

the highest entropy.

Accordingly, the conditional entropy of Y given X, i.e., ),|( XYEntropy is

denoted by:

b

a

d

cdxxypxypxpXYEntropy )|(log)|()()|( 2

where, )|( xyp denotes the conditional probability of Y given X.

3.2.2 Uncertainty Reduction

In this thesis, uncertainty level is measured by entropy. An essential property of

entropy is that entropy is directly proportional to the degree of uncertainty (or

randomness) of the sensory observations.

20

The relations between sensory observations can be categorized into independent,

positively dependent and negatively dependent. Thus, two sensor observations are

independent if the level of uncertainty does not change after they are combined.

Similarly, two sensor observations are defined to be positively dependent if the level of

uncertainty is reduced after they are combined; in contrast, if the level of uncertainty is

increased after they are combined, negative dependence is defined.

In order to better quantify this correlation relationship, we introduce the concept

of entropy correlation coefficient (ECC), which is defined as:

( ) ( )

( )

The ECC values are in the range of [-1, 1].

From the point view of entropy, if the self-entropy is equal to the conditional

entropy, i.e., )|()( XYEntropyXEntropy , observations are independent or irrelevant

( ); and if the self-entropy is larger than the conditional entropy, i.e.,

),|()( XYEntropyXEntropy observations are positively dependent or relevant

( , - ); and if the self-entropy is smaller than the conditional entropy, i.e.,

),|()( XYEntropyXEntropy observations are negatively dependent or relevant

( , -) . The closer the coefficient is to either −1 or 1, the stronger the

correlation between the observations.

It is known that two complete independence sets of sensory observations are of

little practical significance for information fusion, the system uncertainty level is

21

determined by the strength contrast between positively dependent observations and

negatively dependent observations. As a result, the modeling of the positive and negative

dependence becomes an integral part of information fusion tasks and is of particular

interest in this research. A reasonable approach or basic principle is to retain the

positively dependent observations with maximum extent and avoid the negatively

dependent observations with minimum extent in order to tremendously reduce the system

uncertainty level.

22

4. EXPERIMENTS AND RESULTS

4.1 Experiment Design

4.1.1 Definition of Lane Change

Fundamental to understanding the premise of the study is the definition of the

term “lane change”. A basic type of lane changing is developed when a vehicle wants to

overtake slower vehicles. To be more realistic with the consideration of traffic road

condition, our lane changing course consists of five steps as shown in Figure 6: (1)

initiate lane change attempt, (2) add lane, (3) pass, (4) drop lane, and (5) end lane change.

At a high level, we are distinguishing between two classes: lane keeping and lane

changing.

23

Figure 6 Sketch map of lane changing process

4.1.2 Experiment Set-Up

Twelve drivers took part in a simulator based driving experiment. They were

graduate students from northeastern university, six of them were male and six of them

were female. Participants were first informed about the goal of the study and were told to

drive like they normally did. Each subject completed the scenario as depicted in Figure 6,

during which driving performance data were recorded via the smart wheel system, and

simultaneously, driver eye gaze movement were recorded by the eye tracking system as

shown in Figure 7.

t0 t1 t2 t3 t4

(1) Initial lane change attempt

t0 t1 t2 t3 t4

(2) Added lane

t0 t1 t2 t3 t4

(3) Passing

t0 t1 t2 t3 t4

(4) Lane drop

t0 t1 t2 t3 t4

(5) Lane change end

24

Figure 7 Experiment set-up

4.1.3 Experiment Data

The driving data from 12 drivers are used in this study (see Appendix). Each

driver data is segmented into lane following and lane changing tasks. The performance

data that are considered include steering angle, gas pedal pressure, and vehicle heading

angle, as well as vehicle velocity and acceleration. Eye tracking data can be separated

into saccade and fixation. Saccade means the driver is interested in several objects in

front of him or she, but he or she is not paying too much attention to an object. Fixation

suggests the driver is paying attention to a particular object or region in front of him or

her. The longer the fixation duration, the higher interest and concentration the driver has

on that particular object or area. Thus, the original collected gaze points are filter into

Lane keeping (start)

Lane keeping (pass)

Lane changing

(added lane)

Lane changing

(lane drop)

Driving Simulator

Smart-wheel system

Eye tracking system

Fixation

25

fixations which offers a much easier and more interpreter form for lane change intention

analysis. The conducted data are listed in Table 1. From top to bottom, they are fixation

location, fixation duration, steering angle, gas pressure, heading angle, velocity,

acceleration) (LK: Lane keeping; LC: lane changing). The horizontal axis shows the

elapsed time from the start of the lane change. The whole time for lane change was

around 40 seconds, which was the average of the above-mentioned experiment. The

vertical axis shows the steering wheel angle (upper part) and the vehicle lateral position

(lower part).

26

Table 1 Samples of eye gaze and performance data

27

4.2 Three-Stage Nested Design

The objective of this design is to find out the source of the variability in the inputs,

which assumedly affects the model performance. If significant results from differences

among inputs do exist, it is able to achieve the optimum result by selecting the best

“supplier”. A proposed three-stage nested design is illustrated in Figure 8. This design

has three stages that six levels of factor C (different combinations of input data) at the top,

two levels of factor S (data structure) nested under each level of C, and three levels of

factor T (time window) under each level of S.

Figure 8 A three-stage nested design

4.2.1 Data Fusion/Combination (C)

The principle for data combination is through entropy correlation coefficient

(ECC). The ECC indicates the degree of linear dependence between the variables.

Steering angel is chosen as a reference since it presides over others for lane change. As

28

shown in Table 2, steering angle’s ECC with gas pressure is 0.0795; the highest ECC

value (0.9762) is between steering angle and eye gaze; the lowest ECC value is between

steering angle and heading angle. The closer the ECC is to either -1 or 1, the stronger the

correlation between the variables. That means correlation between steering angle and eye

gaze has the maximum positively dependent, while correlation between steering angel

and heading angle has the maximum negatively dependent.

In accordance with the basic rule of uncertainty reduction by entropy, data from

steering angle and heading angle should not be combined together, and eye gaze data is

supposed to play a lot of help on the performance. In addition, it is assumed that multi-

features is superior to single feature. Based on the above two considerations, six different

data combinations are under design. They are: C1 (Steering angle), C2 (Heading angle),

C3 (Steering angle + Gas pressure + Velocity + Acceleration), C4 (Heading angle + Gas

pressure + velocity + acceleration), C5 (Steering angle + Gas pressure + Velocity +

Acceleration + Eye fixation), and C6 (Heading angle + Gas pressure + Velocity +

Acceleration + Eye fixation), of which, C1 or C2 contains single feature, and C3 or C4 or

C5 or C6 contains multi features, and C5 and C6 even includes eye fixation data. C1 and

C3 or C2 and C4 correspond to prove if multi-features is superior to single feature, while

C5 and C6 includes eye fixation data. There are three hidden comparisons, that is,

compassion of single feature and multi features (C1 vs. C5 or C2 vs. C6), comparison of

steering angle and heading angle on performance (C3 vs. C4 or C5 vs. C6), and

comparison of eye gaze on performance (C3 vs. C5 or C4 vs. C6).

29

Table 2 Entropy correlation coefficients

Steering

angle Gas

pressure Heading

angle Velocity Acceleration Eye Gaze

Ste

erin

g

an

gle

0 0.0795 -0.9543 0.1749 0.6592 0.9762

4.2.2 Data Representation (S)

A simple approach to data representation is to input the entire feature sets by

order. In such an approach, feature sets (X) contains steering angle (x1), gas pressure (x2),

heading angle (x3), velocity (x4), acceleration (x5), eye fixation (x6),…, etc.

Another alternative approach is to include some form of relationship (e.g., mean

and variance) measures between feature values rather the values themselves. There are

two reasons for the usage of mean and variance. First of all, mean/variance effectively

captures the change in the feature values which is very critical to learn specific patterns,

and they are useful in reducing the side effects caused by noisy data. Another reason is to

reduce the size of feature sets, thereby decreasing the computational complexity. In this

study, the mean (μ) and variances (σ) of the feature values over the samples is used to

replace the original feature values.

Figure 9 shows two data representation. In Data Representation S1 (non-

overlapping), a sample dataset is equally divided into multipart decided by time length,

each of which producing a mean and a variance. In Data Representation S2 (overlapping)

displays a different structure that inter-parts are overlapping with each other. The

30

motivation of overlapping structure is to testify the fact that the division of lane change

may not happen in sharp boundary and the change may be transitional.

Figure 9 Two data representation structure

However, each sectional sample consists of multiple samples, and each sample

with its own label. Using the definition of lane change, each sample on the training data

can be labeled lane changing (LC) or lane keeping (LK). Another issue is how to label

each section in the data stream.

Here is an observation: the prediction of a classification label for any sample only

depends on the preceding sample on test, and the latest sample always contains the latest

information. Thus, the last sample is the best indication of what the driver is currently

trying to do. In other words, if the last sample in a section of size N is marked LK, the

entire section of size N is labeled as LK and vice versa.

)(

)(

)(

)(

)(

)(

1

2

2

1

1

1

1

n

mi

n

i

mi

i

mi

i

tx

tx

tx

tx

tx

tx

n

i

n

i

i

i

i

i

2

2

1

1

1

1

2

2

1

1

n

i

n

i

i

i

i

i

(S1) non-overlapping Raw data (S2) overlapping

31

Figure 10 Sample labeling

As shown in Figure 10, a data stream of size N is defined by m samples at times

},{ ,2,1 niii ttt . The label of sample at n

it is used to label the entire section of size n. A

moving widow of constant size is used, where, the tail sample is dropped and the head

sample is added thus maintaining a constant size.

4.2.3 Time Window (T)

Longer lane changes see a smooth transition in feature values like straight motion

trajectory, whereas short ones have a relatively abrupt transition.

One issue is that lane change does not have fixed time length. Moreover, temporal

nature makes performance data or eye gaze data change as a function of time. For

example, steering angle has a smooth transition, whereas gas pressure has a relatively

abrupt transition as shown in Table 1. As lane change is always considered as a continual

smooth maneuver rather than an isolated action, the change in the feature values with

32

respect to time is more critical than the individual feature values. Thus, the input to SVM

should be a series of samples within an interval. The size of interval is left to

experimentation, specially, three time window (0.6 s, 1.2 s, 1.8 s) over which data are

summarized are compared.

4.3 Results Evaluation

As mentioned above, a total of 36 combinations are trained and the model

performance is evaluated via a receiver operating characteristic (ROC) curve, which is a

graphical representation of the tradeoff between the true positive rates and false positive

rates as shown in Figure 11.

Figure 11 ROC curves of different combinations

33

It is also common to calculate the area under the ROC convex hull (AUC) and use

AUC for model comparison in machine learning community [Hanley & McNeil, 1983].

Table 3 lists all AUC’s values for each combination after training applied in SVM.

Usually, the closer AUC to 1, the better level training performance. At first glance, this

resultant table let readers feel very complex and difficult to interpret. More explanations

will be given in the following subsections.

34

Table 3 Summary of AUC for each model after training

Factors AUC

C1

S1

T1 0.7449

T2 0.6324

T3 0.6011

S2

T1 0.7390

T2 0.6761

T3 0.6631

C2

S1

T1 0.6824

T2 0.6102

T3 0.5283

S2

T1 0.7224

T2 0.5744

T3 0.6336

C3

S1

T1 0.7642

T2 0.6667

T3 0.6980

S2

T1 0.8973

T2 0.7646

T3 0.8091

C4

S1

T1 0.7305

T2 0.6612

T3 0.6818

S2

T1 0.8621

T2 0.7543

T3 0.7328

C5

S1

T1 0.7707

T2 0.6713

T3 0.6443

S2

T1 0.9028

T2 0.7597

T3 0.8293

C6

S1

T1 0.7688

T2 0.6411

T3 0.5989

S2

T1 0.8898

T2 0.7413

T3 0.8043

35

4.3.1 Comparison of fused multiple feature and single feature

Experiment results in Figure 12 clearly demonstrate that the performance data

fused feature achieves better classification performance than the single feature only.

Figure 12 Comparison of combined features and single feature under ROC

4.3.2 Comparison of data combination (C)

In section 4.2.1, we define various data combinations: C1 (Steering angle), C2

(Heading angle), C3 (Steering angle + Gas pressure + Velocity + Acceleration), C4

(Heading angle + Gas pressure + velocity + acceleration), C5 (Steering angle + Gas

pressure + Velocity + Acceleration + Eye fixation), and C6 (Heading angle + Gas

pressure + Velocity + Acceleration + Eye fixation). In Figure 13, we can find that the

performance of a combination containing heading angle data is worse than that of a

36

corresponding combination containing steering angle data. This difference is more

pronounced for individual comparisons. This also testify our previous view that data from

heading angle is negative to the system performance.

In addition, among those combinations which contain steering angle data, i.e., C1,

C3 and C5, C1 and C3 exclude eye gaze data, while C5 does include eye gaze. Figure 14

denotes that eye gaze plays a positive role on fusion performance. The reason is that eye

gaze and performance data come from different modalities using different equipments,

both of which provide complimentary sources, and these sources are not redundant.

4.3.3 Comparison of time window (T)

Three time windows (C1: 0.6s, C2: 1.2s. C3: 1.8s) are compared under two levels

of data representation. No matter under what data representation structure, C1 achieves

better confidence level that the other two, as shown in Figure 15 and Figure 16.

It is thought that longer sequence may contain more comprehensive and stable

time relations between lane change, and they are expected to yield better performance.

However, since the total quantity of training data is fixed, the number of training samples

decreases with the increase of window size, thus, longer window size may undermine

models’ performance because there are fewer training instances. In our experiment,

smaller window (C1: 0.6 s) provides better performance.

37

4.3.4 Comparison of data structure (S)

By comparing Figure 17 and Figure 18, we conclude that overlapping data

representation structure (S2) achieves better performance. The rationale for this finding is

that driver lane changing behavior is continuous, and the changes may has no strict

divisions and not happen always near the two ends of the time window.

Figure 13 Comparison of data combination

38

Figure 14 Comparison of eye gaze effect on performance

Figure 15 Comparison of time window under non-overlapping data structure (S1)

39

Figure 16 Comparison of time window under overlapping data structure (S2)

Figure 17 Comparison of data representation under the same time window (T1)

40

Figure 18 Comparison of data representation under the same time window (T2)

Figure 19 Comparison of true positive rate of three-stage nested design at 5% false

positive rate

41

5. CONCLUSIONS AND DISCUSSIONS

5.1 Conclusions

Based on prior driver lane change intention research, we are motivated to

determine if eye gaze and performance data are useful for driving intention inferring, and

furthermore which one or combination is the more informative cue. Using driver

performance data (steering angle, gas pressure, velocity, and acceleration) and eye gaze

data recorded in a driving simulator, a driver intention recognition model is developed

that takes into account the dynamic characteristics of vehicle/driver, and its fundamental

performance is analyzed. Results show the achievement of 88.78% accuracy with a time

length of 0.6 seconds at 5% false alarm as shown in Figure 19, which demonstrate this

proposed model have excellent power in terms of early prediction with respect to time,

and such model can improve the performance of driver assistance system due to its early

prediction.

To read driver intention, this thesis illustrates the use of machine learning method

(support vector machine) to recognize driver’s lane changing maneuvers. Through the

participation of twelve subjects, the support vector machine method successfully

classifies their driving maneuvers into two classes: lane keeping and lane changing.

To improve recognition performance, this thesis illustrates the use of entropy

based fusion method to discard those largely negative dependent data to decrease the

redundancy level of input data. Although heading angle data is also recorded during

experiment, heading angle data is finally excluded through the analysis of entropy

42

correlation coefficient and data combination results. So, in the end, only four categories

of performance data (steering angle, gas pressure, velocity, and acceleration) are selected.

Another promising contribution of this thesis is the incorporation of eye gaze data.

“The eye is the mirror of the soul, and the soul is the mirror of our thoughts.” Vehicle

related performance data only passively reflect driver psychological change, while eye

gaze actively reflects driver inner world (e.g. intention).

Since SVM works like a “”black box”, its accuracy level greatly depends on its

input and its kernel function. As RBF kernel function is determined, input variability

factors are subdivided into three aspects: different level of data combination, different

level of data representation structure, and different level of time window. Through three-

stage nested design, we end up experiment to filter out the best “suppliers” to SVM. In

general, less time scale length generates better recognition than longer time interval. This

is because smaller time scale can better captures the lane subtle change. Our proposed

overlapping data representation structure could be well used to simulate the transition of

lane change.

43

5.2 Discussions

The preliminary result of the analysis suggests that there is a potential for other

types of indicators and combinations of indicators to predict the intention to change lanes.

Driving situation consists of three entities: vehicle, driver and environment. It is

reasonable to fuse data from these three aspects to achieve better recognition result. A

full range of driver behaviors like turns, stops, parking etc. can be tested on the same

frameworks to observe the recognition capabilities.

This thesis only considers the simple case for lane change, that distinguishing

between lane keeping and lane changing at a high level. There are actually more

complicated cases to consider. For example, lane change intent is caused by driver’s

distracted driving, that is, a driver unintentionally drifts his car near across lane boundary.

With regard to safety in driving, it is important to distinguish driving patterns between

intended driving and distracted driving.

44

Appendix

Summary of performance data (from top to bottom, they are acceleration, gas

pressure, steering angle and velocity) of twelve subjects.

Subject # 1

45

Subject # 2

Subject # 3

46

Subject # 4

Subject # 5

47

Subject # 6

Subject # 7

48

Subject # 8

Subject # 9

49

Subject # 10

Subject # 11

50

Subject # 12

51

Reference

1. National Highway Traffic Safety Administration (NHTSA) Annual Report, “Calendar

Year 2009 Traffic Fatalities Continue Record Downward Trend”, March 2010.

http://www-nrd.nhtsa.dot.gov/Pubs/811291.PDF

2. World Health Organization, 2004. World Report on Road Traffic Injury Prevention.

Geneva.

3. Bljncoe, L.J., Seay, A.G., Zaloshnja, E., Miller, T.R., Luchter, S., and Spicer, R.S.

(2002) The Economic Impact of Motor Vehicle Crashes 2000. National Highway

Traffic Safety Administration Technical Report, May, 2002.

4. An, X.J., Wu, M., and He, H. (2006) A Novel Approach to Provide Lane Departure

Warning Using Only One Forward-Looking Camera. Proc. International Symp.

Collaborative Technologies and Systems, 2006 (CTS 2006), May 2006, pp. 356-362,

doi: 10.1109/CTS.2006.9.

5. Parker, D., Lajunen, T., and Summala, H. (2002) Anger and Aggression among

Drivers in Three European Countries. Accident Analysis and Prevention, vol. 34, no.

2, March, 2002, pp. 229-235.

6. Geddes, N.D. (1989) Understanding Human Operators’ Intention in Complex

Systems. Ph.D. Dissertation, Georgia Institute of Technology, Atlanta, GA.

7. Chovan, J.D., Tijerina, L., Alexander, G., and Hendricks, D. (1994) Examination of

Lane Change Crashes and Potential IVHS Countermeasure. Washington DC:

National Highway Traffic Safety Administration.

8. Ahmed K.I., Ben-Akiva, M.E., Koutsopoulos, H.N., and Mishalani, R.G. (1996)

Models of Freeway Lane Changing and Gap Acceptance Behavior. International

symposium on transportation and traffic theory, Lyon, France, 24-25 July 1996, pp.

501-115.

9. Mammar, S., Glaser, S., and Netto, M. (2006) Time to Line Crossing for Lane

Departure Avoidance: a Theoretical Study and an Experimental Setting. IEEE

Transactions on Intelligent Transportation Systems, vol. 7, no. 2, 2006, pp. 226-241,

doi: 10.1109/TITS.2006.874707.

52

10. Fazio, J., Michaels, R., Reilly, W., Schoen, J., and Poulis, A. (1990) A Behavioral

Model of Freeway Exiting, Washington, D.C.: Transportation Research Board, 69th

Annual Meeting, January, 1990.

11. Pentland, A., and Liu, A. (1999) Modeling and Prediction of Human Behavior.

Neural Computation, vol. 11, 1999, pp. 229-242.

12. Kuge, N., Yamamura, T. and Shimoyama, O. (2000) A Driver Behavior Recognition

Method Based on a Driver Model Framework. Intelligent Vehicle Systems (SP-1538).

13. Salvucci, D.D. (2004) Inferring Driver Intent: A Case Study in Lane-Change

Detection. In Proceedings of the Human Factors Ergonomics Society 48th Annual

Meeting.

14. Coue, C., Fraichard, T., Bessiere, P., and Mazer, E. (2002) Multi-Sensor Data Fusion

using Bayesian Programming: an Automotive Application. In IEEE / RSJ

International Conference on Intelligent Robots and System, vol. 1, Lausanne,

Switzerland, pp. 141-146.

15. Wei, H., Meyer, E., Lee, J. and Feng, C. (2000) Characterizing and Modeling

Observed Lane-Changing Behavior: Lane-Vehicle-Based Microscopic Simulation on

Urban Street Network. Journal of the Transportation Research Board, vol. 1710, pp.

104-113.

16. Salvucci, D.D., and Liu, A. (2002) The Time Course of a Lane Change: Driver

Control and Eye Movement Behavior. Transportation Research Part F, vol. 5, pp.

123-132.

17. McCall, J.C., Trivedi, M.T., Wipf, D., and Rao, B. (2005) Lane Change Intent

Analysis Using Robust Operators and Sparse Bayesian Learning. IEEE Computer

Society Conference on Computer Vision and Pattern Recognition (CVPRW'05) -

Workshops, 2005, pp. 59.

18. Tijernia, L., and Hetrick, S. (1997) Analytical Evaluation of Warning Onset Rules for

Lane Change Crash Avoidance Systems. Proceedings of the Human Factors and

Ergonomics Society 40th Annual Meeting, pp. 949-953.

19. Ji, Q., Lan, P., and Looney, C. (2006) A Probabilistic Framework for Modeling and

Real-Time Monitoring Human Fatigue. IEEE Transactions on Systems, Man, and

Cybernetics – Part A: Systems and Humans, vol. 36, no. 5, September, 2006.

53

20. Beierholm, U., Kording, K., Shams, L., and Ma, W.J. (2007) Comparing Bayesian

Models for Multisensory Cue Combination without Mandatory Integration. Advances

in Neural Information Processing Systems.

21. Markin, M., Harris, C., Bernhardt, M., Austin, J., Bedworth, M., Greenway, P.,

Johnston, R., Little, A., and Lowe, D. (1997) Technology Foresight on Data Fusion

and Data Processing. Publication of the Royal Aeronautical Society.

22. Hall, D.L., and Llinas, J. (1998) Overview of Data Fusion. Proceedings of the

International Symposium on Circuits and Systems, June 1, 1998.

23. Alami, R., Charila, R., Fleury, S., Ghallab, m., and Ingrand, F. (1998) An

Architecture for Autonomy. International Journal of Robotic Research, vol. 17, no. 4,

pp. 315-337, April 1998.

24. Bedworth, M. and O'Brien, J. (1999) The Omnibus Model: a New Model of Data

Fusion. Proc. 2nd Intl. Conf. on Information Fusion, Sunnyvale.

25. Elmenreich, W. and Pitzek, S. (2001) The Time-Triggered Sensor Fusion Model. In

Proceedings of the 5th IEEE International Conference on Intelligent Engineering

Systems, pp. 297–300, Helsinki–Stockholm–Helsinki, Finland, September 2001.

26. Chung, A.C.S., and Shen, H.C. (2000) Entropy-Based Markov Chains for Multi-

Sensor Fusion. Journal of Intelligent and Robotic System, vol. 29, pp. 161-189.

27. Gunatilaka, A.H., and Baertlein, B.A. (2001) Feature-Level and Decision-Level

Fusion of Noncoincidently Sampled Sensors for Land Mine Detection. IEEE

Transaction on Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 577-589.

28. Cortes, C. and Vapnik, V. (1995) Support-Vector Network. Machine Learning, vol.

20, pp. 273-297.

29. Huang, X. and Lin, Y. (2008) A vision-Based Hybrid Method for Facial Expression

Recognition. In Proceedings of the 1st International Conference on Ambient Media

and Systems, Quebec, Canada, February 11-14, 2008. ICST (Institute for Computer

Sciences Social-Informatics and Telecommunications Engineering), ICST, Brussels,

Belgium, pp. 1-7.

30. Chang, C.C. and Lin, C.J. (2001) LIBSVM : a Library for Support Vector Machines,

2001.

54

31. Jaynes, E.T. (1988) The Relation of Bayesian and Maximum Entropy Methods.

In Maximum-Entropy and Bayesian Methods in Science and Engineering, vol. 1,

Kluwer Academic Publishers, pp. 25-29.

32. Brown, C. D., and Davis, H. T. (2006) Receiver Operating Characteristic Curves and

Related Decision Measures: a Tutorial. Chemometrics and Intelligent Laboratory

Systems, vol. 80, pp. 24-38

33. Salvucci, D. D. (2004) Inferring Driver Intent: a Case Study in Lane-Change

Detection. In Proceedings of the Human Factors Ergonomics Society 48th Annual

Meeting.

34. Salvucci, D. D., and Liu, A. (2002) The Time Course of a Lane Change: Driver

Control and Eye-Movement Behavior. Transportation Research Part F, vol. 5, pp.

123-132.

35. Salvucci, D. D., and Siedlecki, T. (2003) Toward a Unified Framework for Tracking

Cognitive Processes. Proceedings of the 25th Annual Conference of the Cognitive

Science Society. Mahwah, NJ: Lawrence Erlbaum Associates.

36. Hanley, J.A., and McNeil, B.J. (1983) A Method of Comparing the Areas under

Receiver Operating Characteristic Curves Derived from the Same Cases. Radiology,

vol. 148, no. 3, pp. 839-43.

55

ACKNOWLEDGEMENTS

This thesis is completed under the guidance of my graduate academic advisor

Professor Yingzi Lin, who supports and directs me through my whole study at

Northeastern University. I would like to take this opportunity to thank Prof. Lin who

makes this thesis a success.

I thank everyone who ever worked with me in the Intelligent Human Machine

Systems Laboratory; it is my pleasures to work with your guys. My heartfelt thanks to

John Zhu, Hongjie Leng, and Hua Cai for their working together to conduct the

experiment.

I would also like to thank Professor Chris Zhang, it is a great honor to work in

your lab at University of Saskatchewan.

Finally, I would like to thank my family for so many years of meticulous care.