Upload
truongliem
View
247
Download
4
Embed Size (px)
Citation preview
http://www.iaeme.com/IJCIET/index.asp 793 [email protected]
International Journal of Civil Engineering and Technology (IJCIET) Volume 8, Issue 11, November 2017, pp. 793–802, Article ID: IJCIET_08_11_081
Available online at http://http://www.iaeme.com/ijciet/issues.asp?JType=IJCIET&VType=8&IType=11
ISSN Print: 0976-6308 and ISSN Online: 0976-6316
© IAEME Publication Scopus Indexed
REAL-TIME DRIVER FATIGUE OR
DROWSINESS DETECTION SYSTEM USING
FACE IMAGE STREAM
Ilakiyaselvan N, Thomas Abraham J V, Muralidhar A, Kalluri Vinay Reddy
School of Computing Science and Engineering, VIT, Chennai - 600127 Tamilnadu, India
ABSTRACT
According to the Road Accident Statistics in India report presented by the NDTV
in 2016. One death for every 4 minutes because of a road accident in India. The major
reasons for the following accidents are happening due to the Drowsiness and driver
fatigue nature during the driving. Lack of sleep is one of the major reasons for
drowsiness. The boundary between feeling sleepy and established sleep is known as
drowsiness. Driver Fatigue is one major factor for 20% global accidents. There are
many system attempted to identify the drowsiness of the driver in a driving simulation
by recording the Non-visual signals and some systems concentrated on the developing
a face recognition technique while driving and alerting the driver by rising the alarm
in computer software application based interface. This research is aimed to build a
real-time application which can be used to detect the fatigue or drowsiness in driving
conditions and alert the driver whenever drowsiness is detected. The application is
able Predict the behavior of driver by measuring the visual measurements in the
driving conditions in a feasible model which can be used by the all types of
commercial and personal vehicles.
Keywords: Drowsiness, Driver fatigue, Face recognition
Cite this Article: Ilakiyaselvan N, Thomas Abraham J V, Muralidhar A, Kalluri
Vinay Reddy, Real-Time Driver Fatigue or Drowsiness Detection System Using Face
Image Stream, International Journal of Civil Engineering and Technology, 8(11),
2017, pp. 793–802
http://www.iaeme.com/IJCIET/issues.asp?JType=IJCIET&VType=8&IType=11
1. INTRODUCTION
In study, it is found that despite of developing the heavy-end safety measures on the road
development. Accidents are still happening in a large frame across the world. The major
numbers of accidents are due to the drowsiness of the driver [6]. In order to reduce the
accidents a real-time monitoring system has to develop to detect the visual features of the
driver which will measure the fatigue or drowsiness during the driving environment [7]. It is
found that if a driver going in a speed of 100 Km/h and falls asleep for just four seconds
without the control of the driver the vehicle is going to travel a distance of 111 meters in a
Ilakiyaselvan N, Thomas Abraham J V, Muralidhar A and Kalluri Vinay Reddy
http://www.iaeme.com/IJCIET/index.asp 794 [email protected]
Highway [8]. At that speed and distance travelled results to a likely crash. The drivers who
are falling under this category are young drivers, Shift workers in industry which includes the
heavy vehicle drivers and the drivers with the sleep disorders. Study conducted by the
Adelaide Centre for sleep research put some interesting facts regarding drowsiness [8]. They
are like a person who has been awake for the 17 hours faces the same risk of a crash as a
person who has a Blood Alcohol Content reading of 0.05/100ml. They are therefore twice as
likely to have an accident as a person with Zero blood alcohol content who is not fatigues.
The present technology in detection of drowsiness have been classified into four types: [9].
• Measuring Visual Features like eyes, head movement, yawning.
• Measuring Non-Visual Features like EEG and ECG.
• Vehicle position in Lane Monitoring and Steering Pattern Monitoring.
The systems that is dependable on body sensors for measure parameters like brain
activity, heart rate, skin conductance and muscle activity which are difficult to the drivers to
have access to them which causes little uncomforting during driving making them
contactable. The aim of the research that has been taken here is to understand what the real
conditions of driving needs to be followed and what are the expectations from the driver while
using the application[10] [11]. Developing an Android based application which can be placed
in the vehicle without distracting the user can help the driver to come out of the fatigue or
drowsiness while driving. The present work is concentrated on the detection of open and
closed state of human eye using Google Vision library which will parallel help to detect the
facial landmarks. Our method is able to detect eye closure and predict with maximum
accuracy even in the presence of face and eye titling and slightly variations in the light
available around the vehicle.
2. RELATED WORK
The development of drowsiness detection system started back in 1990’s. Considering the
accidents that are happening researchers Hiroshi Ueno with the rest of the team started
analyzing the major reasons for the accidents and found fatigue is one such reason. Images are
used to detect the drowsiness. Increase of time to detect eye state and keeping limited
resources of hardware. The system came up with a little reduced Alertness.[12]
Later many researchers have done significant work to detect drowsiness keeping available
technology. One such attempt which showed significant features extraction done by
researcher Antoine picot, Sylvi and Alice Caplier where they worked on complete visual signs
of eyes through high frame rate video. By recorded video analysis they are able to detect the
drowsiness from the videos but accuracy and efficiency has not improved to the expected
value but proposed the basic algorithm to detect the fatigue.[13]
Later in 2012 to present use enhanced algorithms like viola-johns Object detection
Framework[6], openCV[15] and PERCLOS[6], [9], [16] taken major turn in detecting the
drowsiness many researchers have put forward there work based on eye closure detection.
From all the related work study the requirement of the driver has been analyzed has[4]:
• It should be able to fit in any kind of personal and commercial vehicles which includes
heavy trucks.
• The application should not cause any distraction to the driver. The negative and false
alarm rate should be minimum throughout the journey.
• The system should be able to work in real-time. Able to work in both day and night
time.
Real-Time Driver Fatigue or Drowsiness Detection System Using Face Image Stream
http://www.iaeme.com/IJCIET/index.asp 795 [email protected]
3. PROPOSED SOLUTION TO DETECT DROWSINESS AND SYSTEM
ARCHITECTURE
Figure 1 System Architecture Drowsiness Detection System
To build a real-time face and eye tracking application which can detect the driver fatigue
or drowsiness in driving conditions. The application is embedded and contactless to the driver
for the comfort while driving. It is able to detect the drowsiness and alert the user with in a
threshold time.
It is designed in such a way the tilted action from the face will go along with the detection.
This will make driver advantage of moving inside the vehicle. The application is able to
classify the state of the eye during the driving condition. Based on the percentage of eye
closure the drowsiness alert has been classified into three types: Awake Event, Slightly
Drowsy, and Drowsy. The application is built on the android platform. Whenever the eyelids
are closed to more than 80%. The application will raise an alarm and alert the Driver as
Ilakiyaselvan N, Thomas Abraham J V, Muralidhar A and Kalluri Vinay Reddy
http://www.iaeme.com/IJCIET/index.asp 796 [email protected]
shown in figure 1.. The application is built upon Google Mobile Vision Library[17] and used
Google play Services.
4. APPLICATION DEVELOPMENT AND APPROACH
The modules in the project are broadly classified in to four stages of development as:
• Image Capturing
• Face Movement Detection
• Eye Motion Detection
• Drowsiness Detection and Alertness
Each stage will generate on sort of output which will be input to other stage of Module
4.1. Image Capturing
It requires permission from the user to access the front camera. It should be able to swap
width and height sizes when in portrait rotate by 90 degrees during the movement in face.
Camera hardware
The camera should be able to produce at least four frames per second through camera which
will be accessed by the android device. During night time Near-infrared light can be used to
keep it ready mode to capture the image. The light sensitive can be ranging from 750-2000
nm. For this application in order to achieve high accuracy nearly 60 fps powered camera is
being used. The application even have capability to run four frames per second[17].
4.2. Face Movement Detection
The face movement detection is crucial and important to recognize the facial expression of the
driver in driving condition. An overlay has to be formed to confirm the single face in the
frame which is near to the camera. Multiple faces cannot be detected during face movement
detection.
Figure 2 Face Movement detection
Haar like features in Face Movement Detection
These features are generally used to detect the face and eye in the captured image.
Rectangular haar-like feature is the difference between the summation pixels of areas which
are inside the rectangle at any position and scale within the original image. The advantage of
Real-Time Driver Fatigue or Drowsiness Detection System Using Face Image Stream
http://www.iaeme.com/IJCIET/index.asp 797 [email protected]
using haar like features over raw pixels values is that it can reduce/increase the in-class/out-
of-class variability, which means the classification.
Application of Haar Classifier
The frames that are obtained from camera which includes area of resolution of 640*480
height and width respectively. The Haar like features are applied to the image which captures
through the application in every location and at every scale. From the face geometry the
region of interest is selected from the face region. Now in this ROI Haar cascade for closed
eyes is applied if it detects a closed eye a counter increments and camera fetches next frame
and it is processed. If closed eye is not detected Haar classifier for open eyes are checked.
After it is complete it takes the next frame from the camera and processes it. Every time the
PERCLOS value is calculated as the ratio between numbers of closed eyes detected and
number of eyes open found. From the obtained threshold of PERCLOS, fatigue level decision
is taken, and can be used to alarm the driver.
Tilted Face Detection with Affine Transformation Matrix
The original Haar cascade technique applied for face detection detects upright face only. If
there is a moderate amount of tilt of face it will not be detected, consequently eyes will also
be not detected in such a frames. Since the angle in which driver faces camera can vary with
the driving conditions, it is necessary to detect the tilted face. In order to detect we have
implemented a method based on affine transformation matrix.
Affine Transformation Matrix
An affine transformation linear 2-D geometric function which can map variables of an input
image into new variables by applying linear combination of translation, rotation. The
advantages of using affine transformation are that it preserves collinearity and ratio of
distance. These two properties assures that the affine transformed faces will be detected by the
Haar classifiers[18]. In 2-D graphics, for rotation by angle θ counter clockwise about the
origin written in matrix form as:
����′� = ��� − �� �� �� � �
���
The rotation matrix can be found for an “n” dimensions image once it its size, centre and
angle of rotation needed are known. This is implemented in the algorithm.
Steps used to Detect Tilted Face
• First Haar classifier for the face is applied
• If face is not detected then entire captured images is transformed or rotated to close
wise and counter clockwise.
• After the detection the co-ordinates of face are mapped to original image after de-
rotation.
• The eye detection method gives de-rotated eye region.
• It marks further processing more accurate. These steps improves the angle of tilt for
face detection more than 45 degree, and it returns the ROI for eye detection in a de-
rotated form, it further enhances the eye rotation rate.
Ilakiyaselvan N, Thomas Abraham J V, Muralidhar A and Kalluri Vinay Reddy
http://www.iaeme.com/IJCIET/index.asp 798 [email protected]
Face Tracking
Searching for face in every frame in every scale increases the computational complexity. The
real-time performance of the algorithm can be implemented if we use the temporal
information. If the position and size of face is known accurately in a frame, then we can select
ROI around that position where we can find the face in subsequent frame. The computational
complexity is less since the search region is reduced. Application of KLT algorithm is used
for face tracking[19].
Step 1: Track the Face of the driver
Step 2: Make a copy of the points to be used for computing and locating ROI of the
geometric.
Step 3: Transformation between the points in the previous and the current frames
respectively to the user movement.
Step 4: Get the next Frame in the video sequence.
Video_Frame = Get_Next_Frame (VideoFileReader)
Step 5: Track the points in the ROI. (Need at least 2 points along the axis)
Step 6: Estimate the geometric transformation between the old points and the new points
and eliminate outliers using linear translation. (Minimum Four Frames are required to
calculate).
Step 7: Insert a bounding rectangle box around the object being tracked.
Step 8: Display tracked points.
Video Points = insertMarker (VideoFrame, VisibleParts, ‘+’… ‘Colour’, ‘White’);
Step 9: Reset the points and display the annotated frame using android application.
4.2. Eye Motion Detection.
Once face is localized next step is to detect the position of eye. Subsequently the eye detected
is classified to open or close[15]. The detection of eye in the face region is modeled as an
object detection problem. Haar classifier based eye detection on original obtained camera
frame. One user-defined classifiers for open eye and closed eye are used in this detection
process. The classifier for open and close are trained with a database of positive and negative
images are taken into consideration. The ROI selection is done and the detection of eye is
performed in the localized region[20].
4.3. Detect Drowsiness using PERCLOS Calculation.
The number of open and closed eyes over one minute windows are calculated and PERCLOS
values are found. Different Stages of opening and closing eyes values and variables are taken
into consideration. They are Awake, Drowsy, Eyes Closed, Eyes Opened, Likely drowsy,
Normal eye blink, pending slow eye lid closure, slow eye lid closure events in declaring to
check the PERCLOS value for one minute. Threshold for detecting drowsiness is fixed as:
With Drowsy= 0.8 Sec and With Likely Drowsy= 0.15 Sec
PERCLOS will always compare between the previous event and the actual event.
PERCLOS= (Dm - Da)/ Dm
Dm: It is the frame numbers captured in one minute
Da: It is the frames of the eye belong to attentive category.
Dm - Da: It is the number of eye frame belonging to the inattentive category.
Real-Time Driver Fatigue or Drowsiness Detection System Using Face Image Stream
http://www.iaeme.com/IJCIET/index.asp 799 [email protected]
5. WORKING MODEL OF THE SYSTEM
The application is developed using android platform and Google Mobile Vision API[17]. The
below screenshots of the android application which states the different stages of drowsy and
active states that may cause to particular person while driving. Apart from raising the alarm
the usage of colour based alertness is done each and every colour has their own representation
of alertness.
Green colour (Fig.3) indicates the active state of the driver. Yellow colour (Fig.4)
indicates the likely drowsy state of the driver. Red colour (Fig.5) indicates the drowsy nature
beyond threshold of the driver. The application works even in the tilted position while driving
to the extent of 45 degrees so it can be real time even during the driving conditions. Face
tilted has to be detected because the drivers are not going to stay in ideal condition while
driving. There will be a motion in driving and it has to ready to go along with the road
because little vibrations. The up-ward and down-ward movement of face will be detected. If
in-case there some drivers lean down for up while feeling drowsy. The possibilities of feeling
in those directions are more. More chances of accidents occur at that situation.
Figure 3 Active State Figure 4 Likely Drowsy Figure 5 Drowsy State
Figure 6 Right tilt of the face Figure 7 Left tilt of the face
Ilakiyaselvan N, Thomas Abraham J V, Muralidhar A and Kalluri Vinay Reddy
http://www.iaeme.com/IJCIET/index.asp 800 [email protected]
6. FUTURE WORK AND CONCLUSION
The application that developed supports and works in android mobile phones and tablets. In
the current android market place the app is supported in 97.4% android devices. The proposed
and developed application can help to detect the drowsiness and likely drowsiness of the
driver. By analyzing different stages of the opening and closing eyes the drowsiness is
detected. The use of application can detect the drowsiness and alert the driver is sleeping
conditions and alert the driver by considering the speed of the vehicle at 80-100 km/hr the
threshold is fixed and the same used to make the alarm. A lot of the work can be developed
towards detecting the drowsiness using all types of body motion sensors and eye detection
sensor and enabling semi-automatic brakes to the vehicle whenever the driver is about to feel
sleepy. Other issues which can improve the detection system is to provide with good NIR
lighting facility to the mobile devices which can detect the faces in the night time. This real-
time application can definitely make impact in reducing number of accidents occurring and
increase the road safety of the Driver.
REFERENCES
[1] NDTV, Road Accident Statistics in India, 2016. [Online]. Available:
http://sites.ndtv.com/roadsafety/important-feature-to-you-in-your-car-5/.
[2] H. Kang, Various Approaches for Driver and Driving Behavior Monitoring : A Review,
pp. 616–623, 2013.
[3] K. Srijayathi and M. Vedachary, Implementation of the Driver Drowsiness Detection
System, vol. 2, no. 9, pp. 1751–1754, 2013.
[4] Pressreader, Driver fatigue Global Accidents, 2017. [Online]. Available:
https://www.pressreader.com/south-africa/pretoria-news-
weekend/20161217/281535110633688.
[5] C. T. Lin, C. J. Chang, B. S. Lin, S. H. Hung, C. F. Chao, and I. J. Wang, A real-time
wireless brain-computer interface system for drowsiness detection, IEEE Trans. Biomed.
Circuits Syst., vol. 4, no. 4, pp. 214–222, 2010.
[6] T. Pradhan, A. N. Bagaria, and A. Routray, Measurement of PERCLOS using eigen-eyes,
4th Int. Conf. Intell. Hum. Comput. Interact. Adv. Technol. Humanit. IHCI 2012, pp. 0–3,
2012.
[7] L. M. Bergasa, J. M. Buenaposada, J. Nuevo, P. Jimenez, and L. Baumela, Analysing
driver’s attention level using computer vision, IEEE Conf. Intell. Transp. Syst.
Proceedings, ITSC, pp. 1149–1154, 2008.
[8] V. S. Government, Fatigue Statistics. [Online]. Available: http://www.tac.vic.gov.au/road-
safety/statistics/summaries/fatigue-statistics.
[9] J. J. Yan, H. H. Kuo, Y. F. Lin, and T. L. Liao, Real-time driver drowsiness detection
system based on PERCLOS and grayscale image processing, Proc. - 2016 IEEE Int.
Symp. Comput. Consum. Control. IS3C 2016, pp. 243–246, 2016.
[10] S. Begum, Intelligent driver monitoring systems based on physiological sensor signals: A
review, IEEE Conf. Intell. Transp. Syst. Proceedings, ITSC, no. Itsc, pp. 282–289, 2013.
[11] M. Ben Dkhil, A. Wali, and A. M. Alimi, Drowsy Driver Detection by EEG Analysis
Using Fast Fourier Transform, pp. 313–318.
[12] H. Ueno, M. Kaneda, and M. Tsukino, Development of drowsiness detection system,
Proc. VNIS’94 - 1994 Veh. Navig. Inf. Syst. Conf., pp. 15–20, 1994.
[13] A. Picot, S. Charbonnier, and A. Caplier, Drowsiness detection based on visual signs:
blinking analysis based on high frame rate video, 2010 IEEE Instrum. Meas. Technol.
Conf. Proc., pp. 801–804, 2010.
Real-Time Driver Fatigue or Drowsiness Detection System Using Face Image Stream
http://www.iaeme.com/IJCIET/index.asp 801 [email protected]
[14] Q. Wu, B. X. Sun, B. Xie, and J. Zhao, A PERCLOS-based driver fatigue recognition
application for smart vehicle space, Proc. - 3rd Int. Symp. Inf. Process. ISIP 2010, pp.
437–441, 2010.
[15] J. F. Xie, M. Xie, and W. Zhu, Driver fatigue detection based on head gesture and
PERCLOS, 2012 Int. Conf. Wavelet Act. Media Technol. Inf. Process. ICWAMTIP 2012,
pp. 128–131, 2012.
[16] G. C. vision R. Team, Google Mobile Vision, 2017. [Online]. Available:
https://developers.google.com/vision/.
[17] S. Darshana, D. Fernando, S. Jayawardena, S. Wickramanayake, and C. De Silva,
Efficient PERCLOS and gaze measurement methodologies to estimate driver attention in
real time, Proc. - Int. Conf. Intell. Syst. Model. Simulation, ISMS, vol. 2015–Septe, pp.
289–294, 2015.
[18] P. R. Tabrizi and R. A. Zoroofi, Open/closed eye analysis for drowsiness detection, 2008
1st Int. Work. Image Process. Theory, Tools Appl. IPTA 2008, 2008.
[19] C. Bila, F. Sivrikaya, M. A. Khan, and S. Albayrak, Vehicles of the Future: A Survey of
Research on Safety Issues, IEEE Trans. Intell. Transp. Syst., vol. PP, no. 99, pp. 1–20,
2016.
[20] T. Hayami, K. Matsunaga, K. Shidoji, and Y. Matsuki, Detecting drowsiness while
driving by measuring eye movement- A pilot study, IEEE Conf. Intell. Transp. Syst.
Proceedings, ITSC, vol. 2002–Janua, no. September, pp. 156–161, 2002.
[21] Suganya G, Premalatha M, Bharathiraja S and Rohan Agrawal, A Low Cost Design to
Detect Drowsiness of Driver, International Journal of Civil Engineering and Technology,
8(9), 2017, pp. 1138–1149
[22] Jyoti Verma, Vineet Richariya, Face Detection and Recognition Model Based on Skin
Colour and Edge Information for Frontal Face Images, International Journal of Computer
Engineering & Technology (IJCET), Volume 3, Issue 3, October - December (2012), pp.
384-393
[23] Walaa M Abdel-Hafiez, Mohamed Heshmat, Moheb Girgis, Seham Elaw, A New Face
Recognition Scheme for Faces with Expressions, Glasses And Rotation, International
Journal of Computer Engineering & Technology (IJCET), Volume 5, Issue 4, April
(2014), pp. 11-23
[24] Ram Ratan Ahirwal, Sagar Nachankar, Y. K. Jain, A Novel Method of Average Filtering
for Removing Noise and Face Recognition, International Journal of Computer
Engineering & Technology (IJCET), Volume 5, Issue 8, August (2014), pp. 55-70
[25] Shashikant Sharma, Kota Solomon Raju, Application of Gaussian Filter with Principal
Component Analysis Algorithm for The Efficient Face Recognition, International Journal
of Electronics and Communication Engineering & Technology (IJECET), Volume 4,
Issue 7 (2013), pp. 244-251
[26] S. K. Hese, M. R. Banwaskar, Appearance Based Face Recognition by Pca and Lda,
International Journal of Electronics and Communication Engineering & Technology
(IJECET), Volume 4, Issue 2, March – April, 2013, pp. 48-57
[27] Archana H. Sable, Dr. Girish V. Chowdhary, A Two Phase Algorithm for Face
Recognition in Frequency Domain, International Journal of Computer Engineering &
Technology (IJCET), Volume 4, Issue 6, November - December (2013), pp. 127-135
[28] Smt. Mahananda D. Malkauthekar, Smt. Shubhangi D. Sapkal, Comparison of
Mahalanobis and Manhattan Distance Measures in Pca Based Face Recognition,
International Journal of Computer Engineering & Technology (IJCET), Volume 5, Issue
5, May (2014), pp. 01-11
[29] Keyur Shah, Vijay Ukani, Efficient Face Recognition System using Hybrid Methodology,
International Journal of Advanced Research In Engineering and Technology (IJARET),
Volume 5, Issue 4, April (2014), pp. 179-189
Ilakiyaselvan N, Thomas Abraham J V, Muralidhar A and Kalluri Vinay Reddy
http://www.iaeme.com/IJCIET/index.asp 802 [email protected]
[30] Abhishek Choubey, Girish D. Bonde, Face Recognition across Pose with Estimation of
Pose Parameters, International Journal of Electronics and Communication Engineering &
Technology (IJECET), Volume 3, Issue 1, January- June (2012), pp. 311-316
[31] Bilal Salih Abed Alhayani, Prof. Milind Rane, Face Recognition System by Image
Processing, International Journal of Electronics and Communication Engineering &
Technology (IJECET), Volume 5, Issue 5, May (2014), pp. 80-90
[32] Mrs. Manisha Bhisekar, Prof. Prajakta Deshmane, Image Retrieval and Face Recognition
Techniques: Literature Survey, International Journal of Electronics and Communication
Engineering & Technology (IJECET), Volume 5, Issue 1, January (2014), pp. 52-58
[33] Prof. B.S PATIL, Prof. A.R YARDI, Real Time Face Recognition System Using Eigen
Faces, International Journal of Electronics and Communication Engineering &
Technology (IJECET), Volume 4, Issue 2, March – April, 2013, pp. 72-79
[34] J. V. Gorabal, Manjaiah D. H, Texture Analysis for Face Recognition, International
Journal of Graphics And Multimedia (IJGM), Volume 4, Issue 2, May - December
2013,pp 20-30