Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Long Range Gaze Estimation with
Multiple Near-infrared Emitters
International Automatic Control Conference CACS 2017
Zhi-Xiang Liu, Zheng-Hong Ma, Ming-Chih Ho,
Jia-Yush Yen, Yung-Yaw Chen
智慧型微創手術內視鏡機器人系統研發-子計畫二:微創手術內視鏡機器人智慧型導航系統設計 計畫編號:MOST 104-2221-E-002-136-MY3
智慧型微創手術內視鏡機器人系統研發-子計畫二: 微創手術內視鏡機器人智慧型導航系統設計 計畫編號:MOST 104-2221-E-002-136-MY3
1
Outline
Introduction
Related works
– 3D eye gaze estimation
– Interpolation based eye gaze estimation
Methods
– Process of algorithm
– Multiple NIRs compensation method
Experiment and Results
Conclusions
References
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
2
Introduction
Gaze trackers capture human’s visual attention.
Applications:
– Human-computer interaction, human behavior research, eye
disease diagnosis…etc.
Difficulties in gaze estimation:
– degenerate accuracy under head motion
– large distance applications (larger than 1.0 m)
Goal:
– develop a long range eye signal control system that allow free
head motion.
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
3
Based on geometric relationship.
Identify line of eye sight.
Advantage: flexible, free head motion
Disadvantage: complicated calibration
Related work of 3D method
Fig.1 3D eye model 智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
4
Related work of interpolation method
Interpolation gaze estimation consists of two major parts:
– Feature extraction
Pupil / Iris center
Reference point: corneal reflection
– Determine gaze mapping functions
Obtain mapping relation from image coordinate to screen .
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
5
Overview of Gaze Estimation Algorithm
Methods
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
6
Process 1: Pupil Extraction
Fig.2 Bright pupil image Fig.3 Dark pupil image
Fig.4 pupil contour 智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
7
Process 2: Corneal Reflections extraction
Flow Chart:
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
8
Process 4: Mapping Function
Construct a mapping from image coordinate to screen
coordinate via PCCR (pupil center – corneal reflection)
vector.
The different mapping functions were investigated.
Second order polynomial is used as mapping function.
Ramanauskas, N. (2015)
Calibration technique:
1. Linear calibration model using 5 calibration points
2. Second order polynomial using 9 calibration points
3. Second order polynomial using 25 calibration points
4. Linear interpolation using 25 calibration points
5. Model based mapping
Averaged Mapping Error
(Second order polynomial)
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
9
Calibration Procedure
Calibration points on screen.
C1
C4
C7
C2
C5
C8
C3
C6
C9
Calibration Points
Coordinate
C1 (𝐶𝑥1, 𝐶𝑦1)
C2 (𝐶𝑥2, 𝐶𝑦2)
C3 (𝐶𝑥3, 𝐶𝑦3)
C4 (𝐶𝑥4, 𝐶𝑦4)
C5 (𝐶𝑥5, 𝐶𝑦5)
C6 (𝐶𝑥6, 𝐶𝑦6)
C7 (𝐶𝑥7, 𝐶𝑦7)
C8 (𝐶𝑥8, 𝐶𝑦8)
C9 (𝐶𝑥9, 𝐶𝑦9)
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
10
Multiple NIRs compensation method
Motivation: Using multiple NIRs to effectively reduce
the error under head motion situation.
Construct a mathematical for simulation.
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
11
Multiple NIRs compensation method
Two factors related to gaze point estimation error under
head motion circumstances:
– 1. Eye position
– 2. Gaze position.
Dividing the gaze range into 3 areas.
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
12
Multiple NIRs compensation method
Begin with estimating rough gaze point by using (G, D)
mapping function.
Choose the NIR and mapping function nearest to the
rough estimated gaze point , then compute the gaze point
again.
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
13
Experiment Setting
9 calibration points and 16 testing points
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
14
Experiment 1
Multiple NIR
Single NIR
MeanError(px)
82.36 87.58
std(px) 46.45 60.35
MeanError(cm)
2.22 2.36
MeanError
0.84 0.90
std 0.47 0.62
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
15
Experiment 2
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:NSC 101-2221-E-002-146-MY3
16
Results
X MOVE FB(1st) NFB(1st) FB(2nd) NFB(2nd) FB(avg.) NFB(avg.) improve percentage(%)
NO MOVE 0.72 0.72 0.81 0.79 0.765 0.755 -1.324503311
8cm 0.9 0.88 0.75 0.84 0.825 0.86 4.069767442
16cm 1.05 1.23 0.94 1.09 0.995 1.16 14.22413793
24cm 1.42 1.21 1.21 1.01 1.315 1.11 -18.46846847
32cm 1.02 1.64 1.14 1.79 1.08 1.715 37.02623907
40cm 1.11 1.63 1.07 1.64 1.09 1.635 33.33333333
48cm 1.41 1.66 1.48 1.6 1.445 1.63 11.34969325
56cm 1.71 1.82 1.48 1.59 1.595 1.705 6.451612903
64cm 1.58 1.79 1.43 1.47 1.505 1.63 7.668711656
72cm 1.27 1.58 1.12 1.01 1.195 1.295 7.722007722
Overall AVG. 1.219 1.416 1.143 1.283 1.181 1.3495 12.48610597
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
17 智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:NSC 101-2221-E-002-146-MY3
18
Experiment 3
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:NSC 101-2221-E-002-146-MY3
19
Result
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:NSC 101-2221-E-002-146-MY3
Z MOVE(cm) FB(1st) NFB(1st) FB(2nd) NFB(2nd) FB(avg.) NFB(avg.) improve percentage(%)
0 0.67 0.51 0.61 0.53 0.64 0.52 -23.07692308
8 0.67 0.8 0.58 1.01 0.625 0.905 30.93922652
16 1.28 2.44 1.51 2.79 1.395 2.615 46.65391969
24 1.87 3.42 1.9 3.37 1.885 3.395 44.47717231
32 2.5 3.85 2.97 4.18 2.735 4.015 31.88044832
Overall avg. 1.398 2.204 1.514 2.376 1.456 2.29 26.17476875
20 智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:NSC 101-2221-E-002-146-MY3
21
Conclusions
Our system successfully reduces the head motion
error about twenty percent.
Our system maintain the error within one degree over
the range of 16cm head movement in both x and z
direction.
We propose a new system configuration that can be
used in long rang and free head motion applications.
智慧型微創手術擴增實境系統研發-總計畫兼子計畫二:智慧型微創手術焦點區域即時影像系統
計畫編號:MOST 104-2221-E-002-136-MY3
22
References [1] Z. X. Liu, "A Novel Configuration and Algorithm of Long Distance Gaze
Estimation System for Gaze Point Error Compensation.", Department of Electrical
Engineering, College of Electrical Engineering and Computer Science, National
Taiwan University, Master Thesis, July 2016.
[2] Suh, K. H., Kim, Y.-J., Kim, Y., Ko, D., & Lee, E. C. (2015). Monocular Eye
Tracking System Using Webcam and Zoom Lens Advanced Multimedia and
Ubiquitous Engineering (pp. 135-141): Springer.
[3] Ramanauskas, N. (2015). Calibration of video-oculographical eye-tracking
system. Elektronika ir Elektrotechnika, 72(8), 65-68.
[4] Jixu Chen,Qiang Ji “3D Gaze Estimation with a Single Camera without IR
Illumination” ,2008 IEEE
計畫編號:MOST 104-2221-E-002-136-MY3 智慧型微創手術內視鏡機器人系統研發-子計畫二:微創手術內視鏡機器人智慧型導航系統設計
23
Thank you for your attention