Upload
cecily-foster
View
219
Download
0
Embed Size (px)
DESCRIPTION
Chang Gung University & National Taiwan University 3 Original Idea for the Problem Capture Image Land map We are interested in developing a System- on-Chip, Soc, which can capture image as well as produce vehicle lane map at the same time.
Citation preview
1
Automobile Lane Detection System-on-Chip Integrated with Mixed
Signal Mode CMOS Image Sensor
IEEE 9th International Symposium on Consumer Electronics (ISCE 2005)
Pei-Yung Hsiao¹, Hsien-Chein Cheng¹, Chun-Wei Yeh¹, Shih-Shinh Huang², and Li-Chen Fu²
¹ Department of Electronic Engineering Chang Gung University
² Department of Electrical Engineering National Taiwan University
2
Chang Gung University & National Taiwan University
OUTLINE
The Problem & Motive Brief Introduction to Algorithms Architecture and Circuits description Simulation and Results Conclusion
3
Chang Gung University & National Taiwan University
Original Idea for the Problem
Capture Image
Land map
We are interested in developing a System-on-Chip, Soc, which can capture image as well as produce vehicle lane map at the same time.
4
Chang Gung University & National Taiwan University
Motive The areas of Intelligent Transportation
System, ITS, include lane detection, obstacle recognition, vehicle detection, car following, etc.
Our goal in this investigation is to develop a CMOS imager to achieve real-time image capture and lane detection, simultaneously, for intelligent automotive driver awareness/assistance system.
5
Chang Gung University & National Taiwan University
Automotive IC Design
Vehicle Detection & Tracking
Driver Assistance System
Lane Departure Prevention
Automobile Lane Detection SoC
Built in
The target chip can be defined as a component device for intelligent vehicles.
6
Chang Gung University & National Taiwan University
Widespread Applications
The proposed imager without demanding extra ADC circuits for signal transformation is a single low-cost and compact chip for used in the thousands of consumer electronics not limited to ITS.
7
Chang Gung University & National Taiwan University
Introduction(1/4) From the referenced literatures, there are a lot
of vision-based lane detection algorithms proposed in recent 10 years [1-6] (1995-2004).
In 1995, Kluge and Lakshmanan [3] proposed the LOIS (Likelihood of Image Shape) lane detection, which is able to detect lanes even in situations with shadows or broken lanes by using a stochastic optimization procedure.
8
Chang Gung University & National Taiwan University
Introduction (2/4) In 1995, Broggi [5] proposed an edge-based ro
ad detection algorithm, while it is effective only for the well-painted road.
In 1999, Takahashi, etc. [4] divided the parameter space of the lane model to generate the lane marking patterns and then applied the voting scheme to find the lane boundary.
9
Chang Gung University & National Taiwan University
Introduction (3/4) In 2004, Huang, etc [1] proposed an on-board
vision system for lane recognition and front-vehicle detection to enhance driver's awareness.
Regarding to high recognition rate and hard-wired regularity, we adopted Peak-Finding based lane detection algorithm from Huang, etc [1], which has high recognition rate about 96%.
10
Chang Gung University & National Taiwan University
Introduction (4/4) According to Huang’s algorithm, we also
developed an auto-regulated threshold circuit to automatically adjust the threshold for the lane detector to adapted to different weather conditions.
11
Chang Gung University & National Taiwan University
Architecture and Circuit description Our chip can be divided into three parts, such
as analogue capturing and processing circuits, digital processing circuits and digital control unit.
The analogous circuits include 2-D pixel cell array, CDS module, 1-D Gaussian filter and Peak-Finding module.
The digital processing circuits are composed of Line Point Allocation module, column selector and row selector.
12
Chang Gung University & National Taiwan University
SOC for real-time image capture and lane detection
CMOS Image Sensor
Array
Gaussian Filter Module
Peak-Finding Module
Control
Unit
Lane-Point Finding Module
Row Selector
Lane-Point Output
Analogous Processing circuits
Digital Processing Circuits
CDS Circuit
Upper Region
Column
Selector
13
Chang Gung University & National Taiwan University
Pixel Cell & Sensor Array The developed sensor array consists of two types of p
ixel cells. Our sensor array prototype is made up of 64*64 effecti
ve pixels. The upper region containing 16*64 pixels is ignored in
back-end processing to promote the computing efficiency.
The other regions are horizontally partitioned into three sub-regions. Each sub-region consists of 16 rows.
The 12th row in the sub-region or in the upper region is defined as a 1-D sample array. Consequently, we have four 1-D sample arrays in our sensor array.
14
Chang Gung University & National Taiwan University
Reset
Photodiode
SEL
IpIpd
X
Reset
Photodiode
SEL
IpIpd
X
Ips
1.10 i-2,10 i-1,10 i,10 i+1,10 i+2,10Row Selector
1,11 i-2,11 i-1,11 i,11 i+1,11 i+2,11
1,12 i-2,12 i-1,12 i,12 i+1,12 i+2,12
1,64 i-2,64 i-1,64 i,64 i+1,64 i+2,64
CDS Circuit
Column SelectorCLK
Gaussian Filter (Current)
Gaussian Filter (Preivois)
IG(i,j)
IG(i-1,j)
Ips
Ip(i-1,j)Ip(1,j) Ip(i-2,j) Ip(i,j) Ip(i+1,j)
Ip(i+2,j)
64,10
64,11
64,12
64,64
Ip(64,j)
1,1 i-2,1 i-1,1 i,1 i+1,1 i+2,1 64,1
Normal cell
Sampling cell
15
Chang Gung University & National Taiwan University
Dual 1-D Gaussian Filters Each 1-D Gaussian filter includes 64 Gaussian
mask cells and a current divider. Each Gaussian mask cell consists of 3 current mirrors in 7 transistors and two OR gates.
The Gaussian filter module is used for smoothing the selected pixel by referring to a couple of right and left neighbors to eliminate noisy points in the original image.
16
Chang Gung University & National Taiwan University
Current Gaussian Filter
Previous Gaussian Filter
IG(i,j)
IG(i-1,j)
Ip(1,j) Ip(i,j) Ip(64,j)
S(1)
S(3)
S(2)
S(i)
S(i+
1)
S(i+
2)
S(i-2
)
S(i-1
)
S(64
)
S(63
)
S(62
)
S(1)
S(2)
S(63
)
S(62
)
S(61
)
S(60
)
S(i-1
)
S(i)
S(i+1
)
S(i-3
)
S(i-2
)
Dual 1-D Gaussian Filters
17
Chang Gung University & National Taiwan University
Peak-Finding Module (1/3) The 1st part of the Peak-Finding Module can a
ccumulate and average current, Iavg, from the aforementioned sample arrays, Ips.
The averaged current from sample array, Iavg, was generated according to the following equation.
),*1612(1 64,1
1,0
jiIpsn
In
ji
avg
18
Chang Gung University & National Taiwan University
IG(i,j)IG(i-2,j)
Auto-Regulated Threshold Circuit
Vrefn
Iavg
M1 M2
Isth
Threshold Mapping Circuit
Pp(i,j)
Threshold Mapping Circuit
Threshold Mapping Circuit
Threshold Mapping Circuit
Threshold Mapping Circuit
Vref
ITH
Vbias Ibias
Iavg
From Sample Array
Vcn
Ips
Irefn
P1 P2 P3 P4 P5 P6 P7 P8 P9
P73 P72 P71 P(i-1,j) P(I,j) P68 P67 P66 P65P101 P100
P129 P130 P131 P132 P133 P134 P135 P136 P137P102 P103
P38 P37
P35 P36
Pp(i,j)
Lb(i,j)Lane-Point Output
Peak-Point Output
Peak-Finding Module
Line-Point Allocation
19
Chang Gung University & National Taiwan University
Peak-Finding Module (2/3) The 2nd part of PFM is called as auto-regul
ated threshold circuit. It compares average current, Iavg, with four preconfigured currents, Irefn, and then produces the threshold current, Isth.
The total threshold current, ITH can be noted by the following equation.
, where
4
1n
sthbiasTH III
20 )(
2
2TCnn
M
Msth VV
LWI cu
20
Chang Gung University & National Taiwan University
Peak-Finding Module (3/3) Inside the auto-regulated threshold circuit, each
threshold mapping circuit control a threshold current. It can be noted by the following equation:
According to the 3rd part of the PFM, If the current pixel is a peak point, the output value will be 1, otherwise, it should be 0.
otherwise
jiIIjiIifjiP
GTHG
p
,0),1(),(,1
),(
otherwise
IIifV
avgrefn
Cn
,0,1
20 )( 5
1
1
1Tref
m
nn
M
Mrefn VV
R
RLWI
m
cu
, where
21
Chang Gung University & National Taiwan University
Line-Point Allocation The Lane-Point Allocation Module expressed a
s the following equation is composed of two digital functions, such as the line segment filter and the lane point selector.
Lane points, Lb(i,j), are obtained, and represented by only one pixel width in each row.
])1,(1,[,,1,3
3
3
3
mn
jmiPjniPjiPjiPjiL ppppb
22
Chang Gung University & National Taiwan University
1 2 64i (Col)
1 1
1 2 64
2 2
1 2 64
63 63
1 2
64 64
64 1 2
1
1 642 4097 4098
1,1 1,164,64Peak-Point
Pp(i,j)Lane-Boun Lb(i,j)
63,641,6464,6363,631,6364,6263,21,264,163,1---
--- --- 64,163,162,1 62,2 64,6263,62 64,6363,6362,63 64,6463,6462,64
Timing Cycle 65 66 128 3969 3970 4032 4033 4044 4096
i (Row)
clk R
clk C
Reset
The clock frequency of the Row Selector (clk R) is 64 times of the Column Selector (clk C). In this case, the frequency of the Column Selector is 25MHz and the frequency of the Row Selector is 0.78MHz.
Timing Diagram
23
Chang Gung University & National Taiwan University
HSPICE Simulation (1/2)
(a)(b)(c)
Peak-Point
(a) is the output current of the current Gaussian Filter. (b) is the output current of the previous Gaussian Filter. (c) is the results of the Peak-Finding Module.
1.5us
24
Chang Gung University & National Taiwan University
HSPICE Simulation (2/2)(a)
(b)
(c)
(d)
(a) is the simulation results of the current Gaussian Filter. (b) is the simulation results of the previous Gaussian Filter. (c) is the simulation result of the Peak-Finding Module. (d) is the simulation results of the Lane-Point Allocation Module.
5us 25us
25
Chang Gung University & National Taiwan University
Software Simulation in C
(a) Original image (512x512 > 320x240[1])
(b) Lane map points generated by Peak-Finding algorithm.
(c) Original image (64x64)
(d) Lane map points generated by Peak-Finding algorithm.
From [1], noises are to be removed by the followed post processing.
26
Chang Gung University & National Taiwan University
Experimental Results
(a) Original image in 32 * 32
(c) Result generated by our chip
(b) Result generated by software
27
Chang Gung University & National Taiwan University
Chip Layout in 64 * 64
28
Chang Gung University & National Taiwan University
Specification of The Proposed CMOS Imager
Item ValuesPixel Count 64(H) X 64(V)Pixel Size 18.45 um(H) X 21.8 um (V)Aperture Size 12.45 um(H) X 9.6 um (V)Fill Factor 29.7 %Image Size 1217.7 um (H) X 1455.05 um
(V)Chip Size 2191.4 um (H) X 2389.8 um
(V)Operation Clock 25MHzOperation Voltage 3.3 VPower consumption 159.4mW
29
Chang Gung University & National Taiwan University
Conclusion Our investigation includes a 2-D image sensor
array embedded with four modularized circuits: ----- four 1-D sample array by different pixel cell
design for accumulating the sampled currents; ----- dual 1-D Gaussian filers coupling as an
analogue image smoothing module; ----- an analogue design for Peak-Finding function
associating with a novel auto-regulated threshold operation;
----- a sophisticated digital implementation for Lane-Point allocation.
30
Chang Gung University & National Taiwan University
Conclusion, cont. A new current-mode mixed signal
design of CMOS image sensor integrated with Peak-Finding based lane detection algorithm is developed.
The proposed low-cost and one compact chip solution can grab the road images from the real world and successfully detect the lane markers simultaneously, in real time.
31
Chang Gung University & National Taiwan University
Reference (1/4)[1] Shih-Shinh Huang, Chung-Jen Chen, Pei-Yung Hsiao, and Li-
Chen Fu, “On-Board Vision System for Lane Recognition and Front-Vehicle Detection to Enhance Driver's Awareness”, IEEE International Conference on Robotics and Automation, vol. 3, 26 April – 1 May, 2004, pp.2456–2461.
[2] Yue Wang, Eam Khwang Teoh and Dinggang She, “ Lane detection using B-snake”, , International Conference on Information Intelligence and Systems, 31 Oct. - 3 Nov., 1999, pp.438 – 443.
[3] Kluge, K., and S. Lakshmanan,, “A Deformable-Template Approach to Lane Detection”, in I. Masaky, editor, Proceedings IEEE Intelligent Vehicle’95, Detroit, 25-26 Sept., 1995, pp.54-59.
[4] Takahashi, A., Ninomiya, Y., Ohta, M., and Tange, K., “A Robust Lane Detection Using Real-Time Voting Processor”, IEEE/IEEJ/JSAI International Conference on Intelligent Transportation Systems, 5-8 Oct., 1999, pp.577–580.
32
Chang Gung University & National Taiwan University
Reference (2/4)[5] A. Broggi, “Robust Real-Time Lane and Road Detection in
Critical Shadow Conditions,” Computer Vision, 1995, Proceedings., International Symposium on, 21-23 Nov. 1995 pp.353-358
[6] Li, Q., Zheng, N., and Cheng, H.,“ Springrobot: A Prototype Autonomous Vehicle and Its Algorithms for Lane Detection”, IEEE Transactions on Intelligent Transportation System, vol. 5, no. 4, Dec., 2004, pp.300-308.
[7] Coulombe, J., Sawan, M. and Wang, C., “Variable Resolution CMOS Current Mode Active Sensor”, IEEE International Symposium on Circuits and Systems, vol. 2, 28-31 May, 2000, pp.293 – 296.
[8] Tabet, M., Hornsey, R.,“ CMOS Image Sensor Camera with Focal Plane Edge Detection”, Canadian Conference on Electrical and Computer Engineering, vol. 2, 13-16 May, 2001, pp.1129 – 1133.
33
Chang Gung University & National Taiwan University
Reference (3/4)[9] Pei-Yung Hsiao, Yu-Chun Hsu, Wen-Ta Lee, Chia-Chun Tsai,
and Chia-Hao Lee, ”An Embedded Analog Spatial Filter Design of The Current-Mode CMOS Image Sensor”, IEEE Transactions on Consumer Electronics, vol. 50, no. 3, Aug., 2004, pp.945–951.
[10] N. Yang and G. Jianhong, “ A 256x256 Pixel Smart CMOS Image Sensor for Line Based Stereo Vision Applications”, IEEE Journal of solid state circuits, vol. 35, no. 7, July, 2000, pp.1055-1061
[11] Yuan, J. and Svensson, C., “High-speed CMOS circuit technique,” IEEE J. Solid-state Circuits, vol. 24, no. 2, 1989, pp. 62-70.
[12] Byungsoo Chang, Joonbae Park and Wonchan Kim,“A 1.2 GHz CMOS dual-modulus prescaler using new dynamic D-type flip-flops”, IEEE Journal of Solid-State Circuits, vol. 31, no. 5, May, 1996. pp.749-752
34
Chang Gung University & National Taiwan University
Reference (4/4)[13] Fish, A and Yadid-Pecht, o.,”CMOS current/voltage mode
winner-take-all circuit with spatial filtering”, The IEEE International Symposium on Circuits and Systems, vol. 3, 6-9 May, 2001, pp.636-639.
[14] Nakamura, J., Pain, B., Nomoto, T., Nakamura, T. and Fossum, E.R., “On-focal-plane signal processing for current-mode active pixel sensors”, IEEE Transactions on Electron Devices, vol. 44, no. 10, Oct., 1997, pp.1747 – 1758