Proximity Sensing Based on Dynamic VisionSensor for Mobile Devices

Jae-Yeon Won, Hyunsurk Ryu, Tobi Delbruck, Fellow, IEEE, Jun Haeng Lee, and Jiang Hu, SeniorMember, IEEE,

Abstract—Dynamic Vision Sensor (DVS) is a sensor whichdetects temporal contrast of brightness and has the fastestresponse time compared to conventional frame-based sensorswhich detect static brightness per every frame. The fastestresponse time allows fast motion recognition which is a veryattractive function as a view of consumers. Especially, its lowpower consumption due to the event-based processing is a keyfeature for mobile applications. In recent Smartphone basedon touch screen, a proximity sensor is equipped to preventmalfunction due to undesired contacts with skin while calling. Inaddition, the main processor stops operation of the touch screenand turns display off when any object is close to the proximitysensor to achieve minimizing power consumption. Consideringthe importance of the power consumption and reliable operations,it is certain that proximity sensing is an essential part in touchscreen-based Smartphone. In this paper, a design of proximitysensing utilizing DVS is proposed. It can estimate the distancefrom DVS to an object by analyzing the spatial information ofthe reflection of additional light source. It also uses a patternrecognition based on time domain analysis of the reflectionduring turning on of the light source to avoid wrong proximitydetection by noises such as other light sources and motions. Thecontributions of the proposed design are in three parts. First,it calculates accurate distance in real time only with spatialinformation of the reflection. Second, the proposed design caneliminate environmental noises by using pattern matching basedon time domain analysis while conventional optical proximitysensors, which are mainly used in Smartphone, are very sensitiveto environmental noises due to that they use the total amountof brightness for certain period. Third, our design replacesconventional proximity sensors with holding additional benefitsthat it utilizes the advantages of DVS.

Index Terms—Proximity Sensor, Dynamic Vision Sensor, Pat-tern recognition, Smartphone, Mobile Devices.

I. INTRODUCTION

THE mobile phone has evolved from an electrical devicewhich had only a purpose of communication to an essen-

tial device with an important role to manage all of our dailyneeds. Especially, market share of Smartphone that are mobilephones with computing functions has increased over the worldbecause of its various properties such as communication,internet media, mass media, video camera and etc. In addition,

Copyright c©2014 IEEE. Personal use of this material is permitted. How-ever, permission to use this material for any other purposes must be obtainedfrom the IEEE by sending a request to pubs-permissions@ieee.org

This work was supported in part by Samsung Advanced Institute ofTechnology, Samsung Electronics Co. Ltd., Republic of Korea.

J.-Y. Won is with the department of Electrical and Computer Engineering,Texas A&M University, TX 77843 USA (email: jaeyeon9@neo.tamu.edu).

H. Ryu is with Samsung Advanced Institute of Technology, Yongin 446-712Korea (email: eric ryu@samsung.com).

T. Delbruck is with Institute of Neuroinformatics, University of Zurich, andETH Zurich, Zurich, CH-8057, Switzerland (email: tobi@ini.phys.ethz.ch).

J. H. Lee is with Samsung Advanced Institute of Technology, Yongin 446-712 Korea (email: junhaeng2.lee@samsung.com).

J. Hu is with the department of Electrical and Computer Engineering, TexasA&M University, TX 77843 USA (email: jianghu@ece.tamu.edu).

many sensors are being used in the mobile phones to providemore useful features such as motion recognition [1], [18], [20].

Recently, Dynamic Vision Sensor (DVS) which mimicshuman optic nerve has been researched as a power efficientand fast responding sensor by Lichtsteiner et al [5], [14]. Itdetects only the changes of brightness, and has the fastestresponse speed among image sensors so far. Its fast responsetime enables DVS to be used in various applications such asmotion recognition. Meanwhile, a proximity sensor is alsoequipped to avoid wrong operations by unintended contactswith skin while calling in a touch screen-based Smartphone[12]. Among various proximity sensors, optical proximitysensor is commonly used. And a research has been proposed toexpand its functionality for additional features such as motionrecognition [3]. It outputs infrared light for certain periodsand measures the absolute amount of the lights coming backby reflection. It assumes that an object is close if the lightscoming back is over certain amount. However, it loses detailedinformation by measuring the aggregated absolute values oflights and hard to differentiate environmental noises. Also,it is very sensitive to quality and setup angle of transparentplastic cover for protection and becomes a main reason thatthe fraction defective goes up at the production stage.

The proposed proximity sensing design which utilizes timedomain analysis is robust to the environmental noises, andcan decrease fraction defective in production process. Theproposed design is constructed with a DVS and a light sourcephysically. It suggests two main algorithms to detect proximity.One is distance estimation based on the spatial information ofthe reflection and the other is temporal pattern recognitionof DVS events with an additional light source. A DVS eventis generated when the DVS detects that brightness of apixel is changed. The reflection due to the light source isplaced at different spatial position according to distance tothe reflecting object. It can calculate distance based on thespatial information of DVS events. Also, temporal patternrecognition is used to remove the environmental noises dueto other light sources. It analyzes the temporal patterns of theevents during turning the light source on or off. In addition,when the proposed proximity sensor is equipped and replacesconventional proximity sensor, various and attractive featuressuch as motion recognition using DVS can be utilized as well.

This paper explains conventional proximity sensors in sec-tion II. The concept of DVS and the proposed design forproximity sensing which utilizes DVS is explained in sectionIII. Section IV shows experiment results and performanceanalysis. The conclusions are drawn in section V.

II. CONVENTIONAL PROXIMITY SENSORS

Various proximity sensors have been researched and theirapplications are various as well. Magnetic proximity sensor isone of non-contact sensors and generates magnetic waves [11],

1

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS , FEBRUARY 2015

[13]. If an object is close to the magnetic proximity sensor, aninduced current flows in the object due to the magnetic wave.And the sensor detects impedance changes of the detection coilby the induced current of the object. This sensor detects prox-imity of only metals and is affected easily by other magneticsubstances. Inductive proximity sensor and capacitive sensorare also used to detect surface of metallic objects [8], [17].They are constructed with various switches which have ferritecore, oscillator, detector and solid state switch. If an objectis close from the sensor, the switch is shorted. Otherwise, itis opened. They are mainly used for metal detectors, trafficlights, car washes, and checking occupancy.

Optical proximity sensor is constructed with an emitterwhich is a light source and a receiver. It is mainly used inthe applications of Smartphone [7]. The receiver can perceiveexistence of light. The light source emits light and the receiversuch as photo-transistor operates according to the amount oflight reflected back. In case that an object is far from theproximity sensor, the amount of light back is almost zeroand it decides that the object is not close to the sensor.Because it uses the absolute amount of aggregated light forcertain period, however, the accuracy is not high and it is hardto differentiate the effects by environmental noises that areunintended brightness changes. Ultrasonic proximity sensoralso has an emitter and a transmitter [2], [9]. Instead oflight for optical proximity sensor, it uses ultrasound waveas a source. It observes travel time of the wave which isreflected back by objects. It can estimate the distance withthe time measured, but it is suitable for measuring relativelylong distance. In addition, various approaches for specificapplications have been conducted [4], [6], [10], [15], [19].

Therefore, we propose a design of proximity sensing whichis robust to noises and has suitable for mobile applications.

III. PROPOSED PROXIMITY SENSOR DESIGN

Dynamic Vision Sensor generates events, which we termDVS events here, when DVS detects the change of brightnessof certain pixels and it is over certain threshold. The DVSevents include the type of event (ON or OFF) and its addresswhich is spatial information. Regarding the type of event, DVScan detect whether the brightness is increased or decreased.We term on-events for the DVS events when the brightness isincreased and off-events for the DVS events when the bright-ness is decreased. The proposed method uses an additionallight source to generate intended brightness changes. When anobject is at certain distance from DVS, it detects the brightnesschanges of the reflection on the object through controllingthe light source. The spatial address information and temporalpatterns of the reflections are used in the proposed method. Insection III-A and III-B, it is explained that DVS can measuredistance of an object through modulation by an additionalsynchronized light source and the spatial information of thereflections. However, only with the spatial information by thesynchronized light, it cannot differentiate whether the eventsare caused by the light source or other sources if there aremany light sources around. To minimize the effect of theother light sources which are regarded as noises, this paperalso proposes a method by analyzing the temporal pattern ofthe reflection in section III-C. It utilizes the pattern of thereflected light during the time when the light source is turnon or off. Regarding the integration of these two methods, it

determines the proximity which means an object is close toDVS only when those two methods determine the proximity.

A. Real time Distance EstimationThe proximity sensing design proposed in this paper is

constructed with a DVS and a light source such as laser or lightemitted diode (LED). Main controller turns the light source onand off periodically and changes the brightness enough to bedetected by DVS. When an object is at certain distance fromDVS, the light is reflected from the object. DVS detects thechanges of the brightness and outputs the spatial informationof the reflection. When the distance to an object varies, thespatial position which is detected through DVS also varies.The proposed method calculates the distance based on thespatial information of the DVS events. To measure the distancemore accurately, rectilinear light source is required becausethe diffusion of light is less. In this section, it assumes that anideal rectilinear light source is used as a light source in theproposed proximity sensing design. In addition, it explains amethod to use more realistic light source in section III-B.

S

α1 α2

d

object

DMIN

DVS Light Source

a

b

c

rL rR

ζ

TMIN

(a) Settings of a DVS and a light source

Reflection

sL sR

(b) Spatial address

Fig. 1. An illustration on how the design works.Fig. 1 illustrates how the proposed proximity sensor works.

The distance, S and the angle, α2 between the light sourceand DVS can be adjusted for various applications. If thevalue of S is small, the variation of the address of the pixelswhich are detected by the DVS is not large enough while thedistance of an object from DVS is changed. In other words,it is preferred to set the distance large enough to derive largevariations of the spatial information. Although larger S allowsmore distinct distance estimation, the distance is limited bythe design of applications because applications have their ownsize limitation. Also, larger S affects on the blind distance,DMIN which means cannot be detected by DVS. The angleα2 is related to DMIN and the minimum size of an object thatcan be detected by the DVS. The larger α2 derives the largervariance of the address of the pixels, but cannot detect smallobjects at the center of the DVS while decreasing DMIN . If anobject which should be detected are large enough, the variableα2 can be large enough because it does not need to considersmall objects. Therefore, the design parameter S and the angleα2 are set according to the specifications of the applications.

The angle α1 is viewing angle of DVS, and given by thespecification of the DVS itself. Fig. 1 also describes variablesto calculate the distance to an object from DVS, d. In Fig. 1(a)which is side view of the design, the reflected image is locatedwith the rate of rL and rR which are physical distance amongthe total area that can be detected by DVS. Fig. 1(b) showsspatial address of the reflection in the DVS output. And, (1) isestablished where sL and sR are the distance from the bothsides to the reflection in the DVS output. Equation (2) to (8)

2

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS , FEBRUARY 2015

explain how to calculate the distance d by using sL and sR.The variable a, b, and c can be calculated as shown in (2), (3)and (4), respectively.

rL : rR = sL : sR (1)

a =S

tanα2(2)

b = d− a = d− S

tanα2(3)

c = b · tanα2 =

(d− S

tanα2

)· tanα2 = d · tanα2−S (4)

The physical distances, rL and rR, are calculated as shownin (5) and (6). Equation (7) is also derived by using (1), (5)and (6).

rL = d · tanα1 − c (5)

rR = d · tanα1 + c (6)

(d · tanα1 − c) : (d · tanα1 + c) = sL : sR (7)

In (8), a distance from DVS to the object, d, can be acquiredbecause S, α1 and α2 are set by the specification and sR andsL are the results observed which are known values.

d =S(sL+ sR)

sL(tanα1 + tanα2)− sR(tanα1 − tanα2)(8)

Considering that α1 is fixed by the specification of DVSitself, S and α2 are parameters that can be adjusted appro-priately to applications. In addition, variable S has certainlimitation to be adjusted due to the size of the application.Thus, variable α2 is a main adjustable parameter and deter-mines detectable size, T and detectable distance, D. Withinthe given specifications, α2 has a lower and a higher bound.Within the settings as shown in Fig. 1(a), minimum detectablesize, TMIN where the object is at distance ξ is acquired by(9). The size of an object which can be detected, T should begreater than or equal to TMIN .

TMIN = 2(ξ · tanα2 − S) ≤ T (9)

Also, the minimum distance which can be detected withinthe settings of Fig. 1(a), DMIN , is calculated by (10). Thedetectable distance, D is greater than or equal to the givenspecification, DMIN .

DMIN =S

tanα1 + tanα2≤ D (10)

Assuming that α1, ξ, T , D and S are fixed by the specifi-cations, the angle between the DVS and the light source, α2 isbound as shown in (11) and can be adjusted for applicationswithin the range.

tan−1

(S

D− tanα1

)≤ α2 ≤ tan−1

((T

2+ S

)1

ξ

)(11)

B. Applying Various Light SourcesIn the proposed design, the more rectilinear propagation and

the less diffusion of the light source provide more accuratespatial information of the reflection. In other words, it detectsmore distinct spatial information when the light source whichhas more rectilinear property such as when laser is used.Considering production cost and productivity, however, it maygive more benefits when the less rectilinear light source suchas light emitted diode (LED) is used. Therefore, this section

λ

DVS

LED

Lens

d1

d2

d3

R1

R2

R3

(a) Focal point

R3 R1

R2

Active Region

(b) DVS output

Fig. 2. Minimization of light diffusion using a convex lens.

explains a method to use less rectilinear light sources withmore diffusion of light for realistic equipment.

In the example of LED as a light source which has morediffusion of light, it can focus lights diffused through convexlens to a specific point. Fig. 2 shows that a convex lensconcentrates the light diffused from the LED to one spot ata certain distance, d2. Although it concentrates the light toone spot at a certain distance through a lens, it cannot avoidthe diffusion of light at other distances. To decide the certaindistance which has focal point, it is placed at the mediandistance of the critical range, λ to measure the distance inthe critical range accurately and minimize the error due to thelight diffusion. For the distance out of the critical range, it doesnot have to acquire the spatial information of the reflectionbecause it can recognize that an object is at a far distanceonly with little events by reflected light.

Fig. 2(a) shows how to decide the focal point to minimizethe diffusion of the light using a convex lens. When the criticalrange, λ which is desired to be observed is the difference ofd1 and d3, it sets the focal point at the median point of d1 andd3. Therefore, the focal point, d2 is at (d1 + d3)/2. Althoughit sets the focal point to middle point, the diffusion of the lightstill exists in the critical range. In that case, it calculates onededicate spatial information of the reflection diffused throughaveraging all spatial information in the diffused area. Also, itcan eliminate other DVS events out of the active range becauseit averages DVS events only in the active region. Fig. 2(b)shows the active region which is the area that reflections ofthe light source can be placed. If an object is further than thecenter point, its reflection is placed on the left active regionof the DVS output. If it is closer than the center point, itsreflection is placed on the right side of active region. Thus,the distance can be measured based on the spatial informationof the DVS events even when the reflection of light is diffused.

C. Pattern Recognition for Light Source Turn-On/Off Time

This section explains how the temporal pattern of the reflec-tion is used to increase the robustness of proximity detection.In the proposed technique, the light source is modulated byswitching it on and off. Then, the observation of correspondingreflection pattern can tell if the measured light is reflectedfrom the source or from ambient light noises. The brightnesschanges of the light are generated for rising time (Tr) whenthe light source is turn on and for falling time (Tf ) when it isturn off. When the light intensity is changed, DVS generateson-events which are DVS events have positive polarity for thebrighten pixels. Also, it generates off-events which are theDVS events have negative polarity for the darken pixels. Toacquire realistic rising time and falling time of the light source,photo detector is used to measure the transition time of the

3

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS , FEBRUARY 2015

Fig. 3. Brightness transition of the LED used in this paper

light source. Fig. 3 shows a LED control signal and brightnessof the LED for the transition time measured by photo detectorand digital storage oscilloscope. The rising time, Tr of theLED is set to 1 ms. Therefore, the controller turns the LEDon for 1 ms and turns the LED off for remaining period.

When the light source is turned on and off, the transitionof the DVS events are shown in Fig. 4. The x-axis is timefrom turning a light source on and the y-axis is the numberof the DVS events. In Fig. 4, the number of on-events hasbeen acquired for 100 times of the light on/off operation.Only on-events are observed because the transition of the on-events by turning the light source on is more obvious than theoff-events while minimizing the complexity of the algorithm.The max and min represent maximum and minimum valuesof the number of events among 100 times at each time. Inthis experiment, the transition of the light is measured a 2msinterval, during which the light is on for 1ms and off forthe other 1ms. To acquire the pattern for 2 ms, the maincontroller counts the number of events per 100 us for 2 ms. Thepattern shown in Fig. 4(a) was acquired by the experiment thatDVS observed the light source directly without any reflectedobject to minimize reflecting effects. In addition, it forms acertain pattern of on-events which can be used to differentiateother effects except by the intended light source control. Also,certain patterns of reflection by an object which is at certaindistance from DVS with the light source control are shownin Fig. 4(b). The distance to an object is set to 50 mm,and the light source and DVS direct to the reflecting object.The patterns which are shown in Fig. 4(b) are defined asstandard patterns which is used to differentiate whether theDVS event occurs due to the controlled light source or noises.The distance setting for standard pattern can be varied bythe specifications and the 50 mm used in the experiment isreasonable for mobile devices. As shown in Fig. 4(b), thepattern can differentiate itself from noise ambient light. Inother words, where an object is close to the light source, thepattern shows a similar pattern to the standard pattern whileit is regarded as by other light sources which are noises if apattern observed has a different type to the standard pattern.Even though the patterns vary slightly according to the kindsof objects and distance, the patterns are formed quite similarlyto the standard pattern. The variations of the patterns byobjects and distance will be discussed more in section IV.

The reflecting patterns are various according to reflectingobject, background light, the distance to objects and etc. Itis hard to determine that the pattern belongs to the standardpattern only with the absolute number of DVS events. Thispaper proposes an efficient algorithm to determine whetherthe temporal pattern matches the standard pattern only with acommon rule which utilizes the transition aspect of the pattern.

Fig. 5 explains how to determine the proximity based on

0

50

100

150

200

250

300

0.0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

maxmin

(a) By the light source

0

20

40

60

80

100

0.0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(b) By the reflected light

Fig. 4. The patterns of the DVS on-events.

0

20

40

60

80

100

120

0.0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

(a) Original temporal pattern

0

20

40

60

80

100

120

0.0 0.5 1.0 1.5 2.0

4-A

vera

ged

on

-even

ts

Time (ms)

(b) 4-Averaged pattern

-30

-20

-10

-30

-20

-10

0

10

20

30

40

50

0.0 0.5 1.0 1.5 2.0 T

he a

mo

un

t o

f tr

an

sit

ion

Time (ms)

(c) Transition of the pattern

S0 S1

reset

+ S2

S3

S4S5

SFST

+

+

-

--

+

- - -

++

S6

+-

(d) State diagram

Fig. 5. Proposed pattern recognition.

the transition aspect of the temporal pattern, and the distanceto an object is set to 20 mm. By turning the light sourceon, on-events are generated. Fig. 5(a) shows that there aresome irregular transitions. The irregular pattern may result inmisinterpretation. To avoid the effect of the irregular transition,it uses an averaged pattern as shown in Fig. 5(b). Here othersmoothing functions such as Gaussian function can be used,but the averaging method used in this paper is simple to beimplemented into hardware and effective enough.

ASm =

{1N

∑N−1n=0 CSm+n if m ≤M −N + 1

0 otherwise(12)

Equation (12) describes calculation of the averaged pattern.The ASm is the mth averaged sample and the CSn is thenth current sample. N is the number of samples used inthe averaging and M is the number of maximum samples.Fig. 5(a) shows the original samples, CSn and Fig. 5(b) showsthe averaged samples, ASm. AS0 is regarded as 0. Consideringthat larger averaging causes more information loss, M is setto 20 and N is set to 4 to acquire the smoothing pattern whilekeeping the original pattern. Fig. 5(c) shows the transition ofthe averaged samples, and is acquired by difference of ASm

and ASm + 1. A few positive transitions are followed by afew negative transitions, and this property is used as a commonrule to decide whether the DVS events are caused by an objectwith the intended light control or noises. In the example ofFig. 5(c), it shows four consecutive positive transition valuesfrom 0 ms to 0.3 ms and eleven consecutive negative transitionvalues from 0.4 ms to 1.4 ms. The number of positivetransitions and negative transitions can be different accordingto many conditions. However, the property which includes

4

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS , FEBRUARY 2015

-30

-20

-10

0

2

4

6

8

10

12

14

16

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

(a) Original temporal pattern

-30

-20

-10

-4

-2

0

2

4

6

8

0 0.5 1.0 1.5 2.0

Th

e a

mo

un

t o

f tr

an

sit

ion

Time (ms)

(b) Transition pattern

Fig. 6. The patterns of the DVS events by random light.

some consecutive positive and negative transitions is held onregardless of various conditions. The property is experimentedand verified at section IV. For objects at a far distance, it canbe differentiated by the spatial address of the reflection whichis described in section III-A although it satisfies the commonrule. Fig. 5(d) shows a diagram of the proposed method in casethat it checks more than two consecutive positive transitionsand more than three consecutive negative transitions to decidewhether it matches to the standard pattern. ST means it decidesthe object is close while SF indicates the object is not close.As shown in Fig. 5(c), it shows four positive transitions from0 ms to 0.3 ms, and the state of Fig. 5(d) which begins atS0 moves to S3 because there is no negative transition. And,it has negative transitions from 0.4 ms to 1.4 ms, and thestate moves to next state whenever it has a negative transitionand finally to ST because it has more than three consecutivenegative transitions.

Fig. 6(a) and 6(b) show original DVS on-events and thetransition of the 4-averaged pattern by random light sources,respectively. It shows irregular DVS events for measuring timebecause the random light is not synchronized by the lightsource control. In the example of Fig. 6(b), there are twopositive transitions at time 0 ms and 0.1 ms and a negativetransition at time 0.2 ms, the state moves from S0 to S2 due tothe first two positive transitions, and then it falls to SF becauseit is followed by a negative transition. Finally, it determinesthat there is no object which is close to DVS because it doesnot belong to the standard pattern by the synchronized lightsource control. Also, a threshold can be used to comparewith the total number of events generated. In other words,even though it matches the standard pattern by the proposedalgorithm, it determines that there is no object if the totalnumber of events is less than the activity threshold becausethe number of events is not enough to decide its proximity.

The proposed method utilizes spatial and temporal patternof the reflection together with light source modulation and aDVS which detects the changes of brightness to determineproximity. The advantages and performance of the proposedmethod are explained with experiment results in section IV.

IV. EXPERIMENT RESULTS

The proposed algorithm has been implemented and evalu-ated on FPGA tool environment. A dynamic vision sensor isconnected to GPIOs of the FPGA and the output of the lightsource is controlled by the FPGA. In the experiments, a 128by 128 asynchronous dynamic vision sensor is used [14].The event latency of the sensor is 15 us at the minimum. Itsviewing angle is total 90◦ which is 45◦ (α1) to left side and45◦ to right side with a lens which has 35 mm C-mount lens/ focal length 4.5 mm. A LED is used as a light source with a

(a) 20 mm (b) 50 mm (c) 100 mm (d) 150 mm

Fig. 7. Estimation of the distance based on the address of the reflection.

lens and the LED are located at horizontally 25 mm (S) apartfrom the DVS at the same vertical position. We also analyzedsmaller setting distance (S), 10 mm, for more various and tinyapplications in section IV-G. The setup angle, α2, is set to30◦ to generate higher sensitivity and fully support detectablerange regarding that the range of α2 is 14◦ ≤ α2 ≤ 36◦ whenthe detectable range is 20 mm to 200 mm by equation (11).The distance to the focal point, d2 is set to 50 mm.A. Distance calculation using the spatial information

Figure 7 shows spatial information of reflections, which isthe DVS output obtained by using a software to visualize theDVS events for certain period. White dots represent lighterpixels while black dots represent darker pixels in the greysquare which is region that DVS can detect. To evaluate onlythe effect of reflections by the synchronized light source,the experiments are executed without any other light sourcesexcept a room light. As shown in Fig. 7(a), the position ofthe reflection is on the right side of the DVS output whenthe reflecting object is at 20 mm away from DVS. When thereflecting object is at 50 mm away from DVS, Fig. 7(b) showsthat the reflected light is at center of the DVS output. In casethat the object is further than 50 mm from DVS, reflectedlights are observed on the left side of the DVS output asshown in Fig. 7(c) and 7(d). The experiments prove that theproposed design can calculate the distance only with the spatialinformation of the reflection. And, the resolution error is lessthan 1%, ±0.45mm when an objest is 50 mm apart and setupangle of light source (α2) is set to 30◦.

B. Robustness to background lightsFig. 8 shows the effects of background lights. Fig. 8(a) is

the temporal pattern by an object at 20 mm apart when thebackground light is turned on. Fig. 8(b) is the temporal patternwhen the background light is turned off. Fig. 8(c) and 8(d)show the transition of 4-averaged pattern. Comparing Fig. 8(b)with 8(a), more DVS events are generated in dim environmentbecause the change of the LED brightness is dominant indark environment. However, the temporal patterns are highlycorrelated each other and share the same trait that a fewpositive transitions are followed by a few negative transitions.In other words, Fig. 8(c) and Fig. 8(d) show that the bothpatterns have four positive transitions and more than threenegative transitions. Therefore, the proposed method can beused to detect proximity in presence of background lights.C. The effect of transparent plastic cover

Fig. 9 shows the effect of the transparent plastic cover forprotection. Fig. 9(a) shows the position of the reflection wherean object is at 50 mm apart and a transparent plastic cover isequipped at 5 mm apart from the DVS. As shown in Fig. 9(a),the transparent plastic cover does not affect the performancebecause it is placed closer than the minimum detectabledistance, DMIN , and the reflection by the cover is not detected

5

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS , FEBRUARY 2015

-30

-20

-10

0

50

100

150

200

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(a) Original pattern w BL

-30

-20

-10

0

50

100

150

200

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(b) Original pattern w/o BL

-30

-20

-10

-40

-20

0

20

40

60

80

100

0 0.5 1.0 1.5 2.0

Th

e a

mo

un

t o

f tr

an

sit

ion

Time (ms)

max

min

(c) Transition pattern w BL

-30

-20

-10

-40

-20

0

20

40

60

80

100

0 0.5 1.0 1.5 2.0 The

amount

of

tran

sition

Time (ms)

max

min

(d) Transition pattern w/o BL

Fig. 8. The effects by background light (BL).

(a) Reflectionwith cover

0

20

40

60

80

100

120

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(b) Temporal patternwith cover

0

20

40

60

80

100

120

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(c) Temporal patternwithout cover

Fig. 9. The effect of a transparent plastic cover.

by the DVS. Fig. 9(b) and Fig. 9(c) show the temporal patternof DVS events with a cover and without a cover, respectively.Even though temporal pattern is affected by a cover, it sharesthe same trait that a few positive transitions are followed bya few negative transitions. Therefore, the proposed methodwhich is temporal pattern recognition with spatial informationis robust to the effect due to the transparent plastic cover.

D. The effect of noisesIn this section, it is experimented that the proposed method

is robust to environmental noises by moving objects and otherlight sources. Fig. 10 and 11 show patterns of the temporalon-events with hand gestures and unintended light sources,respectively. Fig. 10(a) shows temporal on-events without thelight source control. It shows that temporal pattern of handgesture forms a different aspect to the standard pattern whichis shown in Fig. 4(b), and can be regarded as a noise. Theeffects by the hand gesture with intended light control havebeen observed in Fig. 10(b). It shows on-events when a hand ismoving at 50 mm apart with light source control. Even thoughthe light intensity changes by shaking a hand are generated, itdoes not affect patterns by the intended light control becauseit measures only for 2 ms since it begins light control.

The effects of noise lights are analyzed in Fig. 11. Fig. 11(a)shows the effect of random lights without a light source con-trol. The temporal pattern is quite distinct from the standardpattern, and can be recognized as noises easily. Fig. 11(b) showthe effect of random lights with a reflecting object at 20 mmapart with a light source control. Even though random lightsaffect the DVS pattern, the impact for the measuring timeis negligible because the random lights are not synchronized

0

10

20

30

40

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(a) Brightness changes and temporal pattern of a handshaking at 50 mm apart without a light source control

0

50

100

150

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(b) Brightness changes and temporal DVS pattern of a handshaking at 50 mm apart with a light source control

Fig. 10. The effect of hand gestures.

0

20

40

60

80

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(a) Brightness changes and temporal DVS pattern of ran-dom lights without a light source control

0

50

100

150

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(b) Brightness changes and temporal DVS pattern of anobject at 20 mm apart with random lights and a light sourcecontrol

Fig. 11. The effects of random lights.

by the light source control. By increasing the distance to theobject, random lights effects may become dominant relativelycomparing the effect of the light source control. Although theeffects of noises become dominant for objects far away fromDVS, disturbance by the random lights can be recognized byusing the proposed temporal pattern recognition method andregarded as noises.

E. The effect of Reflecting Objects

Fig. 12 shows that temporal patterns of DVS events byvarious objects. There is a large variety of temporal patternsas the objects may have all kinds of different colors, texture,surface roughness, etc. Fig. 12(a) and 12(b) show the effect ofcolor of the reflecting object. White color shows more similarpattern to the standard pattern comparing black color becausethe reflectance of white color is higher than black color. Eventhough different color show slightly different patterns, bothof them share the same trait that a few positive transitions

6

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS , FEBRUARY 2015

-30

-20

-10

0

10

20

30

40

50

60

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(a) White paper

-30

-20

-10

0

10

20

30

40

50

60

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(b) Black paper

-30

-20

-10

0

10

20

30

40

50

60

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(c) Glazed wood

-30

-20

-10

0

10

20

30

40

50

60

0 0.5 1.0 1.5 2.0

Th

e n

um

ber

of

on

-even

ts

Time (ms)

max

min

(d) LCD screenFig. 12. The temporal patterns of various objects.

TABLE ISUCCESS RATE OF PROXIMITY DETECTION.

Scenario I Scenario II Scenario III Scenario IVOptical 100.0% 100.0% 95.5% 42.9%DVSS 100.0% 100.0% 95.7% 15.9%DVSST 100.0% 87.5% 92.1% 85.8%DVSSTL 100.0% 100.0% 94.4% 81.5%

are followed by a few negative transitions. Other reflectingobjects, a glazed wood and a LCD screen are experimentedin Fig. 12(c) and 12(d), respectively. The glazed woodshows almost the same as standard pattern, but the LCDscreen distorts the light extensively due to its inner structure.Regarding that it focuses on human hands or face, the proposeddesign for proximity sensing is useful although there are somedistortions at some artificial objects.F. Performance Analysis

In this section, the performance of the proposed design isevaluated. The distance to decide the proximity is set to 50mm. In other words, it determines that an object is close whenit is closer than 50 mm. We use a sequence of more than twopositive transitions followed by three negative transitions as apattern to identify the reflections from the modulated light.

Table. I shows the success rate of proximity detection. Thesuccess rate is defined as the number of correct detectionover the number of detection trials. For the detection trial,we conducted 300 times of proximity detection. We analyzedthree proposed methods through four different scenarios.

• DVSS : Only spatial information of the reflection applied.• DVSST : Temporal pattern recognition shown in Fig. 5

with spatial information of the reflection.• DVSSTL: Allows a single negative transition among the

consecutive positive transitions in DVSST .� Scenario I: A hand not moving at 30 mm apart from DVS

with room lights.� Scenario II: Same settings to Scenario I w/o room lights.� Scenario III: A hand shaking at 30 mm apart from DVS.� Scenario IV: An electrical fan at 300 mm apart from DVS

and the reflection by the fan is placed on the right sideof DVS output intentionaly.

Scenario I and II are for normal operations at day and nighttime, respectively. Correct detection of Scenario I is that theobject is close. It shows 100% success rate of proximity

detection. Regarding the scenario I is the most commonsituation of calling, it proves that the proposed design is veryeffective. The success rate in Scenario II can be increasedup to 100% by applying a loose condition which allows asingle negative transition among consecutive positive transi-tions for temporal pattern recognition. Scenario III and IVare to simulate worst situations. In Scenario III, DVS detectsreflection of the light source and effects by a hand shaking.Its correct operations should detect the proximity. It shows94.4% success rate when DVSSTL is applied. It is also highenough regarding the movement of face and hand is notmuch while calling. Scenario IV is close to the worst case.When the reflection of the events is on right side of DVS,it decides the proximity which is undesirable if it uses onlyspatial information, DVSS . By applying DVSST or DVSSTL,it can differentiate the effect of the fan by using temporalpattern matching because the temporal pattern of the fan is notsynchronized by the light control. Therefore, applying spatialinformation and temporal pattern of the reflection together, itcan determine the proximity by using light source control.

In addition, we experimented the common optical proximitysensing method which is used in Smartphone application [16].The conventional optical proximity sensor detects proximitybased on aggregated brightness change by an additional lightsource. The brightness change is calculated as a difference ofbrightness when the additional light source is turn on and off.Thus, the conventional method also shows effective successrate (100.0%) of proximity detection regardless of backgroundlights. In Scenario IV, the conventional method shows only42.9% success rate to the environmental noises while theproposed DVSSTL shows 81.5% success rate by temporalpattern recognition. Thus, the proposed method is more robustto environmental noises than conventional optical method asshown in Scenario IV.

G. Analysis of setting distance

For more tiny applications, we analyzed the effect of settingdistance, S, between the light source and DVS. Smartphoneof 50 mm wide or larger are very common and 25 mm settingused in this work is suitable to the dimension. Also, large Sis preferred to generate higher sensitivity (distance/pixel) dueto large variation according to the distance to the object asdescribed in section III-A. In some tiny applications, however,S of 25 mm may be too large. Thus, we described the effectof smallersetting distance. Smaller S can be also used forproximity sensing, but it comes with lower sensitivity due toless variation according to the distance to the object.

10

0 m

m

DVS Light Source

20

mm

(S) 25 mm

α2

(a) Setting distance, S = 25 mm

10

0 m

m

DVS Light Source

20

mm

(S) 10 mm

α2

(b) Setting distance, S = 10 mmFig. 13. The analysis of setting distance.

Fig. 13 illustrates examples of 25 mm and 10 mm settingdistance while other settings are same in the paper. Weassumes that an object moves between 20 mm to 100 mmfrom DVS for analysis and the range of movement is suitableto analyze because the proximity design focuses close object.

7

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS , FEBRUARY 2015

In Fig. 13(a), the variation of angle is 52◦ when an objectmoves between 20 mm and 100 mm from DVS and settingdistance is 25 mm. As an example of small distance, Fig. 13(b)shows that the variation of angle is 21◦ for same movementwhen setting distance is 10 mm. Also, sensitivity of each casesare 2.6 mm/pixel and 1.1 mm/pixel, respectively. Therefore,smaller setting distance between DVS and a light source canbe also used even though it reduces sensitivity.

V. CONCLUSION

The proposed proximity sensing design which utilizes a dy-namic vision sensor (DVS) can detect the proximity of an ob-ject with providing robustness to interference from unintendedother light sources and a transparent plastic cover which isequipped for protection. The fast response time of DVS allowsthe proposed design to measure the distance to the objectfrom the DVS in real time based on the spatial information ofthe DVS events. Also, the proposed method can change thecritical range which is measurable distances by changing theangle of the light source. The more rectilinear light source ispreferred to measure the distance more accurately. This paperalso mentioned how to use more diffused light source such asLEDs for the sake of cost-effectiveness. Comparing with theconventional optical proximity sensors which are vulnerableto environmental noises, the proposed method is robust tonoises because it uses temporal pattern of the DVS eventswith synchronizing the light source control. It is experimentedthat the proposed design can achieve up to 100% accuracy atnormal scenarios by applying a common rule with a loosecondition and over 80% accuracy even at the worst scenarios.As an additional advantage, the proposed design can replacethe conventional proximity sensor and utilize benefits of DVSitself when it is not used for proximity detection.

REFERENCES

[1] E. Y. Ahn, J. H. Lee, T. Mullen, and J. Yen, “Dynamic vision sensorcamera based bare hand gesture recognition,” in Proc. of the IEEESymposium on Computational Intelligence for Multimedia, Signal andVision Processing, 2011, pp. 52–59.

[2] C. Canali, G. D. Cicco, B. Morten, M. Prudenziati, and A. Taroni, “Atemperature compensated ultrasonic sensor operating in air for distanceand proximity measurements,” IEEE Tran. Ind. Electron., vol. 29, no. 4,pp. 336–341, Nov. 1982.

[3] H. Cheng, A. M. Chen, A. Razdan, and E. Buller, “Contactless gesturerecognition system using proximity sensors,” in Proc. of the IEEE Int.Conf. on Consum. Electron., 2011, pp. 149–150.

[4] W. Chung, H. Kim, Y. Yoo, C. Moon, and J. Park, “The detection andfollowing of human legs through inductive approaches for a mobile robotwith a single laser range finder,” IEEE Tran. Ind. Electron., vol. 59, no. 8,pp. 3156–3166, Aug. 2012.

[5] T. Delbruck and P. Lichtsteiner, “Photoarray for detecting time-dependent image data,” Patent US 20 080 135 731 A1, 6 12, 2008.

[6] W. Hortschitz, H. Steiner, M. Sachse, M. Stifter, F. Kohl, J. Schalko, A.Jachimowicz, F. Keplinger, and T. Sauter, “Robust precision position de-tection with an optical mems hybrid device,” IEEE Tran. Ind. Electron.,vol. 59, no. 12, pp. 4855–4862, Dec. 2012.

[7] T. Inari and N. Aoki, “A proximity sensor using power variation of alaser diode caused by returned light and its characteristics of stability,” inProc. of the IEEE Annual Conf. on Ind. Electron., 2005, pp. 2114–2118.

[8] M. Jagiella, S. Fericean, and A. Dorneich, “Progress and recent realiza-tions of miniaturized inductive proximity sensors for automation,” IEEESensors J., vol. 6, no. 6, pp. 1734–1741, Dec. 2006.

[9] S. J. Kim and B. K. Kim, “Dynamic ultrasonic hybrid localizationsystem for indoor mobile robots,” IEEE Tran. Ind. Electron., vol. 60,no. 10, pp. 4562–4573, Oct. 2013.

[10] Y. K. Kim, Y. Kim, Y. S. Jung, I. G. Jang, K. Kim, S. Kim, and B. M.Kwak, “Developing accurate long-distance 6-dof motion detection withone-dimensional laser sensors: Three-beam detection system,” IEEETran. Ind. Electron., vol. 60, no. 8, pp. 3386–3395, Aug. 2013.

[11] K. Koibuchi, K. Sawa, T. Honma, T. Hayashi, K. Ueda, and H.Sasaki, “Eddy-current type proximity sensor with closed magnetic circuitgeometry,” IEEE Tran. Magn., vol. 43, no. 4, pp. 1749–1752, Apr. 2007.

[12] N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. T.Campbell, “A survey of mobile phone sensing,” IEEE Commun. Mag.,vol. 48, no. 9, pp. 140–150, Sep. 2010.

[13] J. Li, C. C. Jobes, and J. L. Carr, “Comparison of magnetic fielddistribution models for a magnetic proximity detection system,” IEEETrans. Ind. Appl., vol. 49, no. 3, pp. 1171–1176, May-June 2013.

[14] P. Lichtsteiner, C. Posch, and T. Delbruck, “A 128x128 120 db 15uslatency asynchronous temporal contrast vision sensor,” IEEE J. Solid-State Circuits, vol. 43, no. 2, pp. 566–576, Feb. 2008.

[15] F. Marino, P. D. Ruvo, G. D. Ruvo, M. Nitti, and E. Stella, “Hiper3-d: An omnidirectional sensor for high precision environmental 3-dreconstruction,” IEEE Tran. Ind. Electron., vol. 59, no. 1, pp. 579–591,Jan. 2012.

[16] G. Milette and A. Stroud, Professional Android Sensor Programming.John Wiley & Sons, NJ, 2012.

[17] T. Mizuno, T. Mizuguchi, Y. Isono, T. Fujii, Y. Kishi, K. Nakaya, M.Kasai, and A. Shimizu, “Extending the operating distance of inductiveproximity sensor using magnetoplated wire,” IEEE Tran. Magn., vol. 45,no. 10, pp. 4463–4466, Oct. 2009.

[18] M. Murakami, J. K. Tan, H. Kim, and S. Ishikawa, “Human motionrecognition using directional motion history images,” in Proc. of theInt. Conf. on Control Automation and Systems, 2010, pp. 1512–1514.

[19] R. Richa, M. Balicki, R. Sznitman, E. Meisner, R. Taylor, and G.Hager, “Vision-based proximity detection in retinal surgery,” IEEETrans. Biomed. Eng., vol. 59, no. 8, pp. 2291–2301, Aug. 2012.

[20] W. Wei and A. Yunxiao, “Vision-based human motion recognition: Asurvey,” in Proc. of the Int. Conf. on Intelligent Networks and IntelligentSystems, 2009, pp. 386–389.

Jae-Yeon Won received the B.S. degree in electrical and electronic en-gineering from Yonsei University, Seoul, Korea, in 2007 and the M.S.degree in electrical engineering and computer science from Seoul NationalUniversity, Seoul, in 2009. He is currently working toward the Ph.D. degreein electrical and computer engineering Department, Texas A&M University,College Station, TX since 2010. His research interests include hardware designand power optimization for Chip Multi Processor (CMP) and neuromorphiccomputing.Hyunsurk (Eric) Ryu gradated from POSTECH (B.S., 1992, M.S., 1994,Ph.D., 1998). He has been a Member of Technical Staff in Samsung AdvancedInstitute of Technology, Samsung Electronics, after receiving his Ph.D.Recently he has expanded his research interests to bio-mimic computing,communication and cognitive applications.Tobi Delbruck (M’89-SM’06-F’13) received his BSc in physics and appliedmathematics from UCSD in 1983 and a PhD from Caltech in 1993. Heworked for 5 years on electronic imaging at Arithmos, Synaptics, NationalSemiconductor, and Foveon. He is professor at ETH Zurich in the Inst. of Neu-roinformatics. He is a fellow of IEEE. He has been awarded 6 IEEE awards,including the 2006 ISSCC Jan Van Vessem Outstanding European PaperAward. He co-organizes the Telluride Neuromorphic Cognition Engineeringsummer workshop and the demonstration sessions at ISCAS, and is incomingchair of the CAS Sensory Systems Technical Committee and associate editorof the IEEE Transactions of Biomedical Circuits and Systems. His currentinterests include bio-inspired and neuromorphic sensory processing.Jun Haeng Lee received the B.S., M.S., and Ph.D. degrees in electricalengineering from the Korea Advanced Institute of Science and Technology(KAIST), Daejeon, Korea, in 1999, 2001, and 2005, respectively. He iscurrently with Samsung Advanced Institute of Technology (SAIT) of SamsungElectronics Co. Ltd., Gyeonggi-do, Republic of Korea. His current researchinterests include neuromorphic engineering and biologically plausible modelfor artificial intelligence.Jiang Hu (M’01-SM’07) received the B. S. degree in optical engineering fromZhejiang University, China, in 1990, the M. S. degree in physics in 1997, andthe Ph. D. degree in electrical engineering from the University of Minnesotain 2001. He has been with IBM Microelectronics from January 2001 to June2002. Currently, he is an associate professor in the Department of Electricaland Computer Engineering at the Texas A&M University. His research interestis on Computer-Aided Design for VLSI circuits and systems, especially onlarge scale circuit optimization, clock network synthesis, robust design andon-chip communication. He received a best paper award at the ACM/IEEEDesign Automation Conference in 2001, an IBM Invention AchievementAward in 2003 and a best paper award at the IEEE/ACM InternationalConference on Computer-Aided Design in 2011. He has served as technicalprogram committee member for DAC, ICCAD, ISPD, ISQED, ICCD, DATE,ASPDAC, ISLPED and ISCAS, technical program chair and general chair forthe ACM International Symposium on Physical Design, and associated editorfor IEEE Transactions on CAD and ACM Transactions on Design Automationof Electronic Systems.

8

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS , FEBRUARY 2015