IEEE TRANSACTIONS ON ROBOTICS, VOL. 34, NO. 3 ...ncr.mae.ufl.edu/papers/TRO18b.pdf806 IEEE TRANSACTIONS ON ROBOTICS, VOL. 34, NO. 3, JUNE 2018 utilize a predictor similar to that developed

IEEE TRANSACTIONS ON ROBOTICS, VOL. 34, NO. 3, JUNE 2018 805

Target Tracking in the Presence of IntermittentMeasurements via Motion Model Learning

Anup Parikh , Rushikesh Kamalapurkar , and Warren E. Dixon

Abstract—When using a camera to estimate the pose of a movingtarget, measurements may only be available intermittently, due tofeature tracking losses from occlusions or the limited field of viewof the camera. Results spanning back to the Kalman filter havedemonstrated the utility of using a predictor to update state es-timates when measurements are not available, but target velocitymeasurements or a motion model must be known to implement apredictor for image-based pose estimation. In this paper, a novelestimator and predictor are developed to simultaneously learn amotion model, and estimate the pose, of a moving target from amoving camera. A stability analysis is provided to prove conver-gence of the state estimates and function approximation withoutrequiring the restrictive persistent excitation condition. Two exper-iments illustrate the performance of the developed estimator andpredictor. One experiment involves a stationary camera observinga mobile robot with sporadic feature tracking losses, and a sec-ond experiment involves a quadcopter moving between two mobilerobots on a road network.

Index Terms—Adaptive methods, estimation, switched systems,target tracking.

I. INTRODUCTION

ANUMBER of advances in imaging and computer visionhave enabled geometric reconstruction of features through

image feedback. Due to the projection in the imaging process,and the resulting scale ambiguity, typical approaches exploitmultiple views of the scene, as well as scale information, to re-cover the Euclidean geometry, e.g., stereo vision with a knownbaseline or structure from motion (SfM) with known cameramotion. For online reconstruction (i.e., recursive methods), us-

Manuscript received June 17, 2017; revised December 17, 2017; acceptedFebruary 19, 2018. Date of publication May 22, 2018; date of current versionJune 6, 2018. This paper was recommended for publication by Associate EditorF. Kanehiro and Editor C. Torras upon evaluation of the reviewers’ comments.This research was supported in part by a Task Order contract with the AirForce Research Laboratory, Munitions Directorate at Eglin AFB, in part by theUS National Geospatial Intelligence Agency under Grant HM0177-12-1-0006,and in part by the US Department of Agriculture National Robotics Initiativeunder Grant 2013-67021-21074. Any opinions, findings, and conclusions orrecommendations expressed in this paper are those of the author(s) and do notnecessarily reflect the views of the sponsoring agency. (Corresponding author:Anup Parikh.)

A. Parikh is with Robotics Research & Development, Sandia National Labo-ratories, Albuquerque, NM 87123 USA (e-mail:,[email protected]).

R. Kamalapurkar is with the School of Mechanical and Aerospace En-gineering, Oklahoma State University, Stillwater OK 74078 USA (e-mail:,[email protected]).

W. E. Dixon is with the Department of Mechanical and Aerospace En-gineering, University of Florida, Gainesville, FL 32611-6250 USA (e-mail:,[email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TRO.2018.2821169

ing a single camera, a number of observer/filtering techniqueshave been developed to solve the SfM problem (i.e., determinethe relative Euclidean coordinates of an object with respect to acamera) for a stationary object and moving camera (cf. [1]–[7]),a moving object with a stationary camera (cf. [7]–[12]), as wellas for the case where both the object and camera are in motion(cf. [13] and [14]). In all cases, velocity information of eitherthe camera or target, or both, is used to inject scale informationinto the system.

Previously developed SfM observers rely on continuous mea-surement availability to show convergence of the estimates.However, in many applications, image feature measurementsmay be intermittently unavailable due to feature tracking losses,occlusions, limited camera field of view (FOV), or even thefinite frame rate of the camera, which result in intermittent mea-surements. In this paper, a novel estimator and predictor aredeveloped and shown to converge to within an error bound de-spite the intermittent measurements. The estimation error obeysdifferent dynamics when operating in different modes (i.e., whenmeasurements are available versus when measurements are un-available). Switched systems theory is used to analyze the over-all stability and performance of the system despite the differentdynamics associated with the different modes. Switched sys-tems’ methods are necessary due to the well-known result thatswitching between stable systems can lead to instability [15].The problem is exacerbated in this paper, as shown in the anal-ysis, because the estimation error dynamics are unstable whenmeasurements are unavailable. Therefore, additional analysisis necessary to demonstrate that, despite intermittent measure-ments, the overall switched estimation error dynamics are stable.

Numerous results have been developed for feature tracking inthe presence of intermittent visibility of the target. For example,[16] and [17] describe methods for learning a motion modelonline for feature motion prediction. Similarly, in [18] and[19] Kalman or particle filters are used to estimate feature mo-tion and predict feature coordinates while occluded. In contrast,[20]–[22] use visual context to increase the robustness of featuretrackers to occlusions. For the SfM problem, a technique that isrobust to occlusions or feature tracking losses is developed in[23]; however, only the shape of the object is recovered, and notthe three-dimensional (3-D) position due to the orthogonal pro-jection model used. In contrast to such results, the full 6 degreeof freedom (DOF) pose of the target is estimated in this paper,and the estimates are shown to be stable.

Many of the probabilistic approaches for SfM, or the associ-ated simultaneous localization and mapping (SLAM) problem,

1552-3098 © 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

https://orcid.org/0000-0003-3120-1721

https://orcid.org/0000-0001-9963-5361

https://orcid.org/0000-0002-5091-181X

mailto:[email protected]



806 IEEE TRANSACTIONS ON ROBOTICS, VOL. 34, NO. 3, JUNE 2018

utilize a predictor similar to that developed in this paper orcircumvent the intermittent sensing issue by only updatingstate estimates when new measurements are available (see [24]and [25] for an overview). However, these approaches arebased on either linearizations of the nonlinear dynamics (cf.[26]–[31]), and therefore only show local convergence, or aresample based (e.g., [32] and [33]), and therefore, can only showoptimal estimation in the limit as the number of samples ap-proach infinity. Much of the recent literature on target trackinghas focused on using suboptimal algorithms for tracking usingsimplified motion models (e.g., constant velocity, constant turnrate, etc.), with a focus on reduced complexity and improv-ing practical performance, and do not analyze estimation errorgrowth due to model uncertainty or show estimation error con-vergence [34], [35]. Some methods explicitly handle occlusions,though they either assume availability of range measurementsand only estimate position, therefore rendering the system lin-ear (cf. [36]–[38]), or only estimate relative depth ordering anddo not consider the pose estimation problem, e.g., [39]. Othermethods learn a model of the target motion online using functionapproximation methods (cf. [40]–[47]), though do not providea convergence analysis. Conversely, the full nonlinear dynam-ics are analyzed in this paper, resulting in an arbitrarily largeregion of attraction around the zero estimation error trajectory(i.e., the estimator converges for any set of initial conditionsprovided gains and controller parameters are sufficiently large,rather than linearization-based approaches that rely on suffi-ciently small initial conditions on the estimation error, yieldinglocal convergence results), and the proposed estimator–predictorstructure has computing requirements that can be met by typicalor low-end modern computers (see Section VII). Furthermore,convergence and consistency proofs of probabilistic estimatorstypically require knowledge of the probability distribution of theuncertainty in the system and result in convergence in mean or inmean square. In comparison, analysis of deterministic observerstypically assume boundedness and some level of smoothness ofdisturbances, and yield asymptotic or exponential convergence.The primary contribution of this paper is in the developmentand analysis of a novel estimator and predictor that ensures con-vergence to an ultimate bound as well as online learning of amotion model of the target using a deterministic framework.

Our previous results have shown stability of the position esti-mation error during intermittent measurements [48], [49], pro-vided dwell time conditions are satisfied. Dwell time conditionsspecify the minimum amount of time a single mode or sys-tem must be active before switching to another mode to main-tain system stability, and reverse dwell time conditions specifythe maximum amount of time a system can remain active tomaintain system stability. These conditions must be met at ev-ery switch from one mode to another. In the context of targettracking with intermittent measurements, the dwell time condi-tions specify the minimum contiguous duration the target mustremain in view, and reverse dwell time conditions specify themaximum contiguous duration the target can remain out of view.In [49], a zero-order hold is performed on the state estimateswhen measurements are unavailable, resulting in growth of theestimation error based on the trigonometric tangent function,and an ultimately bounded estimation error result. Since the

tangent function is unbounded for finite arguments, (reverse)dwell time conditions are necessary at every period in whichmeasurements are (un)available to ensure stability. In [48], apredictor was used to update the state estimates when measure-ments are unavailable. This results in exponential growth of theestimation errors when measurements are unavailable, allow-ing the use of average dwell times for stability. Average dwelltime conditions are easier to satisfy than other dwell time condi-tions since they only restrict the average mode durations ratherthan every duration. A downside of the approach in [48] is thatthe use of a predictor requires knowledge of a motion modelof the target to generate target velocity signals utilized in thepredictor.

In this paper, the target motion model is learned online. Anumber of adaptive methods have been developed to compen-sate for unknown functions or parameters in the dynamics; how-ever, parameter estimates may not approach the true parameterswithout persistent excitation (PE) [50]–[52]. The PE conditioncannot be guaranteed a priori for nonlinear systems (as apposedto linear systems, e.g., [50, Th. 5.2.1]) and is difficult to checkonline, in general. Recently, a technique known as concurrentlearning (CL) was developed to use recorded data for onlineparameter estimation [53]–[55] with an alternative excitationcondition. In CL, input and state derivatives are recorded andused similar to recursive least squares to establish a negativedefinite parameter estimation error term in the Lyapunov analy-sis, and hence, a negative definite Lyapunov derivative provideda finite excitation condition is satisfied. However, state deriva-tives can be noisy, and require extensive filter design and tuningto yield satisfactory signals for use in CL. A further contribu-tion of this paper is that the CL technique is reformulated interms of an integral (ICL), removing the need for state deriva-tives, while preserving convergence guarantees. Compared totraditional adaptive methods that utilize PE to ensure parameterconvergence, and hence, exponential stability, ICL only requiresexcitation for a finite period of time, and the excitation conditioncan be checked online.

In this paper, data are recorded online when measurementsare available (i.e., the target is in view of the camera). Usingthe ICL technique, a motion model of the target is learned andused in a predictor to estimate target pose when it is not visibleto the camera. A stability analysis is provided to show thatthis estimation and prediction with learning scheme yields anestimate of the target pose that converges to within an arbitrarilysmall bound around the true target pose.

Two experiments are included to illustrate the performanceof this estimator. One experiment involves a stationary cameraobserving a mobile robot moving according to a static vec-tor field. The results of this experiment demonstrate that thelearning component of the estimator quickly converges and ac-curately predicts the robot motion. The second experiment in-volves a moving camera observing two mobile robots movingalong a road network. Due to the limited FOV and resolutionof the camera, measurements of both robots are not availablesimultaneously. Despite the intermittent measurements, and thestochastic motions of the robots at various points on the roadnetwork, the results demonstrate the feasibility of the developedapproach for tracking the pose of multiple targets.

PARIKH et al.: TARGET TRACKING IN THE PRESENCE OF INTERMITTENT MEASUREMENTS VIA MOTION MODEL LEARNING 807

Fig. 1. Reference frames and coordinate systems of a moving camera observ-ing a moving target.

II. SYSTEM DYNAMICS

Fig. 1 is used to develop the image kinematics. In Fig. 1,FG denotes a fixed inertial reference frame with an arbitrarilyselected origin and Euclidean coordinate system, FQ denotes areference frame fixed to the moving object, with an arbitrarilyselected origin and Euclidean coordinate system, and FC de-notes a reference frame fixed to the camera. The right-handedcoordinate system attached to FC has its origin at the principalpoint of the camera, e3 ∈ E3 axis pointing out and collinear withthe optical axis of the camera, e1 ∈ E3 axis aligned with the hor-izontal axis of the camera, and e2 � e3 × e1 ∈ E3 . The vectorsrq ∈ E3 and rc ∈ E3 represent the vectors from the origin ofFG to the origins of FQ and FC , respectively. The kinematicsof the coordinates of the relative position vector expressed inthe camera coordinate system are

x = vq − vc − ω×c x (1)

where x ∈ R3 denotes the position of the origin of FQ withrespect to the origin of FC (i.e., the relative position of the ob-ject with respect to the camera), vq �

[vq1 vq2 vq3

]T ∈ R3

is the velocity of the origin of FQ with respect to the ori-gin of FG (i.e., the inertial linear velocity of the object),vc �

[vc1 vc2 vc3

]T ∈ R3 is the linear velocity of the ori-gin ofFC with respect to the origin ofFG (i.e., the inertial linearvelocity of the camera), and ωc �

[ωc1 ωc2 ωc3

]T ∈ R3 isthe angular velocity of FC with respect to FG (i.e., the iner-tial angular velocity of the camera), all expressed in the cameracoordinate system. Also, ()× : R3 → R3×3 represents the skewoperator, defined as

p× �

⎡

⎣0 −p3 p2p3 0 −p1−p2 p1 0

⎤

⎦ .

In the following analysis, the quaternion parameterizationwill be used to represent orientation. Let q ∈ H be the unitquaternion parameterization of the orientation of the objectwith respect to the camera, which can be represented in the4-D vector space R4 using the standard basis 1, i, j, k asq �

[q0 qT

v

]T ∈ S4 , where Sr �{x ∈ Rr |xT x = 1

}, and

q0 and qv represent the scalar and vector components of q.Based on this definition, a vector expressed in the object

coordinate system ξq ∈ R3 can be related to the same vec-tor expressed in the camera coordinate system ξc ∈ R3 , asξc = q · ξq · q, where () : S4 → S4 represents the unit quater-

nion inverse operator defined as q �[q0 −qT

v

]Twith identity

q · q = q · q =[1 0 0 0

]T, and (·) : R4 ×R4 → R4 rep-

resents the Hamilton product,1 with property qa · qb ∈ S4 forqa , qb ∈ S4 . The Hamilton product can be expressed in blockmatrix notation as

qa · qb =[

qa0 −qTav

qav qa0I3 + q×av

]qb

where Ig ∈ Rg×g is the identity matrix. The kinematics for therelative orientation of the object with respect to the camera are(see [56, Ch. 3.4] or [57, Ch. 3.6])

q =12B (q) (ωq − q · ωc · q) (2)

where B : S4 → R4×3 is defined as

B (ξ) �[ −ξT

v

ξ0I3 + ξ×v

]

and has the pseudoinverse property B (ξ)T B (ξ) = I3 (see [56,Ch. 3.4]).

III. ESTIMATION OBJECTIVE

The primary goal in this paper is to develop a pose estima-tor/predictor that is robust to intermittent measurements. Thedesign strategy is to filter the pose measurements when theyare available and predict future poses when measurements areunavailable (e.g., the object is not visible to the camera). How-ever, a predictor based on (1) and (2) would require linear andangular velocities of the object to be known. The novelty in thispaper is to learn a model of the object velocities when measure-ments are available, and use the model in the predictor whenmeasurements are not available. To this end, a stacked posestate η (t) ∈ R7 is defined as η (t) �

[xT (t) qT (t)

]Tand

the following assumptions are utilized.Assumption 1: Measurements of the relative pose of the tar-

get are available from camera images when the target is in view.Remark 1: The projection of a 3-D scene onto a 2-D sen-

sor during the imaging process results in scale ambiguity [58,Ch. 5.4.4]. In typical SfM observers, target velocity is used toinject scale into the system and recover the full Euclidean co-ordinates of the target. However, in the scenario considered inthis paper, the target velocities are unknown. To resolve the am-biguity, a known length scale on the target can be used, andby exploiting Perspective-n-Point (e.g., [59]–[65]) or homogra-phy (e.g., [66] and [67]) solvers, the pose of the target can berecovered.

1For brevity, a slight abuse of notation will be utilized throughout the paper.For v1 , v2 ∈ R3 and q ∈ S4 , the equation v2 = q · v1 · q can be written pre-

cisely as qv 2 = q · qv 1 · q, where qv 1 �[

0 vT1

]Tand qv 2 �

[0 vT

2

]T.

In other words, an R4 quaternion qv 1 is derived from an R3 vector v1 by settingthe scalar part of qv 1 to zero and setting the vector part of qv 1 as equal to v1 .Similarly, the resulting vector v2 is derived from the vector component of qv 2 .


Assumption 2: The object velocities are locally Lipschitzfunctions of the object pose and not explicitly time dependent,i.e., vq (t) = φ1 (ρ (η (t) , t)) and ωq (t) = φ2 (ρ (η (t) , t)),where φ1 , φ2 : R7 → R3 are bounded and ρ : R7 × [0,∞)→R7 is a known, bounded, and locally Lipschitz function.

Remark 2: This assumption ensures there exists some func-tion that can be learned, i.e., the object velocities do not meanderarbitrarily. Moreover, via the Stone–Weierstrass theorem [68],it ensures that universal function approximators [e.g., neuralnetworks (NN)] can be used to estimate the object velocities toan arbitrary level of accuracy. The Stone–Weierstrass theoremonly guarantees the estimate is accurate over a compact set,hence dependence on the state is allowed since it is bounded viaAssumption 3 below, but exclusion of an explicit dependenceon time is required since the interval t ∈ [0,∞) is consideredin the analysis. The velocities can change with time, since thestate of the object can change with time; however, the map-ping between the object state and the object velocity is assumedto be static. This assumption holds in cases of, e.g., projectileor orbital motion, pursuit-evasion games, as well as simplisticmodels of vehicles moving along a road network, e.g., the proofof concept experiments provided in Section VII.

Assumptions analogous to Assumption 2 are implicit in ma-chine learning and function approximation contexts. Intuitively,if an explicit and unknown time dependence is allowed in thefunction to be estimated, there is no guarantee that the data usedto approximate the function, and hence, the function estimate,will be valid in the future. For example, in [47], Campbell et al.describe a scenario of tracking a target with a finite set of behav-iors and use a nonparametric approach to learn an anomalousbehavior. This type of target motion could be learned using ourapproach if the velocity maps φ1 and φ2 were piecewise-in-timestatic. For such a case, the analysis in the Section VI can be ex-panded to include switching due to changing target behavior.However, if the target exhibited new behavior (i.e., a new statein the Markov model) at every timestep, there would be no hopein learning the overall target behavior, since the past data wouldprovide no insight into future behavior.

In some scenarios, information beyond the object pose (e.g.,traffic levels, time of day, weather, etc.) may be relevant inpredicting the target behavior. These auxiliary states can beconsidered in the function approximation to capture a widerclass of possible target behavior without violating technical re-quirements underpinning learning. The auxiliary states can beincluded either directly if they are measurable, or by using anobserver, hidden Markov model, etc. to generate state estimatesif the auxiliary states are not measurable.

Remark 3: In some applications, the velocity field of the tar-get is expected to be dependent on the target’s pose with respectto the world, rather than its relative pose with respect to the cam-era. The function ρ is used to transform the relative pose to itsworld pose by using the camera pose with respect to the world. Inother applications, the velocity field is expected to rely solely onthe relative pose (e.g., a pursuit-evasion scenario in an obstacle-free environment, where the evader’s motion would only bedependent on its pose with respect to the pursuer/camera) orthe camera pose is unknown, in which case ρ can be taken as

the identity function on η (t). As shown in the following, thesecoordinate transformations are embedded in the bases of thefunction approximation.

Assumption 3: The state η (t) is bounded, i.e., η (t) ∈ X ,where X ⊂ R7 is a convex, compact set.

Remark 4: In estimation, for the state estimates to convergeto the states while remaining bounded, the states themselvesmust remain bounded. This is analogous to the requirement ofbounded desired trajectories in control problems.

In this development, the unknown motion model functions φ1and φ2 are approximated with a neural network, i.e.,

[vq (t)

12 B (q (t)) ωq (t)

]=[

φ1 (ρ (η (t) , t))12 B (q (t)) φ2 (ρ (η (t) , t))

]

= WT σ (ρ (η, t)) + ε (ρ (η, t)) (3)

where σ : R7 → Rp is a known, bounded, locally Lipschitz,vector of basis functions, W ∈ Rp×7 is a matrix of the un-known ideal weights, and ε : R7 → R7 is the function approxi-mation residual, which is locally Lipschitz based on the locallyLipschitz properties of vq (t), ωq (t), B (q (t)), ρ (η (t) , t), andσ(·), and is a priori bounded with a bound that can be madearbitrarily small based on the Stone–Weierstrass theorem, i.e.,ε � supη∈X , t∈[0,∞) ‖ε (ρ (η, t))‖, where ‖·‖ denotes the Eu-clidean norm. Note that if W is known, φ2 (ρ (η (t) , t)) can beapproximated by premultiplying by 2BT (q (t)) and utilizingthe pseudoinverse property of B (q (t)).

To quantify the estimation objective, let

η (t) � η (t)− η (t) (4)

denote the estimation error, where η (t) ∈ R7 contains the po-sition and orientation estimates. Also, let

W (t) � W − W (t) (5)

denote the parameter estimation error, where W (t) ∈ Rp×7 isthe estimate of the ideal function approximation weights. Basedon these definitions, the kinematics in (1) and (2) can be rewrit-ten as

η (t) = WT σ (ρ (η (t) , t)) + ε (ρ (η (t) , t)) + f (η (t) , t)(6)

where f : R7 × [0,∞)→ R7 is a known function defined as

f (η (t) , t) � −[

vc (t) + ωc (t)× x (t)12 B (q (t)) (q (t) · ωc (t) · q (t))

]

.

IV. ESTIMATOR DESIGN

The following sections detail the estimator. The estimator issummarized in Algorithm 1, where δt refers to the loop timestep.

A. Update

Based on the subsequent stability analysis, during the periodsin which measurements are available, the position and orienta-tion estimate update laws are designed as

˙η (t) = W (t)T σ (ρ (η (t) , t)) + f (η (t) , t) + k1 η (t)

+ k2sgn (η (t)) (7)


Algorithm 1: Algorithm for Estimator.

Input: η (0), W (0)Output: η (t), W (t)

Initialization:Initialize {t1 , ..., tN }, {Y1 , ...,YN }, {F1 , ...,FN },{Δη1 , ...,ΔηN } to 0Estimator loop:while target tracking do

if target is in view thenη (t)←∫ t

t−δt [W (τ)T σ (ρ (η (τ) , τ))+f (η (τ) , τ)+k1 η (τ) + k2sgn (η (τ))]dτ

W (t)←∫ t

t−δt

[proj(Γσ (ρ (η (τ) , τ)) η (τ)T

+kC LΓN∑

i=1YT

i (Δηi −Fi − YiW (τ)))]

dτ

Y (t)← ∫ t

t−Δt σT (ρ (η (τ) , τ)) dτ

F (t)← ∫ t

t−Δt fT (η (τ) , τ) dτData Selection:for i = l to N do

λi←λmin

{

Y (t)T Y (t) +N∑

j=1, j �=i

YTj Yj

}

end fork ← arg max

i{λi}

if λk > λmin

{N∑

i=1YT

i Yi

}then

tk ← tYk ← Y (t)Fk ← F (t)Δηk ← ηT (t)− ηT (t−Δt)

end ifelse

η (t)← ∫ t

t−δt [proj(W (τ)T σ (ρ (η (τ) , τ))+ f (η (τ) , τ))]dτ

W (t)←∫ t

t−δt

[proj(kC LΓ

N∑

i=1YT

i (Δηi

−Fi − YiW (τ)))]dτ

end ifend whilereturn η (t), W (t)

where sgn(·) is the signum function. To facilitate the designof an ICL update law, let Y (t) �

∫ t

t−Δt σT (ρ (η (τ) , τ)) dτ

andF (t) �∫ t

t−Δt fT (η (τ) , τ) dτ , where Δt ∈ R is a positiveconstant denoting the size of the window of integration. The ICLupdate law for the motion model approximation parameters isdesigned as

˙W = proj

(Γσ (ρ (η (t) , t)) η (t)T

+ kCLΓN∑

i=1

YTi

(Δηi −Fi − YiW (t)

))

(8)

where proj (·) is a smooth projection operator (see [69,Appendix E], [70, Remark 3.7]) with bounds based on the statebounds and velocity bounds of Assumptions 2 and 3, N ∈ N0 ,kC L ∈ R, and Γ ∈ Rp×p are constant, positive definite, andsymmetric control gains, Δηi � ηT (ti)− ηT (ti −Δt), Fi �F (ti), Yi � Y (ti), and ti represents past time points, i.e.,ti ∈ [Δt, t] at which measurements are available. The princi-pal goal behind this design is to incorporate recorded input andtrajectory data to identify the ideal weights. The time points ti ,and the corresponding Δηi , Fi , and Yi that are recorded andused in (8) are referred to as the history stack. As shown in thesubsequent stability analysis, the parameter estimate learningrate is related to the minimum eigenvalue of

∑Ni=1 YT

i Yi , mo-tivating the use of the singular value maximization algorithm in[54, Ch. 6] for adding or replacing data in the history stack.

To gain additional insight into the adaptive update law designin (8), the integral of the transpose of (6) is

∫ t

t−Δt

ηT (τ) dτ =∫ t

t−Δt

σT (ρ (η (τ) , τ)) Wdτ

+∫ t

t−Δt

εT (ρ (η (τ) , τ)) dτ

+∫ t

t−Δt

fT (η (τ) , τ) dτ.

Using the Fundamental Theorem of Calculus and simplifyingyields

ηT (t)−ηT (t−Δt)=Y(t)W + E(t)+ F(t) ∀t ∈ [Δt,∞)(9)

where ∀t ∈ [Δt,∞), E (t) �∫ t

t−Δt εT (ρ (η (τ) , τ)) dτ . Usingthe relation in (9), the update law in (8) can be simplified as

˙W = proj

(

Γσ (ρ (η (t) , t)) η (t)T + kC LΓN∑

i=1

YTi YiW (t)

+kCLΓN∑

i=1

YTi Ei

)

(10)

for all t > Δt, where Ei � E (ti). Taking the time derivative of(4), substituting (6) and (7), and simplifying yields the followingclosed-loop error dynamics when measurements are available

˙η (t) = W (t)T σ (ρ (η (t) , t))− k1 η (t) + ε (ρ (η (t) , t))

− k2sgn (η (t)) . (11)

B. Predictor

During periods when measurements are not available, thestate estimates are simulated forward in time using

˙η (t) = proj(W (t)T σ (ρ (η (t) , t)) + f (η (t) , t)

). (12)

Similarly, the recorded data continues to provide updates to theideal weight estimates via

˙W (t) = proj

(

kCLΓN∑

i=1

YTi


))

(13)


which can be simplified as

˙W (t) = proj

(

kCLΓN∑

i=1

YTi YiW (t) + kCLΓ

N∑

i=1

YTi Ei

)

.

(14)Taking the time derivative of (4), substituting (6) and (12), and

simplifying yields the following closed-loop dynamics whenmeasurements are not available

˙η (t) = W (t)T σ (ρ (η (t) , t)) + f (η (t) , t)− f (η (t) , t)

+ W (t)T (σ (ρ (η (t) , t))− σ (ρ (η (t) , t)))

+ ε (ρ (η (t) , t)) . (15)

V. IMPLEMENTATION AND DATA SELECTION

The integration time window Δt can be selected small relativeto the time scale of the dynamics (see Section VII for examples)to reduce the adverse effects of noise and function approxi-mation error. After time Δt (i.e., t > Δt), the signals Y (t),F (t), and ηT (t)− ηT (t−Δt) are available. For initializationof the history stack, the values of Y (t), F (t), and ηT (t)−ηT (t−Δt) can be saved at every time step until N values have

been recorded, and hence,∑N

i=1 YTi


)

can be calculated. However, typically the data collected duringinitialization is not sufficiently rich (i.e., do not satisfy As-sumption 4). Therefore, a procedure similar to that described in[54, Ch. 6] can be used for replacing data in the history stack.

Specifically, if λmin

{Y (t)T Y (t) +

∑Ni=1,i �=j YT

i Yi

}>λmin

{∑Ni=1 YT

i Yi

}for some j ∈ {1, 2, . . . , N}, where λmin {·}

refers to the minimum eigenvalue of {·}, then replace tj ,Yj ,Fj ,and Δηj with t, Y (t), F (t), and ηT (t)− ηT (t−Δt), respec-

tively. In this way, λmin

{∑Ni=1 YT

i Yi

}is always increasing. If

the system trajectories are sufficiently exciting (i.e., satisfy As-

sumption 4), λmin

{∑Ni=1 YT

i Yi

}will be strictly greater than

zero in finite time, at which point new data are not needed, andhence, the system trajectories no longer need to be exciting.

VI. ANALYSIS

The system considered in this paper operates in two modes.The evolution of a Lyapunov-like function is developed inLemma 1 for the mode when measurements are available and theupdate is used. Similarly, the evolution of a Lyapunov-like func-tion is developed in Lemma 2 for the mode when measurementsare unavailable and the predictor is active.

In addition to the switching that occurs as measurements be-come intermittently unavailable, in the following stability anal-ysis, time is partitioned into two phases. During the initial phase,insufficient data have been collected to satisfy a richness con-dition on the history stack. In Theorem 1, it is shown that thedesigned estimator and adaptive update law are still sufficientfor the system to remain bounded for all time despite the lack ofdata. After a finite period of time, the system transitions to thesecond phase, where the history stack is sufficiently rich and theestimator and adaptive update law are shown, in Theorem 2, to

asymptotically converge to an arbitrarily small bound. To guar-antee that the transition to the second phase happens in finitetime, and therefore, the overall system trajectories are ultimatelybounded, we require the history stack be sufficiently rich aftera finite period of time, as specified in the following assumption.

Assumption 4:

∃λ, T > 0 : ∀t ≥ T, λmin

{N∑

i=1

YTi Yi

}

≥ λ (16)

where λmin {·} refers to the minimum eigenvalue of {·}.The condition in (16) requires that the system be sufficiently

excited, though is weaker than the typical PE condition sinceexcitation is only needed for a finite period of time. Specifically,PE requires∫ t+Δt

t

σ (ρ (η (τ) , τ))σT (ρ (η (τ) , τ)) dt≥αI >0 ∀t > 0

(17)whereas Assumption 4 only requires the system trajectories tobe exciting up to time T (at which point

∑Ni=1 YT

i Yi is fullrank), after which the exciting data recorded during t ∈ [0, T ]is exploited for all t > T . Another benefit of the developmentin this paper is that the excitation condition is measurable (i.e.,

λmin

{∑Ni=1 YT

i Yi

}can be calculated), whereas in PE, Δt is

unknown, and hence, an uncountable number of integrals wouldneed to be calculated at each of the uncountable number of timepoints t in order to verify PE.

To facilitate the following analysis, let tonn and toff

n denotethe nth instance at which measurements become available andunavailable, respectively. Then, during t ∈ [ton

n , toffn ) measure-

ments are available and the estimator is active, whereas duringt ∈ [toff

n , tonn+1) measurements are unavailable and the predictor

is active. The duration of contiguous time each of these modesare active is denoted Δton

n � toffn − ton

n and Δtoffn � ton

n+1 − toffn ,

respectively, and the total amount of time each of these modesis active between switching instances a and b are denotedT on (a, b) �

∑bi=a Δton

i and T off (a, b) �∑b

i=a Δtoffi , respec-

tively. Also, ξ (t) �[

η (t)T vec(W (t)

)T]T

∈ R7+7p de-

notes a stacked state and parameter error vector, where vec (·)denotes a stack of the columns of (·).

To facilitate the Lyapunov-based analysis in Lemmas 1 and2, as well as Theorems 1 and 2, consider the Lyapunov functioncandidate V : R7+7p → R defined as

V (ξ (t)) � 12η (t)T η (t) +

12

tr(W (t)T Γ−1W (t)

). (18)

The function in (18) can be bounded as β1 ‖ξ (t)‖2 ≤V (ξ (t)) ≤ β2 ‖ξ (t)‖2 , where tr (·) denotes the matrixtrace operator, β1 � 1

2 min{1, λmin

(Γ−1

)}, and β2 �

12 max

{1, λmax

(Γ−1

)}. Also, due to the projection operator

in (8) and (13), and since W is a constant, W (t) is bounded,and V (ξ (t)) ≤ c2 + c3 ‖η (t)‖2 , where c2 , c3 ∈ R>0 are pos-itive constants.

Lemma 1: The estimator in (7) and (8) remains boundedduring t ∈ [ton

n , toffn ).


Proof: Taking the time derivative of (18) during t ∈[ton

n , toffn ), substituting (10) and (11), and simplifying yields

V (ξ (t)) ≤ −k1 ‖η (t)‖2 + c1

where c1 ∈ R>0 is a positive constant. Using the bounds on V ,V can be bounded as

V (ξ (t)) ≤ −k1

c3V (ξ (t)) +

(k1c2 + c1

c3

).

Using the Comparison Lemma [71, Lemma 3.4]

V (ξ (t)) ≤ V (ξ (tonn )) exp [−λ (t− ton

n )] +(

c2 +c1

k1

)

(19)∀t ∈ [ton

n , toffn ), where λ � k1

c3.

After sufficient data have been gathered (i.e., t ∈ [tonn , toff

n ) ∩[T,∞), where T was defined in Assumption 4)

V (ξ (t)) ≤ V (ξ (tonn )) exp [−λT (t− ton

n )] + cUB (20)

where λT � min{k1 ,λC L }β2

, cUB � c1 β2min{k1 ,λCL} , λCL � kCLλmin{∑N

i=1 YTi Yi

}, and λCL > 0 based on Assumption 4. �

Remark 5: Note that c1 is based on a bound on the data inthe history stack Yi , the CL gain kCL, and the bound on functionapproximation error ε, and therefore, cannot be arbitrarily de-creased through gain tuning. However, the ultimate error boundafter sufficient data has been gathered, cUB can be made arbitrar-ily small by increasing the gains k1 and kCL, and by decreasingε, e.g., increasing the number of neurons in the NN.

Lemma 2: The predictor in (12) and (13) remains boundedduring t ∈ [toff

n , tonn+1).

Proof: Taking the time derivative of (18) during t ∈[toff

n , tonn+1), substituting (14) and (15), and simplifying yields

V (ξ (t)) ≤ c4 ‖ξ (t)‖2 + c5

where c4 , c5 ∈ R>0 are positive constants. Using bounds on V ,V can be bounded as

V (ξ (t)) ≤ c4

β1V (ξ (t)) + c5 .

Using the Comparison Lemma [71, Lemma 3.4]

V (ξ (t))≤V(ξ(toffn

))exp

[c4

β1

(t− toff

n

)]∀t ∈ [toff

n , tonn+1)

(21)which remains bounded for all bounded t. �

Theorem 1: The estimator and predictor in (7), (8), (12),and (13) remain bounded provided there exists a k <∞, andsequences {Δton

n }∞n=0 and{Δtoff

n

}∞n=0 such that

c4

β1T off (nk, (n + 1) k) < λT T on (nk, (n + 1) k) ∀n ∈ N.

(22)

Proof: Consider a single cycle of losing and regaining mea-surements, i.e., t ∈ [ton

n , tonn+1). Based on (19) and (21)

V(ξ(tonn+1

)) ≤ V (ξ (tonn )) exp

[c4

β1Δtoff

n − λΔtonn

]

+(

c2 +c1

k1

)exp

[c4

β1Δtoff

n

]. (23)

Using (23), the evolution of V over k cycles is

V(ξ(ton(n+1)k

))≤ c6V (ξ (ton

nk )) + c7

where c6 , c7 ∈ R>0 are positive, bounded constants, and c6 <1 based on (22). Let {sn}∞n=0 be a sequence defined by therecurrence relation

sn+1 = M (sn )

with initial condition s0 = V (ξ (ton0 )), where M : R→ R is de-

fined as M (s) � c6s + c7 . Since c6 < 1, M is a contraction [72,Definition 9.22], and therefore, all initial conditions s0 approachthe fixed point s = c7

1−c6[72, Th. 9.23]. Since the sequence {sn}

upper bounds V in the sense that V (ξ (tonnk )) ≤ sn , V is also ul-

timately bounded. However, V may grow within[tonnk , ton

(n+1)k

]

since the dwell time condition in (22) is specified over k cyclesrather than a single cycle, and therefore, the ultimate bound ofξ, which is based on the ultimate bound of V is

lim supt‖ξ (t)‖ ≤ β1

c7

1− c6exp

(c4

β1T off

max

)

where T offmax � sup

nT off (nk, (n + 1) k). �

Theorem 2: After sufficient data are collected, i.e., t ∈[T,∞), the estimator and predictor in (7), (8), (12), and (13)converge to a bound that can be made arbitrarily small providedthere exists a k <∞, and sequences {Δton

n }∞n=0 and {Δtonn }∞n=0

such that (22) is satisfied.Proof: The proof follows similarly to the proof of Theorem 1.

Consider a single cycle of losing and regaining measurementsafter sufficient data has been collected, i.e., t ∈ [ton

n , tonn+1) ∩

[T,∞). Based on (20) and (21)

V(ξ(tonn+1

)) ≤ V (ξ (tonn )) exp

[c4

β1Δtoff

n − λT Δtonn

]

+ cUB exp[

c4

β1Δtoff

n

]. (24)

Using (24), the evolution of V over k cycles is

V(ξ(ton(n+1)k

))≤ c8V (ξ (ton

nk )) + c9

where c8 , c9 ∈ R>0 are positive, bounded constants, and c8 < 1based on (22). By using the same contraction arguments as inTheorem (1), the ultimate bound of ξ is

lim supt‖ξ (t)‖ ≤ β1

c9

1− c8exp

(c4

β1T off

max

)

where T offmax � sup

nT off (nk, (n + 1) k). �


Remark 6: The fundamental difference between Theorems 1and 2, and hence the need for sufficiently rich data, is the controlover the ultimate error bound. In Theorem 1, c7 is based on c2 ,which is based on the projection bound on the ideal functionapproximation weight errors, which is a priori determined, andtherefore, the ultimate error bound cannot be decreased. In The-orem 2, c9 is based on cUB which can be made arbitrarily smallby, for example, increasing the number of neurons in the NN.

Remark 7: The dwell time condition in (22) is similar toan average dwell time condition, but only over k cycles. Thecondition requires that, over k cycles, the total amount of timethat the stable subsystem is active (i.e., the target in view),scaled by the decay rate of the stable subsystem, is greater thanthe total amount of time that the unstable subsystem is active(i.e., the target not in view), scaled by the error growth rate.This is a relaxed condition compared to typical (i.e., singlecycle) forward and reverse dwell time conditions as it allowsflexibility in allocating time in the subsystems over k cycles.For example, if a large amount of time is spent observing thetarget in the first of k cycles, relatively little time is needed withthe target in view in the remaining k − 1 cycles to still satisfy(22) and ensure error convergence. With single cycle dwell timeconditions, any surplus time spent observing the target beyondwhat is necessary to satisfy the dwell time condition has nobenefit in the sense of relaxing the dwell time requirements ofsubsequent cycles.

VII. EXPERIMENTS

Experiments were performed to verify the theoretical resultsand demonstrate the performance of the developed estimationand prediction scheme with online model learning. In the first ex-periment (Section VII-A), a stationary camera observed a targetmoving along a smooth vector field. In the second experiment(Section VII-B), a moving camera observed two targets movingalong a road network. In both experiments, a Clearpath RoboticsTurtleBot 2 with a Kobuki base was utilized as a mobile vehiclesimulant (i.e., the target). A fiducial marker was mounted on themobile robot, and a corresponding tracking software library (see[73] and [74]) was used to repeatably track the image featurepixel coordinates, as well as provide target pose measurements,when the target was in the camera FOV. A NaturalPoint, Inc.OptiTrack motion capture system was used to record the groundtruth pose of the camera and target at a rate of 360 Hz. The poseprovided by the motion capture system was also used to esti-mate the linear and angular velocities of the camera necessaryfor the estimator, where the current camera velocity estimateswere taken to be the slope of the linear regression of the 20 mostrecent pose data points. The same procedure was used to cal-culate the linear and angular velocities of the target for groundtruth and comparison with the learned model.

For both experiments, radial basis functions (RBF) were usedin the NN, with parameters selected based on the descriptionprovided in the subsequent sections. Estimator gains were se-lected as k1 = 3, k2 = 0.1, kC L = 1, and Γ = I , and the inte-gration window was selected as Δt = 0.1 s. Further discussionon how the parameters and gains were selected is provided inSection VII-C.

Fig. 2. During the first experiment, the target was commanded to follow avector field of this form.

A. Target Motion Along a Smooth Vector Field

In the first experiment, a stationary camera observed a tar-get moving in a vector field of the form shown in Fig. 2. AnIDS UI-3060CP camera was used to capture 1936× 1216 pixelresolution images at a rate of 60 frames per second. The func-tion ρ (η, t) introduced in Assumption 2 was used to determinethe estimated 2-D position of the target in the world coordinatesystem using the camera pose (see Remark 3). For this exper-iment, 81 kernels were used in the NN, with means arrangedin a uniform 9 × 9 grid across the vector field (see Fig. 2) andcovariance selected as Σk = 0.3I2 . A total of N = 600 datapoints were saved in the CL history stack. During the first 60 sof the experiment, target visibility was maintained to quicklyfill the CL history stack. Data were added at a rate of approxi-mately 1 sample per second (the rate at which the data selectionalgorithm described in Section V could be executed, which isconsidered approximate since the operating system does notnecessarily have deterministic execution cycles), resulting in10% of the history stack filled at the end of the initial learningphase. After the initial phase, periodic measurement loss wasinduced artificially by intermittently disregarding pose mea-surements and switching to the predictor. The dwell times foreach period were selected randomly as Δton

n ∼ U (15, 30) andΔtoff

n ∼ U (10, 20). The results of this experiment are shown inFig. 3 through Fig. 7. As shown in Figs. 3 and 4, the predictorinitially performs poorly; however, prediction significantly im-proves as more data are acquired. Boundedness of the unknownparameter estimates is validated in Fig. 5. Figs. 6 and 7 demon-strate that once sufficient data are acquired, the NN output tracksthe motion of the target well, therefore reducing the need forlarge feedback and sliding mode gains, as well as accuratelypredicting target motion when measurements are unavailable.Accurate motion prediction is achieved despite the target devi-ating from the prescribed vector field due to the nonholonomicconstraints on the mobile robot, as well as random disturbances


Fig. 3. Relative position estimates for the first experiment with a stationarycamera.

Fig. 4. Relative orientation estimates for the first experiment with a stationarycamera.

such as wheel slip. The norm of the position root-mean-squareerror (RMSE) vector was 0.25 m and the orientation RMSE was24.4◦ for this experiment, considering data after 200 s (i.e., afterinitial data collection).

For comparison, a modified version of the estimator was im-plemented on the same data collected during the first experiment.The modification represents the case of using an ideal predictor,analogous to one of the scenarios considered in [48]. Specifi-cally, the feedforward NN terms were replaced with the actualtarget velocities in this modification, leading to the position es-timates shown in Fig. 8. Despite the improved performance,

Fig. 5. Evolution of the NN ideal weight estimates during the first experimentwith a stationary camera.

Fig. 6. Output of the NN compared with ground truth linear velocities for thefirst experiment with a stationary camera. The jump in the ground truth signal atapproximately 450 s was caused by inaccurate numerical velocity approximationwhen the motion capture system temporary lost track of the target.

the target velocities are typically not available, and therefore,this design may not be implementable in many applications.However, through the learning scheme developed in this pa-per, the estimator performance quickly approaches that of theideal scenario, without requiring target motion information. Forcomparison, the norm of the position RMSE vector was 0.03 mand the orientation RMSE was 1.2◦ for this experiment usingthe ideal predictor, confirming the obvious notion that perfectvelocity prediction results in better performance.

An extended Kalman filter (EKF) with a constant velocitymodel was also tested for comparison. The measurement dataand visibility times were the same as in the first experiment,resulting in the position estimates shown in Fig. 9. As expected,


Fig. 7. Output of the NN compared with ground truth orientation rates (i.e.,12 B (q (t)) ωq (t)) for the first experiment with a stationary camera.

Fig. 8. Relative position estimates for the experiment with a stationary camerausing a modified observer with an ideal predictor. Although performance issatisfactory, this design requires unmeasurable target velocity information, andtherefore is not implementable in many applications.

the EKF performs well when the actual target velocities areconstant, in comparison to our estimator during the beginningof the experiment, since our estimator is still learning the mo-tion model. However, as the target velocity changes, as wouldbe common in many practical scenarios, our model-learningapproach outperforms the EKF, even during periods where thetarget velocity is constant. One way to improve the initial per-formance of our estimator (e.g., while the model parameters arestill being learned) would be to include the velocity of the target

Fig. 9. Relative position estimates for the experiment with a stationary camerausing a EKF with a constant velocity model. Without a good motion model, theEKF quickly diverges when the target is not in view.

Fig. 10. Overall setup of the second experiment, with a quadcopter observingthe targets as they move along a road network.

in the state, as is done in the EKF. However, the augmented statespace would require a larger NN, and therefore more data, andwould require estimating an acceleration motion model ratherthan a velocity motion model, which may not perform, as wellas a velocity motion model. With the EKF, the norm of the po-sition RMSE vector was 1.67 m and the orientation RMSE was57.5◦ for this experiment, both of which are much larger thanthe corresponding errors using our model learning approach.

B. Multiple Targets on a Road Network

A second experiment was performed to demonstrate the uti-lization of results developed in this paper to an application.Specifically, the goal of this experiment was to use a singlemoving camera to estimate the pose of two targets independentlymoving along an unknown road network (shown in Figs. 10 and11). At intersections in the road network, the targets randomlyselected a direction to travel, hence violating Assumption 2. Inthis experiment, a camera on-board a Parrot Bebop 2 quadcopterplatform was used to capture 640× 368 pixel resolution images(see Fig. 12), which were wirelessly streamed to an off-board


Fig. 11. During the second experiment, both targets traveled along this net-work, randomly selecting turns at intersections.

Fig. 12. View from the flying quadcopter. Only one target is visible in theFOV at a time.

computer at 30 frames per second. The function ρ (η, t) wasaugmented to also output the estimated target heading in theworld coordinate system, and the NN was composed of 172kernels, with mean positions evenly spaced along the roadsat 0.37 m intervals, mean headings parallel to the road, andcovariance Σk = 0.1I3 . Two independent instances of the esti-mator developed in this paper were used to estimate the targetposes, one for each target; however, for simplicity, since thetargets share a common road network, the CL history stack wasshared between the two estimators, with a total of N = 2000data points saved in the stack. Independent history stacks couldalso have been used. During the initial phase, the quadcopterwas commanded to follow a single target for approximately300 s, therefore acquiring enough data to reasonably approx-imate a motion model of the targets along the road network.After the initial phase, the quadcopter was commanded to fol-low whichever target was closest to an intersection, since this iswhere the assumptions are violated, i.e., a deterministic functionapproximator would not be expected to accurately approximatea stochastic function. After the target selected a direction, andleft the intersection, the predictor for this target is activated, andthe quadcopter follows the other target. This strategy matches a

Fig. 13. Position estimates of target 1 expressed in world coordinates for thesecond experiment with a quadcopter observing two moving targets.

Fig. 14. Orientation estimates of target 1 relative to the world coordinatesystem for the second experiment with a quadcopter observing two movingtargets.

reasonable strategy one might employ in a real world scenario:observe a target at intersections or other areas where the targetcan act randomly, but once the target has selected a direction, asufficiently learned predictor is expected to perform well, andthe observer can move on to other targets.

The results of this experiment are shown in Fig. 13 throughFig. 16, and a video demonstrating the experiment is avail-able at https://www.youtube.com/watch?v=QCIQtsQdhsM.Figs. 13–16 show the true and estimated pose of the targetsin world coordinates, thus, demonstrating that after sufficientdata are collected, the target pose can be accurately estimatedeven if the target remains outside the camera FOV for significant


Fig. 15. Position estimates of target 2 expressed in world coordinates for thesecond experiment with a quadcopter observing two moving targets.

Fig. 16. Orientation estimates of target 2 relative to the world coordinatesystem for the second experiment with a quadcopter observing two movingtargets.

durations, despite significant delay due to the wireless transmis-sion of the images, as well as the decreased measurement accu-racy compared to the first experiment due to the low-resolutioncamera.

C. Parameter Selection

As with many function approximation techniques, the param-eters used for the NN are dependent on the specific application.For the experiments discussed in the preceding sections, com-monly used RBFs were selected as the kernel since they exhibit

local similarity (i.e., for the experiments, nearby points in thestate space are expected to have similar velocity values, andRBFs have increasingly similar activation for increasingly sim-ilar inputs). As demonstrated by the results, RBFs performed sat-isfactorily, and therefore, more exotic kernels were not consid-ered, but could be explored for other applications as necessary.

The primary concern for selecting the number and distri-butions of the kernels is to ensure that the relevant parts of thestate space (i.e., the areas where the targets are expected to movethrough) have nonzero kernel activation. The secondary concernis to match the density and parameters of the kernels (e.g., meanand variance for RBFs) to the complexity of the underlying vec-tor field to be approximated. In other words, regions of the vectorfield that are expected to have large spatial derivatives shouldbe approximated with a dense distribution of kernels, each withrelatively little extent (low variance for RBFs). If little is knownabout where the target may travel or how aggressively it maymaneuver, a conservative approach can be taken, where a verylarge number of kernels can be distributed over a large sectionof the state space, and then packing the kernels densely and withtight spatial extent. It is not surprising that with no knowledge ofthe operational space the resulting conservative approach wouldhave increased memory and computational requirements.

In each experiment, the centers of the RBFs were distributedacross the vector field and road network, respectively. For thesecond experiment, where the output of ρ also included the targetheading, the kernel centers were doubled, one for each of the twodirections parallel to the road. Kernel centers were separated bya distance approximately 0.3 m and had variance of 0.3I2 and0.1I3 for the first and second experiment, respectively, wherethe smaller covariance was selected for the second experimentdue to the tight turns in the road map. In both experiments, theseinitial parameter values performed satisfactorily, suggesting thisapproach is insensitive to NN tuning.

The two experiments use two different imaging sensors withvarying capabilities. In the first experiment, a stationary high-resolution camera is used to capture images at a high frame rate,and images are transferred over a wired connection with lowlatency and without compression. In the second experiment, amoving low-resolution camera is used to capture images at arelatively lower frame rate, and the images are transferred overa wireless connection with high latency and lossy compression.The experiments demonstrate the viability of our approach inboth cases using almost identical estimator parameters (withminor differences in the NN kernel parameters, as describedpreviously), suggesting insensitivity to estimator gains despite,e.g., frame drops, delay, lower measurement accuracy due to thelower image resolution, etc.

Minor tuning of the gains (k1 , k2 , kCL, and Γ) was requiredbeyond the initial values of 1 or I . Since signum functionsare known to produce high-frequency chatter, k2 was set to0.1, and k1 was set to 3.0 to yield a desirable estimate conver-gence rate when measurements were available. The integrationtime window Δt was selected to be approximately equal to thetimescale of changes in the target velocity. Our initial selectionof Δt = 0.1 resulted in satisfactory estimator performance, andtherefore, was not adjusted.


D. Discussion

Beyond the restriction on the target behavior to ensure learn-ing is possible based on Assumption 2, the preceding exper-iments demonstrate that the remaining assumptions are eithereasily satisfied or do not significantly hinder estimator perfor-mance when not met. As discussed in Remark 1, Assumption1 (i.e., availability of pose measurements) can be satisfied bycurrently available computer vision techniques and minimal do-main knowledge. Assumption 3 (boundedness of the target pose)is easily satisfied in any practical scenario. Assumption 4 (suf-ficient richness in the data) is required to ensure NN parameterestimate convergence; however, predictor performance may besufficient without it. For example, in the preceding experiments,the calculated minimum eigenvalue of the history stack re-mained within the floating point precision floor of the computersystem, suggesting the history stack is not full rank. Despite thatFigs. 6 and 7 demonstrate satisfactory predictor performance.

VIII. CONCLUSION

An adaptive observer and predictor were developed to esti-mate the relative pose of a target from a camera in the presenceof intermittent measurements. While measurements are avail-able, data are recorded and used to update an estimate of thetarget motion model. When measurements are not available, themotion model is used in a predictor to update state estimates.The overall framework is shown to yield ultimately bounded es-timation errors, where the bound can be made arbitrarily smallthrough gain tuning, increasing data richness, and function ap-proximation tuning. Experimental results demonstrate the per-formance of the developed estimator, even in cases of stochastictarget motion where the assumptions are violated.

Although the experiments demonstrate a robustness to moder-ate violations of Assumption 2 (i.e., the targets in the experimentwith the road network do not always follow a static vector field),estimation and prediction where a model does not exist (e.g., thetarget follows a nonperiodic, time-varying trajectory) is still anopen problem. However, it may be possible to use the methodsdeveloped in this paper to track objects undergoing a wider classof motions through relaxation of Assumption 2. First, states be-yond just the object pose (e.g., traffic levels) can be used in theneural network to predict the object velocities. This can be usedto account for varying object behavior without relying on ex-plicit time dependence. Furthermore, object velocity models canbe expanded to allow for piecewise constant mappings betweenthe object state and object velocity. This way, data collected toestimate the model is ensured to be valid for at least a finiteperiod of time, while divergence of the measurements from pre-diction can be used as an indication that new data need to becollected, or that other components of the system (e.g., featuretracking) are producing erroneous output. This approach couldallow the tracked objects to switch their mode of operation.

REFERENCES

[1] D. Braganza, D. M. Dawson, and T. Hughes, “Euclidean position esti-mation of static features using a moving camera with known velocities,”in Proc. IEEE Conf. Decis. Control, New Orleans, LA, USA, Dec. 2007,pp. 2695–2700.

[2] O. Dahl, Y. Wang, A. F. Lynch, and A. Heyden, “Observer forms forperspective systems,” Automatica, vol. 46, no. 11, pp. 1829–1834, Nov.2010.

[3] N. Zarrouati, E. Aldea, and P. Rouchon, “So(3)-invariant asymptotic ob-servers for dense depth field estimation based on visual data and knowncamera motion,” in Proc. Amer. Control Conf., Montreal, QC, Canada,Jun. 2012, pp. 4116–4123.

[4] A. Chiuso, P. Favaro, H. Jin, and S. Soatto, “Structure from motion causallyintegrated over time,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24,no. 4, pp. 523–535, Apr. 2002.

[5] R. Spica, P. R. Giordano, and F. Chaumette, “Active structure from motion:Application to point, sphere, and cylinder,” IEEE Trans. Robot., vol. 30,no. 6, pp. 1499–1513, Dec. 2014.

[6] A. Dani, N. Fischer, Z. Kan, and W. E. Dixon, “Globally exponentiallystable observer for vision-based range estimation,” Mechatronics, vol. 22,no. 4, pp. 381–389, 2012.

[7] W. E. Dixon, Y. Fang, D. M. Dawson, and T. J. Flynn, “Range identificationfor perspective vision systems,” IEEE Trans. Autom. Control, vol. 48,no. 12, pp. 2232–2238, Dec. 2003.

[8] D. Karagiannis and A. Astolfi, “A new solution to the problem of rangeidentification in perspective vision systems,” IEEE Trans. Autom. Control,vol. 50, no. 12, pp. 2074–2077, Dec. 2005.

[9] X. Chen and H. Kano, “A new state observer for perspective systems,”IEEE Trans. Autom. Control, vol. 47, no. 4, pp. 658–663, Apr. 2002.

[10] L. Ma, C. Cao, N. Hovakimyan, C. Woolsey, and W. E. Dixon, “Fastestimation and range identification in the presence of unknown motionparameters,” IMA J. Appl. Math., vol. 75, no. 2, pp. 165–189, 2010.

[11] G. Hu, D. Aiken, S. Gupta, and W. Dixon, “Lyapunov-based range iden-tification for a paracatadioptric system,” IEEE Trans. Autom. Control,vol. 53, no. 7, pp. 1775–1781, Aug. 2008.

[12] X. Chen and H. Kano, “State observer for a class of nonlinear systems andits application to machine vision,” IEEE Trans. Autom. Control, vol. 49,no. 11, pp. 2085–2091, Nov. 2004.

[13] A. Dani, Z. Kan, N. Fischer, and W. E. Dixon, “Structure and motionestimation of a moving object using a moving camera,” in Proc. Amer.Control Conf., Baltimore, MD, USA, 2010, pp. 6962–6967.

[14] A. Dani, N. Fischer, and W. E. Dixon, “Single camera structure and mo-tion,” IEEE Trans. Autom. Control, vol. 57, no. 1, pp. 241–246, Jan. 2012.

[15] D. Liberzon, Switching in Systems and Control. Cambridge, MA, USA:Birkhauser, 2003.

[16] M. Sznaier and O. Camps, “Motion prediction for continued autonomy,”in Encyclopedia of Complexity and Systems Science. New York, NY, USA:Springer, 2009, pp. 5702–5718.

[17] M. Sznaier, O. Camps, N. Ozay, T. Ding, G. Tadmor, and D. Brooks,“The role of dynamics in extracting information sparsely encoded in highdimensional data streams,” in Dynamics of Information Systems, vol. 40,New York, NY, USA: Springer, 2010, pp. 1–27.

[18] T. Ding, M. Sznaier, and O. Camps, “Receding horizon rank minimizationbased estimation with applications to visual tracking,” in Proc. IEEE Conf.Decis. Control, Cancun, Mexico, Dec. 2008, pp. 3446–3451.

[19] M. Ayazoglu, B. Li, C. Dicle, M. Sznaier, and O. I. Camps, “Dynamicsubspace-based coordinated multicamera tracking,” in Proc. IEEE Int.Conf. Comput. Vis., Barcelona, Spain, Nov. 2011, pp. 2462–2469.

[20] M. Yang, Y. Wu, and G. Hua, “Context-aware visual tracking,” IEEETrans. Pattern Anal. Mach. Intell., vol. 31, no. 7, pp. 1195–1209, Jul.2009.

[21] H. Grabner, J. Matas, L. V. Gool, and P. Cattin, “Tracking the invisible:Learning where the object might be,” in Proc. IEEE Conf. Comput. Vis.Pattern Recognit., San Francisco, CA, USA, Jun. 2010, pp. 1285–1292.

[22] L. Cerman, J. Matas, and V. Hlavac, “Sputnik tracker: Having a companionimproves robustness of the tracker,” in Proc. Scand. Conf., Oslo, Norway,Jun. 2009, pp. 291–300.

[23] B. R. Fransen, O. I. Camps, and M. Sznaier, “Robust structure frommotion and identified dynamics,” in Proc. IEEE. Int. Conf. Comput. Vis.,Oct. 2005, pp. 772–777.

[24] H. Durrant-Whyte and T. Bailey, “Simultaneous localization and mapping:Part I,” IEEE Robot. Autom. Mag., vol. 13, no. 2, pp. 99–110, Jun. 2006.

[25] T. Bailey and H. Durrant-Whyte, “Simultaneous localization and mapping(SLAM): Part II,” IEEE Robot. Autom. Mag., vol. 13, no. 3, pp. 108–117,Sep. 2006.

[26] M. W. M. G. Dissanayake, P. Newman, S. Clark, H. F. Durrant-Whyte,and M. Csorba, “A solution to the simultaneous localization and mapbuilding (SLAM) problem,” IEEE Trans. Robot. Autom., vol. 17, no. 3,pp. 229–241, Jun. 2001.

[27] G. P. Huang, A. I. Mourikis, and S. I. Roumeliotis, “A quadratic-complexity observability-constrained unscented Kalman filter for SLAM,”IEEE Trans. Robot., vol. 29, no. 5, pp. 1226–1243, Oct. 2013.


[28] J. Sola, A. Monin, M. Devy, and T. Vidal-Calleja, “Fusing monocularinformation in multicamera SLAM,” IEEE Trans. Robot., vol. 24, no. 5,pp. 958–968, Oct. 2008.

[29] J. A. Hesch, D. G. Kottas, S. L. Bowman, and S. I. Roumeliotis, “Con-sistency analysis and improvement of vision-aided inertial navigation,”IEEE Trans. Robot., vol. 30, no. 1, pp. 158–176, Feb. 2014.

[30] T. Lupton and S. Sukkarieh, “Visual-inertial-aided navigation for high-dynamic motion in built environments without initial conditions,” IEEETrans. Robot., vol. 28, no. 1, pp. 61–76, Feb. 2012.

[31] M. Kaess, A. Ranganathan, and F. Dellaert, “iSAM: Incremental smooth-ing and mapping,” IEEE Trans. Robot., vol. 24, no. 6, pp. 1365–1378,Dec. 2008.

[32] M. Montemerlo, S. Thrun, D. Koller, and B. Wegbreit, “FastSLAM 2.0:An improved particle filtering algorithm for simultaneous localization andmapping that provably converges,” in Proc. Int. Joint Conf. Artif. Intell.,2003, pp. 1151–1156.

[33] G. Grisetti, C. Stachniss, and W. Burgard, “Improved techniques for gridmapping with Rao-Blackwellised particle filters,” IEEE Trans. Robot.,vol. 23, no. 1, pp. 34–46, Feb. 2007.

[34] E. Mazor, A. Averbuch, Y. Bar-Shalom, and J. Dayan, “Interacting mul-tiple model methods in target tracking: A survey,” IEEE Trans. Aerosp.Electron. Syst., vol. 34, no. 1, pp. 103–123, Jan. 1998.

[35] S. S. Blackman, Multiple-Target Tracking With Radar Applications.Norwood, MA, USA: Artech House, 1986.

[36] K. Granstrom and U. Orguner, “A PHD filter for tracking multiple ex-tended targets using random matrices,” IEEE Trans. Signal Process.,vol. 60, no. 11, pp. 5657–5671, Nov. 2012.

[37] K. Wyffels and M. Campbell, “Negative observations for multiple hypoth-esis tracking of dynamic extended objects,” in Proc. Amer. Control Conf.,2014, pp. 642–647.

[38] K. Wyffels and M. Campbell, “Negative information for occlusion rea-soning in dynamic extended multiobject tracking,” IEEE Trans. Robot.,vol. 31, no. 2, pp. 425–442, Apr. 2015.

[39] A. Andriyenko, S. Roth, and K. Schindler, “An analytical formulation ofglobal occlusion reasoning for multi-target tracking,” in Proc. IEEE Int.Conf. Comput. Vis. Workshop, 2011, pp. 1839–1846.

[40] H. Wei et al., “Camera control for learning nonlinear target dynamicsvia Bayesian nonparametric Dirichlet-process Gaussian-process (DP-GP)models,” in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., 2014, pp. 95–102.

[41] W. Lu, G. Zhang, S. Ferrari, and I. Palunko, “An information potentialapproach for tracking and surveilling multiple moving targets using mobilesensor agents,” in Proc. SPIE 8045 Unmanned Syst. Technol. XIII, 2011,pp. 1–13.

[42] J. Joseph, F. Doshi-Velez, A. S. Huang, and N. Roy, “A Bayesian non-parametric approach to modeling motion patterns,” Auton. Robot., vol. 31,no. 4, pp. 383–400, 2011.

[43] M. K. C. Tay and C. Laugier, “Modelling smooth paths using Gaussianprocesses,” in Field and Service Robotics (Series Springer Tracts in Ad-vanced Robotics vol. 42). Berlin, Germany: Springer, 2008, pp. 381–390.

[44] J. Ko and D. Fox, “Gp-Bayesfilters: Bayesian filtering using Gaussianprocess prediction and observation models,” Auton. Robot., vol. 27, no. 1,pp. 75–90, 2009.

[45] D. Ellis, E. Sommerlade, and I. Reid, “Modelling pedestrian trajectorypatterns with Gaussian processes,” in Proc. IEEE Int. Conf. Comput. Vis.Workshops, 2009, pp. 1229–1234.

[46] B. T. Morris and M. M. Trivedi, “A survey of vision-based trajectorylearning and analysis for surveillance,” IEEE Trans. Circuits Syst. VideoTechnol., vol. 18, no. 8, pp. 1114–1127, Aug. 2008.

[47] T. Campbell, S. Ponda, G. Chowdhary, and J. P. How, “Planning underuncertainty using Bayesian nonparametric models,” in Proc. AIAA Guid.Navig. Control Conf., Aug. 2012, pp. 1–24.

[48] A. Parikh, T.-H. Cheng, H.-Y. Chen, and W. E. Dixon, “A switched systemsframework for guaranteed convergence of image-based observers withintermittent measurements,” IEEE Trans. Robot., vol. 33, no. 2, pp. 266–280, Apr. 2017.

[49] A. Parikh, T.-H. Cheng, R. Licitra, and W. E. Dixon, “A switched systemsapproach to image-based localization of targets that temporarily leave thecamera field of view,” IEEE Trans. Control Syst. Technol., pp. 1–8, Oct.,2017, doi: 10.1109/TCST.2017.2753163.

[50] P. Ioannou and J. Sun, Robust Adaptive Control. Englewood Cliffs, NJ,USA: Prentice-Hall, 1996.

[51] K. Narendra and A. Annaswamy, Stable Adaptive Systems. EnglewoodCliffs, NJ, USA: Prentice-Hall, 1989.

[52] S. Sastry and M. Bodson, Adaptive Control: Stability, Convergence, andRobustness. Upper Saddle River, NJ, USA: Prentice-Hall, 1989.

[53] G. V. Chowdhary and E. N. Johnson, “Theory and flight-test validation ofa concurrent-learning adaptive controller,” J. Guid. Control Dyn., vol. 34,no. 2, pp. 592–607, Mar. 2011.

[54] G. Chowdhary, “Concurrent learning for convergence in adaptive con-trol without persistency of excitation,” Ph.D. dissertation, Georgia Inst.Technol., Atlanta, GA, USA, Dec. 2010.

[55] G. Chowdhary, T. Yucelen, M. Muhlegg, and E. N. Johnson, “Concurrentlearning adaptive control of linear systems with exponentially convergentbounds,” Int. J. Adapt. Control Signal Process., vol. 27, no. 4, pp. 280–301,2013.

[56] H. Schaub and J. Junkins, Analytical Mechanics of Space Systems (AIAAEducation Series). New York, NY, USA: AIAA, 2003.

[57] D. H. Titterton and J. L. Weston, Strapdown Inertial Navigation Technol-ogy (Series IEE Radar, Sonar, Navigation and Avionics Series), 2nd ed.London, U.K.: The Institution of Electrical Engineers, 2004.

[58] Y. Ma, S. Soatto, J. Kosecka, and S. Sastry, An Invitation to 3-D Vision.New York, NY, USA: Springer, 2004.

[59] B. M. Haralick, C.-N. Lee, K. Ottenberg, and M. Nolle, “Review and anal-ysis of solutions of the three point perspective pose estimation problem,”Int. J. Comput. Vision, vol. 13, no. 3, pp. 331–356, Dec. 1994.

[60] R. Horaud, B. Conio, O. Leboulleux, and B. Lacolle, “An analytic solutionfor the perspective 4-point problem,” in Proc. IEEE Conf. Comput. Vis.Pattern Recognit., 1989, pp. 500–507.

[61] D. DeMenthon and L. S. Davis, “Exact and approximate solutions of theperspective-three-point problem,” IEEE Trans. Pattern Anal. Mach. Intell.,vol. 14, no. 11, pp. 1100–1105, Nov. 1992.

[62] X.-S. Gao, X.-R. Hou, J. Tang, and H.-F. Cheng, “Complete solution clas-sification for the perspective-three-point problem,” IEEE Trans. PatternAnal. Mach. Intell., vol. 25, no. 8, pp. 930–943, Aug. 2003.

[63] L. Quan and Z. Lan, “Linear N-point camera pose determination,” IEEETrans. Pattern Anal. Mach. Intell., vol. 21, no. 8, pp. 774–780, Aug. 1999.

[64] T. Q. Phong, R. Horaud, A. Yassine, and P. D. Tao, “Object pose from2-d to 3-d point and line correspondences,” Int. J. Comput. Vision, vol. 15,no. 3, pp. 225–243, 1995.

[65] D. DeMenthon and L. Davis, “Model-based object pose in 25 lines ofcode,” Int. J. Comput. Vision, vol. 15, pp. 123–141, 1995.

[66] N. R. Gans, A. Dani, and W. E. Dixon, “Visual servoing to an arbitrarypose with respect to an object given a single known length,” in Proc. Amer.Control Conf., Seattle, WA, USA, Jun. 2008, pp. 1261–1267.

[67] K. Dupree, N. R. Gans, W. MacKunis, and W. E. Dixon, “Euclidean calcu-lation of feature points of a rotating satellite: A daisy chaining approach,”AIAA J. Guid. Control Dyn., vol. 31, pp. 954–961, 2008.

[68] M. H. Stone, “The generalized weierstrass approximation theorem,” Math.Mag., vol. 21, no. 4, pp. 167–184, 1948.

[69] M. Krstic, I. Kanellakopoulos, and P. V. Kokotovic, Nonlinear and Adap-tive Control Design. New York, NY, USA: Wiley, 1995.

[70] W. E. Dixon, A. Behal, D. M. Dawson, and S. Nagarkatti, NonlinearControl of Engineering Systems: A Lyapunov-Based Approach. Boston,MA, USA: Birkhauser, 2003.

[71] H. K. Khalil, Nonlinear Systems, 3rd ed. Upper Saddle River, NJ, USA:Prentice-Hall, 2002.

[72] W. Rudin, Principles of Mathematical Analysis. New York, NY, USA:McGraw-Hill, 1976.

[73] S. Garrido-Jurado, R. Munoz-Salinas, F. Madrid-Cuevas, and M. Marin-Jimenez, “Automatic generation and detection of highly reliable fiducialmarkers under occlusion,” Pattern Recognit., vol. 47, no. 6, pp. 2280–2292, Jun. 2014.

[74] Aruco library, 2014. [Online]. Available: http://www.uco.es/investiga/grupos/ava/node/26

Anup Parikh received the B.S. degree in mechan-ical and aerospace engineering, the M.S. degree inmechanical engineering, and the Ph.D. degree inaerospace engineering, all from University of Florida,Gainesville, FL, USA.

He is currently a Postdoctoral Researcher with theRobotics R&D group, Sandia National Laboratories,Albuquerque, NM, USA. His primary research inter-ests include Lyapunov-based estimation and controltheory, machine vision, and autonomous systems.

http://dx.doi.org/10.1109/TCST.2017.2753163

http://www.uco.es/investiga/grupos/ava/node/26.

http://www.uco.es/investiga/grupos/ava/node/26.


Rushikesh Kamalapurkar received the M.S. andPh.D. degrees from the Department of Mechan-ical and Aerospace Engineering, University ofFlorida, Gainesville, FL, USA, in 2011 and 2014,respectively.

After working for a year as a Postdoctoral Re-search Fellow with Dr. W. E. Dixon, he was selectedas the 2015–2016 MAE Postdoctoral Teaching Fel-low. In 2016, he joined the School of Mechanical andAerospace Engineering, Oklahoma State University,Stillwater, OK, USA, as an Assistant Professor. He

has authored/coauthored 3 book chapters, over 20 peer-reviewed journal papers,and over 20 peer-reviewed conference papers. His primary research interest isintelligent, learning-based optimal control of uncertain nonlinear dynamicalsystems.

Dr. Kamalapurkar’s work has been recognized by the 2015 University ofFlorida Department of Mechanical and Aerospace Engineering Best Disserta-tion Award and the 2014 University of Florida Department of Mechanical andAerospace Engineering Outstanding Graduate Research Award.

Warren E. Dixon (F’16) received the Ph.D. degreefrom the Department of Electrical and Computer En-gineering, Clemson University, Clemson, SC, USA,in 2000. He was a Research Staff Member and EugeneP. Wigner Fellow with Oak Ridge National Labora-tory (ORNL) until 2004, when he joined the Depart-ment Mechanical and Aerospace Engineering, Uni-versity of Florida, Gainesville, FL, USA. His mainresearch interest has been the development and ap-plication of Lyapunov-based control techniques foruncertain nonlinear systems.

Dr. Dixon has received notable recognition of his work, including the 2009and 2015 American Automatic Control Council (AACC) O. Hugo Schuck (BestPaper) Award, the 2013 Fred Ellersick Award for Best Overall MILCOM Pa-per, the 2011 American Society of Mechanical Engineers (ASME) DynamicsSystems and Control Division Outstanding Young Investigator Award, the 2006IEEE Robotics and Automation Society (RAS) Early Academic Career Award,and NSF CAREER Awards (2006-2011). He is an ASME Fellow.

Documents

IEEE TRANSACTIONS ON ROBOTICS, VOL. 34, NO. 3 ...ncr.mae.ufl.edu/papers/TRO18b.pdf806 IEEE TRANSACTIONS ON ROBOTICS, VOL. 34, NO. 3, JUNE 2018 utilize a predictor similar to that developed