Information-driven learning, distributed fusion, and planningARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation ... This year:

Value of

Information

1st year review. UCLA 2012

VOI

Year 2 review. ARL, Sept 9 2013

ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation

Value of

Information


ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation

VOI


Information-driven learning, distributed fusion, and planning

Co-PI Alfred Hero University of Michigan

Value of

Information


VOI

Year 2 review. ARL, Sept 9 2013 1st year review. UCLA 2012

Value of

Information VOI


Our main thrust this year

• Progress 1: Information-driven learning: – Learning structure in high dimension: Kronecker PCA – For spatio-temporal sources Kronecker PCA captures information

much more efficiently than standard (low-rank) PCA

• Progress 2: Distributed information fusion: – Distributed inference: local 2nd order nbd marginalization – Performs as well as global fusion w/o message passing

• Progress 3: Human-in-the-loop planning and processing

– Cooperative human-machine 20 questions framework – Human adds early information gain for target detection

Quantify and optimize VoI by: dimensionality reduction, nearest neighbor aggregation, and human-in-the-loop.

Value of

Information


VOI


Value of

Information VOI


Progress 1: Information driven learning Last year: KGlasso – MSE scaling laws

Tsiligkaridis, Hero, Zhou, 2012

Una

chie

vabl

e re

gion

• KGlasso has best scaling law in n,p • Task: Estimate spatio-temporal covariance

0 10 20 30 40 50 60 70 80 90 100

20 uncorrelated sequences

Time index i

Value of

Information


VOI


Value of

Information VOI


Progress 1: Information-driven learning This year: Kronecker PCA (See poster)

[1] Tsiligkaridis & H TSP 2013

Standard PCA Kronecker PCA

||𝐂 − 𝐂𝑟 ||𝐹2 ≤ 𝐦𝐦𝐦𝒓𝒓𝒓𝒓 𝐑 ≤𝒓

||𝐑 − 𝑹 𝐂 ||𝑭𝟐 + 𝑪𝒓 𝒑𝟐 + 𝒒𝟐

𝒓

Theorem [1]: For pq x pq covariance C the MSE of Kronecker PCA approximation Cr to C is bounded

r r

Deficiency: Single KP may not be adequate fit to covariance A Solution: Use a sum of KPs to approximate covariance -> K-PCA

Property (Pistsianis and Van Loan 1992): K-PCA is complete expansion Our K-PCA algorithm: Spectral solution to convex minimization problem

Value of

Information


VOI


Value of

Information VOI


Progress 1: Information-driven learning Kronecker PCA (See poster)

[1] Tsiligkaridis & H TSP 2013

Kronecker spectrum Eigenspectrum

Value of

Information


VOI


Value of

Information VOI


Progress 2: Distributed information fusion in sensor networks

Meng,Wei,Wiesel,H, AISTATS 2013(NP Award)

||𝐂 − 𝐂𝑟 ||𝐹2

≤ 𝐦𝐦𝐦𝒓𝒓𝒓𝒓 𝐑 ≤𝒓

||𝐑 − 𝑹 𝐂 ||𝑭𝟐 + 𝑪𝒓 𝒑𝟐 + 𝒒𝟐

𝒓

Theorem [1]: For pq x pq sample cov C the MSE of Kronecker PCA of rank r is bounded

Objective: predict states measured by SN Model: Gauss-Markov random field:

Standard: Iterative ML by message passing Proposed: Non-iterative by 2-NN relaxation

Value of

Information


VOI


Value of

Information VOI


Progress 3: Human-in-the-loop processing Cooperative localization (See poster)

Tsiligkaridis, Sadler & Hero, ICASSP 2013

Human MSE gain ratio

Optimal queries are equalizing bisection rules

Human advantage: can account for context Human limitation: limited visual accuity

Value of

Information


VOI


Value of

Information VOI


Summary of year 2 activities

• This year’s research directly impacts – Information-driven learning

• Kronecker PCA provides much better fit to spatio-temporal data than standard PCA. VoI for prediction/detection/classification is improved.

– Distributed information fusion • Second-order neighborhood information has nearly as high

value as global information about SN. – Information exploitation

• Inclusion of human-in-the-loop provides up to 15% MSE gain in early iterations. Value of human-provided information characterized by resolution acuity parameter kappa.

Value of

Information


VOI


Value of

Information VOI


Ongoing and future focus areas and collaborations

• Working towards a unified VoI theory of sensing of spatio-temporal processes: – VoI-driven mission planning with target-dependent payoffs

(UM/MIT) – (Poster today - Mu, Newstadt, How, H) – Information theoretic bounds and algorithms for

learning/fusion/planning that account for side-information (human inputs, low rank, sparse, Kronecker structure)

– Information geometric theory of VoI (UM/ASU)

• Applications to anomaly detection in video, SNs and radar (UM/OSU/AFRL).

• Experimental validation studies: – Software defined radar testbed (UM/OSU/ASU/MIT). – Refined human models and experiments (UM/UCSD)

Value of

Information


VOI


Value of

Information VOI


Publications in year 2 • T. Tsiligkaridis, A.O. Hero, S. Zhou, "Convergence properties of

Kronecker graphical lasso algorithms," IEEE Trans on SP, 2013 • T. Tsiligkaridis N and A.O. Hero, ``Covariance Estimation in High

Dimensions via Kronecker Product Expansions,'' IEEE Trans on SP, 2013.

• D. Wei and A.O. Hero, "Multistage adaptive estimation of sparse Signals," IEEE Journal of Selected Topics in Signal Processing, 2013.

• Dennis Wei and Alfred O. Hero, III, "Adaptive spectrum sensing and estimation," ICASSP 2013, Vancouver.

• Z. Meng, D. Wei, A. Wiesel, A.O. Hero, "Distributed Learning of Gaussian Graphical Models via Marginal Likelihoods,“ AISTAT.

• Theodoros Tsiligkaridis and Alfred O. Hero III, "Low Separation Rank Covariance Estimation using Kronecker Product Expansions," IEEE ISIT 2013, Istanbul.

• Theodoros Tsiligkaridis, Brian M. Sadler and Alfred O. Hero III, "A collaborative 20 questions model for target search with human-machine interaction," ICASSP 2013, Vancouver.

Documents

Information-driven learning, distributed fusion, and planningARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation ... This year: