VOI Year 3 review. UCB, Nov. 20 2014 Year 3 review. UCB, Nov. 20 2014 VOI Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation [http://wiki.eecs.umich.edu/voimuri] ARO W911NF-11-1-0391 Program manager: Liyi Dai Investigators: Al Hero (PI), Raj Nadakuditi, John Fisher, Jon How, Alan Willsky, Randy Moses, Emre Ertin, Angela Yu, Michael Jordan, Stefano Soatto, Doug Cochran 3 rd VoI MURI Review 2014
Value-centered Information Theory for Adaptive Learning, Inference, Tracking,
and Exploitation
[http://wiki.eecs.umich.edu/voimuri]
ARO W911NF-11-1-0391Program manager: Liyi Dai
Investigators: Al Hero (PI), Raj Nadakuditi, John Fisher, Jon How, Alan Willsky, Randy Moses, Emre Ertin, Angela Yu, Michael Jordan, Stefano Soatto, Doug Cochran
3rd VoI MURI Review 2014
Presenter
Presentation Notes
Team carefully composed based on skills and track record in building theory and finding its practical application in adaptive sensing (information collection). Statistical machine learning: To develop applicable theory for quantifying value of information and algorithms that maximize value. Distributed information fusion: to apply this theory to information aggregation and delivery for complex high dimensional networked scenarios. Information exploitation: to take this aggregated information and use it to control sensor actions for optimizing mission tasks such as autonomous vision, UAV strike planning, and multilayer multiscale situational awareness NB: We consider "information collection" or "information gathering" in addition to "sensing". There is a general DoD interest in using combinations of hard and soft data data sources for inference. Hero: (PI) information theoretic learning, adaptive detection/estimation/tracking, statistical performance analysis, large scale information fusion, variable selection and DR. Nadakuditi: spectral theory of high dimensional matrices, free probability, DR in sample starved regimes, non-commutative info theory Moses: wide area sensor networks, localization and tracking, radar SP, radar imaging Ertin: Statistical Signal Processing, Distributed and Sequential Detection, with applications to Sensor Networks and Radar Systems, predator-prey pursuer games. WSN testbed. Fisher: multimodality information fusion, large scale graphical models, networked sensor planning. How: autonomous navigation and distributed control. Multiple platform UAV testbed. Willsky: sparse signal representations, dictionary learning, hierarchical multiresolution dyamical representations. Jordan: statistical machine learning theory, non-parametric Bayes methods, graphical models, asymptotic statistics, performance modeling. Soatto: robotic vision systems and control, shape theory, actionable information. Cochran: integrated sensing and processing, statistical manifolds and information geometry Yu: cognitive sciences, multiple agent learning
Information extraction - data to information (fusion of soft and hard data) Information exploitation – information to action (deploy new sensor) Quality of information – quantified uncertainty (entropy) Value of information – exploitation utility (did action have desired effect?) Example of mission information: a report of a supposed threat exists in a staging area. The threat is likely to be class A, B, or C and has prior probability P(A), P(B), P(C). Example of mission objective: investigate the threat and report back nature and level of threat (e.g., certain, immediate and deadly) Example of mission support task: Support the mission by detecting and classifying hard and soft targets in the staging area. Note: VoI for mission objective may be different than VoI for a mission task. Definition: Information-driven learning – learning that accounts for context, physics-based models, and mission objectives. Our goal: Taking into account the mission, the constraints on assets, and specific objective we seek to develop theory and algorithms that learn how to robustly extract relevant information from complex multimodal data. Relevant tasks include detection, estimation, classification, tracking, strike and confirm strike. The theory must account for factors such as Small sample size –Nadakuditi/Hero/Jordan; Poorly specified target and clutter models –Moses/Jordan/Soatto, Feedback control actions – How/Soatto/Cochran Hostile or adversarial environments – Ertin Computation/communication constraints – How/Fisher Distributed sensing resources – Moses/Hero/Nadakuditi/How/Fisher Time-critical decision making – How/Soatto/Cochran (control actions) Jordan/Hero/Nadakuditi (learning rates) Fisher/Moses/Hero (computational/communication complexity) To do this we will develop Systematic Analytical Framework that blends physics-based models (quantify robustness in real world conditions), learning-based algorithms (generalize beyond physics model using data), and context based assessments (use soft information to condition the model). Example of physics-based model: target RCS generates diffuse and specular radar reflections giving rise to a Chi-squared distribution (Swerling class II model). Example of learning based model: a kernel density estimate or kNN cluster-tree (Hartigan) of feature distribution. A SVM classifier learned from sample instances. Example of contextual information: urban vs rural area, reports of insurgent sightings in area, historical record of IED’s on a certain stretch of road.
• To derive a comprehensive set of principles for task-specific information extraction, distributed information fusion, and information exploitation that can be used to design the next generation of autonomous and adaptive sensing systems.
• Specific objectives:
• Develop analytical frameworks for quantifying value of information. • Study fundamental tradeoffs for information collection and fusion• Develop info processing algorithms with performance guarantees • Validate theory and algorithms on sensing testbeds at MIT, OSU,
UCSD and UCLA
• Technical approach: value-centered information theory, machine learning and control.
Our MURI’s principal aim
Presenter
Presentation Notes
Statistical machine learning: To develop applicable theory for quantifying value of information and algorithms that maximize value. Distributed information fusion: to apply this theory to information aggregation and delivery for complex high dimensional networked scenarios. Information exploitation: to take this aggregated information and use it to control sensor actions for optimizing mission tasks such as autonomous vision, UAV strike planning, and multilayer multiscale situational awareness Mostly directly out of the proposal. This is just to get the ball rolling.
Info sources: Collected sensor data: raw data has no meaning without information about relevancy to decision task Target/clutter signatures: informs the processing about likely models for observed data. Contextual information: scenario contingent target/clutter models, data quality, time-location, historical records, et Decision tasks: Each task is related but not necessary best served by same information pattern generated by the sensing policy Processing constraints: These restrict the type of policies, and hence the quality and value of information made available to decisionmaker. Time&energy: information that is stale has no value. Information that cannot be collected within the time-energy constraint will be stale. Computation&Memory: this can be traded for accuracy as can the others. Notoriously hard to deal with this constraint in rigorous manner – but Chandrasekharan and Jordan have a nice restricted problem where they get mathematical results. Also note sources of "soft" data and constraints on humans-in-the-loop as data processors
There are many mature fields that have grown around the problem of information collection, decisionmaking with practical constraints. Before our MURI there was no theory that addresses all three simultaneously in the context of distributed sensing, fusion, and planning.
• Information-driven structure learning and representation• Learning in graphical models latent variable structures [Hero, How, Fisher] • Trade-offs between complexity and performance [Ertin, Fisher, Hero, Jordan]• Representations of information for video analysis [Hero, Soatto]
• Distributed information fusion• Decentralized learning and local information aggregation [Ertin, Hero, Jordan, Moses]• Subspace processing and fusion of information [Ertin, Jordan, Nadakuditi]• Robust information-driven fusion [Ertin, Fisher, Hero]• Multimodality fusion with missing or unreliable information [Hero]
• Active information exploitation for resource management• VoI proxies for resource management [Ertin, Hero, Cochran]• Information geometric foundations [Cochran, Hero]• Models of human and human-machine interaction [Hero, Yu]• Mission adaptive planning with convex proxies [Hero, How]
Research highlights
Presenter
Presentation Notes
Statistical machine learning: To develop applicable theory for quantifying value of information and algorithms that maximize value. Distributed information fusion: to apply this theory to information aggregation and delivery for complex high dimensional networked scenarios. Information exploitation: to take this aggregated information and use it to control sensor actions for optimizing mission tasks such as autonomous vision, UAV strike planning, and multilayer multiscale situational awareness Mostly directly out of the proposal. This is just to get the ball rolling.
• Application domains being considered– Space-time adaptive processing (STAP) (Ertin, Cochran, Hero, Zelnio)– Moving target estimation in SAR (Hero, Zelnio)– Robust video motion segmentation (Nadakuditi, Hero)– Object and feature recognition in video (Soatto)– Fusion of WAMI and LIDAR imagery (Fisher) – Intruder detection with multi-modality sensors (Hero, Nasrabadi)– Human collaboration and HMI modeling (Yu, Hero, Sadler)– Wide area search and mission adaptive planning (How, Hero) – Social media, crowdsourcing and text streams (Hero and Jordan, Kaplan)
• Experimental model building and validation– Factors influencing human performance in collaborative tasks (Yu)– Radar test-bed to validate VoI proxies and associated algorithms (Ertin)
Presenter
Presentation Notes
According to Kuhn the development of a science is not uniform but has alternating ‘normal’ and ‘revolutionary’ (or ‘extraordinary’) phases. The revolutionary phases are not merely periods of accelerated progress, but differ qualitatively from normal science. Normal science does resemble the standard cumulative picture of scientific progress, on the surface at least. Kuhn describes normal science as ‘puzzle-solving’ (1962/1970a, 35–42).
• The committee noted current efforts addressing relevant subproblems of Value of Information (VoI). Team needs to coordinate research activities toward establishing a rigorous, foundational framework of value of information for distributed fusion. A unifying theme is needed...Research coordination has occurred on several levels and several subgroups have been working together on foundational themes. This will be reported today in the talks and posters
Presenter
Presentation Notes
�Collaborations of which I am aware Cochran, Fisher, Hero: a theory of VoI proxies How, Hero: a theory of mission-adaptive planning Cochran, Hero, Nadakuditi: VoI separation theorems Jordan, Hero: fundamental performance tradeoffs (privacy,comms,computation,etc) Yu, Hero: human interaction models in VoI Fisher and Soatto: VoI in active vision Moses and Ertin: decentralized inference, radar testbed Classification: UNCLASSIFIED Caveats: NONE Al, Please find below government feedback. Please disseminate to the team. Thanks, Liyi ��Comments and recommendations: ��1. The government committee was pleased to see that the MURI team produces high quality, fundamental research results. The technical work was excellent. This is a strong team. ��2. The work on tradeoff between time and risk (UCB) was excellent and particularly novel. ��3. The committee noted current efforts addressing relevant subproblems of Value of Information (VoI). Team needs to coordinate research activities toward establishing a rigorous, foundational framework of value of information for distributed fusion. A unifying theme is needed. One possibility is to define an overarching, unifying problem so that every team member could address part(s) of the problem. The problem should be formulated in a general manner so that the appropriate value of information metrics are not trivially self-evident by the context of the task (e.g., the task is not simply to track a target where localization error is an obvious metric). ��4. The committee had a lot of discussion on defining appropriate metrics for value of information. Research efforts thus far have focused on increasing the usefulness of information through mutual information, visibility, KL divergence or other measures by improving the amount of new information, or maximizing the information content in future measurements. Because VoI is task dependent, one possibility is to investigate the principles of defining an appropriate metric for a given task or a targeted user (human or machine). Another possibility is to establish approaches and methods to adaptively define or evaluate value of information. One example along this direction is the VoI based resource management work as reported by MIT last year. There is a need to address more general problems. ��5. The committee had active discussion about the types of data that should be considered, particularly how to handle soft data. There is a strong interest within ARL on VoI to support cyber research. The team has existing relevant expertise: UCB has worked with soft data for decades and the research naturally covers soft data. UMich also reported results applicable to soft data. On the other hand, a sensor for this MURI could be any data collection capability including soldier as a sensor or a soft data collection node. The team is still at an early stage toward establishing a theory of VoI. To achieve the MURI objective, it was suggested that the highest priority of research be to establish a framework first using whatever type of data the team is familiar with. Generalization of the framework to include soft data could be done later with possible modifications. The team is certainly encouraged to include soft data in research. ��6. UCSD is new to the team and offers complementary research agenda. The focus of research should be on human aspects of value of information, going beyond attention detection and eye tracking. For example, it would be of interest to investigate how to assign value of information for human-machine interaction. Other examples might be of the form: experiments to evaluate what makes human models pliable (the effective samples associated with human priors), and required machine certainties (second order model statistics) required for affecting a decision makers actions. ��7. Resource management of networked sensors should consider realistic assumptions such as including constraints on communication bandwidth, computing, storage, connectivity, or power assumption. The team (J How, MIT) reported VoI results under communication and computing constraints last year. While the government committee understands the tradeoff between complexity and elegance of a theory, it is desirable for research efforts to focus on theory and methods that are applicable to problems of practical complexity as opposed to results that are valid only under overly simplified assumptions. ��8. Future research is encouraged to place more emphasis on multimodality sensors, particularly visual modality. Quantifying the relationship of sensor geometry and expected task dependent information gain for visual data (perhaps conditioned on the spectral content of the object(s) of interest) would also be of interest. ��9. There was a mention of evaluating the value of lost information. ��10. The government committee discussed the use of real data for research validation and motivation. It was noted that existing real data may not be useful to the team's research agenda because the MURI is intended to establish a new theory/framework that currently does not exist or was not considered in data collection. The team will set up a testbed, under the support of a DURIP grant, to address the need of creating realistic scenarios for research validation, verification, and problem motivation. MURI PM (Liyi Dai) will coordinate with ARDEC and MURI team to explore the feasibility of using ARDEC's multimodal sensor network testbed. �
Previous government comments 5• …the highest priority of research be to establish a framework
first using whatever type of data the team is familiar with. This has been and continues to be the team’s approach to model validation in building our framework for VoI.
• Generalization of the framework to include soft data could be done later with possible modifications. The team is certainly encouraged to include soft data in research. We are developing general theory capable of handing hard and soft data. Several coPIs will report on soft data in the talks and posters.
• UCSD is new to the team and offers complementary research agenda. The focus of research should be on human aspects of value of information, going beyond attention detection and eye tracking. We report results on modeling human behavior in problem solving experiments. As shown in Angela Yu’s ARL presentation last June, UCSD’s results generate models for the team’s efforts on human-in-the-loop and collaborative learning.
• While the government committee understands the tradeoff between complexity and elegance of a theory, it is desirable for research efforts to focus on theory and methods that are applicable to problems of practical complexity as opposed to results that are valid only under overly simplified assumptions. Our team recognizes the importance of model validation and applying our results to real world data. The team has achieved this through designing experiments, disciplinary collaborations, and technology transitions.
Previous government comment 10• The government committee discussed the use of real data
for research validation and motivation….Real data from various sources, including ARL, AFRL, and co-PI labs, is being used to validate models and results. We expect to coordinate with ARDEC to explore other multimodal data sources.
• The team will set up a testbed, under the support of a DURIP grant, to address the need of creating realistic scenarios for research validation, verification, and problem motivation. We anticipate collection of radar data from the DURIP testbedto begin by the end of the calendar year.
VOI1. Gene T. Whipps, Emre Ertin, Randolph L. Moses, "A consensus based decentralized EM for a mixture of factor
analyzers“2. Diyan Teng and Emre Ertin, “Learning to Aggregate Information for Sequential Inferences”3. Nithin Sugavanam, Emre Ertin, “Optimal measurement matrix design for structured signals in clutter and
noise”4. K. Greenewald, A.O. Hero, "Regularized Block Toeplitz Covariance Matrix Estimation via Kronecker Product
Expansions“5. H.W. Chung, T. Tsiligkaridis, B. Sadler, A. Hero, “Collaborative 20 questions”6. Z. Meng, B. Erikson, A.O. Hero, "Learning Latent Variable Gaussian Graphical Models" 7. Zhang, Chen, Zhou and Jordan: Spectral methods meet EM: A provably optimal algorithm for crowdsourcing8. Zhang, Wainwright and Jordan: Lower bounds on the performance of polynomial-time algorithms9. Nishihara, Jegelka and Jordan: On the convergence rate of decomposable submodular function minimization10. J. Duchi, M. Jordan, M. Wainwright and Wibisono: Optimal rates for zero-order optimization: the power of two
function evaluations11. Brian E. Moore, Raj Rao Nadakuditi, and Jeffrey A. Fessler, “Improved robust PCA using low-rank denoising
with optimal singular value shrinkage”12. L. Crider and D. Cochran, “Value of Information Sharing in Network Signal Detection”13. S. Zhang, A. Tran and A. Yu, “Personality and Behavioral Predictors of Human Exploration/Exploitation Behavior
in a Bandit Task”14. B. Mu and J. How, “ Focused information gathering with an application in SLAM”15. Josh Hernandez, Konstantine Tsotsos, Gottfried Graber, and Stefano Soatto, “"Scene Segmentation with Dense
Reconstruction from Monocular Video“16. Konstantine Tsotsos and Stefano Soatto, “"Robust Visual-Inertial Navigation and Mapping"
![Einsatzfähig unter Linux / Windows-XP! Fertig installiert ...wp13255653.server-he.de/ilco3/images/Beschreibung... · Kickstart-2.3.2 [Restore-Backup] Joomla-Wiki [Wiki] Phoca-Galerie](https://img.dokumen.tips/doc/110x75/5fa18abeb41bf0376f6509d5/einsatzfhig-unter-linux-windows-xp-fertig-installiert-kickstart-232-restore-backup.jpg)