NATO ARW, Velingrad, Bulgaria, 2006 1 Methods for Data and Information Fusion Kiril Alexiev, Iva Nikolova [email protected] Tel: 9796620; 0898 898 616 25A,

NATO ARW, Velingrad, Bulgaria, 2006 1

Methods for Data and Information Fusion

Kiril Alexiev, Iva Nikolova

[email protected]

Tel: 9796620; 0898 898 616

25A, Acad.G.Bonchev Str., Sofia 1113, Bulgaria

Institute for Parallel Processing - Bulgarian Academy of Science

mailto:[email protected]

Correct decision making (taking) in the security sector mainly depends on information, received from multiple sources. Often, the information is insufficient, unreliable and contradictive.

Methods for Data and Information Fusion 2

Institute for Parallel Processing - Bulgarian Academy of Sciences

Architecture of sensor network


user

CommunicationCommunication

sensornode

routing data

sensordata

sensornoderouting

data

sensordata

query

sensordata


Definition of Data and Information Fusion

Wikipedia:

Sensor fusion is the combining of sensory data such that the resulting information is in some sense better than would be possible when these sources were used individually.

Better = more accurate, more complete, or more dependable



Definition of Data and Information Fusion

Authors remark:

In definition:

“combination of data” is not very suitable phrase. We have to find better one, for example “simultaneously processed data”



Benefits from Fusion Process

The first and the most important remark is that fusion process is necessary most of all to reduce (to filter) input information through its integration (merging) and generalization.

Fusion process is necessary to improve accuracy.Fusion process is necessary to reduce

uncertainty.



Structure of Data and Information Fusion (JDL) Level 0: Preliminary data processing – pixel or signal

level data association and characterization.

Level 1: Data alignment, association, tracking and identification.

Level 2: Situation assessment.

Level 3: Threat assessment.

Level 4: Process Refinement includes adaptive processing through performance evaluation and decision or resource and mission management.





Paper's classification by fusion level

2%

64%

15%

11%5% 3% Preprocessing

Level 1

Level 2

Level 3

Level 4

Others



Paper's classification by sensor type

40%

15%4%15%

26% Radar

Visual

Infrared

Acoustic

Others



Paper classification by target type

36%

7%31%

14%

12%

Air targets

Mobile Robots

Ground and/or mobiletargets

Submarine

Others

Level 1

Temporal data fusion Sensor data fusion



The Nearest Neighbor method associates the nearest measurement to the track prediction. The more complicated Global Nearest Neighbor minimizes cluster cost function in measurement distribution.

The probabilistic data association filter (PDAF) and its extension to multiple targets – joint PDAF (JPDAF), solve the same task of measurement identification in a simpler way. In the JPDAF hypotheses are built for the measurements and targets only for the current scan. In this way the number of hypotheses is additionally reduced but the chance of combinatorial explosion in dense target and clutter scenarios still remains.



Data association methods

In Multiple Hypothesis Tracking approach all measurements received at a scan are assigned to initialized targets, new targets or false alarms. A number of hypotheses are generated. Every one supposes a possible assignment scheme between measurements, received in all scans, and the targets - confirmed, new ones or false. Pruning and gating techniques are used to retain the most likely hypotheses and in this way to reduce their number

Finite Set Statistics considers all measurements as measurements from a generalized sensor and all targets as a generalized target of interest. Fusion of information from one and the same sensor but from different moments of time



Data association methods

Identification

Two types of identification: Structural identification – more difficult

Define structure (model), which in the best way corresponds to the observed system (process).

Parametrical identification – a lot of algorithms

Find (calculate) values of parameters, which characterize entirely considered system (process).



Parameter identificationMathematical description

Linear dynamic system (Markovian presentation):

Kalman filter gives optimal solution for Gaussian noises



)()()()(

)()()()()()1(

kwkxkHkz

kvkukGkxkFkx

Markovian - semi Markovian

Linear - non-linear dynamic system

Additive - non additive system noise

Gaussian - non Gaussian system noise

Additive - non additive measurement noise

Gaussian - non Gaussian measurement noise

Description



))0(),...,1(),(()1( xkxkxFkx

))(()1( kxFkx

))(),(()( kwkxHkz

),,( vvmvNv

),,( wwmwNw

))(),(()1( kvkxFkx

The simplest tracking filter, considered in the paper, is alpha-beta filter. It is suitable for tracking of moving with constant velocity targets without steady-state error. The alpha-beta-gamma filter has ability to track even accelerating targets without steady-state error.

Kalman filter is a classical optimal estimating algorithm for dynamical linear system with Gaussian measurement and system noise. The modification of Kalman filter - Extended Kalman filter is developed for non-linear systems. The EKF gives particularly poor performance on highly non-linear functions because only the mean is propagated through the non-linearity. The unscented Kalman filter (UKF) uses a deterministic sampling technique to pick a minimal set of sample points (called sigma points) around the mean.



The theoretically most powerful approach for manoeuvring targets tracking is known to be Interacting Multiple Models estimator. Generalized Pseudo-Bayesian (GPB) estimators different orders, Fixed structure IMM, Variable Structure IMM, Probabilistic Data Association IMM are variants. The most important feature is that all these estimators use in parallel several models for modelling of the estimated system.

Particle filters, also known as Sequential Monte Carlo methods (SMC), are sophisticated model estimation techniques based on simulation. Particle filters generate a set of samples that approximate the filtering distribution to some degree of accuracy. Sampling Importance Resampling (SIR) filters with transition prior as importance function are commonly known as bootstrap filter and condensation algorithm.



Temporal data fusion (Alan Steinberg)



NN Nearest Neighbor PF Particle FilterF Alpha-Beta Filter PDAF Probabilistic Data Association FilterKF Kalman Filter JPDAF Joint Probabilistic Data Association FilterEKF Extended Kalman Filter FISST Finite Set StatisticsIMM Interacting Multiple Model filter Y/N Good/Poor Choice;



Alan Steinberg

Fully Centralized Measurement Fusion Architecture



Fully Centralized Trajectory Fusion Architecture



Distributed Decision Fusion Architecture



Simple Example When both sources are reliable, there is a consensus and

it is reasonable to find solution in the cross-section of and - sets of corresponding sources: . If the two sources do not agree, we have . The hypothesis for reliability sources is no longer credible and three other hypotheses appear: 1) First source is correct, the second is incorrect; 2) First source is incorrect, but second is incorrect; 3) Both sources are incorrect. How to find the correct hypothesis? As a precaution, all available information is kept and we hold up .



21 DDx 21 DDx

Example – continue It is obvious that the first fusion method is the most

informative because the information is refined to the intersection of sets given by each source. It is also the most “risky” approach because the real value of is assumed to be inside a smaller set than the two initial sets. The second fusion method is more reliable since all the information given by the two sources is preserved. The drawback of such an approach is a loss of accuracy since the set assumed to contain , is larger than each of the initial sets.



Homogeneous sensor fusion



AND Operator. This method transforms the output of the sensors in a binary yes/no consensus operating with logical AND. After that thresholds are applied to find the result. The procedure is very simple, intuitive and fast, if the values of thresholds are determined in advance. The method does not take into account the degree of confidence of each sensor.

Weighted Average. This method takes a weighted average of available sensor data and uses it as the fused value. Usually the weights are proportional to accuracy of sensors or to credibility of sensor information.

Voting. The voting schemes main advantage is computation efficiency. Voting involves the derivation of an output data object from a collection of n input data objects, as prescribed by the requirements and constraints of a voting algorithm. The voting algorithms can be quite complex in terms of content and structure of the input data objects and how they handle the votes (weights) at input and output.

Sensor fusion Bayesian Theory. The use of Bayesian inference theory is widely

spread for the fusion of redundant information. The most known method is the Kalman Filter, that is optimal in a statistical sense (it presents the least square error). Bayesian theory is also used to establish the weights linking the sensors in a weighted average fusion architecture. Moreover, some reductions of superbayesian methods to probabilistic evidence combination formulas have been provided. Some problems arise in a Bayesian framework: I) it does not distinguish between “lack of evidence” and “disbelief”; ii) practical difficulties in setting the apriori probabilities: noninformative priors can cause a wrong bias of further reasoning; iii) it assumes that the knowledge sources are consistent.



Sensor Fusion Information Theory. Mutual information, in the

form of the Kullback-Leiber divergence, has been used in [12] as a way of combining probabilistic masses (sensor outputs). This is yet another method of fusing two probabilities, this time with a non-bayesian law, adding some information on average image values (e.g. depending on lighting conditions). The local maximum of the mutual information is then taken as the fused value.



Sensor Fusion Belief Theory. Dempster-Shafer evidential

reasoning is used to compute the belief of a given event from two or more assessments provided by different knowledge sources at a symbolic level. This theory is based on the premise that each source of information provides only a partial belief about a proposition. Problem – redistribution of conflicts.

Dezert Smarandache Theory(DSmT). DSmT is analogous to Dempster-Shafer evidential reasoning theory but overcomes some drawbacks of this theory



Sensor fusion Fuzzy Reasoning. Fuzzy sets and variables are used to

deal with real-world models where the usual ideal mathematical assumptions are inappropriate. Under the fuzzy framework, the possibility theory has emerged to represent imprecision in terms of fuzzy sets and to quantify uncertainty through four proposed notions: possibility, necessity, plausibility, and credibility distributions .

Geometric Methods, e.g. using uncertainty ellipsoids. Parametrical identification – if we know model, we can estimate parameters;



Level 2,3,4 fusion



Belief Propagation Nets Markov Random Fields Factor Graphs Game theory

New research direction

New tracking filters – may be FISST, may be new one

Increased interest in image fusion methods - improvement of existing, search for new ones.

Increased interest on higher level fusion – not only theoretical but engineering approach

Decision level methods for fusion – like Dezert –Smarandache Theory or new ones.



Documents

NATO ARW, Velingrad, Bulgaria, 2006 1 Methods for Data and Information Fusion Kiril Alexiev, Iva Nikolova [email protected] Tel: 9796620; 0898 898 616 25A,