View
35
Download
0
Category
Tags:
Preview:
DESCRIPTION
Ontology and Human Intelligences in Optimization and Fusion. Moises Sudit October 28, 2013. Gadenfors Conceptual Spaces. Consider a situation where you are walking through the woods:. - PowerPoint PPT Presentation
Citation preview
Moises SuditOctober 28, 2013
Ontology and Human Intelligencesin Optimization and Fusion
Slide 2
Gadenfors Conceptual Spaces
Consider a situation where you are walking through the woods:
Associationist: Travel through one small part at a time, understand features (rocks, rivers, trees, etc.), learn as we go, clear path for next time we travel…
Conceptual: “overhead view” understanding of geometry of paths as features come together (N,S,E,W)…
Symbolic: Semantic street names and directions (left, right, etc.) are given to paths, thus we gain independence from the features…
Conceptual Spaces
• Overview• A conceptual space consists of a set of geometric domains and
their associated metrics and corresponding similarity measures• A concept is a collection of property regions within these domains,
the correlations (i.e., co-occurrences) between these properties, and their salience weights
• Each concept is additionally characterized by a set of forbidden domain-property pairs
• A query is a set of points, one in each domain, describing its attributes
D1 D2 DK
ConceptualSpace
Domains
Properties
1( , )s x y 2 ( , )s x y ( , )Ns x y
Introduction to Conceptual Spaces
Armor Level
# Wheels
Amphibious
Domains•# Wheels•Armor Level•Amphibious
Concepts•Tank (0, Heavy, No)•LAV (>6, Light, Yes)•Truck (4-6, Light, No)•Jeep (>6, Light, No)
Benefits•Allows similarities btw objects to be calculated•More flexible than First Order Logic•Transparent
>6
4-6
0Light Medium HeavyNo
Yes
Properties•# Wheels(0, 4-6, >6)•Armor Level (Light, Medium, Heavy)•Amphibious(Yes, No)
Observations (Wheels, Armor, Amphibious)
Introduction To Conceptual Spaces
Armor Level
# Wheels
Amphibious
(0, Heavy, No)(4, Light, No)(4, Light, Yes)
d1d2
Slide 6
Two Models for Conceptual Spaces• Single Observation Mathematical Model
• Only one observation is made on one object• This object is compared to each individual concept in the library
(world) to determine which it is most similar to• Multiple Observation Mathematical Model
• Multiple observations are made from either a single sensor or multiple sensors
• Observations may not necessarily be of the same object• This handles “The Association Problem” in Data Fusion
• Each observation is compared to each concept to determine which it is most similar to
Slide 7
Single Observation Model
j j k
Set of ConceptsSet of Domains
Subset of Domains of concept k for k C
Set of properties of Domain j for j D
=Set of (i,j) and (i ,j ) that mutually exclusive,
where i P and i P for j,j D
k
j
k
CD
D
P
I
, k Csimilarity of property j within domain i
1,0, . .
ij
ij
s
if property j from domain i is consideredx
o w
Concept encoded in set of constraintsObserved Object appears only in the objective function
1 1 2 2
1
1 1 2 2
max
. . 1
1 , , ,
i
ij iji j
n
ijj
ki j i j
s x
s t x i
x x i j i j I
x B
We prove a lemma showing that any sized finite set of mutually exclusive properties can be broken into pairs.
Slide 8
Example: Decision Variables11 12 13 14 15 16
21 22 23
31 32 33
41
Color: red(x ), white(x ),brown(x ), black(x ), yellow(x ),grey(x )Shape: rectangular(x ),short & round(x ), tall & thin(x )Sound: quick "boom"(x ),explosion(x ), humming(x )RelativeSize: small(x 42 43
51 52
61
),medium(x ),big(x )Motion: drives(x ), walks(x )Smell:gaseous(x )
We set up a library of 4 concepts (Bomb, Auto, Human, Gas Tank). Each utilize some of the same domains/properties and some different ones.
We run them against 4 observed objects and see how our model works.
Slide 9
Example: ConceptsBomb:Color: red, white, brown,
blackShape: rectangular, short &
roundSound: “boom”, explosionRelative Size: small11 12 13 14
21 22
31 32
41
11 22
13 22
14 21
22 31
11 32
13 32
111
1111111
x x x xx xx xxx xx xx xx xx xx xx B
Auto:Color: black, yellowShape: rectangular, short & roundSound: hummingRelative Size: smallMotion: drives
14 15
21 22
33
41
51
14 21
15 22
11
111
11
x xx xxxxx xx xx B
Slide 10
Example: Concepts (cont.)Human:Color: white, blackShape: short & round, tall &
thinRelative Size: largeMotion: walks
Gas Tank:Color: black, greyRelative Size: mediumSmell: gaseous
12 14
22 23
42 43
52
12 42
12 23
14 43
14 22
111
11111
x xx xx xxx xx xx xx xx B
14 16
42
61
111
x xxxx B
Slide 11
Multiple-Observation Model
j
Set of ConceptsSet of Domains
Subset of Domains of concept k for k CSet of Observations
Set of properties of Domain j for j D
=Set of (i,j) and (i ,j ) that mutually exclusive,
where i P a
k
j
k
CD
DO
P
I
j knd i P for j,j D , k Cmaximum number of distinct concepts that could be observed
pedigree of domain j by observation o for j D and o O
similarity of property i of domain j by observation
jo
jio
m
p
s
jo for i P , j D and o O
1 if property i of Domain j is associated with observation o
for i ,0 Otherwise
1 if observation o is associated with concept k for 0 Otherwise
j jio
ok
x P j D and o O
k C and o Oy
1 if observation o is associated with concept k for 0 Otherwisek
k Cz
Slide 12
Multiple-Observation Model (cont.)
j
j j jo io io
o O j D i P
Max p s x
. : , ,
1 , with
jk
jo
jio ok
i P
j joio
i P
st x y j k o
x j o P
2 {( , ), ( , )} , ,j j kio i o okx x y i j i j I k o
1okk C
y o
ok ko O
kk C
y O z k
z m
, , 0 1 , , ,jio ok kx y z or i j k o
Constrains the number of properties selected in each domain.
Constrains cross-domain property disallowed pairings.
Allows only one concept to be selected for each observation.
Constrains the number of objects being observed by the sensory system.
Maximizes property similarities based on sensor reports.
Slide 13
What do we have?• A hybrid Conceptual Space/Integer
Programming model that can:• Consider multiple observations by multiple sensors• Account for the pedigree of each sensor in
accordance to its ability to sense each specific property/domain
• The ability to change the number of allowed objects being observed (m)
• All of these capabilities are captured within a single, mathematical model using proven optimization techniques
• How well does it work?• Emotion Recognition (compared against Support
Vector Machine)• Automatic ICON Identification for CPOF
Emotion Recognition
Slide 15
Emotion Recognition through Conceptual Spaces
True Emotions
False Emotions
Fear
Sadness
Anger
Enjoyment
We are taking the BB3 Data and classifying pictures into one of 8 concepts:
4 true emotions and 4 false emotions (attempted deceit)
Slide 16
Emotion Recognition• Process
• Images are obtained and analyzed automatically in terms of facial features
• Facial features are considered in classification of images into emotions, both true emotions and falsified emotions
• Parts of the ProcessComponent Description Workload
Major Component
s (MC)
Measurements and distances between parts of the face – used to determine which Action Units exist.
CUBS
Action Units (AU)
50+ defined – technique for measurement of facial movement (Facial Action Coding System).
Ekman & Friesen
Emotions Existence and combination of AU’s help define a person’s true emotion and may be able to depict deceit as well.
Our Work
• Major Components Action Units Emotions
Emotion Recognition
wrinklesCrows feetLips Compressed……………
AUi
AUj
AUk
AUl
Anger
Enjoyment
Fear
Sadness
Measurable features calculated based on distances between certain points. Determined by automated systems.
Several Major Components combine to form Action Units
The presence of Several Action Units at the same time define emotions.
Slide 18
Conceptual Spaces – Classification Model
1 2
1
2
{ , , , ,, ., , }
{ 1, 2, 4, 6, 12, 15, 23}
{ , },ji i
i
i
C Anger Enjoyment Fear SadnessFalse Anger False Enjoy False Fear False Sadness
D AU AU AU AU AU AU AU
P x x i Dwhere x AU does not occur in Concept kwhere x AU occurs in Concept k
AU1 AU2 AU4 AU6 AU12 AU15 AU23
Concept X11 X12 X21 X22 X31 X32 X41 X42 X51 X52 X61 X62 X71 X72
Anger .5 .5 .5 .5 1 0 1 0 1 0 1 0 0 1
Enjoy. .8 .2 .5 .5 1 0 0 1 0 1 1 0 1 0
Fear 0 1 0 1 1 0 1 0 1 0 1 0 1 0
Sad .5 .5 0 1 .5 .5 1 0 1 0 0 1 1 0
False Anger .5 .5 1 0 .7 .3 1 0 1 0 1 0 1 0
FalseEnjoy.
1 0 1 0 1 0 1 0 .75 .25 1 0 1 0
False Fear .5 .5 1 0 1 0 .75 .25 1 0 1 0 1 0
False Sad 1 0 1 0 1 0 1 0 1 0 .5 .5 0 1
Table below shows the existence of properties in concepts. There are properties that cannot exist together – the constraints handle these.
Slide 19
Observations & Model Results• Observations
• Taken from the BB3 Dataset (CUBS) – 344 images analyzed• Since many images produced the same MC values, we
consolidate into 49 observations• MC’s either occur or they do not {0, 1}• Each AU contains anywhere from 1 to 4 MC’s that suggest
the AU is occurring.
# MC's fitting property j in domain i for observation oTotal # MC's in domain i
jis
• Model Results• 49 observations out of which 7 are conflicting so we deleted them• 42 observations against the 8 concept definitions in IP through CPLEX• Objective Value = 291.50• Solution Time = 0.13 seconds• Of the 42 observations, all 42 were classified correctly!
Multi-Class SVM – Classification Model
Using “SVM Light” (Joachims, T. SVM-Light Multi-Class, 2007. Cornell U.)
42 Obs.
27 Obs. 15 Obs.
Training Set Experiment Set
Slack/Kernel
Linear 2nd Poly. 3rd Poly. Radial Basis
Sigmoid
c = 1 7 7 7 8 8c = 10 7 7
(1.16s)9 (2.64s)
8 7
c = 100 7 10 (4.47s)
11 (10.98s)
7 7
c = 1000 7 12 (16.78s)
12 (40.73s)
N/A 7 (2.55s)
c = 5000 12 12 (39.73s)
12 (155.94s)
N/A 7 (5.44s)
c = 10000 13 12 (39.86s)
12 (156.00s)
N/A 7 (8.08s)
Output in Table:• Value = # correct (of 15)• Time = training time (if > 1.0 sec)
Conceptual Spaces – Classification Model
1 2
1
2
{ , , , ,, ., , }
{ 1, 2, 4, 6, 12, 15, 23}
{ , },ji i
i
i
C Anger Enjoyment Fear SadnessFalse Anger False Enjoy False Fear False Sadness
D AU AU AU AU AU AU AU
P x x i Dwhere x AU does not occur in Concept kwhere x AU occurs in Concept k
AU1 AU2 AU4 AU6 AU12 AU15 AU23
Concept X11 X12 X21 X22 X31 X32 X41 X42 X51 X52 X61 X62 X71 X72
Anger .8 .2 .4 .6 .7 .3 1 0 1 0 1 0 0 1
Enjoy. .5 .5 1 0 1 0 .25 .75 0 1 1 0 1 0
Fear .25 .75 0 1 .5 .5 .75 .25 1 0 1 0 1 0
Sad .55 .45 .3 .7 .675 .325 .85 .15 1 0 0 1 1 0
False Anger 1 0 1 0 0 1 .875 .125 .5 .5 1 0 1 0
FalseEnjoy.
1 0 1 0 1 0 .625 .375 0 1 1 0 1 0
False Fear .5 .5 1 0 .625 .375 .75 .25 1 0 1 0 1 0
False Sad 1 0 1 0 1 0 1 0 1 0 .5 .5 0 1
Used the averages of the xi1 and xi2 values to “train” the concepts below.
SVM’s v. Conceptual Spaces• These should be used under different conditions.
• SVM’s – No a priori knowledge, but trainable data is available
• Conceptual Spaces – A priori knowledge available, no need to train Support Vector
Machines Conceptual Spaces
Parameter ChoicesKernel selectionParameters within each kernelSlack allowance (c-value)
N/A
Model Accuracy (in this example) 13/15 = 86.67% 14/15 = 93.33%
Model Speed (in this example) 0.16 seconds 0.08 seconds
Multi-Class SVM – Classification Model
Using “SVM Light” (Joachims, T. SVM-Light Multi-Class, 2007. Cornell U.)
42 Obs.
42 Obs. 42 Obs.
Training Set Experiment Set
Slack/Kernel
Linear 2nd Poly. 3rd Poly. Radial Basis
Sigmoid
c = 1 30 31 30 31 20c = 10 31 31
(1.44s)32 (3.62s)
36 28
c = 100 31 36 (9.75s)
37 (16.84s)
38 26
c = 1000 37 38 (48.75s)
41 (50.49s)
N/A 25 (4.95s)
c = 5000 37 41 (48.27s)
42 (121.05s)
N/A 25 (9.06s)
c = 10000 39 41 (90.83s)
42 (263.89s)
N/A 21 (13.34s)
Output in Table:• Value = # correct (of 42)• Time = training time (if > 1.0 sec)
SVM’s v. Conceptual Spaces• These should be used under different conditions.
• SVM’s – No a priori knowledge, but trainable data is available
• Conceptual Spaces – A priori knowledge available, no need to train Support Vector
Machines Conceptual Spaces
Parameter ChoicesKernel selectionParameters within each kernelSlack allowance (c-value)
N/A
Model Accuracy (in this example) 42/42 = 100% 42/42 = 100%
Model Speed (in this example) 121.05 seconds (training) 0.11 seconds
CS v. SVM Testing (Observation Dimensionality)
For SVM’s, use 3rd deg. Polynomial and c = 1000
No. Obs. SVM training-time SVM run-time CS run-time
42 2.22 0.08 0.03
420 2,504.15 4.55 0.31
4,200 285,821.11 291.80 6.64
42,000 32,623,621.50 21,048.08 566.39
Run Time Comparison
0
5000
10000
15000
20000
25000
0 10000 20000 30000 40000 50000
Number of Observations
Time
(sec
.)
SVM run-timeCS run-time
CS v. SVM Testing (Concept Dimensionality)
For SVM’s, use 3rd deg. Polynomial and c = 1000
No. Conc. SVM training-time SVM run-time CS run-time
8 2,504.15 4.55 0.31
24 14,568.64 16.70 2.58
80 89,759.09 138.66 59.99
120 167,631.52 255.67 174.33
Run Time Comparison
0
50
100
150
200
250
300
0 20 40 60 80 100 120 140
Number of Concepts
time
(sec
.)
SVM run-time
CS run-time
CPOF ICON Example: Project Overview(Command Post of the Future)
Speech Recognition
Software
[A B C D E] 40% B2 D2
D3
Filter
Domain
Library
[A B2 C D3 E]
AeroText/ Java Class Creation
A B2 C D3 E
Field Soldier TOC Operator/Field Soldier
INFERD
Event Report
Conceptual Spaces Algorithm
INCIDENT
ICON
Known Event
Unknown Event
Key = Input
= OutputText
CPOF Event Icons
1. Bomb2. Drive-by Shooting3. Explosion4. Grenade Attack5. IED (Improvised Explosive Device)
6. Mortar Attack7. Murder
8. Point of Impact9. RPG (Rocket Propelled Grenade)
10. Sniping11. VBIED (Vehicle-Borne IED)
12. PBIED (Person-Borne IED)
Process Flow“Shark 6, this is Oscar Two Delta. Contact Left. Over“Oscar Two Delta, this is Shark 6, over"Location - Mike Rome 05742371, over“Roger, Over One WIA from pistol shot, estimate enemy force of 5, in pursuit, overHeading south from CP1 on route 7 at high speed”Roger, 1 WIA. OverRequest QRF to location 38 SMB xxxxxyyyyy.RogerRequest immediate medevac at Checkpoint 2.Roger, deploying medical personnel. Over.
Mission background in
situ DB
In situ database•Communications ElectronicsOperations Instructions (CEOI)•Patrol Orders•Intelligence Preparation of the Battlefield •Call signs/code names•Channels•Location•Organizational constructs
Shark 6 = Fallujah TOCOscar 2 D = CINC ACF A (Lt. Wayne Demerol’s unit, 5 men)Mike Romeo 05742371 = Grid 38SMB428489021538 SMB xxxxxyyyyy = lat/long surface marker buoyCP1 = Grid abcdefg, temporary checkpoint buildingWIA = wounded in ActionQRF = quick reaction forceRemove extraneous (Over, roger, swear words)
ICON DB
PastSpot/SIGACT
Reports
• Created SPOT reportpeople = Wayne Demarolpeople = enemy number = 5place = 38SMB4284890215place = 38.889556, – 77.0352546organization = cinc acf aorganization = fallujah tocevent = injury number = 1event = pursuitevent = shootingrecord = firearm item = pistolevent = deploy medical personnelContext: time = 1349; date = 05092005
Infuse implicit information
Extract entities, events and relationshipsand Context
1.date = 0913492.event = direct fire3.icon =4.Attributes• affiliation = hostile• target = US• weapon class =
light• WIA=15. confidence = .856. Translated text
Fuzzy Matchingof attributes and events;Confidencelevel;Icon creation
Representative speech-to-text output, including confidence score
Fuzzy Context Search
Finishing the Example (cont.)
Snipin
g
Most Likely Icon at End of Time Four
Bomb
Snipin
g
VBIED
Conceptual
SpacesAlgorith
m
EVENT REPORT:Weapon: GunPersonnel: GroupEvent: Ambush
EVENT REPORT:Weapon: BombPersonnel: NoneEvent: Explosion
EVENT REPORT:Weapon: GunPersonnel: GroupEvent: Skirmish
EVENT REPORT:Weapon: Personnel: VehicleEvent:
Recommended