Upload
mendel
View
31
Download
0
Tags:
Embed Size (px)
DESCRIPTION
BIO-INSPIRED AND COGNITIVE COMPUTING for data mining, tracking, fusion, financial prediction, language understanding, web search engines, and diagnostic modeling of cultures. IEEE 2007 Fall Short Course Holiday Inn, Woburn MA 7:00 – 9:00 pm, Nov. 8, 15, 22, Dec. 6. Leonid Perlovsky - PowerPoint PPT Presentation
Citation preview
BIO-INSPIRED AND COGNITIVE COMPUTINGfor
data mining, tracking, fusion, financial prediction, language understanding, web search engines, and
diagnostic modeling of cultures
Leonid PerlovskyVisiting Scholar, Harvard University
Technical Advisor, AFRL
IEEE 2007 Fall Short Course
Holiday Inn, Woburn MA
7:00 – 9:00 pm, Nov. 8, 15, 22, Dec. 6
OUTLINE
1. Cognition, Complexity, and Logic
2. The Knowledge Instinct -Neural Modeling Fields and Dynamic Logic
3. Language
4. Integration of cognition and language
5. High Cognitive Functions
6. Evolution of cultures
7. Future directions
DETAILED OUTLINE
1. Cognition – integration of real-time signals and a priori knowledge
1.1. physics and mathematics of the mind1.2. genetic argument for the first principles1.3. the nature of understanding1.3.1. concepts: “chair”1.3.2. hierarchy1.4. combinatorial complexity (CC) – a fundamental
problem?1.5. CC since 1950s1.6. CC vs. logic1.6.1. formal, multivalued and fuzzy logics1.6.2. dynamic logic1.6.3. Aristotle vs. Godel + Alexander the Great1.7. mathematics vs. mind1.8. structure of the mind: concepts, instincts, emotions,
behavior1.9. the knowledge instinct1.9.1. need for learning1.9.2. “knowledge emotion” = aesthetic emotion
2. Modeling Field Theory (NMF) of cognition2.1. the knowledge instinct = max similarity2.2. Similarity as likelihood, as information2.3. Dynamic Logic (DL)
2.4. applications, examples, exercises 2.4.1 clustering2.4.2 tracking and CRB
- example of tracking below clutter- complexity NMF vs. MHT- models
2.4.3 recognition- example of pattern in image below clutter- complexity NMF vs. MHT- models
2.4.4 fusion- example of fusion, navigation, and detection below clutter- models
2.4.5 prediction- financial prediction- models
2.5. block-diagrams2.7. hierarchical structure
3. Language3.1. language acquisition and complexity3.1.2. language separate from cognition3.1.3. hierarchy of language3.1.4. application
- search engine based on understanding
3.2. NMF of language
3.2.1 differentiating non-differentiable qualitative functions
DETAILED OUTLINECONTINUATION
4. Integration of cognition and language4.1. language vs. cognition4.2. past: AI and Chomskyan linguistics4.3. integrated models4.3. integrated hierarchies4.4. Humboldt’s inner linguistic form
5. Prolegomena to a theory of the mind5.1. higher cognitive functions5.2. from Plato to Lock5.3. from Kant to Grossberg5.4. NMF vs. Buddhism5.5. NMF vs. neuro-biology5.5. NMF dynamics: elementary thought process5.6. consciousness and unconscious5.7. aesthetics and beauty5.8. intuition5.9. why Adam was expelled from paradise5.9. symbols and signs 5.9.1. NMF theory of symbols5.9.2. symbolic AI: “bewitchment”5.10. why Adam was expelled from paradise
6. Evolution of Culture 6.1. Culture and language6.2. KI: differentiation and synthesis6.3. “Spiritual” cultural measurements6.4. Mathematical modeling and
predictions6.4.1. dynamic culture6.4.2. traditional culture6.4.3. terrorist’s consciousness6.4.4. interacting cultures6.5. Evolution of concepts and emotions6.6. Creativity6.7. Disintegration of cultures 6.8. Emotions in language6.9. English vs. Arabic 6.10. Synthesis 6.11. Differentiation of emotions6.12. Role of music in evolution of the
mind and culture
7. Future directions & publications
7.1. Science and Religion7.2 Predictions and testing7.3. Future directions7.4 Publications
INTRODUCTION
Nature of the mind
PHYSICS AND MATHEMATICS OF THE MINDRANGE OF CONCEPTS
Logic is sufficient to explain mind– [Newell, “Artificial Intelligence”, 1980s]
No new specific mathematical concepts are needed– Mind is a collection of ad-hoc principles, [Minsky, 1990s]
Specific mathematical constructs describe the multiplicity of mind phenomena– “first physical principles of mind”– [Grossberg, Zadeh, Perlovsky,…]
Quantum computation– [Hameroff, Penrose, Perlovsky,…]
New unknown yet physical phenomena– [Josephson, Penrose]
GENETIC ARGUMENTSFOR THE “FIRST PRINCIPLES”
Only 30,000 genes in human genome– Only about 2% difference between human and apes– Say, 1% difference between human and ape minds– Only about 300 proteins
Therefore, the mind has to utilize few inborn principles – If we count “a protein per concept”– If we count combinations: 300300 ~ unlimited => all concepts
and languages could have been genetically h/w-ed (!?!)
Languages and concepts are not genetically hardwired– Because they have to be flexible and adaptive
COGNITION
• Understanding the world around– Perception– Simple objects– Complex situations
• Integration of real-time signals and existing (a priori) knowledge – From signals to concepts– From less knowledge to more knowledge
EXAMPLE
Example: “this is a chair, it is for sitting”
Identify objects– signals -> concepts
What in the mind help us do this? Representations, models, ontologies?– What is the nature of representations in the
mind?– Wooden chairs in the world, but no wood in the
brain
VISUAL PERCEPTION
Neural mechanisms are well studied– Projection from retina to visual cortex (geometrically accurate) – Projection of memories-models
• from memory to visual cortex– Matching: sensory signals and models– In visual nerve more feedback connections than feedforward
• matching involves complicated adaptation of models and signals
Difficulty– Associate signals with models– A lot of models (expected objects and scences)– Many more combinations: models<->pixels
Association + adaptation– To adapt, signals and models should be associated– To associate, they should be adapted
ALGORITHMIC DIFFICULTIES A FUNDAMENTAL PROBLEM?
Cognition and language involve evaluating large numbers of combinations– Pixels -> objects -> scenes
Combinatorial Complexity (CC) – A general problem (since the 1950s)
• Detection, recognition, tracking, fusion, situational awareness, language…
• Pattern recognition, neural networks, rule systems…
Combinations of 100 elements are 100100
– This number ~ the size of the Universe • > all the events in the Universe during its entire life
CC was encountered for over 50 years
Statistical pattern recognition and neural networks: CC of learning requirements
Rule systems and AI, in the presence of variability : CC of rules
– Minsky 1960s: Artificial Intelligence– Chomsky 1957: language mechanisms are rule systems
Model-based systems, with adaptive models: CC of computations
– Chomsky 1981: language mechanisms are model-based (rules and parameters)
Current ontologies, “semantic web” are rule-systems– Evolvable ontologies : present challenge
COMBINATORIAL COMPLEXITY SINCE the 1950s
CC AND TYPES OF LOGIC
CC is related to formal logic– Law of excluded middle (or excluded third)
•every logical statement is either true or false– Gödel proved that logic is “illogical,” “inconsistent” (1930s)
– CC is Gödel's “incompleteness” in a finite system
Multivalued logic eliminated the “law of excluded third”– Still, the math. of formal logic– Excluded 3rd -> excluded (n+1)
Fuzzy logic eliminated the “law of excluded third”– Fuzzy logic systems are either too fuzzy or too crisp– The mind fits fuzziness for every statement at every step => CC
Logic pervades all algorithms and neural networks – rule systems, fuzzy systems (degree of fuzziness), pattern recognition, neural networks (training uses logical statements)
LOGIC VS. GRADIENT ASCENT
Gradient ascent maximizes without CC– Requires continuous parameters– How to take gradients along “association”?
• Data Xn (or) to object m• It is a logical statement, discrete, non-differentiable
– Models / ontologies require logic => CC
Multivalued logic does not lead to gradient ascent
Fuzzy logic uses continuous association variables, but no parameters to differentiate– A new principle is needed to specify gradient ascent along
fuzzy associations: dynamic logic
DYNAMIC LOGIC
Dynamic Logic unifies formal and fuzzy logic– initial “vague or fuzzy concepts” dynamically
evolve into “formal-logic or crisp concepts”
Dynamic logic– based on a similarity between models and
signals
Overcomes CC of model-based recognition – fast algorithms
ARISTOTLE VS. GÖDEL logic, forms, and language
Aristotle– Logic: a supreme way of argument– Forms: representations in the mind
Form-as-potentiality evolves into form-as-actuality Logic is valid for actualities, not for potentialities (Dynamic Logic)
– Thought language and thinking are closely linked Language contains the necessary uncertainty
From Boole to Russell: formalization of logic– Logicians eliminated from logic uncertainty of language– Hilbert: formalize rules of mathematical proofs forever
Gödel (the 1930s) – Logic is not consistent
Any statement can be proved true and false
Aristotle and Alexander the Great
OUTLINE
• Cognition, complexity, and logic- Logic does not work, but the mind does
• The Mind and Knowledge Instinct- Neural Modeling Fields and Dynamic Logic
• Language
• Integration of cognition and language
• Higher Cognitive Functions
• Future directions
STRUCTURE OF THE MIND
Concepts – Models of objects, their relations, and situations– Evolved to satisfy instincts
Instincts– Internal sensors (e.g. sugar level in blood)
Emotions– Neural signals connecting instincts and concepts
• e.g. a hungry person sees food all around
Behavior– Models of goals (desires) and muscle-movement…
Hierarchy– Concept-models and behavior-models are organized in a “loose”
hierarchy
THE KNOWLEDGE INSTINCT
Model-concepts always have to be adapted– lighting, surrounding, new objects and situations
– even when there is no concrete “bodily” needs
Instinct for knowledge and understanding– Increase similarity between models and the world
Emotions related to the knowledge instinct– Satisfaction or dissatisfaction
• change in similarity between models and world
– Related not to bodily instincts• harmony or disharmony (knowledge-world): aesthetic emotion
REASONS FOR PAST LIMITATIONS
Human intelligence combines conceptual understanding with emotional evaluation
A long-standing cultural belief that emotions are opposite to thinking and intellect– “Stay cool to be smart”– Socrates, Plato, Aristotle– Reiterated by founders of Artificial Intelligence [Newell,
Minsky]
Neural Modeling Fields (NMF)
A mathematical construct modeling the mind– Neural synaptic fields represent model-concepts– A loose hierarchy of more and more general concepts– At every level: bottom-up signals, top-down signals– At every level: concepts, emotions, models, behavior– Concepts become input signals to the next level
NEURAL MODELING FIELDSbasic two-layer mechanism: from signals to
concepts
Signals– Pixels or samples (from sensor or retina)
x(n), n = 1,…,N
Concept-Models (objects or situations) Mm(Sm,n), parameters Sm, m = 1, …;
– Models predict expected signals from objects
Goal: learn object-models and understand signals (knowledge instinct)
THE KNOWLEDGE INSTINCT
The knowledge instinct = maximization of similarity between signals and models
Similarity between signals and models, L– L = l ({x}) = l (x(n))
– l (x(n)) = r(m) l (x(n) | Mm(Sm,n))
– l (x(n) | Mm(Sm,n)) is a conditional similarity for x(n) given m
• {n} are not independent, M(n) may depend on n’
CC: L contains MN items: all associations of pixels and models (LOGIC)
n
m
SIMILARITY
Similarity as likelihood
– l (x(n) | Mm(Sm,n)) = pdf(x(n) | Mm(Sm,n)),– a conditional pdf for x(n) given m– e.g., Gaussian pdf(X(n)|m) = G(X(n)|Mm,Cm) = 2-d/2 detCm
-1/2 exp(-DmnTCm
-1 Dmn/2); Dmn = X(n) – Mm(n)
– Note, this is NOT the usual “Gaussian assumption” • deviations from models D are random, not the data X• multiple models {m} can model any pdf, not one Gaussian model
– Use for sets of data points
Similarity as information
– l (x(n) | Mm(Sm,n)) = abs(x(n))*pdf(x(n) | Mm(Sm,n)),– a mutual information in model m on data x(n)– L is a mutual information in all model about all data– e.g., Gaussian pdf(X(n)|m) = G(X(n)|Mm,Cm)
– Use for continuous data (signals, images)
DYNAMIC LOGIC (DL) non-combinatorial solution
Start with a set of signals and unknown object-models– any parameter values Sm
– associate object-model with its contents (signal composition)
– (1) f(m|n) = r(m) l (n|m) / r(m') l (n|m')
Improve parameter estimation– (2) Sm = Sm + f(m|n) [ln l (n|m)/Mm]*[Mm/Sm]
• ( determines speed of convergence)
– learn signal-contents of objects
Continue iterations (1)-(2). Theorem: MF is a converging system
- similarity increases on each iteration- aesthetic emotion is positive during learning
'm
n
OUTLINE
• Cognition, complexity, and logic- Logic does not work, but the mind does
• The Mind and Knowledge Instinct- Neural Modeling Fields and Dynamic Logic- Application examples
• Language
• Integration of cognition and language
• Higher Cognitive Functions
• Future directions
APPLICATIONS
Many applications have been developed– Government– Medical– Commercial (about 25 companies use this technology)
Sensor signals processing and object recognition– Variety of sensors
Financial market predictions– Market crash on 9/11 predicted a week ahead
Internet search engines– Based on text understanding
Evolving ontologies for Semantic Web
Every application needs models– Future self-evolving models: integrated cognition and language
APPLICATION 1 - CLUSTERING
Find “natural” groups or clusters in data Use Gaussian pdf and simple models
l (n|m) = 2-d/2 detCm-1/2 exp(-Dmn
TCm-1 Dmn/2); Dmn = X(n) – Mm(n)
Mm(n) = Mm; each model has just 1 parameter, Sm = Mm
– This is clustering with Gaussian Mixture Model
For complex l(n|m) derivatives can be taken numerically For simple l(n|m) derivatives can be taken manually
– Simplification, not essential
Simplify parameter estimation equation for Gaussian pdf and simple modelsln l (n|m)/Mm = (-Dmn
TCm-1 Dmn) /Mm = Cm
-1 Dmn + DmnT
Cm-1 = 2 Cm
-1 Dmn, (C is symmetric)
Mm = Mm + f(m|n) Cm-1 Dmn
…
In this case, even simpler equations can be derived samples in class m: Nm = f(m|n); N = Nm
rates (priors): rm = Nm / N
means: Mm = f(m|n) X(n) / Nm
covariances: Cm = f(m|n) Dmn * DmnT / Nm
- simple interpretation: Nm, Mm, Cm are weighted averages. The only difference from standard mean and covariance estimation is weights f(m|n), probabilities of class m
These are iterative equations, f(m|n) depends on parameters; theorem: iterations converge
n
n
n
n
m
EXERCISE (1) – HOME WORK
Code a simple NMF/DL for Gaussian Mixture clustering
(1) Simulate data– Specify true parameters, say:
• Dimensionality, d = 2• Number of classes, 3, m = 1, 2, 3; • Rates rm= 0.2, 0.3, 0.5; • Number of samples, N = 100; Nm= N*rm= 20, 30, 50; • Means, Mm= …; 2-d vectors, (0.1, 0.1), (0.2, 0.8) (0.8, 0.2) • Covariances, Cm = unit matrixes
– Call a function that generates Gaussian data
(2) Run NMF/DL code to estimate parameters (rm, Mm, Cm) and association probabilities f(m|n)
– Initiate parameters to any values, say each r = 1/3; means should not be the same, say M = (0,0), (0,1), (1,1); covariances should be initiated to larger values than uncertainties in the means (squared), say each C = diag(2,2).
– Run iterations: estimate f(m|n), estimate parameters… until, say (changes in M) < 0.01
(3) Plot results (2-d plots)– Plot data for each class in different color/symbols– Plot means– Plot covariances
• 2- ellipses around their means: (X-M)TC-1(X-M) = 2
APPLICATION 2 - TRACKING
1) Example
2) Complexity of computations
3) Exercise
4) Cramer-Rao Bounds
Example 2: GMTI Tracking and Detection
Below Clutter
18 dB improvement
0 1 kmCross-Range
Ra
ng
e1
km0
(a)True
Tracks
b
Ra
ng
e1
km0
Initial state of model 2 iterations
5 iterations 9 iterations 12 iterations Converged state
DL starts with uncertain knowledge and converges rapidly on exact solution
EXAMPLE (2) note page
Detection and tracking targets below clutter: (a) true track positions in 0.5km x 0.5km data set; (b) actual data available for detection and tracking (signal is below clutter, signal-to-clutter ratio is about –2dB for amplitude and –3dB for Doppler; 6 scans are shown on top of each other, each contains 500 data points). Dynamic logic operation: (c) an initial fuzzy model, the fuzziness corresponds to the uncertainty of knowledge; (d) to (h) show increasingly improved models at various iterations (total of 20 iterations). Between (c) and (d) the algorithm fits the data with one model, uncertainty is somewhat reduced. There are two types of models: one uniform model describing clutter (it is not shown), and linear track models with large uncertainty; the number of track models, locations, and velocities are estimated from the data. Between (d) and (e) the algorithm tried to fit the data with more than one track-model and decided, that it needs two models to ‘understand’ the content of the data. Fitting with 2 tracks continues till (f); between (f) and (g) a third track is added. Iterations stopped at (h), when similarity stopped increasing. Detected tracks closely correspond to the truth (a). Complexity of this solution is low, about 106 operations. Solving this problem by MHT (template matching with evaluating combinations of various associations – this is a standard state-of-the-art) would take about MN = 101700 operations, unsolvable.
TRACKING AND DETECTION BELOW CLUTTER (movie, same as above)
DL starts with uncertain knowledge, and similar to human mind does not sort through all possibilities, but converges rapidly on exact solution
3 targets, 6 scans, signal-to-clutter, S/C ~ -3.0dB
TRACKING EXAMPLEcomplexity and improvement
•Technical difficulty-Signal/Clutter = - 3 dB, standard tracking requirements
15 dB-Computations, standard hypothesis testing ~ 101700,
unsolvable
• Solved by Dynamic Logic-Computations: 2x107
-Improvement 18 dB
EXERCISE (2)
Develop NMF/DL for target trackingStep 1 (and the only one)
– Develop models for tracking– Model: where do you expect to see the target– Parameters, Sm: position xm and velocity vm
– Mm(xm, vm , n) = xm + vm * tn
– Time, tn is not a parameter of the model, but data– More complex: Keplerian trajectories– Note: typical data are 2-d () or 3-d (R,)
• Models are 3-d (or 2-d, if sufficient)• 3-d models might be “not estimatable” from 2-d data
– linear track in (x,y,z) over a short time appears linear in () – then, use 2-d models, or additional data, or longer observation
period
CRAMER-RAO BOUND (CRB)
Can a particular set of models be estimated from a particular (limited) set of data?– The question is not trivial
• A simple rule-of-thumb: N(data points) > 10*S(parameters)• In addition: use your mind: is there enough information in the data?
CRB: minimal estimation error (best possible estimation) for any algorithm or neural neworks, or… – When there are many data points, CRB is a good measure
(=ML=NMF)– When there are few data points (e.g. financial prediction) it might be
difficult to access performance• Actual errors >> CRB
Simple well-known CRB for averaging several measurementsst.dev(n) = st.dev(1)/√n
Complex CRB for tracking:– Perlovsky, L.I. (1997a). Cramer-Rao Bound for Tracking in Clutter and Tracking
Multiple Objects. Pattern Recognition Letters, 18(3), pp.283-288.
EXERCISE (2 cont.) HOMEWORK
Homework: code NMF/DL for tracking, simulate data, run the code, and plot results– When simulating data: add sensor error and clutter
• track + sensor errors: X(n,m) = xm + vm * tn + mn
• sensor error: use sensor error model (or simple Gaussian) for m,n
• clutter: X(n,m+1) = c,n ; use clutter model for c,n, or a simple uniform or Gaussian; simulate required number of clutter data points
– Do not forget to add clutter model to your NMF/FL code• l (n|m=clutter) = const; the only parameter, rc = expected proportion of
clutter
Note: the resulting system is “track-before-detect” or more accurately “concurrent detection and tracking”– does not require a standard procedure
• (1) detect, (2) associate, (3) track– association and detection is obtained together with tracking– DL requires no combinatorial searches, which often limits “track-
before-detect” performance
APPLICATION 3
FINDING PATTERNS IN IMAGES
IMAGE PATTERN BELOW NOISE
Object Image
y
Object Image + Clutter
y
x x
Multiple Hypothesis Testing (MHT) approach:try all possible ways of fitting model to the data
PRIOR STATE-OF-THE-ARTComputational complexity
For a 100 x 100 pixel image:
Number of Objects Number of Computations
1 1010
2 1020
3 1030
NMF MODELS
Information similarity measure lnl (x(n) | Mm(Sm,n)) = abs(x(n))*ln pdf(x(n) | Mm(Sm, n))
n = (nx,ny)
Clutter concept-model (m=1) pdf(X(n)|1) = r1
Object concept-model (m=2… ) pdf(x(n) | Mm(Sm, n)) = r2 G(X(n)|Mm (n,k),Cm)
Mm (n,k) = n0 + a*(k2,k); (note: k, K require no estimation)
2/
2/
Kk
Kk
ONE PATTERN BELOW CLUTTER
Y
X
SNR = -2.0 dB
DYNAMIC LOGIC WORKING
y (m
) R
ange
x (m) Cross-range
DL starts with uncertain knowledge, and similar to human mind converges rapidly on exact solution
• Object invisible to human eye
• By integrating data with the knowledge-model DL finds an object below noise
Three objects in noise object 1 object 2 object 3 SCR - 0.70 dB -1.98 dB -0.73 dB
MULTIPLE PATTERNS BELOW CLUTTER
y y
x x
3 Object Image + Clutter3 Object Image
a b c d
fe hg
IMAGE PATTERNS BELOW CLUTTER (dynamic logic iterations see note-text)
a b c d
fe hg
IMAGE PATTERNS BELOW CLUTTER (dynamic logic iterations see note-
text)
Logical complexity = MN = 105000, unsolvable; DL complexity = 107
S/C improvement ~ 16 dB
APPLICATION (3) note page
‘smile’ and ‘frown’ patterns: (a) true ‘smile’ and ‘frown’ patterns shown without clutter; (b) actual image available for recognition (signal is below clutter, signal-to-clutter ratio is between –2dB and –0.7dB); (c) an initial fuzzy model, the fuzziness corresponds to the uncertainty of knowledge; (d) to (h) show increasingly improved models at various iterations (total of 22 iterations). Between (d) and (e) the algorithm tried to fit the data with more than one model and decided, that it needs three models to ‘understand’ the content of the data. There are several types of models: one uniform model describing the clutter (it is not shown), and a variable number of blob models and parabolic models, which number, locations, and curvatures are estimated from the data. Until about (g) the algorithm ‘thought’ in terms of simple blob models, at (g) and beyond, the algorithm decided that it needs more complex parabolic models to describe the data. Iterations stopped at (h), when similarity stopped increasing. Complexity of this solution is moderate, about 1010 operations. Solving this problem by template matching would take a prohibitive 1030 to 1040 operations. (This example is discussed in more details in [[i]].) [i] Linnehan, R., Mutz, Perlovsky, L.I., C., Weijers, B., Schindler, J., Brockett, R. (2003). Detection of Patterns Below Clutter in Images. Int. Conf. On Integration of Knowledge Intensive Multi-Agent Systems, Cambridge, MA Oct.1-3, 2003.
MULTIPLE TARGET DETECTIONDL WORKING EXAMPLE
x
yDL starts with uncertain knowledge, and similar to human mind does not sort through
all possibilities like an MHT, but converges rapidly on exact solution
COMPUTATIONAL REQUIREMENTS COMPARED
Dynamic Logic (DL) vs. Classical State-of-the-art Multiple Hypothesis Testing (MHT)
Based on 100 x 100 pixel image
108 vs. 1010
2x108 vs. 1020
3x108 vs. 1030
Number of Objects
Number of Computations
DL vs. MHT
1
2
3
• Previously un-computable (1030), can now be computed (3x108 )
• This pertains to many complex information-finding problems
APPLICATION 4
SENSOR FUSIONConcurrent fusion, navigation, and detection
below clutter
SENSOR FUSION
The difficult part of sensor fusion is association of data among sensors– Which sample in one sensor corresponds to which sample in
another sensor?
If objects can be detected in each sensor individually– Still the problem of data association remains– Sometimes it is solved through coordinate estimation
• If 3-d coordinates can be estimated reliably in each sensor– Sometimes it is solved through tracking
• If objects could be reliably tracked in each sensor, => 3-d coordinates
If objects cannot be detected in each sensor individually– We have to find the best possible association among multiple
samples– This is most difficult: concurrent detection, tacking, and fusion
NMF/DL SENSOR FUSION
NMF/DL for sensor fusion requires no new conceptual development
Multiple sensor data require multiple sensor models– Data: n -> (s,n); X(n) -> X(s,n)– Models Mm(n) -> Mm(s,n)
PDF(n|m) is a product over sensors– This is a standard probabilistic procedure, another
sensor is like another dimension– pdf(m|n) -> pdf(m|s,n)
Note: this solves the difficult problem of concurrent detection, tracking, and fusion
s
Source: UAS Roadmap 2005-
2030
UNCLASSIFIED
CONCURRENT NAVIGATION, FUSION, AND DETECTION
multiple target detection and localization based on data from multiple micro-UAVs
A complex case–detection requires fusion (cannot be done with one sensor)–fusion requires exact target position estimation in 3-D–target position can be estimated by triangulation from multiple views–this requires exact UAV position
• GPS is not sufficient• UAV position - by triangulation relative to known targets
–therefore target detection and localization is performed concurrently with UAV navigation and localization, and fusion of information from multiple UAVs
Unsolvable using standard methods. Dynamic logic can solve because computational complexity scales linearly with number of sensors and targets
GEOMETRY: MULTIPLE TARGETS, MULTIPLE UAVS
UAV 1
X1 = X01 + V1t
X1=(X1,Y1,Z1)
UAV m
Xm=(Xm,Ym,Zm)
Xm = X0m + Vmt
CONDITIONAL SIMILARITIES (pdf) FOR TARGET k
Data from UAV m, sample number n, where βnm = signature position and fnm = classification feature vector:
Similarity for the data, given target k:
wheresignature position
classification features
Note: Also have a pdf for a single clutter component pdf(wnm| k=0) which is uniform in βnm, Gaussian in fnm.
Compute parameters that maximize the log-likelihood
Data Model and Likelihood Similarity
Total pdf of data samples is the summation of conditional pdfs (summation over targets plus clutter)
(mixture model)
UAV parameters target parameters classification feature parameters
Concurrent Parameter Estimation / Signature Association (NMF iterations)
Note1: bracket notation
Note2: proven to converge (e.g. EM algorithm)
Note 3: Minimum MSE solution incorporates GPS measurements
FIND SOLUTION FOR SET OF “BEST” PARAMETERS BY ITERATING BETWEEN…
Parameter Estimation and Association Probability Estimation (Bayes rule)
(probability that sample wnm was generated by target k)
Sensor 1 (of 3): Models Evolve to Locate Target Tracks in Image Data
Sensor 2 (of 3): Models Evolve to Locate Target Tracks in Image Data
Sensor 3 (of 3): Models Evolve to Locate Target Tracks in Image Data
NAVIGATION, FUSION, TRACKING, AND DETECTION (this is the basis for the previous 3 figures, all fused in x,y,z, coordinates;
double-click on the blob to play movie)
Model Parameters Iteratively Adapt to Locate the Targets
Error vs. iteration# (4 targets)
Estimated Target Position vs. iteration# (4 targets)
Parameter Estimation Errors Decrease with Increasing Number of UAVs in the Swarm
Error falls off as ~ 1/√M, where M = # UAVs in the swarm
(Note: Results are based upon Monte Carlo simulations with synthetic data)
Error in Parameter Estimates vs. clutter level and # of UAVs in the swarm
Target position UAV position
APPLICATION 5
DETECTION IN SEQUENCES OF IMAGES
DETECTION IN A SEQUENCE OF IMAGES
Signature + low noise level (SNR= 25dB)
Signature + high noise level (SNR= -6dB)
signature is present, but is obscured by noise
DETECTION IN IMAGE SEQUENCETEN ROTATION FRAMES WERE USED
Iteration 10 Iteration 100 Iteration 400
Iteration 600Compare with
Measured Image (w/o noise)
Upon convergence of the model, important parameters are estimated, including center of rotation, which will next be used for spectrum estimation.
Four model components were used, including a uniform background component. Only one component became associated with point source.
TARGET SIGNATURE
Extracted from low noise image
Extracted from high noise image
APPLICATION 6
• Radar Imaging through walls- Inverse scattering problem - Standard radar imaging algorithms (SAR) do not work
because of multi-paths, refractions, clutter
SCENARIO
RADAR IMAGING THROUGH WALLS
Standard SAR imaging does not work
Because of refraction, multi-paths and clutter
Estimated model, work in progress
Remains:
-increase convergence area
-increase complexity of scenario
-adaptive control of sensors
DYNAMIC LOGIC / NMF
INTEGRATED INFORMATION: objects; relations;situations; behavior
Data and Signals
Dynamic Logic
combining conceptual analysis
with emotional evaluation
MODELS
- objects- relations- situations- behavioral
CLASSICAL METHODOLGYno closure
Result: Conceptual objects
signals Input: World/sceneSensors / Effectors
MODELS/templates •objects, sensors•physical models
Recognition
NMF: closurebasic two-layer hierarchy: signals and concepts
Result: Conceptual objects
signals
Input: World/scene
Sensors / Effectors
Correspondence / Similarity measures
MODELS •objects, sensors•physical models
Attention / Actionsignals Sim.signals
APPLICATION 7
• Prediction
- Financial prediction
PREDICTION
Simple: linear regression– y(x) = Ax+b– Multi-dimensional regression: y,x,b are vectors, A is a m-x– Problem: given {y,x}, estimate A,b
Solution to linear regression (well known)– Estimate means <y>, <x>, and x-y covariance matrix C– A = Cyx Cxx
-1; b = <y> - A<x>
Difficulties– Non-linear y(x), unknown shape– y(x) changes regime (from up to down)
• and this is the most important event (financial prediction)
– No sufficient data to estimate C • required ~10*dx+y
3 data points, or more
NMF/DL PREDICTION
General non-linear regression (GNLR)– y(x) = f(m|n) ym(x) = f(m|n) (Amx+bm)
– Amand bm are estimated similar to A,b in linear regression with the following change: all (…) are changed into f(m|n)(…)
For prediction, we remember that f(m|n) = f(m|x)
Interpretation– m are “regimes” or “processes”, f(m|x) determines influence of
regime m at point x (probability of process m being active)
Applications– Non-linear y(x), unknown shape– Detection of y(x) regime change (e.g. financial prediction or control)
• Minimal number of parameters: 2 linear regressions; f(m|n) are functions of the same parameters
• Efficient estimation (ML)• Potential for the fastest possible detection of a regime change
m
m
n
n
FINANCIAL PREDICTION Efficient Market Hypothesis
Efficient market hypothesis, strong: – no method for data processing or market analysis will bring
advantage over average market performance (only illegal trading on nonpublic material information will get one ahead of the market)• Reasoning: too many market participants will try the same
tricks
Efficient market hypothesis, week: – to get ahead of average market performance one has to do
something better than the rest of the world: better math. methods, or better analysis, or something else (it is possible to get ahead of the market legally)
FINANCIAL PREDICTIONBASICS OF MATH. PREDICTION
Basic idea: train from t1 to t2, predict and trade on t2+1; increment: t1->t1+1, t2->t2+1; …– Number of data points between t1 to t2 should be >> number
of parameters
Decide on frequency of trading, it should correspond to your psychological makeup and practical situation– E.g. day-trading has more potential for making (or losing) a
lot of money fast, but requires full time commitment
Get past data, split into 3 sets: (1)developing, (2)testing, (3)final test (best, in real time, paper trades)
After much effort on (1), try on (2), if work, try on (3)
DETAILS OFFINANCIAL PREDICTION
Develop “best” mathematical technique for market prediction– Takes a lot of effort– Test in “up” and “down” markets– If your simulated portfolio goes up and up smoothly (you are
constantly making money in computer simulation), look for an error• Simple errors in computing return• Include spread and commission in return• “Illegal” training: using (t2+1) information for training (or trading)
Technical details– What to optimize in training/development?
• Performance (return or cum. return), ROR = ($end - $start) / $start • Sharpe (return/risk ratio) Sh = (ROR – RORmarket) / std(RORmarket)
– Sh > 1 ok, Sh > 3 look for errors (beware)
– Include penalizing factor for free parameters (Akaike, Statistical Learning Theory (Vapnik), Ridge regression)
• Ridge regression: min [ (y(t) – p(t))2 + a (p(t) – <p(t)>)2 ]
FINANCIAL MARKET PREDICITION
Recommended Portfolios vs. Marketsportfolio gains: rec-sp = 6.2%, rec-nq = 7.4% (vs. markets sp = 1.8%, nq = -4.2% loss)
risk measures: gain/st.dev = 3.6, 4.0 (vs mkts 0.35, -.45), average exosure = 14% (vs. mkt 100%)
94.00%
96.00%
98.00%
100.00%
102.00%
104.00%
106.00%
108.00%
110.00%
12/30/2005 1/29/2006 3/1/2006 3/31/2006 5/1/2006 5/31/2006 7/1/2006
date
cu
mu
lati
ve g
ain
% (
VA
MI)
SP weeklyVAMI
NQ weeklyVAMI
RecSP+T weeklyVAMI
RecNQ+T weeklyVAMI
BIOINFORMATICS
Many potential applications– combinatorial complexity of existing algorithms
Drug design– Diagnostics: which gene / protein is responsible
• Pattern recognition• Identify a pattern of genes responsible for the condition
– Relate sequence to function• Protein folding (shape)• Relate shape to conditions
Many basic problems are solved sub-optimally (combinatorial complexity) – Alignment– Dynamic system of interacting genes / proteins
• Characterize• Relate to conditions
NMF/DL FOR COGNITIONSUMMARY
Cognition– Integrating knowledge and data / signals– Evolution from vague to crisp
Knowledge = concepts = models
Knowledge instinct = similarity(models, data)
Aesthetic emotion = change in similarity
Emotional intelligence– combination of conceptual knowledge and emotional evaluation
Applications– Recognition, tracking, fusion, prediction…
OUTLINE
• Cognition, complexity, and logic
• The Mind and Knowledge Instinct
• Language
• Integration of cognition and language
• Higher Cognitive Functions
• Future directions
LANGUAGE
Integration of language-data and language-models– Speech, text– Language acquisition / learning– Search engines
Language is similar to cognition – specific language data– NMF of language
LANGUAGE ACQUISITIONAND COMPLEXITY
Chomsky: linguistics should study the mind mechanisms of language (1957)
Chomsky’s language mechanisms– 1957: rule-based– 1981: model-based (rules and parameters)
Combinatorial complexity– For the same reason as all rule-based and
model-based methods
HIERARCHY OF LANGUAGE
Speech is a (loose) hierarchy of objects – Words are “made of” language sounds, phonemes– Phrases are “made of” words– …
Text is a (loose) hierarchy of objects – Letters, words, phrase,…
Meanings of language objects – Language objects acquire meanings in the hierarchy– Phonemes acquire meaning in words– Words acquire meaning in phrases– Phrases acquire meaning in paragraphs,…
APPLICATION: SEARCH ENGINE BASED ON UNDERSTANDING
Goal-instinct: –Find conceptual similarity between a query and text
Analyze query and text in terms of concepts
“Simple” non-adaptive techniques –By keywords–By key-sentences = set of words
• Define a sequence of words (“bag of words”)• Compute coincidences between the bag and the document• Instead of the document use chunks of 7 or 10 words
How to learn useful sentences?
APPLICATION: LEARN LANGUAGE UNDERSTANDING
Goal-instinct: – Find conceptual similarity between a query and text
Analyze query and text in terms of concepts
Learn and identify model-concepts in texts – Words from a dictionary– Hierarchy
• phrases made up of words, paragraphs of phrases…
Language instinct knowledge instinct
OUTLINE
• Cognition, complexity, and logic
• The Mind and Knowledge Instinct
• Language- NMF of language
• Integration of cognition and language
• Higher Cognitive Functions
• Future directions
NMF OF LANGUAGE basic two-layer hierarchy: words and phrase-concepts
Words and Concept-Models– words w(n), n = 1,…,N;
– model-phrase Mm(Sm,n), parameters Sm, m = 1, …;
– Simplistic “bag”-model: Mm(Sm,n) = Sm = {nm ; wm,1 ;wm,2 ;…wm,s}
• nm is a position of the model-center in the text {s/2… (N-s/2)}
Goal: learn phrase-models– associate sequences n with models m and find parameters Sm
– learn word-contents of phrases (and grammatical relationships)
Maximize similarity between words and models, L– Likelihood L = l(w(n))
– l(w(n)) = r(m) l(w(n) | Mm(Sm,n))
– CC: L contains MN items: all associations of words and models
n
m
DYNAMIC LOGIC (non-combinatorial solution)
Start with a large body of text and unknown phrase-models– any parameter values Sm – associate fuzzy phrase-model with its contents (words)– (1) f(m|n) = r(m) l(n|m) / r(m') l(n|m')
Improve parameter estimation– (2) Sm = Sm + f(m|n) [lnl(n|m)/Mm]*[Mm/Sm]
• ( determines speed of convergence)
– learn word-contents of phrases (and grammatical relationships)
Continue iterations (1)-(2). Theorem: NMF is converging
- similarity increases on each iteration - aesthetic emotion is positive during learning
'm
n
DIFFERENTIATION OF QUALITATIVE FUNCTIONS
Differentiation of bag-model– The “bag”-model is non-differentiable– This is a principal moment, learning non-differentiable models
requires sorting through combinations– Lead to combinatorial complexity
How to differentiate – Non-continuous, non-differentiable, qualitative functions
Also essential for the hierarchy – Higher level are made up of “bags” of lower level concepts
“QUALITATIVE” DERIVATIVE
Define fuzzy conditional partial similarity as– l(n|m) = Σs G(e(n,m,s) , m )
– e(n,m,s) is a distance between n and the word wm,s in Mm(n) that matches w(n), counted in the number of words (if no match, e(n,m,s) = S/2+1)
– Fuzziness is determined by phrase-model length, S
and matching st.dev. m = S / 3
Parameter estimation– Initialize with a large S and any values for { wm,s } – On every iteration compute e(n,m,s) and m
– From every model delete the least likely word– Reduce the phrase length S by 1– Thus, “most likely” words are gradually selected for each model– Details in Perlovsky, L.I. (2006). Symbols: Integrated Cognition and Language. Book Chapter in A. Loula, R. Gudwin, J. Queiroz, eds.,
Computational Semiotics. Idea Group, Hershey, PA.
OUTLINE
• Cognition, complexity, and logic
• The Mind and Knowledge Instinct
• Language
• Integration of cognition and language
• Higher Cognitive Functions
• Future directions
LANGUAGE vs. COGNITION
• “Nativists”, - since the 1950s- Language is a separate mind mechanism (Chomsky)- Pinker: language instinct
• “Cognitivists”, - since the 1970s- Language depends on cognition- Talmy, Elman, Tomasello…
• “Evolutionists”, - since the 1980s- Hurford, Kirby, Cangelosi…- Language transmission between generations
• Co-evolution of language and cognition
WHAT WAS FIRST COGNITION OR LANGUAGE?
How language and thoughts come together?
Conscious – “final results” ~ logical concepts
Language seems completely conscious– However, a child at 5 knows about “good” and “bad” guys– Are these conscious concepts?
Unconscious – fuzzy mechanisms of language and cognition
Logic:– Same mechanisms for L. & C.– Did not work
Sub-conceptual, sub-conscious integration
INTEGRATEDLANGUAGE AND COGNITION
Where language and cognition come together?– A fuzzy concept m has linguistic and cognitive-sensory models
• Mm = { Mmcognitive,Mm
language };
– Language and cognition are fused at fuzzy pre-conceptual level• before concepts are learned
Understanding language and sensory data– Initial models are fuzzy blobs
– Language models have empty “slots” for cognitive model (objects and situations) and v.v.
– Child’s learning
– Language participates in cognition and v.v.
L & C help learning and understanding each other – Help associating signals, words, models, and behavior
INNER LINGUISTIC FORM HUMBOLDT, the 1830s
In the 1830s Humboldt discussed two types of linguistic forms– words’ outer linguistic form (dictionary) – a formal designation– and inner linguistic form (???) – creative, full of potential
This remained a mystery for rule-based AI, structural linguistics, Chomskyan linguistics– rule-based approaches using the mathematics of logic make
no difference between formal and creative
In NMF / DL there is a difference – static form of learned (converged) concept-models– dynamic form of fuzzy concepts, with creative learning
potential, emotional content, and unconscious content
OUTLINE
• Cognition, complexity, and logic
• The Mind and Knowledge Instinct
• Language
• Integration of cognition and language
• Higher Cognitive Functions
• Future directions
HIGHER COGNITIVE FUNCTIONS
Abstract models are at higher levels of hierarchy– More vague-fuzzy, less conscious
At every level– Bottom-up signals are recognized lower-level-concepts
– Top-down signals are vague concept-models
– Behavior-actions (including learning-adaptation)
Similarity measures
Models
Action/Adaptation
Models
Action/AdaptationSimilarity measures
objects
situations
meanings
TOWARD A THEORY OF THE MIND
From Plato to physics of the mind– A mathematical theory describing “first
principles” of the mind – Corresponding to existing data and
making testable predictions
FROM PLATO TO LOCKE
Realism: – Plato: ability for thinking is based on a priori Ideas– Aristotle:
• ability for thinking is based on a priori (dynamic) Forms• an a priori form-as-potentiality (fuzzy model) meets
matter (signals) and becomes a form-as-actuality (a concept)
• actualities obey logic, potentialities do not
Nominalism– Antisthenes (contemporary of Plato)
• there are no a priori ideas, just names for similar things– Occam (14th c.)
• Ideas are linguistic phenomena devoid of reality– Locke (17th c.)
• A newborn mind is a “blank slate”
FROM KANT TO GROSSBERG
Kant: three primary inborn abilities– Reason = understanding (models of cognition)– Practical Reason = behavior (models of behavior)– Judgment = emotions (similarity)– “We only know concepts, not things-in-themselves”
Jung– Conscious concepts are developed based on inherited structures of the
mind, archetypes, inaccessible to consciousness– Realists vs. nominalists = introverts vs. extroverts
Chomsky– Inborn structures, not “general intelligence”
Grossberg– Models attaining a resonant state (winning the competition for signals)
reach consciousness
DL & Aristotle– A-priori models are vague-fuzzy and unconscious – “Understood” models in a resonant state are crisper and more
conscious
NMF AND BUDDHISM
Fundamental Buddhist notion of “Maya” – the world of phenomena, “Maya”, is meaningless deception
– penetrates into the depths of perception and cognition
– phenomena are not identical to things-in-themselves
Fundamental Buddhist notion of “Emptiness” – “consciousness of bodhisattva wonders at perception of emptiness
in any object” (Dalai Lama 1993)
– any object is first of all a phenomenon accessible to cognition
– value of any object for satisfying the “lower” bodily instincts is much less than its value for satisfying higher needs, knowledge instinct
– Bodhisattva’s consciousness is directed by the knowledge instinct
– concentration on “emptiness” does not mean emotional emptiness, but the opposite, the fullness with highest emotions related to the knowledge instinct, beauty and spiritually sublime
MIND VS. BRAIN
We start understanding how to relate the mind to the brain– Which neural circuits of the brain implement which functions
of the mind
NMF DYNAMICS
A large number of model-concepts compete for incoming signals
Uncertainty in models corresponds to uncertainty in associations f(m|n)
Eventually, one model (m') wins a competition for a subset {n'} of input signals x(n), when parameter values match object properties, and f(m'|n) values become close to 1 for n{n'} and 0 for n{n'}
Upon convergence, the entire set of input signals {n} is divided into subsets, each associated with one model-object
Fuzzy a priori concepts (unconscious) become crisp concepts (conscious) – dynamic logic, Aristotelian forms, Jungian
archetypes, Grossberg resonance Elementary thought process
CONSCIOUSNESS AND UNCONSCIOUS
Jung: conscious concepts and unconscious archetypes
Grossberg: models attaining a resonant state (winning the competition for signals) reach consciousness
NMF: fuzzy mechanisms (DL) are unconscious, crisp concept-models, adapted and matched to data are conscious
AESTHETIC EMOTIONS
Not related to bodily satisfaction
Satisfy instincts for knowledge and language – learning concepts and learning language
Not just what artists do
Guide every perception and cognition process
Perceived as feeling of harmony-disharmony– satisfaction-dissatisfaction
Maximize similarity between models and world– between our understanding of how things ought to be
and how they actually are in the surrounding world; Kant: aesthetic emotions
BEAUTY
Harmony is an elementary aesthetic emotion– higher aesthetic emotions are involved in the
development of more complex “higher” models
The highest forms of aesthetic emotion, beautiful – related to the most general and most important models– models of the meaning of our existence, of our purposiveness– beautiful object stimulates improvement of the highest models
of meaning
Beautiful “reminds” us of our purposiveness– Kant called beauty “aimless purposiveness”: not related to
bodily purposes– he was dissatisfied by not being able to give a positive
definition: knowledge instinct– absence of positive definition remained a major source of
confusion in philosophical aesthetics till this very day
Beauty is separate from sex, but sex makes use of all our abilities, including beauty
INTUITION
Complex states of perception-feeling of unconscious fuzzy processes– involves fuzzy unconscious concept-models– in process of being learned and adapted
• toward crisp and conscious models, a theory– conceptual and emotional content is undifferentiated– such models satisfy or dissatisfy the knowledge
instinct before they are accessible to consciousness, hence the complex emotional feel of an intuition
Artistic intuition – composer: sounds and their relationships to psyche– painter: colors, shapes and their relationships to psyche– writer: words and their relationships to psyche
INTUITION: Physics vs. Math.
Mathematical intuition is about– Structure and consistency within the theory– Relationships to a priori content of psyche
Physical intuition is about– The real world, first principles of its organization, and
mathematics describing it
Beauty of a physical theory discussed by physicists – Related to satisfying knowledge instinct
• the feeling of purpose in the world
WHY ADAM WAS EXPELLED FROM PARADISE?
God gave Adam the mind, but forbade to eat from the Tree of Knowledge– All great philosophers and theologists from time immemorial
pondered this – Maimonides, 12th century
• God wants people to think for themselves• Adam wanted ready-made knowledge• Thinking for oneself is difficult (this is our predicament)
Today we can approach this scientifically – Rarely we use the KI– Often we use ready-made heuristics, rules-of-thumb– Both are evolutionary adaptations– Cognitive effort minimization is opposite to the KI
2003 Nobel Prize in Economics – People’s choices are often irrational
GOD, SNAKE, and fMRI
Majority often make irrational, heuristic choices (CEM-type)
Stable minority is rational (KI-type)
fMRI – KI-type think with cortex (uniquely human)– CEM-type think with amygdala (animals)
God demands us being humans
“Snake’s apple” pulled Adam back to animals
SYMBOL
“A most misused word in our culture” (T. Deacon)
Cultural and religious symbols– Provoke wars and make piece
Traffic Signs
SIGNS AND SYMBOLSmathematical semiotics
Signs: stand for something else– non-adaptive entities (mathematics, AI)– brain signals insensitive to context (Pribram)
Symbols– general culture: deeply affect psyche– psychological processes connecting conscious
and unconscious (Jung)– brain signals sensitive to context (Pribram)– signs (mathematics, AI): mixed up– processes of sign interpretation
NMF: mathematics of symbol-processes– elementary thought process
NMF THEORY OF SYMBOLS -mathematical semiotics-
Semiotics studies symbol-content of culture
Example: a written word "chair" – The mind interprets it to refer to something else: an entity in the world, a
specific chair, or the concept "chair" in the mind– The mind, or an intelligent system is an interpreter, the written word is a
sign, the real-world chair is a designatum, and the concept in the interpreter's mind, the internal representation of the results of interpretation is called an interpretant of the sign
– The essence of a sign is that it can be interpreted by an interpreter to refer to something else, a designatum
This is a simplified description of a thinking process, called semiosis– its mechanism is given by the elementary thought process
Elementary thought process involving consciousness and unconscious, concepts and emotions, is a dynamic symbol process – a much more complicated entity than was originally envisioned by founders
of “symbolic AI”
SYMBOLS and “SYMBOLIC AI”
Founders of “symbolic AI” believed that by using “symbolic” mathematical notations they would penetrate into the mystery of mind– But mathematical symbols are just notations (signs)– Not psychic processes
This explains why “symbolic AI” was not successful
This also illustrates the power of language over thinking– Wittgenstein called it
• “bewitchment (of thinking) by language”
SYMBOLIC ABILITY
Integrated hierarchies of Cognition and Language– High level cognition is only possible due to language
– Language is only possible due to cognition
Similarity Action
ActionSimilarity
Similarity Action
ActionSimilarity
cognition language
M
M M
M
grounded in real-world objects
grounded in language
grounded in language
OUTLINE
• Cognition, complexity, and logic
• The Mind and Knowledge Instinct
• Language
• Integration of cognition and language
• Higher Cognitive Functions
• Future directions- Evolution of languages and cultures
CULTURE AND LANGUAGE
• Animal consciousness–Undifferentiated, few vague concepts
–No mental “space” between thought, emotion, and action
• Evolution of human consciousness and culture –More differentiated concepts
–More mental “space” between thoughts, emotions, and actions
–Created by evolution of language
• Language, concepts, emotions–Language creates concepts
–Still, colored by emotions
16-Sep-05 121
EVOLUTION OF CULTURES
• The knowledge instinct–Two mechanisms: differentiation and synthesis
• Differentiation –At every level of the hierarchy: more detailed concepts–Separates concepts from emotions
• Synthesis –Knowledge has to make meaning, otherwise it is useless–Diverse knowledge is unified at the higher level in the hierarchy–Connects concepts and emotions
Connect language and cognitionConnect high and low: concepts acquire meaning at the next level
16-Sep-05 122
DYNAMICS OF DIFFERENTIATION AND
SYNTHESIS• Differentiation, D
–New knowledge comes from differentiating old knowledge,Speed of change of D ~ D
–Differentiation continues if knowledge is useful (emotional)Speed of change of D ~ - S
–Differentiation stops if knowledge is “too” emotionalSpeed of change of D ~ 0, if S id “too large”
• Synthesis, S–Emotional value of knowledge–Emotions per concept diminish with more concepts
Speed of change of S ~ -D–Synthesis grows in the hierarchy (H)
Speed of change of S ~ H
16-Sep-05 123
CULTURAL STATES CAN BE MEASURED
Differentiation– Number of words
Synthesis– Emotions per word
Hierarchy – Social, political, cultural, language
“Material” measures – Demographics, geopolitics, natural resources…– Ignore for a moment
MODELING “spiritual aspects” ofCULTURAL EVOLUTION
Differentiation, synthesis, hierarchy
dD/dt = a D G(S); G(S) = (S - S0) exp(-(S-S0) / S1)
dS/dt = -bD + dH
H = H0 + e*t
DYNAMIC CULTURE
Average synthesis, high differentiation; oscillating solutionKnowledge accumulates; no stability
TRADITIONAL CULTURE
High synthesis, low differentiation; stable solutionStagnation, stability increases
TERRORIST’S CONSCIOUSNESS
Ancient consciousness was “fused”– Concepts, emotions, and actions were one
• Undifferentiated, fuzzy psychic structures
– Psychic conflicts were unconscious and projected outside
• Gods, other tribes, other people
Complexity of today’s world is “too much” for many– Evolution of culture and differentiation
• Internalization of conflicts: too difficult
– Reaction: relapse into fused consciousness• Undifferentiated, fuzzy, but simple and synthetic
The recent terrorist’s consciousness is “fused”– European terrorists in the 19th century– Fascists and communists in the 20th century– Current Moslem terrorists
INTERACTING CULTURES
Knowledge accumulation + stability
1) Early: Dynamic culture affects traditional culture, no reciprocity2) Later: 2 dynamic cultures stabilize each other
FUTURE SIMULATIONS OF EVOLUTION
Genetic evolution simulations (1980s - )– Used basic genetic mechanisms – Artificial Life, evolution models: Bak and Sneppen,
Tierra, Avida
Evolution of cultural concepts – Genes vs. memes (cultural concepts)– Evolution of concepts vs. evolution of genes
• Culture evolves much faster than genetic evolution • Human culture is <10,000 years, likely, no genetic
evolution (?)– Evolution of languages– Concepts evolve from fuzzy to crisp and specific– Concepts evolve into a hierarchy– Concepts are propagated through language
MECHANISMS OF CONCEPT EVOLUTION
Differentiation, synthesis, and language transmission
Differentiation– Fuzzy contents become detail and clear– A priori models, archetypes are closely connected to unconscious
needs, to emotions, to behavior• Concepts have meanings
Cultural and generational propagation of concepts through language– Integration of language and cognition is not perfect
• Language instinct is separate from knowledge instinct
Propagation of concepts through language– A newborn child encounters highly-developed language– Synthesis: cognitive and language models {MC, ML} are connected
individually– No guarantee that language model-concepts are properly integrated
with the adequate cognitive model-concepts in every individual• And we know this imperfection occurs in real life• Meanings might be lost• Some people speak well, but do not quite understand and v.v.
SPLIT BETWEEN CONCEPTUAL AND EMOTIONAL
Dissociation between language and cognition – Might prevail for the entire culture
Words maintain their “formal” meanings– Relationships to other words
Words loose their “real” meanings– Connection to cognition, to unconscious and emotions
Conceptual and emotional dissociate– Concepts are sophisticated but “un-emotional”– Language is easy to use to say “smart” things
• but they are meaningless, unrelated to instinctual life
CREATIVITY
At the border of conscious and unconscious
Archetypes should be connected to consciousness– To be useful for cognition
Collective concepts–language should be connected to – The wealth of conceptual knowledge (other concepts)– Unconscious and emotions
Creativity in everyday life and in high art– Connects conscious and unconscious
Conscious-Unconcious ≠ Emotional-Conceptual– Different slicing of the psyche
DISINTEGRATION OF CULTURES
Split between conceptual and emotional – When important concepts are severed from emotions – There is nothing to sacrifice one’s life for
Split may dominate the entire culture– Occurs periodically throughout history – Was a mechanism of decay of old civilizations– Old cultures grew sophisticated and refined but got
severed from instinctual sources of life• Ancient Acadians, Babylonians, Egyptians, Greeks,
Romans…
– New cultures (“barbarians”) were not refined, but vigorous• Their simple concepts were strongly linked to instincts,
“fused”
EMOTIONS IN LANGUAGE
• Animal vocal tract–controlled by old (limbic) emotional system–involuntary
• Human vocal tract–controlled by two emotional centers: limbic and cortex–Involuntary and voluntary
• Human voice determines emotional content of cultures–Emotionality of language is in its sound: melody of speech
16-Sep-05 135
LANGUAGE:EMOTIONS AND CONCEPTS
• Conceptual content of culture: words, phrases–Easily borrowed among cultures
• Emotional content of culture–In voice sound (melody of speech)–Determined by grammar–Cannot be borrowed among cultures
• English language (Diff. > Synthesis)–Weak connection between conceptual and emotional (since 15 c)–Pragmatic, high culture, but may lead to identity crisis
• Arabic language (Synthesis > Diff.)–Strong connection between conceptual and emotional–Cultural immobility, but strong feel of identity (synthesis)
16-Sep-05 136
SYNTHESIS
People cannot live without synthesis– Feel of wholeness– Meaning and purpose of life
Creativity, life, and vigor requires synthesis– Emotional and conceptual, conscious and unconscious– In every individual
• Lost synthesis and meaning leads to drugs and personal disintegration– In the entire culture
• Lost synthesis and meaning leads to cultural disintegration
Historical evolution of consciousness– From primitive, fuzzy, and fused to differentiated and refined– Interrupted when synthesis is lost– Differentiation and synthesis are in opposition, still both are required– Example: religion vs. science
• Religious synthesis empowered human mind (15 c) and created conditions for development of science (17 c)
• Scientific differentiation destroyed religious synthesis• Evolution of our culture requires overcoming this split, and it is up to us, scientists and engineers
Individual consciousness – Combining differentiation and synthesis– Jung called individuation, “the highest purpose in every life”
MECHANISM OF SYNTHESIS
Integrating the entire wealth of knowledge– Undifferentiated knowledge instinct “likelihood maximization”
• Global similarity– Differentiated knowledge instinct
• Highly-valued concepts• Local similarity among concepts
Highly valued concepts acquire properties of instincts– Affect adaptation, differentiation, and cognition of other concepts– Generate emotions, which relate concepts to each other
Differentiated knowledge instinct – An emergent hierarchy of concept-values – Differentiated emotions connect diverse concepts– We need huge diversity of emotions to integrate conceptual
knowledge => synthesis
DIFFERENTIATION OF EMOTIONS
Historical evolution of human consciousness– Animal calls are undifferentiated
• concept-emotion-communication-action– Ancient languages are highly emotional (Humboldt, Levy-
Brule)
Language evolved toward unemotional differentiation– Nevertheless, most conversations have little conceptual
content• From villages to corporate board-rooms, people talk to
establish emotional contact• Human speech affects recent and ancient emotional centers
– Inflections and prosody of human voice appeals directly to ancient undifferentiated emotional mechanisms
– Accelerates differentiation, but endangers synthesis
Music evolved toward differentiation of emotions– At once: creates tensions and wholeness in human soul
ROLE OF MUSIC IN EVOLUTION OF THE MIND
Melody of human voice contains vital information– About people’s world views and mutual compatibility – Exploits mechanical properties of human inner ear
• Consonances and dissonances
Tonal system evolved (14th to 19th c.) for – Differentiation of emotions– Synthesis of conceptual and emotional– Bach integrates personal concerns with “the highest”
Pop-song is a mechanism of synthesis– Integrates conceptual (lyric) and emotional (melody)– Also, differentiates emotions– Bach concerns are too complex for many everyday needs– Human consciousness requires synthesis immediately
Rap is a simplified, but powerful mechanism of synthesis– Exactly like ancient Greek dithyrambs of Dionysian cult
EVOLUTION vs. INTELLIGENT DESIGN
• Science causal mechanisms
• Religion teleology (purpose)
• Wrong! –In basic physics causality and teleology are equivalent–The principle of minimal energy is teleological–More general, min. action (min. Lagrangian)
• The knowledge instinct –Teleological principle in evolution of the mind and culture –Dynamic logic is a causal law equivalent to the KI–Causality and teleology are equivalent
16-Sep-05 141
PREDICTIONS AND TESTING of NMF/DL theory of the mind
Methods of experimental testing– Neural, psychological, and psycholinguistic labs– Emerging method: simulation of multi-agent evolving systems
General neural mechanisms of the thought process– includes neural mechanisms for bottom-up (sensory) signals, top-down
(“imagination”) model-signals, and the resonant matching between the two – confirmed by neural and psychological experiments
Adaptive modeling abilities – well studied: adaptive parameters are synaptic connections
Instinctual learning mechanisms
Ongoing and future research will confirm, disprove, or suggest modifications to
– similarity measure as a foundation of knowledge and language instincts– mechanisms of model parameterization and parameter adaptation– dynamics of fuzziness during perception/cognition/learning– mechanisms of language and cognition integration– mechanisms of differentiation and synthesis
CURRENT AND FUTURE RESEARCH DIRECTIONS
Physics of the mind– Higher cognitive functions– Intuition, beautiful, sublime, conscious, unconscious,
creativity– Cognition and language
Inverse problems / control for complex physical systems
Robotic systems– Integrated autonomous control, sensing, communication
cognition & language– Collaborative: robots learn from humans– Evolving (improving not obsolete)
Semantic web Interactive environment Evolution of languages and cultures
– Role of music in cognition and evolution– Better understanding among peoples
THE END
Can we describe mathematically and build a simulation model for evolution of all of these abilities?
Can we build robotic systems understanding us, collaborating with us?
PUBLICATIONS
280 publications
OXFORD UNIVERSITY PRESS(2001; 3rd printing)
2007
Neurodynamics of High Cognitive Functionswith Prof. Kozma, Springer
Sapient Systemswith Prof. Mayorga, Springer
Future:
The Knowledge InstinctBasic Books
BACK UP
Why the mind and emotions?
NMF vs. inverse problems
NMF vs. biology of eye
Cognition and understanding
Intelligent agents
WHY MIND AND EMOTIONS?
A lot about the mind can be explained: concepts, instincts, emotions, conscious and unconscious, intuition, aesthetic ability,…– but, isn’t it sufficient to solve mathematical equations or to code a
computer and execute the code?
A simple yet profound question– the answer is in history and practice of science– Newton laws do not contain all of the classical mechanics– Maxwell equations do not exhaust radars and radio-communication– a physical intuition about the system is needed – an intuition about NMF was derived from biological, linguistic,
cognitive, neuro-physiological, and psychological insights into human mind
Practical engineering applications of DL require biological, linguistic, cognitive, neuro-physiological, and psychological insights in addition to mathematics
NMF vs. BIOLOGY OF EYE
Human eye is a part of the brain - Integrated sensor-processor system in both design and in operations
• multi-layer hierarchical system• integrated adaptive optimized resource allocation
- Feedback from higher layers to lower layers is essential • eye-brain neural pathway contains more feedback connections than feedforward
ones- Perception involves matching of retinal projections and model projection-primings
Adaptive, joint optimization of sensor-processor system network - Based on a hierarchy with feedback among layers and modules
NMF hierarchy with feedback - Every layer has 5 basic modules/elements: (1) incoming signals (structured at lower layer, unstructured at the current layer) (2) models(3) similarity measure between signals and models (knowledge instinct)(4) adaptation mechanism(5) outgoing signals (a structure: sign-concept)
NMF VS. INVERSE SCATTERING
Inverse Scattering in Physics– reconstruction of target properties by “propagating back” scattered fields
– usually complicated, ill-posed problems (exception: CATSCAN)
Biological systems (mind) solve this problem all the time– by utilizing prior information (in feedback neural pathways)
Classical Tikhonov’s inversion cannot use knowledge– regularization parameter () is a constant – Morozov’s modification can utilize prior estimate of errors for
Inverse Scattering using NMF– can utilize any prior knowledge ( became an operator)– utilizes prior knowledge adaptively ( depends on parameters)
COGNITION AND UNDERSTANDING
Incoming signals, {x,w} are associated with model-concepts (m)– creating phenomena (of the NMF-mind), which are understood
as objects, situations, phrases,…– in other words signal subsets acquire meaning (e.g., a subset
of retinal signals acquires a meaning of a chair)
Several aspects of understanding and meaning– concept-models are connected (by emotional signals) to
instincts and to behavioral models that can make use of them for satisfaction of bodily instincts
– an object is understood in the context of a more general situation in the next layer consisting of more general concept-models (satisfaction of knowledge instinct)
• each recognized concept-model (phenomenon) sends (in neural terminology: activates) an output signal
• a set of these signals comprises input signals for the next layer models, which ‘cognize’ more general concept-models
• this process continues up and up the hierarchy towards the most general models: models of universe (scientific theories), models of self (psychological concepts), models of meaning of existence (philosophical concepts), models of a priori transcendent intelligent subject (theological concepts)
NMF AND INTELLIGENT AGENTS
Agents are– Autonomy and interaction, communication – Goal-oriented, functional– Sense environment, extracts information – use knowledge, produce new knowledge– embody the concept of life and intelligence
Every NMF model-concept is an ‘agent’– Goal: improve its fitness to the data (improve knowledge)– Significantly autonomous, communicates with other agents– Receives signals from sensors or other agents – Competitive-cooperative interaction (learning)
• cooperation within a model, competition among models
Higher level: Culture, society of NMF agents (minds)– Learn from each other– Communicate, evolve language, culture– Applications: collaborative systems