Embed Size (px)
DESCRIPTION
Human visual system interprets information obtained through eyes to build a model of the surrounding world. This channel is our main source for understanding the world. Walking on a street, reading a book, or watching a movie all rely on our visual system. The relationship between movement, visual perception and language is complex. Movement is a specific focus of this presentation for several reasons. It is a fundamental part of human activities that ground our understanding of the world. Abstract meanings are often constructed as metaphoric extensions of movement schemas. As there is an increasing amount of video and motion tracking data available, formation of semantic models based on movement using computational methods is becoming feasible. In addition to movement, multilinguality and subjectivity of understanding are also addressed.
Citation preview
From Patterns of Movement toSubjectivity of Understanding
Timo Honkela
Aalto University(former Helsinki University of Technology)
Department of Information and Computer ScienceCognitive Systems research group
Finland
UC Santa Barbara, 4 Dec 2012
0Background and
(un)Related Topics
Natural language database interfacewith dependency-based compositional semantics
● H. Jäppinen, T. Honkela, H. Hyötyniemi & A. Lehtola (1988):A Multilevel Natural Language Processing Model. Nordic Journal of Linguistics 11:69-87.
What is the turnover of the ten largest stock exchange companies in forestry?
Morphological analysis
Dependency parsing
Logical analysis
Database query formation
Result from the SQL database
Classical example: Learning meaning from context:
Maps of words in Grimm fairy tales
Honkela, Pulkki & Kohonen 1995
Map of Finnish Science
Chemistry
Physics andengineering
Biosciences
Medicine
Culture and society
WordICA
Timo Honkela, Aapo Hyvärinen, and Jaakko Väyrynen. WordICA - Emergence of linguistic representations for words by independent component analysis. Natural Language Engineering, 16(3):277–308, 2010.
Jaakko J. Väyrynen, Lasse Lindqvist, and Timo Honkela. Sparse distributed representations for words with thresholded independent component analysis. In Proceedings of IJCNN'07, pages 1031–1036, 2007.
Learning taxonomies
Mari-Sanna Paukkeri, Alberto Pérez García-Plaza, Víctor Fresno, Raquel Martínez Unanue and Timo Honkela (2012). Learning a taxonomy from a set of text documents. Applied Soft Computing, 12(3), pp. 1138--1148.
Text mining for peer support
TOPIC ANALYSIS SENTIMENT ANALYSIS
Discussion forum postings, etc.
Selected stories
STYLEANALYSIS
MULTICRITERIA SELECTION PROCESS
User modelingand analysis of
feedback
EVALUATION
User'sinput
(Hon
kela
, Iz
zatd
ust,
Lag
us 2
012)
ICA of wellbeing-related termsin Reddit texts
(Honkela, Izzatdust, Lagus 2012)
Analyzing Complexity of Languages
Markus Sadeniemi, Kimmo Kettunen, Tiina Lindh-Knuutila, and Timo Honkela. Complexity of European Union languages: A comparative approach. Journal of Quantitative Linguistics, 15(2):185–211, 2008.
META-NETNetwork of Excellence
MultilingualWeb
Concept Formation andCommunication - General Theory
Timo Honkela, Ville Könönen, Tiina Lindh-Knuutila, and Mari-Sanna Paukkeri. Simulating processes of concept formation and communication. Journal of Economic Methodology, 15(3):245–259, 2008.
λ : Ci × Cj → R, i ≠ jA distance between two points in the concept spaces of different agents
S: symbol space,The vocabulary of anagent that consists of discrete symbols
: sξ i S∈ i → CAn individual mapping function from symbols to concepts
φi: Si D→An individual mapping from agent i's vocabulary to the signal space D andan inverse mapping φ
1 i from the signal
space to the symbol space
Ci: Ndimensional metric concept space
Observing f1 and after symbol selection process, agent 1 communicates a symbol s*to agent 2 as signal d. When agent 2 observes d, it maps it to some s2
S∈ 2 by using the function φ 11.
Then it maps the symbol to some point in its concept space by using ξ2. If this point is close to its observation f2 in the sense of λ, the communication process has succeeded.
1Patterns of
Movement
Why brains?
● What are the central differences between plants and animals?
“The original need for a nervous system was to coordinate movement, so an organism could go find food, instead of waiting for the food to come to it.”
● An extreme example: A sea squirt transformsfrom an “animal” to a “plant”. It absorbs its own cerebral ganglion that it used to swim about and find its attachment place.
http://goodheartextremescience.wordpress.com/2010/01/27/meet-the-creature-that-eats-its-own-brain/
http://www.fi.edu/learn/brain/
Human movement
David Bailey's thesis (1997):
Verbs related to hand movement
Point of view fromcognitive linguistics
● The meaning of linguistic symbols in the mind of the language users derives from the users' sensory perceptions, their actions with the world and with each other.
● For example: the meaning of the word 'walk' involves● what walking looks like● what it feels like to walk and after having walked● how the world looks when walking
(e.g. objects approach at a certain speed, etc.). ● ...
Abstract vs concrete grounding
Ronald Langacker
Motion capture
AnimationImage analysis
Video analysis
Robotics
Machine learning
Language learning
Socio-cognitive modelingSymbol grounding
Jorma Laaksonen
Tapio Takala
Klaus Förger
Harri Valpola
Oskar Kohonen
Timo Honkela
Reinforcementlearning
Paul WagnerMarkus Koskela
Xi Chen
Learning relations
Kinect
OptiTrack
Multimodally Grounded Language Technology
A project funded by Academy of Finland2011-2014
Timo Honkela as the Principal Investigator
A collaboration betweendepartments of
* Information and Computer Science, and
* Media Technology
Turk on Gesture Interaction
Matthew Turk, UCSB
Potential uses of gesture interaction technologies:
● mouse replacement for user interaction
● video game control● navigation for
visualization● sign language
recognition● automatic transcription
of communication● medical diagnosis and
rehabilitation● sculpting● conducting and playing
music● interactive art
Earlier today:
Labeling movements
From an unpublished manuscript. Experiments by Klaus Förger.
Linking between modalities
Potential uses of the emerging technologies
● Multimodally grounded natural language interaction and machine translation
● Animation based on linguistic instruction● Automated skill instruction
(playing an instrument, learning some sports, etc.)
● Video annotation● Addressing some of the fundamental issues
in traditional AI, cognitive science and philosophy
2Contextuality and
Subjectivity of
Understanding
Meaning is contextual
red winered skinred shirt
Gärdenfors: Conceptual Spaces
Hardin: Color for Philosophers
Meaning is contextual
SNOW -WHITE?
WHITE
Meaning is contextual
● “Small”, “big”● “White house”● “Get”● “Every” - “Every Swede is tall/blond”● etc. etc.
Another comment:
Strict compositionality cannot be assumed
Fuzziness
Learning meaning from context
● Self-Organizing Semantic Maps● Latent Semantic Analysis● Latent Dirichlet Allocation● WordICA● etc. etc.
Classical example: Learning meaning from context:
Maps of words in Grimm fairy tales
Honkela, Pulkki & Kohonen 1995
Meaning is subjective
Meaning is subjective
● Good● Fair● Useful● Scientific● Democratic● Sustainable● etc.
A proper theory ofmeaning has to takethis into account
Gary B. Fogel11th of June, 2012
WCCI 2012
2bMeasuring
Subjectivity of
Understanding
User-specificdifficulty
assessment
Basic architecture of the method
GICA:Grounded
Intersubjective Concept Analysis
Description of the method
Timo Honkela, Juha Raitio, Krista Lagus, Ilari T. Nieminen, Nina Honkela, and Mika Pantzar.
Subjects on objects in contexts: Using GICA method to quantify epistemological subjectivity.
Proceedings of IJCNN 2012, International Joint Conference on Neural Networks, pp. 2875-2883, 2012.
Publication:
Subjectifying: adding subjective views into object-context matrices
Outcome: Subject-Object-Context (SOC) Tensors
Potential sources for subjectification
● Conceptual surveys: ● individual assessment of contextual
appropriateness
● Text mining:● statistics of word/phrase-context patterns
● Empirical psychology:● reaction times, etc.
● Brain research
Flattening: unfolding 3-way tensorfor traditional 2-way analysis
GICA:Grounded
Intersubjective Concept Analysis
Examples of use
OBJECTS:
Relaxation
Happiness
Fitness
Wellbeing
CONTEXTS:
SUBJECTS: Event participants
Data collection
MDS: Objects x Subjects
Fitness
NeRV: Objects x Subjects
Fitness
J. Venna, J. Peltonen, K. Nybo, H. Aidos, and S. Kaski. Information Retrieval Perspective to Nonlinear Dimensionality Reduction for Data Visualization. Journal of Machine Learning Research, 11:451-490, 2010.
NeRV:
SOM: Objects x Subjects
Case 2: State of the Union Addresses
● In this case, text mining is used for populating the Subject-Object-Context tensor
● This took place by calculating the frequencies on how often a subject uses an object word in the context of a context word● Context window of 30 words
Analysis of the word 'health'
Timo Honkela, Ville Könönen, Tiina Lindh-Knuutila, and Mari-Sanna Paukkeri. Simulating processes of concept formation and communication. Journal of Economic Methodology, 15(3):245–259, 2008.
Conclusions (1)
● Languages, including formal languages, should be considered as tools for coordination, storing and sharing knowledge in a compressed form – approximate and relative to the point of view taken
● Constructing a language or symbol system (such as an ontology) is an investment and spreading the language into use in a community is even a larger one
Conclusions (2)
● Making people aware of the differences in the conceptual systems among them may have different applications, e.g., ● Helping in conflict resolution● Promoting interdisciplinary communication● Enhancing participatory processes and
democracy
From TEDxAALTO presentation “Measuring Subjectivity of Meaning – and How it may change our life” with illustrations by Nelli Honkela
1+2Movement and
Subjectivity
goo.gl / UZnvH
goo.gl / UZnvHSurvey address:
Thank y o u !
