View
47
Download
0
Category
Tags:
Preview:
DESCRIPTION
Reading what Machines ‘Think’. Fabio Massimo Zanzotto and Danilo Croce University of Rome “Tor Vergata” Roma, Italy. Prelude. Question This is a fascinating research problem. Can we find a more controlled setting where we can test if this is possible?. - PowerPoint PPT Presentation
Citation preview
Fabio Massimo Zanzotto and Danilo CroceUniversity of Rome “Tor Vergata”Roma, Italy
Reading what Machines ‘Think’
F.M.Zanzotto
University of Rome “Tor Vergata”
Prelude
Brain Activation
Pattern Recognizer
chair
Tom Mitchell, Invited Talk at the Association for Computational Linguistics Conference 2007
Mitchell, T.M., Shinkareva, S.V., Carlson, A., Chang, K.M., Malave, V.L., Mason, R.A., Just, M.A.: Predicting human brain activity associated with the meanings of nouns. Science 320(5880) (May 2008) 1191–1195
QuestionThis is a fascinating research problem. Can we find a more controlled setting where we can test if this is possible?
F.M.Zanzotto
University of Rome “Tor Vergata”
Idea
Cognitive physical object
Cognitive task
Observedimage
Observing a chair
Sorting a vector
BrainComputational
Machine
F.M.Zanzotto
University of Rome “Tor Vergata”
Why investigating the computer side is relevant?• Foundational perspective
– Computers are becoming extremely complex. They are fastly approaching the complexity of human brain
– Computers are controlled machines: their behavior and thier internal organization is known
– Then, computers offer a way to estimate if the claim on the brain side is reachable: if we can read what machines think, we can hope to read what brains think.
Motivation
F.M.Zanzotto
University of Rome “Tor Vergata”
Why investigating the computer side is relevant?• Applicative perspective
Can we develop technologies that “read the computer mind”? This predictive model can have a wide variety of applications, e.g., detecting malicious software, detecting the intentions of hostile computers by looking at their activation patterns.
Motivation
F.M.Zanzotto
University of Rome “Tor Vergata”
• Investigating the computer side: Long term research program
• Sketching the overall observation activity• Virtual Observation of Processes• Experimental Investigation
In the rest of the talk
F.M.Zanzotto
University of Rome “Tor Vergata”
Long-term research program
101010100101110111100001001010101001011101110101101001010101100000111010101010101111101010101110101101111110001101010100101110111100001001010101001011101110101101001010101100000111010101010101111101010101110101101111110000101010111010110111111000010101011101011011111100001010101110101101111110001…
011101111000010101010101
011101111000010101010101111101010101110101101111110001..
.
011101111000010101010101111101010101110101101111
110001...
Physical Memory Chip Physical Memory Dump Virtual Memory Dump(organized in processes)
Process Memory Dump
Physical device activation image
capturer
Virtual activation image
capturer
Virtual activation image
capturer
Virtual activation image
capturer
F.M.Zanzotto
University of Rome “Tor Vergata”
Sorting a vector
Sketching the overall observation activity
Brain Activation
Pattern Recognizer
chair
Sorting a vector
Process Activation
Pattern Recognizer
Building images from processes
Defining feature spaces for images
Observed Phenomena: Processes
F.M.Zanzotto
University of Rome “Tor Vergata”
Observed Phenomena: Processes
F.M.Zanzotto
University of Rome “Tor Vergata”
Given a cognitive activity, the procedure for extracting images from this activity is then the following:– running the process p representing the cognitive
activity c– stopping the process at given states or at given time
intervals – dumping the memory associated with the process M(p)– given a fixed height image and the memory dump, read
incrementally bytes of the memory dump and fill the associated RGB pixel with the read values I(M(p))
Building activation images from processes
F.M.Zanzotto
University of Rome “Tor Vergata”
Process memory in a given time interval
Activation image of the process in a given time interval
where
is the RGB pixel definition of the image
Building activation images from processes
][)( 0 mbbpM
kvv
k
pp
pppMI
,0,
,00,0
))((
][ 23133 h·j)(ih·j)(ih·j)(ii,j b bb p
F.M.Zanzotto
University of Rome “Tor Vergata”
Process: Vector Sorter Initial State
Process: Vector Sorter Final State
Building activation images from processesSmoothing
(more similar to real chip observation)
F.M.Zanzotto
University of Rome “Tor Vergata”
We used three major classes of features• Chromatic feaures
– Capture the color properties of the image determining, an n-dimensional vector representation of the 2D chromaticity histograms
• texture (OP - OGD) features– emphasize the background properties and their
composition.• transformation features (OGD)
Defining feature spaces for images
F.M.Zanzotto
University of Rome “Tor Vergata”
• Experimental Set-up– Collection of activation images– Used Machine Learning algorithms
• Experimental Results
Experimental Evaluation
F.M.Zanzotto
University of Rome “Tor Vergata”
• 3 different “cognitive tasks” (algorithms) – sorting, comparing two strings, visiting a binary tree
• 3 different programming languages – c, java, php
• for each pair algorithm-programming language– 20 different randomly generated input data– 3 snapshots (beginning, middle, end)
Experimental Set-up
F.M.Zanzotto
University of Rome “Tor Vergata”
We randomly splited the final set (540 images) in:• Training: 270 images Testing: 270 imagesTwo classification tasks:• Determining the programming language (3 classes) (lang) • Determining the cognitive task (3 classes) (algo) Used Machine learning Models:• Decision Tree Learners (DecTree)• Naive Bayes
Experimental Set-up
F.M.Zanzotto
University of Rome “Tor Vergata”
Results
Classification accuracy
F.M.Zanzotto
University of Rome “Tor Vergata”
The parallelism between computer and brain/mind is not new in general– Cognitive psychology– Cognitive sciences
We looked this parallelism from an other perspective
Conclusion
F.M.Zanzotto
University of Rome “Tor Vergata”
Future Work
101010100101110111100001001010101001011101110101101001010101100000111010101010101111101010101110101101111110001101010100101110111100001001010101001011101110101101001010101100000111010101010101111101010101110101101111110000101010111010110111111000010101011101011011111100001010101110101101111110001…
011101111000010101010101
011101111000010101010101111101010101110101101111110001..
.
011101111000010101010101111101010101110101101111
110001...
Physical Memory Chip Physical Memory Dump Virtual Memory Dump(organized in processes)
Process Memory Dump
Physical device activation image
capturer
Virtual activation image
capturer
Virtual activation image
capturer
Virtual activation image
capturer
F.M.Zanzotto
University of Rome “Tor Vergata”
Thank you for the attention!
Recommended