From Awareness to Adaptation

Univ. Prof. Dr. Alois Ferscha

Universität Linz, Institut für Pervasive Computing Altenberger Straße 69, A-4040 Linz

[email protected]

Large Scale Recognition and Opportunistic Configuration

Alois Ferscha

AWARENESS Workshop

Faculty of Engineering of the University of Bologna viale Risorgimento 2, 40136 Bologna

January 23, 2012

(supported by the EC under the FP7 FET grant AWARENESS)

From Awareness to Adaptation

Connectedness: Late 90s – Early 2000s

Connecting literally every-“thing” to every-“thing” driven by the availability of “technology to connect” >> Networks of ICT systems (Internet, Mobile Data, Wireless Communications) •  clouds of communicating, miniaturized, cheap, fast, powerful, wirelessly

connected, “always on” systems (omnipresent / total connectivity) •  enabled by the massive availability of miniaturized computing, storage,

communication, and embedded systems technologies •  sensors collect data, passive interaction with environment; actuators control devices,

can modify environment •  sensor-actuator systems invert role of interaction from human to machine

(implicit interaction) •  special purpose computing and information appliances, spontaneously communicating •  organism like capabilities (self-configuration, self-healing, self-optimizing, self-

protecting)

Weiser, M.: The Computer for the 21st Century. Scientific American. September, 1991, 94-104. IEEE Pervasive Computing: Weiser’s Vision 20 Years Later, Vol. 11, No. 1, January-March 2012.

IEEE Pervasive Computing: Weiser’s Vision 20 Years Later, Vol. 11, No. 1, January-March 2012.

Alois Ferscha Awareness >> Adaptation // AWARENESS Workshop, Bologna // January 23, 2012 // 2

Aware interaction among every-“thing”s driven by upcoming context recognition and knowledge processing technologies >> Ecologies of ICT systems •  situation aware, self-aware, context-aware, energy-aware, etc. •  reframing autonomy to be based on “knowledge”, extracted from low level sensor

data captured in a particular situation or over long periods of time •  “autonomic elements” able to capture, interpret, predict, context (from sensors) •  to build up, represent, carry and share knowledge •  to self-describe, -manage, and –organize with respect to the environment •  behaviour >> “knowledge based” monitoring, analyzing, planning and executing •  interaction in spontaneous spatial/temporal contexts, based on proximity, priority,

privileges, capabilities, interests, offerings, environmental conditions, etc.

Awareness: Early – Mid 2000s

Schilit, B., Adams, N. and Want, R.: Context-Aware Computing Applications, Mobile Computing Systems and Applications, 1994. WMCSA 1994, Vol., No., pp.85-90, 8-9 Dec. 1994. Dey, A. K.: Understanding and Using Context. Personal and Ubiquitous Computing 2001. Vol. 5, No 1, 4-7. Kephart, J.O. and Chess, D.M.: The vision of autonomic computing, IEEE Computer, Vol. 36, No.1, pp. 41- 50, Jan 2003.



“Meaningful” interaction of every-“thing”s within a complex system build upon connectedness and awareness, but give meaning to situations / actions >> “Smartness” •  use technical sensors (aligned with 5 human senses) for

vision, audition, gustation, olfaction, tactition to give meaning to situations •  computational perception > to extract meaning ... •  understanding patterns (activity, mobility, „life style“) > to extract meaning ...

•  formalization of human cognitive capabilities: attention, intent, emotions, experience, ...

•  relate meaning to building up belief, acting, expectation, memorizing, learning, developing behaviour, building up knowledge

Smartness: Mid 2000s - Present

Castellani B, Hafferty FW. Sociology and complexity science: A new field of science. New York, NY: Springer; 2009. Pentland, A.S.: Socially Aware Computation and Communication. IEEE Computer Volume 38, Issue 3, March 2005.



Pervasive Adaptation Research Agenda (www.perada.eu)

Request a Printed Copy: http://www.pervasive.jku.at/rab/order/ See fet11 Presentation Videos: http://www.pervasive.jku.at/fet11 Alois Ferscha Awareness >> Adaptation // AWARENESS Workshop, Bologna // January 23, 2012 // 5

Highly complex, orchestrated, cooperative and coordinated systems •  “Ensembles of Digital Artefacts” (FP7 FET), “the power of crowds” •  Rephrased in Weiser-words: “Pervasive ICT will be weaving social and

technological phenomena into the ‘fabric of everyday life of societies. ”

•  "... dynamic network of many … acting in parallel …” “… constantly acting and reacting to what the other are doing …” “… where the control tends to be highly dispersed and decentralized …” “… if there is to be any coherent behavior, it has to arise from competition / cooperation.“ “… overall behavior of the system is the result of a huge number of decisions made every moment by many individuals …" (Castellani, B., Haerty, F.W., 2009).

•  deployment of pervasive computing systems at massive scale poses challenges both in the technological as well as in the societal dimension!

1012 -1013 “things” or “goods” traded in markets 109 humans; 109 nodes on the internet, mobile phones, cars, digital cameras, etc. on planet 107 citizens in megacities, 108 users on Facebook, 108 videos on YouTube, 107 titles on last.fm, ...

”Socio-Technical Fabric”

A. Ferscha, N. Davies, A. Schmidt, N. Streitz: Pervasive Socio-Technical Fabric. Procedia Computer Science. Elsevier, Vol. 7, pp. 88-91, Dec. 2011.


A Critique on “Non-Social” ICT

“working in the small”, severe deficiencies “in the large”


A Critique on “Non-Social” ICT

“working in the small”, severe deficiencies “in the large” Personal Communication (email, social network software, …) >> ?? Personal ICT (smartphones, appliances, gadgets, … ) >> ?? Individual Mobility (automotive vehicles, airplanes, public transp., … ) >> ?? Logistics (transportation of goods, energy, … ) >> ?? Mass Media (print, TV, e-paper, digital video, … ) >> ?? Common: None has been designed on “Social Grounds” !


What are the Social Capabilities of your

Car ? Espresso Machine ? Shoes ?


What is Socially „Capable“ ICT?

What are the Social Capabilities of your

Car ? Espresso Machine ? Shoes ?

“We need to understand that traffic is not just a line of cars: It is a web of connections. A real solution will look at relationships across the entire road network and all the other systems that are touched by it: our supply chains, our environment, our companies, the way people and communities live and work…“

IBM Global Commuter Pain Study, 2010

“The possibility is quite high, that if a lot of people power off all lights at the same time and back on 5 minutes later, pumped hydro storage wont be enough to satisfy the enormous spike, resulting in substations taking their last breath.” Is Brazil’s Mega-Blackout A Harbinger Of Energy Grid Disasters to Come?

Io9-com, 2009

Mass Panic Jan 2006, Mekka Sep 2005, Bagdad May 2001, Accra Apr 2001, Johannesburg Jun 2000, Roskilde Dec 1999, Innsbruck Jul 1990, Mekka Apr 1989, Sheffield Duisburg Loveparade, 2010


What is Socially „Capable“ ICT?

[1] Dey, A. K.: Understanding and Using Context. Personal and Ubiquitous Computing 2001. Vol. 5, No 1, 4-7. [2] Pentland, A.: Socially Aware Computation and Communication. IEEE Computer Volume 38, Issue 3, March 2005, 33–40. [3] Groh, G., et.al.: . Applications for social situation models. In Proc. of the IADIS Int. Conf. on Wireless Applications and Computing. 2010. [4] Lukowicz, P., Pentland, A. and Ferscha, A..: From Context Awareness to Socially Interactive Computing, IEEE Pervasive, Vol. 11, No 1. 2012.

Context “… is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and application themselves.” [1] Seminal work on context modeling, context recognition, context awareness by A. Dey Social Context „... is the additional non-verbal information (e.g. signals from the body language, facial expression, and tone of voice, etc.) transmitted among communicating people, and which is the main determinant of a successful social interaction and engagement.” [2] Seminal work on cognitive and social analysis of human communication by A. Pentland “ .. all social relations, social interactions and social situations which are directly related to or confined to small time-intervals and space-regions around the present time or present location of a person”. [3] Socially Aware Computing „... instead of considering each device for itself, look at them as a socially collaborating collective ...” [4]

Socio-Inspired ICT

Context Aware ICT

Socially Aware ICT

Socially Interactive ICT

Socio- Inspired ICT


Socio-Inspired ICT


Understanding the hidden laws and processes of society Inspire the development of a new wave of robust, trustworthy and adaptive ICT based on the principles of social interactions.

!

Perception range for early lane changes

1,000m

100m 500m

A. Riener, K. Zia, A. Ferscha, C. Beltran Ruiz, J. Minguez Rubio: AmI technology helps to sustain speed while merging – A data driven simulation study on Madrid motorway ring M30. 14th IEEE/ACM International Symposium on Distributed Simulation and Real Time Applications (DS-RT 2010), Fairfax, VA USA, IEEE Computer Society Press, pp. 10, October 2010.

MAIN ROAD

LOWER ROAD

outermost/upper lane middle lane

innermost lane

innermost lane middle lane

outermost lane

MERGING LANE (PM45)

Sensor/Tactor Seat: Supporting Lane Change Safety by Extending the Perception Range

Sensor/Tactor Seat Madrid M30 Motorway

Merging Flow Optimization M30 Motorway Traffic Patterns

Example Socio-Inspired ICT :: Social Cars (SOCIONICAL)


Example Socio-Inspired ICT :: Social Sense App (SOCIONICAL)

Example Socio-Inspired ICT :: Social Displays (SAPERE)

Scale m displays : n users

LSAs

Content User-profiles Presentation Service Scheduling Service Steering Service Destination Field

Eco-Laws

DIFFUSE BOND-TARGET BOND-MATCH CREATE-STUB EXPORT-STUB MERGE-STUB EXPORT-SKEL MERGE-SKEL SHORTEST YOUNGEST

Matcher

Content ! User profile Scheduling Service ! Content Scheduling Service ! Pres. Service Presentation Service ! Content Steering Service ! Scheduling Service Steering Service ! Destination Field

Nodes

Both public and private displays are SAPERE nodes that advertise their presence by pumping a computat ional f ie ld throughout the ecosystem. This fields are used by the steering service for path computation.


Socio-Inspired ICT Research Challenges

Socio-Inspired ICT

Social Values: Ethics Moral Norm Social Awareness Social Behaviour Social Interaction Cognitive Capabilities

Novel ICT:

Microelectronics Sensors / Effectors

Smart Material

Social Media Participatory Platforms.

Big Data

Reality Mining Recommender Systems

Value Based Design

(Respect, Dignity, Trust)

ICT with Social Capabilities (Needs Fundamental Research) Social Adaptiveness, Self-Organization Cooperation, Competition Conflict Resolution Negotiation, Decision Making Reputation Collective Awareness Attraction/Repulsion Flocking/Foraging Morphogenesis/Chemotaxis

ICT w. Cognitive Capabilities (Needs Fundamental Research)

Attention, Perception, Meaning

Belief, Trust Experience Expectation

Empathy Narration

Goal Oriented Behavior

Formal / Computational Models

of

“Social Context” “Social Awareness”

“Principles of Social Interaction” “Social Sense” “Social Norm”


Socio-Inspired ICT Vision

(1) New ICT could contribute to gather direct, accurate and intelligible information on how an individual experiences social relations >> foundational character of a “Social Sense”

(2) New ICT could be designed to gain from “socially inspired” system architectures and operation principles >> foundational character of “Socially Capable/Adaptive ICT”


Technology

From Awareness to Adaptation