1

Physical-Cyber-Social Computing

Amit ShethLexisNexis Ohio Eminent Scholar,

Director, Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled ComputingWright State University, Dayton, OH, USA

Ack: Pramod Ananthram, Cory Henson, and Kno.e.sis Semantic Sensor Web team

Presented at SW2022 @ ISWC2012

2

1999 - 2002

3http://knoesis.wright.edu/library/download/HSK02-SEE.pdfhttp://knoesis.wright.edu/library/download/SBA+02-IC.pdf http://www.google.com/insidesearch/features/search/knowledge.html

Semantic Web: Early Realizations to Wide Adoption

Semantic categorization, association, and inference

Building knowledge base by entity extraction, use of this knowledgefor search, annotation, and personalization.

Semantic enrichment using domain knowledge

Semantic search and knowledge browsing

Creating a “Web of related Information”

Semantics & Semantic Web in 1999-2002

4

Now…12 years later

5

Google’s knowledge graph allows exploration of interconnected things

6

Semantic Web: 10 Years from Now

7

Physical-Cyber-Social Computing

Amit ShethLexisNexis Ohio Eminent Scholar,

Director, Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled ComputingWright State University, Dayton, OH, USA

Ack: Pramod Ananthram, Cory Henson, and Kno.e.sis Semantic Sensor Web team

Presented at SW2022 @ ISWC2012

8

Cyber-Physical Systems

http://www.nsf.gov/pubs/2013/nsf13502/nsf13502.htm

The computational and communication components closely interact with physical components. Enables better cyber-mediated observationof and interaction with physical components.

CPS involves sensing, computing, and actuating components

9

Physical Systems Cyber Systems

Remote heath monitoring

Mobile cardiac outpatient telemetry and real time analytics

Applications integrating telemedicine service

Remote real-time patient monitoring of vitals

Offers sensor solutions for monitoring cardiovascular ailments

Provides solutions to monitor cardio activity

Non-invasive wearable monitoring of vital signals

Cardiovascular patient monitoring

Continuous monitoring of physiological parameters

http://quantifiedself.com/2010/01/non-invasive-health-monitoring/

Physical-Cyber Systems

Health applications and tools that monitor a person physically and connect themto care providers (e.g. doctors)

10

Social SystemsCyber Systems

Cyber-Social Systems

Observations spanning Cyber and Social world – people share their activities, knowledge, experiences, opinions, and perceptions.

Sharing experiences and management of conditions and their treatments

Managing risks and making informed decisions based on gene sequencing

11

Social SystemsPhysical Systems Cyber Systems

Self knowledge through numbers

Involves interactions between all the three components.

Physical-Cyber-Social Systems

Sensors collecting observations form the physical world

http://www.fitbit.com/product/features#social

Social aspect of sharing and friendlycompetition

QS conference where experiences of analyzing and visualizing data from physiological sensors are presented.

This data is stove piped due to fragmentation in sensor data collection services.

Data collected from physiological sensors analyzed in the social context of “similar” people.

Needs significant human involvement in interpretation of physiological observations using their knowledge of the domain and social experiences.

Integration and interaction between physical, cyber, and social components for computation is brittle.

12

What if?

Computations leverages observations form sensors, knowledge and experiences from people to understand, correlate, and personalize solutions.

Physical-Cyber

Social-Cyber

Physical-Cyber-Social

13

Conventional search returns a set of documents for serving the information need expressed as a search query. Answer engine like WolframAlpha provides answers

for a query.

Chances of finding heartburn cases in a year along with their age group

Chances of finding heartburn cases based on ethnicity.

Distribution of heartburn cases based on age, weight, height and BMI

Analyzing this data with sensor observations collected for Mark, who is a white, 65 years of age, 190 pounds, 5’ 10’’ has relative low chances of having a heartburn

Search vs. Solution

14

Social Systems

Physical Systems Cyber Systems

Physical-Cyber-Social Computing

Semantics play a crucial role in bridging the semantic gap between different sensor types, modalities, and observations to derive insights leading to a holistic solution.

15http://www.yourdictionary.com/brain

Determines

Experience

Physical, Cyber and Social

Experiences

Perceptual Inference

Background Knowledge (spanning Physical-Cyber-Social)

Evolves

Influences

Influences

Observes

Influences

Physical-Cyber-Social Computing

We experience the world through perceptions and actions

Experiences evolve our background knowledge

Background knowledge + new observations will enhance our experience

16http://www.yourdictionary.com/brain

Determines

Experience

Experiences

Perceptual Inference

Background Knowledge (spanning Physical-Cyber-Social)

Evolves

Influences

Influences

Observes

Influences

Physical-Cyber-Social Computing

Diastolic blood pressure (BP) between 86 and 90

Asian male has lower thresholds for hypertension

2

3

How are my peers of the same socio-economic-cultural background doing w.r.t. BP?

5

Experiences in managing blood pressure

6

Increase the use of herbs and spices instead of salt

Asian male

1

Shared physiological observations from sensors

Corrective actions to be taken

4

7

17

Rich knowledge of the medical

domain

EMR and PHR

Physiological sensor data from

human population

Cyber

Physical

Current CPS is focused on , Sensing, processing, and actuating components along withcommunication, power consumption, security.

There are silos of knowledge on the cyber world which are under utilized.

Health related experiences shared by humans

Social

Social networks bridge the social interactions in the physical and cyber world.

Humans take decisions based on insights provided by other humans and machines in the physical world

Cyber world connects the physical world which consists of other machines or humans

Physical systems being instrumented with sensors, we have deeper view of physical world

Sensors around, on, and in humans will bridge the physical and cyber world.

Cyber-Physical Social Systems Landscape

18

Sensors around, on, and in humans will bridge the physical and cyber world.

Rich knowledge of the medical

domain

EMR and PHR

Physiological sensor data from

human population

Cyber

Physical

Current CPS is focused on , Sensing, processing, and actuating components along withcommunication, power consumption, and security.

There are silos of knowledge on the cyber world which are under utilized.

Health related experiences shared by humans

Social

Social networks bridge the social interactions in the physical and cyber world.

Mark’s discomfort sensed by: galvanic skin response, heart rate, fit bit, and Microsoft kinect

His phone computes the possible cause being increased intake of fried food.

Mark is experiencing heartburn.Mark advised to go low on fried food.Physical

Sensing

Actuating

Computing

Should Mark be convinced by this?What if fatty food intake was just a coincidence with something serious?

Cyber Physical Systems: Now

19

Sensors around, on, and in humans will bridge the physical and cyber world.

Cyber

Physical

We believe that current CPS should view the physical worldby incorporate solutions form (knowledge) cyber worldwith a lens of social context.

There are silos of knowledge on the cyber world which are under utilized.

Social

Social networks bridge the social interactions in the physical and cyber world.

Mark’s discomfort sensed by: galvanic skin response, heart rate, fit bit, and Microsoft kinect

Physical Cyber Social Computing involves: (1) Comparing physiological observations from people similar to him (age, weight, lifestyle,ethnicity, etc.) (2) Analyzing health experiences of similar people reporting heartburn (3) Incorporating history of ailments of Mark (4) Leveraging medical domain knowledge of diseases and symptoms.•He is advised to visit a doctor since he had a heart condition (from EMR) in the past and heartburns in similar people (social) was a symptom of arterial blockage

Mark is experiencing heartburn.

Alert to contact his doctor.

Physical

Sensing

Actuating

Computing

Rich knowledge of the medical

domain

EMR and PHR

Physiological sensor data from

human population

Health related experiences shared by humans

Physical Cyber Social Computing: A Vision

20

Toward the Vision: Semantic Perception in Khealth

http://knoesis.org/healthApp/

Paper at ISWC: C. Henson, K. Thirunarayan, A. Sheth, An Efficient Bit Vector Approach toSemantics-based Machine Perception in Resource-Constrained Devices

A Khealth application:leveraging low cost sensors toward reducing hospital readmissions for ADHF patients

Khealth will continue toward the vision of Physical-Cyber-Social Computing to understand, correlate, and personalize solutions

Convert large number of observations to semantic abstractions that provide insights and translate into decisions

Semantic Perception as an example component of PCS Computing

ObserveProperty

PerceiveFeature

Explanation

Discrimination

Translating low-level signals into high-level knowledge

Focusing attention on those aspects of the environment that provide useful information

Prior Knowledge

1

2

22

• CPS are stovepipe systems with narrow set of observations of the real world.

• Current CPS do not possess the knowledge support for decision making with Mark’s case.

• Social aspects plays an important role in decision making of CPS.

• The vision of Cyber Physical Social Computing is to provide solutions for these limitations.

CPS Current State of Art: Limitations

26

• Transition form search to solution engines for actionable information.

• Seamless integration of technology involving minimal human involvement.

• Transition form reactive systems (humans initiating information need) to proactive systems (machines initiating information need).

• Sharing of knowledge, experiences, and observations across physical-cyber-social worlds lead to informed decision making.

• Physical Cyber Social Computing plays an important role in this vision. Semantic computing will provide integration and reasoning capabilities needed for PCS computing.

Conclusions

27

Influential visions by Bush, Licklider, Eaglebert, and Weiser.

Amit Sheth,”Computing for Human Experience: Semantics-Empowered Sensors, Services, and Social Computing on the Ubiquitous Web," IEEE Internet Computing, vol. 14, no. 1, pp. 88-91, Jan.-Feb. 2010, doi:10.1109/MIC.2010.4

A. Sheth, Semantics empowered Cyber-Physical-Social Systems, Semantic Web in 2012 workshop at ISWC 2102.

Cory Henson, Amit Sheth, Krishnaprasad Thirunarayan, 'Semantic Perception: Converting Sensory Observations to Abstractions,' IEEE Internet Computing, vol. 16, no. 2, pp. 26-34, Mar./Apr. 2012, doi:10.1109/MIC.2012.20

Cory Henson, Krishnaprasad Thirunarayan, Amit Sheth. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web. Applied Ontology, vol. 6(4), pp.345-376, 2011.

Cory Henson, Krishnaprasad Thirunarayan, and Amit Sheth, 'An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices,' In: Proceedings of 11th International Semantic Web Conference (ISWC 2012), Boston, Massachusetts, USA, November 11-25, 2012.

A bit more on this topic