115
Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan G r o u p f o r User Interface Research University of California Berkeley

Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Embed Size (px)

Citation preview

Page 1: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Confab Tutorial

Jason I. HongChris BeckmannJeff HeerAlan Newberger

G r o u p f o rUser Interface Research

University of CaliforniaBerkeley

Page 2: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 2

Vision

“A hospital Mirror World has a software version of every patient, doctor, bed, room – and every abstract entity that’s important: cash in the bank, drugs on order, and so on. Through permanent sensors and ordinary terminal-based record-keeping, the Mirror World reflects the real one.”

– David Gelernter, Mirror Worlds

Page 3: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 3

Vision

• What if…– …we could create a world model that describes

salient aspects of the world in real-time?– …everyone could contribute to this world model

in the same way that everyone can contribute to the global World Wide Web?

• Such a World Model would greatly facilitate construction of context-aware apps– App developers wouldn’t have to hand craft each

model– Streamline sharing of context data– Uniform façade around diverse sensors and

software APIs

Page 4: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 4

Tutorial Outline

• Context Data Model• Programming Model• Liquid Distributed Querying• The Messy Details

Page 5: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 5

Context Data ModelHigh-Level Rationale

Page 6: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 6

Context Data ModelHigh-Level Rationale

Web(Global Scope)

(Context for people)

Web Service(Specialized API)(Global Scope)

(Context for computers)

Train Info Board(Local Scope)

(Context for people)

Unified API(Network effects)(Limited Scope)

(Context for computers)

Page 7: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 7

Context Data ModelHigh-Level Overview

BeaconWeb

Scraping

AutoDiary

HVAC

Context Data Layer

Context Data Layer

Sou

rces

Sin

ks

Sp

aces

Sensor

PublicDisplay

ManualInput

ContextBrowser

Page 8: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 8

Context Data Model

Alice’s InfoSpaceAlice’s InfoSpace

Loc Activity

Health

Room 525’s InfoSpace

Room 525’s InfoSpacePerso

n

Person

Device

PDA-1138’s InfoSpace

PDA-1138’s InfoSpace

Owner

Page 9: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 9

Context Data ModelDivision of Responsibilities

InfoSpace Server

InfoSpace

Tuple

Analogous to web serversManages a collection of InfoSpacesUnit of administrationUnit of deployment

Analogous to a web site / homepageRepresents context data about an entityRepresents zone of protectionManages collection of context tuplesUnit of ownership and addressing

Analogous to individual web pageRepresents single piece of context dataContains privacy preferences and metadataUnit of storage

Page 10: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 10

Context Data ModelInfoSpaces

• Distributed world model– Each with partial and incomplete knowledge of

world– Each with a different perspective of the world

• Represents three different things– Context data about an entity

• “My name is John”, “I am hungry”

– Context data perceived by that entity• “I am with Alice”, “I am in room 525”

– Context data queried by that entity• “Carol tells me that the dog is in the kitchen”

• Managed by individual represented or by admin– Like a homepage

Page 11: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 11

Context Data ModelInfoSpaces

• TupleSpace meets Web• TupleSpace

– A shared data space– add(), remove(), query(), subscribe(), unsubscribe()– Complexity shifted into data model and query

language

• Web– Leverages existing technology (ex. firewalls)– Leverages well-understood models for administration,

deployment, authoring, and programming– End-user mental model– Independent deployment & anarchic scalability

[Fielding]

Page 12: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 12

Example ModelInfoSpaces

Alice’s InfoSpace

Alice’s InfoSpace

Tuples

Out-Log

In-Log

Out-Subscriptions

In-Subscriptions

Properties

Policies

Page 13: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 13

Tuples

• Represent a discrete piece of context data

• Contains:– Context data– Metadata– History of that data– Privacy information

Page 14: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 14

Context Data ModelTuples

Static Dynamic

Intr

insic

NameHeight

Extr

insic

Room 525 part of Soda

Room TemperatureHospital Bed Empty

Alice is in Room 525People in Room 606

Page 15: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 15

Querying

• XPath is a language for addressing parts of an XML document– Think of an XML document as a tree-structure

• http://12.233.57.65:8080/infospace/jasonh– ?q=//ContextTuple[@datatype='location.room']– &sortby=//ContextTuple/@timestamp-created– &sortorder=descending– &num=2

• Snapshot– Get current state

Page 16: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 16

QueryingXPath Explorer

Page 17: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 17

Operators

• Operators– Small components for transforming data– Extensibility without having to modify the main

code

• In-operators– Check Privacy Tag

• Out-operators– Add Privacy Tag– Clear Sources– Sort

• On-operators– Garbage Collection– Periodic Report

Page 18: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 18

OperatorsExample Data-Flow

Alice’s InfoSpace

Alice’s InfoSpace

In Operators Out Operators

TupleTupleTuple

HTTPFront-end

Page 19: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 19

Sink-SideOverview

Confab Client

Alice’s InfoSpace

Alice’s InfoSpace

Bob’s InfoSpace

Bob’s InfoSpace

Room 22 InfoSpace

Room 22 InfoSpace

InfoSpaceServer

HTTPFront-end

ActiveProperties

Listeners

Page 20: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 20

Sink-SideConfabClient

• ConfabClient– Java client-side API for accessing InfoSpaces– add(), remove(), query(), subscribe(),

unsubscribe()

• ActiveProperties

“lederer.location”

“lederer.activity”

“lederer.temp”

OnDemandQuery

PeriodicQuery

Subscription

“606”

“Napping”

“98.6”

Page 21: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 21

Source-SideSimulator

Page 22: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 22

Putting it all togetherIn / Out Board for Room 410

• Approach 1 – Query each individual

• Approach 2 – Query the room

“lederer.loc”

“klemmer.loc”

“mattkam.loc”

PeriodicQuery

PeriodicQuery

PeriodicQuery

… …

“room410.occupants”PeriodicQuery

Page 23: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 23

The Messy DetailsDownload these packages

• JDK 1.4• Tomcat 4.1.18 Web Server

– http://jakarta.apache.org/builds/jakarta-tomcat-4.0/release/v4.1.18/bin/

• CVS – we like TortoiseCVS, http://www.tortoisecvs.org/

• Ant 1.5.1 Build System– http://ant.apache.org/

• Pageant and Putty Public-Key– http://www.chiark.greenend.org.uk/~sgtatham/putty/

download.html

Page 24: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 24

The Messy DetailsSourceForge

• SourceForge Open Source Repository– Create account at http://sourceforge.net/– Create and upload your public-key– Join Confab dev-team if you want to CVS

commit• http://sourceforge.net/projects/confab/• No team t-shirts yet

– CVS checkout latest snapshot

Page 25: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 25

Page 26: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 26

Privacy

• Layer perspective– Each layer responsible for security and privacy

between layers

• Dataflow perspective– Tuples contain data about usage– Digital rights management

Page 27: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 27

Related Work – Semantic Web / DAML

• Semantic Web has no story for– Individuals managing their data– Handling sensor data and dynamic updates– Where specific pieces of data live

• Confab is simpler– Complexity of Semantic Web is huge barrier to

entry– Start simple

Page 28: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 28

Related Work – Context Toolkit

• Focus first on the data model rather than sensors

• Early mapping of sensor to ontology• Per sensor managment

Page 29: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 29

Related Work – ParcTab System

• Confab is an evolution of ParcTab system

Page 30: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 30

Related Work – EventHeap / iRoom

Page 31: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 31

Related Work –

Page 32: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 32

Related Work –

Page 33: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 33

Motivation

• Modern computers divorced from our reality– Unaware of who, where, and what around them– Mismatch between our expectations and

functionality– Also limits what we can do with computers

• Computers have extremely limited input– Aware of explicit input only– A lot of effort to do simple things (or to

remember)

• Context-Aware Computing– One line of ubiquitous computing research– Making computers more aware of the physical

and social situations they are embedded in

Page 34: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 34

Examples of Using Context

Context TypesExisting Examples Human Concern

Room ActivityAuto Lights On / Off Convenience

Personal Identity & Time

File Systems Finding Info

TimeCalendar Reminders Memory

Activity Finding Info

Safety

Time

Location

Activity

Health Alert

Tag PhotosHistory

Identity

ProximityEfficiency

Service FleetDispatching

Context Types Potential Examples Human Concern

Page 35: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 35

Technology Trends

• Sensors– GPS, Active Badges, Active Bats– Smart Dust– Cameras and microphones

• Recognition algorithms– MSR Radar location from 802.11– Smart Floor footstep force

• Wireless technologies– Bluetooth, 802.11, cell phone

Page 36: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 36

A New Class of Context-Aware Apps

Active Badge(Olivetti)

ParcTabs(Xerox PARC)

Cyberguide(Abowd et al)

Page 37: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 37

A Computational View of Context

• Context as a strategy for building apps• Increasing the number of input channels

into the computer– Pushing towards implicit acquisition of data

• Creating better models– Pushing towards the physical and social

• Using the input and models in useful ways– Proactively taking predictable and meaningful

actions– Tagging other information for future lookup– Passing on more information to people

Page 38: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 38

A Computational View of Context

Au

ton

om

y

Sensing

Lights�

FileSystem� Calendar�

Tag Photos�

Health�

Realtime �Dispatching• Pervasiveness• Inference• Fusion• Models

Page 39: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 39

Two Problems with Context-Awareness

• Scalability– Lots of people, places, things, and sensors– Over long periods of time– Over large geographic distances– Sharing resources (sensors and data)

• Privacy– Tremendous source of valid criticism– Need architecture and mechanisms to

safeguard personal data and make it easy for people to manage

Page 40: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 40

Research Goals and Solution Overview

• Provide network-oriented set of abstractions, mechanisms, and programming model

• Scalability– Data-oriented P2P repositories called

information spaces– Different infospaces federate when needed

• Privacy– Provide suite of mechanisms for app developers– Based on Fair Information Practices and

Information Asymmetry

Page 41: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 41

Talk Overview

MotivationResearch OverviewConfab Architecture – ScalabilityConfab Architecture – PrivacyStatusContext + Whisper thoughtsContext + SpeakEasy thoughts

Page 42: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 42

Architectural Abstractions

• Information Spaces– P2P TupleSpace repositories of context data and

operators– Associated with entities (people, places, things)– Somewhat similar to web servers and home pages

• Context Data– Representation for context data

• Operators– Reusable and composable code operating on data

• Context Queries / Notifications– Simple API for accessing context

Page 43: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 43

Architectural SketchInformation Spaces

Carol's InfoSpace(Desktop)

Information Spaces

Carol's InfoSpace

(PDA)

Soda 525 InfoSpace(Server)

Page 44: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 44

Architectural SketchContext Data

Loc Act ActLoc

Context Data Information Spaces

Page 45: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 45

Architectural SketchOperators

Context Data Information Spaces

TransFilter Log

Operators

Page 46: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 46

Architectural SketchContext Queries

Loc

Context Data Information Spaces

TransFilter

Operators

Query

LocTrans

Page 47: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 47

Architectural SketchContext Notifications

Context Data Information Spaces

Operators

Notification(Standing Query)

Page 48: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 48

Architectural SketchPeering of Information Spaces

Carol's Context when Mobile

Carol's Context in Room 525

Context = Set of Available Info Spaces

Page 49: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 49

Emergency Response Scenario

• Part of a suite of context-aware apps under development for fire or earthquake situations

• Keep track of people in a building– Allow building managers to check if a building

is clear in the event of an evacuation– Allow firefighters to check where people were

• Provide reasonable privacy protection– People don't like to be tracked– Emergency situations relatively rare

Page 50: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 50

Emergency Response Scenario

Registering with the Building's InfoSpace

Building InfoSpace

Carol'sInfoSpace

Smart Dust

User="Carol"Location="525 Soda Hall"Time="Apr 12 1:05PM"

Access Control

User="Carol"Location="5th floor"Age="37 seconds"

Send location info

Logging

User="Carol"Location="in"Age="37 seconds"Blurring

Page 51: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 51

Emergency Response Scenario

Querying during an Emergency

Building InfoSpace

Carol'sInfoSpace

Smart Dust

User="Carol"Location="5th floor"Age="37 seconds"

User="Carol"Location="525 Soda Hall"Time="Apr 12 1:05PM"

Notification Logging

User="Carol"Location="525 Soda"Age="7 seconds"

Page 52: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 52

Layers of InfoSpaces and Context Data

Physical

Logical

View

My Location on PDA

My Location on PC

My Location

My Locationto Strangers

My Locationto Friends

My Locationto Family

Page 53: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 53

Scalability Recap

• Architecture analogous to web– Information spaces are like web servers– Information spaces contain context data– Context data is eventually consistent (helps

availability)

• Differences from web architecture– Each device contains an information space (so

devices can access context even w/o net access)

– Information spaces contain reusable operators for manipulating and protecting context data

Page 54: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 54

Talk Overview

MotivationResearch OverviewConfab Architecture – ScalabilityConfab Architecture – PrivacyStatusContext + Whisper thoughtsContext + SpeakEasy thoughts

Page 55: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 55

Privacy PhilosophyFair Information Practices

• Notice• Choice• Onward Transfer• Access• Security• Data Integrity• Enforcement

Page 56: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 56

Privacy PhilosophyInformation Asymmetry

“In all of human history, no government has ever known more about its people than our government knows about us. And in all of human history, no people have ever been anywhere near as free.” (Brin)

Page 57: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 57

Some Desired Privacy Features

• Intentional ambiguity– "Where is Victoria?" "Chez Panisse" -> "Berkeley" ->

"CA"– Give different answers depending on requestor

• Plausible deniability– "Is Adam busy?" "Yes" or "Unknown" according to

prefs

• Risk Avoidance– "Mark does not trust this person / infospace"

• Tracking– Who has my data? What are they doing with it?– (Also a reverse-privacy issue?)

Page 58: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 58

A Privacy Design Space

• Legal• Social• Economic• Technology

Det

ect

ion

Avo

ida

nce

Pre

ven

tion

Collection Second UseAccess

Th

emes

fo

r M

inim

izin

g A

sym

met

ry

Data Lifecycle

AnonymizationPseudonymization

P3P

RBAC

LocationSupport

Privacy Mirrors

Wearables

User Interfaces for Feedback, Notification, and Consent

Goal: Provide reusable mechanisms that can populate this design space

Page 59: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 59

Privacy Trust Model

• Optimistic– I trust you and your current infospace– Make it easy for others to do "the right thing"

[tm]

• Pessimistic– I don't trust you or your current infospace– Modify the data assuming you will do "the

wrong thing" (more blurring or watermarking)– Or don't send the data to you at all

• Make it easy to support spectrum of trust models between full optimistic and full pessimistic

Page 60: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 60

Two Privacy Mechanisms

• Operators

• Privacy Tags– Preferences for how personal data should be

used– "Don't forward to anyone else" – "Don't fuse with other pieces of data"

Garbage collection Remove or aggregate old data

Blurring Increase ambiguity

Access Control Check authorization

Logging Detection

Filters Remove certain data

Page 61: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 61

Talk Overview

MotivationResearch OverviewConfab Architecture – ScalabilityConfab Architecture – PrivacyStatusContext + Whisper thoughtsContext + SpeakEasy thoughts

Page 62: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 62

Status

• Still in early-to-mid phases– Currently developing initial implementation– JDK, JXTA (Java P2P), XML– Possibly also WSDL, SOAP

• Target applications– SpeakEasy (PARC)– Suite of Emergency Response apps– Possible Educational Technology apps

• "Metrics"– Types of and effectiveness of apps that can be

built– Ease of adoption– Robustness

Page 63: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 63

The Ultimate Metric

Page 64: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 64

Some Context + Whisper Thoughts

• Use location + activity to help determine level of security– Within "safe" boundaries use low security– Within "unsafe" boundaries switch to high

security, provide more feedback, and avoid risky situations (talking to strange computers)

• Boundaries can be based on:– People nearby (Social)– Activity– Location (Physical)

• Use contextual information from sensors and other sources to help determine these boundaries

Page 65: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 65

Some Context + SpeakEasy Thoughts

• Useful context for components– History of usage / Inferred patterns of usage– Location of component

• Useful context for people– Location of person– Personal history of usage / Inferred patterns– Shared history of usage (how others have

used)– Activity

• ie "It looks like you're doing a presentation"• Make it easy, or automate some things• How well can you guess activity from simple data?• How well can you do it over time?

Page 66: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 66

Q & A

Focus, Jason, focus!

Jen MankoffAsst Prof Berkeley

Privacy is good here, but be careful not to fall into the systems tarpit.

Bill SchilitIntel Labs Seattle

Co-director

Page 67: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 67

Q & A

Agree with Bill do I, beware the dark side of systems you must!

YodaJedi Master

Kickass Dude

Page 68: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 68

Q & AGood work, Jason, I think you deserve a raise!

James LandayAssoc Prof Berkeley

My Advisor

This party's started!

Mace WinduJedi Master

Also a Kickass Dude

Page 69: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Jason I. Honghttp://guir.berkeley.edu/cfabric

G r o u p f o rUser Interface Research

University of CaliforniaBerkeley

Thanks to:DARPA ExpeditionsPARCIntel FellowshipNSF ITRYoda

Contextthe circumstances in which an event occurs; a setting; to join; to weave

Page 70: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 70

Q & AMaybe privacy won't be a large issue in the future. Very difficult to say because of the tradeoffs in value, safety, convenience.

One way of evaluating is to describe the design space, and show how your work makes it easy to build in that space.

Page 71: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 71

Q & ABut do we really need ubicomp at all? And if so, how do we build and evaluate it so that it's socially relevant and meaningful?

Maybe context itself isn't really the issue, because activity orders and delineates what is and isn't relevant at any point.

Page 72: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 72

Functional Requirements

• Context Acquisition– Getting the data from a variety of sources

• Context Modeling– Representing the data

• Context Storage and Dissemination– Storing the data– Making the data available when it is needed

• Context Usage– Using the data in a program

Page 73: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 73

Context Data

• Problem: how to represent context data?• Entities

– Like nouns, people, places, and things

• Attributes– Like adjectives or properties, key-value pairs

• Relationships– How one entity relates to another entity

• Aggregates– Actions, Groups of people

Page 74: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 74

Context Data

Person="[email protected]"

Name="Location"Value="Room 525"Schema="Building:Room"Metadata=

Time="1023498143"Time-to-Live="60sec"Source="SmartDust"

Name="Device"Value=http://zzz.comSchema="Device"

En

tity

Attrib

ute

Rela

tion

ship

Page 75: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 75

Key Architectural Abstractions

• Information Spaces– Repositories of context data and operators

• Context Data– Representation for context data

• Operators– Composable code operating on context data

• Context Queries / Notifications– Simple query language (like SQL for DB)– Push / Pull semantics

Page 76: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 76

Information Spaces

• Problem: where to store context data?• Information Spaces analogous to web

servers– Have a unique name– Have an owner– Contain multiple (and not necessarily

related) pieces of data– Can get / put pieces of data (given security

and privacy prefs)

Page 77: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 77

Operators

• Problem: how to manipulate context data in a reusable manner?

• Chainable OperatorsData-type Conversion Ex. Celsius -> Farenheit

Fusion Refine same data type

Composition Merge different data types

Garbage collection Remove or aggregate old data

Blurring Increase ambiguity

Access Control Check authorization

Logging Detection

Filters Remove certain data

Page 78: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 78

Context Queries

• Problem: how to use context data?

Page 79: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 79

Related Work

• Context Toolkit• EventHeap• ParcTab infrastructure

Page 80: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 80

Existing Examples of Using Context

Context TypesExisting Examples Human Concern

Room ActivitySmoke Alarm Safety

Room ActivityAuto Lights On / Off Convenience

Object IdentityBarcode Scanners Efficiency

Personal Identity & Time

File Systems Finding Info

TimeCalendar Reminders Memory

Page 81: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 81

Potential Examples of Using Context

Existing Examples Context Types Potential Examples Human Concern

Activity Convenience

Activity Finding Info

Identity Memory

Identity & Time Safety

Time Efficiency

Identity

Time

Location

Proximity

Activity

History

Health Alert

Auto Cell Phone Off In Meetings

Service FleetDispatching

Tag Photos

Proximal Reminders

Page 82: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 82

Defining ContextAbowd & Dey / Moran & Dourish

• "Any information that can be used to characterize the situation of an entity, where an entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and the application themselves. Context is typically the location, identity, and state of people, groups, and computational and physical objects." (Abowd and Dey)

• "Context refers to the physical and social situation in which computational devices are embedded" (Moran and Dourish)

Page 83: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 83

Defining ContextDistributed Cognition

• Distributed cognition– Need to go beyond physical attributes (ex.

temp)– Look at “state of digital resources, people’s

concepts, task state, social relations, local work culture” (Kirsh)

– Model key attributes and deep structure of whole system (individuals, offices, social structs, work practices)

• Problems– What are the key attributes?– How to represent?

Page 84: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 84

Defining ContextSituated Action

• Situated action– Actions are fluid, moment-by-moment,

improvised, often unplanned, and highly context-dependent

– “[T]he context in which actions take place is what allows people to find it meaningful” (Dourish)

• Problems– Very high-level form of context– Can low-level computer-based context be

useful?– Also, how does this really help us build systems?

Page 85: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 85

Defining ContextPhenomenology

• Phenomenology– Reality consists of objects and events as they are

perceived in human consciousness and not of anything independent of human consciousness.

– Meaning (and hence context) arises from the ways in which we engage with and act within the world

• Problems– Need this level of sophistication to make progress?– How does this help us build systems? – Very wide chasm between philosophy and practice

Page 86: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 86

Defining ContextMy Perspective

• Point #1 – Not clear if we need a solid definition– Operating systems and Artificial Intelligence

• Point #2 – Let's treat it like "information"– Shannon treated it from a mechanical perspective

(i.e. transmission) made great inroads – We are still debating the meaning of "information"– But now we can do it electronically

• Let's treat context from computer perspective– Let designers define context app-by-app– Provide generic reusable mechanisms (like DB)

Page 87: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 87

Privacy

• Privacy is a relatively new concept in society, and is “ultimately a psychological construct, with malleable ties to specific objective conditions” (Grudin)– Convenience, Safety, Efficiency– Ex. Credit cards and cell phones

• Open access to online calendars for efficiency and awareness (Palen)

Page 88: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 88

Designing Context-Aware Systems

• Minimize automatic actions– Probably cost-to-benefit via decision theory (value, error,

correctness)

• Provide feedback– What is being captured?– Why did the system do that?

• Feed-forward– If you do that, then the system will do this

• Confirmation– The system just did the following action

• Endpoint– Context for people or context for computers?

Page 89: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 89

Vision

Context-Aware Computing Today

Page 90: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 90

Vision

Context-Aware Computing in the Future

January

Page 91: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 91

Example ModelOrganizing

Data Sources(Personal, Group, Public)

January

Page 92: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 92

Example ModelOrganizing

End-User Devices, ServicesAnd Applications

Sensors and Beacons(Mobile and Infrastructure)

Data Sources(Personal, Group, Public)

???

Page 93: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 93

Example ModelOrganizing

Context Data Model Layer

Sources

Sinks

Page 94: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 94

Example ModelOrganizing

Confab Client

Alice’s InfoSpace

Alice’s InfoSpace

Bob’s InfoSpace

Bob’s InfoSpace

Room 22 InfoSpace

Room 22 InfoSpace

InfoSpaceServer

HTTPFront-end

ActiveProperties

Page 95: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 95

Example ModelActive Properties

“scott.location”

“scott.activity”

“current-device.room”

Page 96: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 96

Example ModelDivision of Responsibilities

InfoSpace Server

InfoSpace

Tuple

Analogous to web serversManages a collection of InfoSpacesUnit of administration and deploymentUnit of deployment

Analogous to a web site / homepageRepresents context data about an entityRepresents zone of protectionManages collection of context tuplesUnit of ownership and addressing

Analogous to individual web pageRepresents single piece of context dataContains privacy preferences and metadataUnit of storage

Page 97: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 97

Example ModelEvolution of Context-Aware Systems

Page 98: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 98

Example ModelA Predicted Evolution of Context-Aware Systems

Web(Global Scope)

(Context for people)

Web Service(Global Scope)

(Context for computers)

Train Info Board(Local Scope)

(Context for people)

Unified API(Global Scope)

(Context for computers)(Network effects)

Restricted Scope(Global Scope)

(Context for computers)(Network effects)

Page 99: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 99

Example ModelA Predicted Evolution of Context-Aware Systems

Train Info BoardSensor InputManual Input

Page 100: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 100

Example ModelPhysical, Logical, and View

Sinks

DataStores

Sources

Context-AwareApplications

Context Data

Sensors, Beacons,Databases, Web pages

Page 101: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 101

Example ModelIntrinsic and Extrinsic Context

Loc

Page 102: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 102Berkeley CS InfoSpace Server

Example ModelSingle InfoSpace Server

Alice’s InfoSpace

Alice’s InfoSpace

Room 525’s InfoSpace

Room 525’s InfoSpace

HTTPFront-end In-Operators

Out-Operators

On-Operators

Tuple TupleTupleTuple Tuple

InfoSpaceAccess

Page 103: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 103

Example ModelContext Data Model

Alice’s InfoSpaceAlice’s InfoSpace

Room 525’s InfoSpace

Room 525’s InfoSpace

Loc Activity

PDA-1138’s InfoSpace

PDA-1138’s InfoSpace

Person

Person

Device

Owner

Health

Page 104: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 104

Example ModelInfoSpaces

Alice’s InfoSpace

Alice’s InfoSpace

MotionGPS

Health MonitorService

Heartbeat

Personal LocTriggers

AutoDiary

CalorieTracker

Page 105: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 105

Example ModelInfoSpaces

Alice’s InfoSpaceAlice’s InfoSpace

Room 525’s InfoSpace

Room 525’s InfoSpace

Loc Activity

PDA-1138’s InfoSpace

PDA-1138’s InfoSpace

Person

Person

Device

Owner

Health

Page 106: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 106

Example ModelAdding Data

ActiveBadge

WirelessTriangulation

Alice’sLaptop’s InfoSpace

Alice’sLaptop’s InfoSpace

Loc.Tri

Alice’sInfoSpace

Alice’sInfoSpace

Loc.Active

-Badg

eLoc.Tr

i

SUBSCRIBE Loc.*

SUBSCRIBE Loc.*

SUBSCRIBE Loc

Page 107: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 107

Example ModelAdding Data

ActiveBadge

WirelessTriangulation

Alice’sLaptop’s InfoSpace

Alice’sLaptop’s InfoSpace

Loc.Active

-Badg

e

POST

Loc.Tri

POSTLoc.Tr

i

Alice’sInfoSpace

Alice’sInfoSpace

Loc.Active

-Badg

eLoc.Tr

i

Loc.Tri

POST

Page 108: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 108

Example ModelTransforming Data

Acquaintance’sInfoSpace

Acquaintance’sInfoSpace

Type=“Location”User=“Alice”Location=“Berkeley, CA”Time=“Oct 06 1:05 PM”Notify=“[email protected]”Pref=“Do not forward”Time-to-Live=“1 week”

Type=“Location”User=“Alice”Loc=“525 Wozniak Hall”Time=“Oct 06 1:05 PM”Time-to-live=“Forever”

Building’s InfoSpace

Alice’s InfoSpace

Type=“Location”User=“xyzzy”Loc=“5th floor Wozniak Hall”Time=“Oct 06 1:05 PM”Notify=“[email protected]”Pref=“Do not forward”Pref=“Emergency use only”Time-to-Live=“1 hour”

Page 109: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 109

Address

NameOccupied

Hospital

Phone#

Name

Patient

DoctorTemperature

RoomNumber

Room

Page 110: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 110

Example ModelPhysical, Logical, and ViewHospital'

s InfoSpace

Hospital's InfoSpace

Room 525's InfoSpace

Room 525's InfoSpace

Occupied

Temperature

RoomNumber

Room 527's InfoSpace

Room 527's InfoSpace

Occupied

Temperature

RoomNumber

Dr. X's InfoSpace

Dr. X's InfoSpace

Room

Phone#

Name

Room

Room

Doctor

Patient Y's InfoSpace

Patient Y's InfoSpace

Heart RateRoom

Address

Name

Patients

Page 111: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 111

<ContextTuple dataformat="http://badge.hospital.org/map.jsp" datatype="location.room" description="Represents location of an entity" entity-name="Doctor X" timestamp-created="2002.Dec.02 14:28:39 PST"> <Values> <Value value="525" /> </Values>

<Sources> <Source datatype="location.room" link="http://badge.hospital.org/loc.jsp" source="Active Badge" timestamp="2002.Dec.02 14:28:39 PST" out-tid="3F8B4528" in-tid="22A4610F" value="525" /> </Sources>

<PrivacyTags> <Lifetime value="500" /> <Copy value="no" /> <Fuse value="no" /> </PrivacyTags></ContextTuple>

Page 112: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 112

Example ModelInfoSpaces and Tuples

Room 525's InfoSpace

Room 525's InfoSpace

Occupied

Temperature

RoomNumber

525

527

535

537

Dr. X's InfoSpace

Dr. X's InfoSpace Activity

Room

Phone#

Name

PowerMonitor

PersonalDiary

Con

text

Sou

rces

Con

text

Sin

ks

Con

text

Data

Mod

el

Dr. X

Page 113: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 113

D

B

C

A

Page 114: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 114

A CF

Access SecondUse

Alice'sLocation

Alice'sLocation

Page 115: Confab Tutorial Jason I. Hong Chris Beckmann Jeff Heer Alan Newberger G r o u p f o r User Interface Research University of California Berkeley

Feb 18 2003 115

BA