April 13, 2006Jiao Tong University, Shanghai1 Advanced Multimedia and Presence Services using Classical…

April 13, 2006 Jiao Tong University, Shanghai 1

Advanced Multimedia and Advanced Multimedia and Presence Services using Presence Services using Classical and P2P SIPClassical and P2P SIP

Henning Schulzrinne(with Kundan Singh, Ron Shacham, Xiaotao Wu, Jonathan

Lennox and others)Department of Computer Science

Columbia [email protected]


OverviewOverview Quick overview of SIP Ring-and-hope presence-mediated

communications Uses for presence:

Old: “I’m on-line” Location-based services Presence-derived call handling Presence-derived trust

Presence and privacy Service and session mobility Application sharing


Philosophy transitionPhilosophy transitionOne computer,

many users

One computer,one user

Many computers,one user

anywhere,any time

any media

right place (device),right time,right media

~ ubiquitous computing

mainframe era


Evolution of VoIPEvolution of VoIP

“amazing – thephone rings”

“does it docall transfer?”

“how can I make itstop ringing?”

1996-2000 2000-2003 2004-

catching upwith the digital PBX

long-distance calling,ca. 1930 going beyond

the black phone


Collaboration in transitionCollaboration in transition

intra-organization;

small number of systems

(meeting rooms)

inter-organization

multiple technology generationsdiverse end

points

proprietary (single-vendor)

systems

standards-based solutions


SIP OverviewSIP Overview


Internet services – the Internet services – the missing entrymissing entry

Service/delivery

synchronous asynchronous

push instant messagingpresenceevent notificationsession setupmedia-on-demand

messaging

pull data retrievalfile downloadremote procedure call

peer-to-peer file sharing


Filling in the protocol gapFilling in the protocol gap

Service/delivery

synchronous asynchronous

push SIPRTSP, RTP

SMTP

pull HTTPftpSunRPC, Corba, SOAP

(not yet standardized)


SIP as service enablerSIP as service enabler Rendezvous protocol

lets users find each other by only knowing a permanent identifier

Mobility enabler: personal mobility

one person, multiple terminals

terminal mobility one terminal, multiple IP

addresses session mobility

one user, multiple terminals in sequence or in parallel

service mobility services move with user


What is SIP?What is SIP? Session Initiation Protocol protocol that

establishes, manages (multimedia) sessions also used for IM, presence & event notification uses SDP to describe multimedia sessions

Developed at Columbia U. (with others) Standardized by

IETF (RFC 3261-3265 et al) 3GPP (for 3G wireless) PacketCable

About 100 companies produce SIP products Microsoft’s Windows Messenger (≥4.7)

includes SIP


PhilosophyPhilosophy Session establishment & event notification Any session type, from audio to circuit

emulation Provides application-layer anycast service Provides terminal and session mobility Based on HTTP in syntax, but different in

protocol operation Peer-to-peer system, with optional support by

proxies even stateful proxies only keep transaction

state, not call (session, dialogue) state transaction: single request + retransmissions proxies can be completely stateless


Basic SIP message flowBasic SIP message flow


SIP trapezoidSIP trapezoid

SIP trapezoid

outbound proxy

[email protected]: 128.59.16.1

registrar

1st request

2nd, 3rd, … request

voice trafficRTP

destination proxy(identified by SIP URI domain)


SIP message formatSIP message format

SDP

INVITE sip:[email protected] SIP/2.0

Via: SIP/2.0/UDP here.com:5060From: Alice <sip:[email protected]>To: Bob <sip:[email protected]>Call-ID: [email protected]: 1 INVITESubject: just testingContact: sip:[email protected]: application/sdpContent-Length: 147

v=0o=alice 2890844526 2890844526 IN IP4 here.coms=Session SDPc=IN IP4 100.101.102.103t=0 0m=audio 49172 RTP/AVP 0a=rtpmap:0 PCMU/8000

SIP/2.0 200 OK

Via: SIP/2.0/UDP here.com:5060From: Alice <sip:[email protected]>To: Bob <sip:[email protected]>Call-ID: [email protected]: 1 INVITESubject: just testingContact: sip:[email protected]: application/sdpContent-Length: 134

v=0o=bob 2890844527 2890844527 IN IP4 there.coms=Session SDPc=IN IP4 110.111.112.113t=0 0m=audio 3456 RTP/AVP 0a=rtpmap:0 PCMU/8000m

essa

ge b

ody

head

er fi

elds

requ

est l

ine

request response


PSTN vs. Internet TelephonyPSTN vs. Internet Telephony

Signaling & Media Signaling & Media

Signaling Signaling

Media

PSTN:

Internettelephony:

China

Belgian customer,currently visiting US

Australia


SIP addressingSIP addressing Users identified by SIP or tel URIs

sip:[email protected] tel: URIs describe E.164 number, not

dialed digits (RFC 2806bis) tel URIs SIP URIs by outbound proxy A person can have any number of SIP

URIs The same SIP URI can reach many

different phones, in different networks sequential & parallel forking

SIP URIs can be created dynamically: GRUUs conferences device identifiers

(sip:[email protected]) Registration binds SIP URIs (e.g.,

device addresses) to SIP “address-of-record” (AOR)

tel:110 sip:sos@domain

domain 128.59.16.17via NAPTR + SRV


3G Architecture 3G Architecture (Registration)(Registration)

visited IM domain

home IM domain

servingCSCF

interrogating

proxy

interrogating

mobility managementsignaling

registration signaling (SIP)_


SIP is PBX/Centrex readySIP is PBX/Centrex readycall waiting/multiple calls

RFC 3261

hold RFC 3264transfer RFC 3515/Replacesconference RFC 3261/callee

capsmessage waiting message summary

packagecall forward RFC 3261call park RFC 3515/Replacescall pickup Replacesdo not disturb RFC 3261call coverage RFC 3261

from Rohan Mahy’s VON Fall 2003 talk

simultaneous ringing

RFC 3261

basic shared lines

dialog/reg. package

barge-in Join“Take” ReplacesShared-line “privacy”

dialog package

divert to admin RFC 3261intercom URI conventionauto attendant RFC 3261/2833attendant console

dialog package

night service RFC 3261

cent

rex-

styl

e fe

atur

es

boss/admin features

attendant features


A constellation of SIP RFCsA constellation of SIP RFCs

Resource mgt. (3312)Reliable prov. (3262)INFO (2976)UPDATE (3311)Reason (3326)SIP (3261)

DNS for SIP (3263)Events (3265)REFER (3515)

DHCP (3361)DHCPv6 (3319)

Digest AKA (3310)Privacy (3323)P-Asserted (3325)Agreement (3329)Media auth. (3313)AES (3853)

Non-adjacent (3327)Symmetric resp. (3581)Service route (3608)User agent caps (3840)Caller prefs (3841)

ISUP (3204)sipfrag (3240)

Security & privacy

Configuration

Core

Mostly PSTN

Content types

Request routing


An eco system, not just a An eco system, not just a protocolprotocol

SIP

XCAP(config)

RTSP

SIMPLEpolicyRPID….

SDP

XCON(conferencing)

STUNTURNRTP

configures

initiates carries

carriescontrols provide addresses


SIP – a bi-cultural protocolSIP – a bi-cultural protocol

• overlap dialing• DTMF carriage• key systems• notion of lines• per-minute billing• early media• ISUP & BICC interoperation• trusted service providers

• multimedia• IM and presence• location-based service• user-created services• decentralized operation• everyone equally suspect


Context-aware Context-aware communicationcommunication

context = “the interrelated conditions in which something exists or occurs”

anything known about the participants in the (potential) communication relationship

both at caller and calleetime CPLcapabilities caller preferenceslocation location-based call routing

location eventsactivity/availability presencesensor data (mood, bio)

privacy issues similar to location data


GEOPRIV and SIMPLE GEOPRIV and SIMPLE architecturesarchitectures

target locationserver

locationrecipient

rulemaker

presentity

caller

presenceagent watcher

callee

GEOPRIV

SIPpresence

SIPcall

PUBLISHNOTIFY

SUBSCRIBE

INVITE

publicationinterface

notificationinterface

XCAP(rules)

INVITE

DHCP


The role of presenceThe role of presence Guess-and-ring

high probability of failure: “telephone tag” inappropriate time (call

during meeting) inappropriate media (audio

in public place) current solutions:

voice mail tedious, doesn’t scale, hard to search and catalogue, no indication of when call might be returned

automated call back rarely used, too inflexible

most successful calls are now scheduled by email

Presence-based facilitates unscheduled

communications provide recipient-specific

information only contact in real-time if

destination is willing and able

appropriately use synchronous vs. asynchronous communication

guide media use (text vs. audio)

predict availability in the near future (timed presence)

Prediction: almost all (professional) communication will be presence-initiated or

pre-scheduled


Basic presenceBasic presence Role of presence

initially: “can I send an instant message and expect a response?”

now: “should I use voice or IM? is my call going to interrupt a meeting? is the callee awake?”

Yahoo, MSN, Skype presence services: on-line & off-line

useful in modem days – but many people are (technically) on-line 24x7

thus, need to provide more context + simple status (“not at my desk”)

entered manually rarely correct does not provide enough context for directing

interactive communications


Presence data architecturePresence data architecture

rawpresencedocument

createview

(compose)privacyfiltering

draft-ietf-simple-presence-data-model

compositionpolicy

privacypolicy

presence sources

XCAP XCAP

(not defined yet)

depends on watcherselect best sourceresolve contradictions

PUBLISH


Presence data architecturePresence data architecture

candidatepresencedocument

watcherfilter

rawpresencedocument

post-processingcomposition(merging)

finalpresencedocument

differenceto previous notification

SUBSCRIBE

NOTIFY

remove data not of interest

watcher


Presence data modelPresence data model

“calendar” “cell” “manual”

[email protected], video, text

[email protected]

person(presentity)

(views)

services

devices


Rich presenceRich presence More information automatically derived from

sensors: physical presence, movement electronic activity: calendars

Rich information: multiple contacts per presentity

device (cell, PDA, phone, …) service (“audio”)

activities, current and planned surroundings (noise, privacy, vehicle, …) contact information composing (typing, recording audio/video IM, …)


RPID: rich presenceRPID: rich presence<person>

<tuple>

<device>

<activities><class><mood><place-is><place-type><privacy><relationship><service-class><sphere><status-icon><time-offset><user-input>


RPID = rich presenceRPID = rich presence Provide watchers with better information about

the what, where, how of presentities facilitate appropriate communications:

“wait until end of meeting” “use text messaging instead of phone call” “make quick call before flight takes off”

designed to be derivable from calendar information or provided by sensors in the environment

allow filtering by “sphere” – the parts of our life don’t show recreation details to colleagues


CIPID: Contact InformationCIPID: Contact Information More long-term identification of

contacts Elements:

card – contact Information home page icon – to represent user map – pointer to map for user sound – presentity is available


The role of presence for call The role of presence for call routingrouting Two modes:

watcher uses presence information to select suitable contacts

advisory – caller may not adhere to suggestions and still call when you’re in a meeting

user call routing policy informed by presence

likely less flexible – machine intelligence

“if activities indicate meeting, route to tuple indicating assistant”

“try most-recently-active contact first” (seq. forking)

LESS

translateRPID

CPL

PA

PUBLISH

NOTIFY

INVITE


Presence and privacyPresence and privacy All presence data,

particularly location, is highly sensitive

Basic location object (PIDF-LO) describes

distribution (binary) retention duration

Policy rules for more detailed access control

who can subscribe to my presence

who can see what when

<tuple id="sg89ae"> <status> <gp:geopriv> <gp:location-info> <gml:location> <gml:Point gml:id="point1“

srsName="epsg:4326"> <gml:coordinates>37:46:30N 122:25:10W

</gml:coordinates> </gml:Point> </gml:location> </gp:location-info> <gp:usage-rules> <gp:retransmission-allowed>no

</gp:retransmission-allowed> <gp:retention-expiry>2003-06-23T04:57:29Z

</gp:retention-expiry> </gp:usage-rules> </gp:geopriv> </status> <timestamp>2003-06-22T20:57:29Z</timestamp></tuple>


Privacy policy Privacy policy relationshipsrelationships

geopriv-specific presence-specific

common policy

RPID CIPID

future


Privacy rulesPrivacy rules Conditions

identity, sphere time of day current location identity as <uri> or

<domain> + <except>

Actions watcher confirmation

Transformations include information reduced accuracy

User gets maximum of permissions across all matching rules privacy-safe

composition: removal of a rule can only reduce privileges

Extendable to new presence data rich presence biological sensors mood sensors


Example rules documentExample rules document

<identity><id>[email protected]</id></identity>

<sub-handling>allow</sub-handling>

<provide-services> <service-uri-scheme>sip</service-uri-scheme> <service-uri-scheme>mailto</service-uri-scheme></provide-services><provide-person>true</provide-person><provide-activities>true</provide-activities><provide-user-input>bare</provide-user-input>

<rul

eset

>

<rule id=1>

<con

ditio

ns>

<tra

nsfo

rmat

ion

s><a

ctio

ns>


Creating and manipulating Creating and manipulating rulesrules Uploaded in whole or part via XCAP XML not user-visible Web or application UI, similar to

mail filtering Can also be location-dependent

“if at home, colleagues don’t get presence information”

Possibly implementation-defined “privacy levels”


Location-based servicesLocation-based services Finding services based on location

physical services (stores, restaurants, ATMs, …) electronic services (media I/O, printer, display,

…) not covered here

Using location to improve (network) services communication

incoming communications changes based on where I am configuration

devices in room adapt to their current users awareness

others are (selectively) made aware of my location security

proximity grants temporary access to local resources


Location-based SIP Location-based SIP servicesservices Location-aware inbound routing

do not forward call if time at callee location is [11 pm, 8 am]

only forward time-for-lunch if destination is on campus do not ring phone if I’m in a theater

outbound call routing contact nearest emergency call center send [email protected] to nearest branch

location-based events subscribe to locations, not people Alice has entered the meeting room subscriber may be device in room our lab stereo

changes CDs for each person that enters the room

mailto:[email protected]


Location detectionLocation detection

SIP UAGPS

receiver

Bluetooth

DHCPserver

swipecard

activebadge

manually

SUBSCRIBE

NOTIFY

Locationserver

iButton

PUBLISH

WiFi


DHCP for locationsDHCP for locations modified dhcpd (ISC) to generate location information use MAC address backtracing to get location information

DHCPserver

458/17 Rm. 815458/18 Rm. 816

DHCP answer:sta=DC loc=Rm815lat=38.89868 long=77.03723

8:0:20:ab:d5:d

CDP + SNMP8:0:20:ab:d5:d 458/17


Location-based service Location-based service languagelanguage

false

trueNOTIFY

action alert

conditions

proximity

occupancy

time

IM

actions

alert

message

log

call

transfer

join

events

incoming

outgoing

notify

message

subscription


Program location-based Program location-based servicesservices



TrackingTracking


Internet2 WG PIC trialInternet2 WG PIC trial

PUBLISHpresence status

NOTIFYmyselfand others’ locations

SUBSCRIBEto my location


Example: user-adaptive Example: user-adaptive device configurationdevice configuration

“all devices that are in the building”RFC 3082?

PA

devicecontroller

SUBSCRIBEto each room

SUBSCRIBE to configurationfor users currently in rooms

1. discover room URI2. REGISTER as contact for room URI

tftp

HTTP

SLP802.11 signal strength

location

REGISTERTo: 815cepsrContact: alice@cs

SIP

room 815


Session mobilitySession mobility Walk into office,

switch from cell phone to desk phone

call transfer problem SIP REFER

related problem: split session across end devices

e.g., wall display + desk phone + PC for collaborative application

assume devices (or stand-ins) are SIP-enabled

third-party call control


How to find services?How to find services? Two complementary developments:

smaller devices carried on user instead of stationary devices devices that can be time-shared

large plasma displays projector hi-res cameras echo-canceling speaker systems wide-area network access

Need to discover services in local environment SLP (Service Location Protocol) allows querying for services

“find all color displays with at least XGA resolution” slp://example.com/SrvRqst?public?type=printer

SLP in multicast mode SLP in DA mode

Need to discover services before getting to environment “is there a camera in the meeting room?” SLP extension: find remote DA via DNS SRV


Internet

CorrespondentNode (CN)

SIP UA

SLP UA

SIP SM

Local Devices

SLP SA SLP UA

SIP SM SIP UA

SLP DA

Mobile Node (MN)

SLPSIPRTP

SIP UA

Transcoder

Session mobilitySession mobility


Presence for spam Presence for spam preventionprevention VoIP spam (“spit”) and IM spam

(“spim”) likely to be more annoying than email spam

Subscription to another person is indication of mutual trust other person allows me to see their

status trusts me Thus, use watcher list (who is

watching me) as trust vector


Conferencing issuesConferencing issues


Open issues for conferencing Open issues for conferencing standardizationstandardization Multi-device systems and session mobility Conference floor control BFCP

simple RPC-like protocol Centralized conference control (XCON

WG) still struggling with control model

set + get variables (~SNMP) RPC-like model ( SOAP, XML-RPC, NETCONF) send descriptions? forms (XForms?)?


XCON SystemXCON SystemLogical XCON Server

Floor ControlClient

TEMPLATEOf the SYSTEM:•Pre-configured•Initial/Default values

Conf EventNotification Server

Focus

CPCP Client

CCCPClient

CPCPServer

CCCPServer

CallSignaling

Client

TEMPLATE Policy:•Of TYPE RULES

RESERVATION Policy:•Of TYPE RULES

CURRENT Policy:•Of TYPE RULES

RESERVATIONOf the INSTANCE:•Of TYPE CONFERENCE-INFO

STATEOf the CURRENT INSTANCE:•Of TYPE CONFERENCE-INFO

NotificationClient

FloorControl Server

SIP/PSTN/H.323T.120/Etc.

CCCPCPCP SIP NOTIFY/Etc. BFCP

Logical XCON Client


Open issues: application Open issues: application sharingsharing Current: T.120

doesn’t integrate well with other conference control mechanisms

hard to make work across platforms (fonts) ill-defined security mechanisms

Current: web-based sharing hard to integrate with other media, control and record generally only works for Windows mostly limited to shared PowerPoint

Current: vnc whole-screen sharing only can be coerced into conferencing, but doesn’t

integrate well with control protocols


IETF effort: standardized IETF effort: standardized application sharingapplication sharing Remote access = application sharing Four components:

window drawing ops PNG keyboard input mouse input window operations (raise, lower, move)

Uses RTP as transport synchronization with continuous media but typically, TCP allow multicast large group sessions


Peer-to-peer Peer-to-peer approaches for SIPapproaches for SIP

Henning SchulzrinneDept. of Computer Science

Columbia University


ApplicationsApplications Small stand-alone networks

2-50 SOHO, events, emergency coordination may not have access to Internet

infrastructure Corporate size networks

50-1000 single administrator

Global-scale networks 1000-100 million consumer applications serious trust issues


SIP-managedDHT

OpenDHT

Three basic approachesThree basic approaches Full distribution and search

similar to Bonjour scales to small, local networks

DHT built using SIP see Kundan/Schulzrinne and

Cao/Bryan/Lowekamp dedicated to VoIP Skype model

Using an external DHT (Columbia) using OpenDHT as generic service

used by multiple applications can provide mapping or pointer to mapping


P2P-SIP: Implementation in P2P-SIP: Implementation in SIPcSIPc

OpenDHT Trusted nodes Robust Fast enough (<1s)

Identity protection Certificate-based SIP id == email

P2P forCalls, IM, presence, offline message, STUN server discovery and name search


P2P-SIP: What is P2P-SIP: What is OpenDHT?OpenDHT? Service model, unlike earlier library model of Chord or

CAN DHT accessed via SunRPC & XML-RPC Easy deployment and maintenance

200-300 Bamboo DHT nodes on PlanetLab Public DHT service running since April 2004 Many existing applications: i3, CFS, Ostream, HIP,…

DHT API (server side on Bamboo nodes) PUT(key,value,H(secret),ttl) where H() is SHA1 GET(key) (value,H(secret),remaining-ttl) REMOVE(key,H(value),secret,ttl)

ReDiR API (client side for lookup/join/leave) Can build anycast, multicast, range search using this

Fair resource (disk) allocation among clients (IP addr)


Hybrid architectureHybrid architecture

Cross register, or Locate during call

setup DNS, or P2P-SIP hierarchy


ConclusionConclusion Lots of open issues… Motivation for P2P?

Low operational cost? Ease of use? Reliability?

What to use p2p for? identifier lookup (“location server)” locating a NAT traversal server (STUN) media file storage (voice mail) presence conferencing

Some architectural open issues trust (sybil, byzantine failures)

“man-in-the-middle” identifier ownership

prevention? at least, detection reliability


ConclusionConclusion Basic multimedia communication solutions

available scalable, secure, good quality

Media delivery controlled interaction Binary presence rich presence Presence as enabler of loosely-coupled

advanced services location-based services serve as service script inputs trust indicator for spit/spim prevention

Application sharing as missing link P2P mode as complementary model

Documents

April 13, 2006Jiao Tong University, Shanghai1 Advanced Multimedia and Presence Services using Classical…