Embed Size (px)
Citation preview
RTP/RTCP Overview
Hank Peng
Audio/Video Transport (avt)
• Chartered 1992-Mar-05– to specify a protocol for real-time transmission of
audio and video over unicast and multicast UDP/IP• Concluded 2011-Mar-19– RTP, associated profiles, extensions, and payload
formats
Real-time Applications and Infrastructure
• Area– RAI: Real-time Applications and Infrastructure
• Real-time Applications and Infrastructure– avtcore Audio/Video Transport Core Maintenance– avtext Audio/Video Transport Extensions– mmusic Multiparty Multimedia Session Control– p2psip Peer-to-Peer Session Initiation Protocol– payload Audio/Video Transport Payloads– rtcweb Real-Time Communication in WEB-browsers– simple SIP for Instant Messaging and Presence Leveraging Extensions– sipcore Session Initiation Protocol Core– xmpp Extensible Messaging and Presence Protocol
Current Tasks for avt - 1• 1) Maintenance of the core RTP/SRTP/RTCP protocols and the AVP,
SAVP, AVPF, and SAVPF profiles– maintain and enhance the SRTP Profile, with review and input from the
Security Area– specify how to use Explicit Congestion Notification (ECN) with RTP/RTCP– continue to investigate the export of summarized RTP/RTCP information for
operations and monitoring purposes
• 2) Specification and maintenance of payload formats for use with RTP– provide guidelines for payload format design– develop payload formats for new media codecs– review and revise existing payload formats to advance those which are
useful to Draft Standard or Standard, and to declare others as Historic
Current Tasks for avt - 2• 3) Specification of metric blocks for use with the RTCP Extended Report
(XR) framework– provide a mechanism allowing identification data for a set of related metric
reports to be carried separately and identified in those reports by reference– provide report blocks for delay, delay variation, and discard– provide report blocks for burst/gap discard and loss– provide report blocks supporting rapid synchronization of multiple RTP streams– provide report blocks for jitter buffer metrics– provide report blocks for loss concealment metrics
• 4) Extensions to the core protocols to facilitate joining, synchronizing, control, and monitoring of RTP multimedia sessions– develop tools to support rapid acquisition of multimedia streams– specify necessary extensions to support rapid synchronization of multiple RTP
streams
RFCs Related to QoS2.0 Sprint 1• rfc3550 - RTP: A Transport Protocol for Real-Time Applications• rfc3551 - RTP Profile for Audio and Video Conferences with Minimal Control • rfc3611 - RTP Control Protocol Extended Reports (RTCP XR)• rfc4571 - Framing Real-time Transport Protocol (RTP) and RTP Control Protocol (RTCP) Packets over
Connection-Oriented Transport• rfc4585 - Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP-
AVPF)• rfc5104 - Codec Control Messages in the RTP Audio-Visual Profile with Feedback (AVPF)• rfc5117 - RTP Topologies• rfc5285 - A General Mechanism for RTP Header Extensions• rfc5761 - Multiplexing RTP Data and Control Packets on a Single Port• rfc5968 - Guidelines for Extending the RTP Control Protocol (RTCP)• rfc6035 - Session Initiation Protocol Event Package for Voice Quality Reporting• rfc6184 - RTP Payload Format for H.264 Video• rfc6190 - RTP Payload Format for Scalable Video Coding• draft-alvestrand-rmcat-remb - RTCP message for Receiver Estimated Maximum Bitrate• draft-alvestrand-rtcweb-congestion - A Google Congestion Control Algorithm for Real-Time
Communication on the World Wide Web
Transport Functions
• Application Support– Reliability control: loss recover, in-sequence delivery, etc.
• Network Control– Congestion control, rate allocation, etc
• The distinction between the two is not sharp.– Rate allocation and scheduling can be viewed as part of
either one above.• This dual view arises when we contemplate
traditional transports: TCP and UDP
Overview of RTP
• Provides end-to-end delivery services for real-time traffic: interactive audio and video– Payload identification, sequence numbering, timestamping
and delivery monitoring• Runs on top of UDP, and less often, TCP.– RTP does not guarantee delivery or prevent out-of-order
delivery.• Primarily designed to support multiparty multimedia
conferences, typically assumes IP multicast.
Overview – Cont.
• The protocol has two parts.– RTP: carry real-time data– RTP control protocol (RTCP): monitor the quality of service
and to convey information about the participants.• Principles of application level framing and integrated
layer processing. – Is malleable to provide application specific info.– Is typically integrated into the application processing.– Protocol is deliberately not complete. It only contains the
common functions.– A complete specification for an application also includes a
profile and a payload format document.
Example- Multicast audio conference
• Need a multicast address and a pair of ports: one for data and one for (RTCP) control.
• RTP header contains type of audio encoding (such as PCM). Senders can change encoding during the conference.
• RTP header contains timing information. Audio data can be played out as they are produced by the source.
• Senders and receivers multicast reports through RTCP. Packet loss ratio, delay jitter, and other status info can be monitored.
Example – Mixers and Translators• Mixers: a RTP-level entity that receives streams of RTP data
packets from one or more sources and combines them into a single stream.
• A translator forwards RTP packets from different sources separately.
• Mixer is like a new RTP-level source to the receivers.• Translator is more transparent. Receivers can identify
individual sources even though packets pass through the same translator and carry the translator’s network source address.
• Mixer can re-synchronize the incoming stream and generates its own timing info.
Example – Mixers and Translators• Mixers: a RTP-level entity that receives streams of RTP data
packets from one or more sources and combines them into a single stream.
• A translator forwards RTP packets from different sources separately.
• Mixer is like a new RTP-level source to the receivers.• Translator is more transparent. Receivers can identify
individual sources even though packets pass through the same translator and carry the translator’s network source address.
• Mixer can re-synchronize the incoming stream and generates its own timing info.
Example: Translator at Firewall
Translator
Firewall
Translator
On multicastAddress a, port p, p+1
On multicastAddress b, port q, q+1
Inside Firewall
Note that UDP or TCP connections terminates at Firewall.
Some RTP Definitions• Transport address: network address + port• RTP session: communications on a pair of transport addresses
(data + control)• Synchronization source (SSRC): the source of a stream of RTP
packets– Identified by 32-bit SSRC identifier.– All packets from the same SSRC form a single timing and sequencing
space. Receivers group packets by SSRC for playback.– Not dependent on network address.– Examples: all packets from a camera; from a mixer; for layered
encoding transmitted on separate RTP sessions a single SSRC is used for all layers.
– A participant need not use the same SSRC for all RTP sessions in a multimedia session.
RTP Fixed Header
P: PaddingX: Header ExtensionCC: CSRC countM: Marker of record boundary
PT: Payload type; mapping can be
specified by profile of the application
Sequence number: for each packet can be used by the receiver to detect loss or restore sequence.
RTP Fixed Header – Cont.
• Timestamp– Reflects sampling instant of the first byte of data– Clock frequency can be specified by profile of payload
format documents for the application.– Example: for fixed-rate audio, clock may increment by one
for each sampling period.
• SSRC: chosen randomly for each synchronization source; with the intent that no two synchronization sources in the same session have the same SSRC.
Profile-Specific Modifications to the RTP Header
• Marker bit and payload type are interpreted according to the application’s profile.
• Moreover, the byte containing them can be redefined by the profile.
• If a particular class of application needs additional functionality, the profile should define additional fixed fields following SSRC.
• If X bit is 1, exactly one header extension follows CSRC list (if present). – Variable length– Used to experimental purpose
RTCP• Primary function is to provide feedback on the quality
of data distribution.– Through sender and receiver reports;– For adaptive encoding (adaptive to network congestion);– Can be used to diagnose faults
• RTCP carries a persistent transport-level identifier for an RTP source, called canonical name, CNAME.– Receivers use CNAME to keep track of each participant– And to synchronize related media streams (with the help of
NTP)• Passes participant’s identification for display.
RTCP Packets
• SR: sender reports; sending and reception stat.
• RR: receiver reports; for reception statistics from multiple sources.
• SDES: source description item, include CNAME• BYE: indicates end of participation• APP: application specific functions
Compound RTCP Packets
• A compound RTCP packet contains multiple RTCP packets of the previous types.
• Example:
SR Packet
SR Packet – Cont.• RC: receiver report count• Length: in 32-bit words – 1• NTP ts: wallclock time, used to calculate RTT• RTP ts: in unit and offset of RTP ts in data packets. Can be used with NTP
ts for inter-media synchronization.• Fraction lost: since the last RR or SR packet was sent. Short term loss ratio.• LSR: last SRT time stamp; middle 32 bits of NTP timestamp.• DLSR: delay since last SR; expressed in 1/65536 seconds between
receiving the last SR packet from SSRC_n and sending this report. Source SSRC_n can compute RTT using DLSR, LSR and the reception time of the report, A.RTT = A – LSR – DLSR
• An application’s profile can define extensions to SR or RR packets
SDES Packet
CNAME Item in SDES Packet
• Mandatory• Provides a persistent identifier for a source.• Provides a binding across multiple media used by one
participant in a set of related RTP sessions. CNAME should be fixed for that participant.
• SSRC is bound to CNAME• Example: [email protected]; [email protected], etc.
Other Items in SDES Packet
• NAME: user name• EMAIL:• PHONE:• LOC: location• TOOL: application or tool name• NOTE: notice/status
BYE: Goodbye RTCP Packet
• Mixers should forward the BYE packet with SSRC/CSRC unchanged.
• Reason for leaving: string field; e.g., “camera malfunction”
APP RTCP Packet
• Subtype: allows a set of APP packets to be defined under one unique name.
• Name: unique name in the scope of one this application.
Conclusions - I
• RTP defines transport support for common functions of real-time applications.– Timing information: sampling period and NTP– Synchronization source for playback– Payload types (encoding)– Quality reports: short-term and long-term packet loss, and jitters.– Participants indication: CNAME, NAME, EMAIL, etc.– Multicast distribution support– Conversion: mixers and translators
• Extensible protocol by profile payload format documents• Customizable to application or application classes
Conclusion – II
• Separation of control and data stream (analogous to out-band signaling)– Data header overhead is small.– Can accomplish complex control features.– Complexity of the protocol/algorithm is not so
bad, because there is little hard guarantee (It relies on TCP or application for hard guarantees).
Conclusions – III
• Congestion control is not defined in baseline document, but may be defined by application’s profile.– Leads to application-specific congestion control or
adaptation
• RTP can be considered as user-space transport entities, but does not run as stand-alone process.
• Mixers and translators are stand-alone processes. They terminate TCP or UDP connections.
RTP Algorithms - I
• RTCP packets generation: need to limit the control traffic– Control traffic takes 5% of data traffic bandwidth (not
defined)– ¼ of the RTCP bandwidth is used by senders– Interval between RTCP packets scales linearly with the
number of members in the group.– Each compound RTCP packet must include a report packet
and a SDES packet for timely feedback.
RTP Algorithms - II
• SSRCs are chosen randomly and locally and can collide.
• Loops introduced by mixers and translators– A translator may incorrectly forward a packet to the same
multicast group from which it has received the packet.– Parallel translators.
• Collision avoidance of SSRC and loop detection are entangled.
Example of A Profile Document
• RTP data header: – use one marker bit– No additional fixed fields– No RTP header extensions are defined.
• RTCP– No additional RTCP packet types.– No SR/RR extensions are defined– SDES use: CNAME is sent every reporting interval, other
items should be sent only every fifth reporting interval.
rfc3551 - RTP Profile for Audio and Video Conferences with Minimal Control
PayloadTypes
rfc3611 - RTP Control Protocol Extended Reports (RTCP XR)
• Supplements the six statistics that are contained in SR and RR
• Support multicast inference of network characteristics (MINC) or voice over IP (VoIP) monitoring
• Convey information used for network management
Report Block categories
• Packet-by-packet block– Loss RLE RB– Duplicate RLE RB– Packet Receipt Times RB
• Reference time related block– Receiver Reference Time RB– DLRR RB
• Summary metric block– Statistics Summary RB– VoIP Metrics RB
XR Packet Format
Extended RB Framework
Loss RLE RB
Duplicate RLE RB
Packet Receipt Times RB
Receiver Reference Time RB
DLRR – Delay since last Receiver Report
Statistics Summary
VoIP Metrics RB
rfc4585 - Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based
Feedback (RTP-AVPF)• Receivers may use the base mechanisms of RTCP to report packet
reception statistics and thus allow a sender to adapt its transmission behavior in the mid-term
• RTP-AVPF enables receivers to provide, statistically, more immediate feedback to the senders and thus allows for short-term adaptation and efficient feedback-based repair mechanisms to be implemented– Retransmission– FEC– media-specific mechanisms - reference picture selection
• Modification/addition– Early RTCP messages and algorithms allowing for low-delay feedback in
small multicast groups– general-purpose feedback messages
Definitions• Regular RTCP mode
– RTCP messages are sent following the rules of RFC3550, minimal interval of 5s
• Early RTCP mode– receiver is often (but not always) capable of reporting events of interest
back to the sender close to their occurrence• Immediate Feedback mode
– each receiver is, statistically, capable of reporting each event of interest immediately back to sender
• Feedback (FB) message– convey information about events observed at a receiver
• Feedback (FB) threshold– indicates the transition between Immediate Feedback and Early RTCP mode
Modes of Operation
• an indication of whether or not the receiver will, on average, be able to report all events to the sender in a timely fashion
• Immediate Feedback mode– group size is below the FB threshold– N<=B*T/R, where N: average number of events per interval
T, B: RTCP bandwidth fraction, R: average RTCP packet size • Early RTCP mode– N > B*T/R
• Regular RTCP Mode
Modes of Operation
Format of RTCP Feedback Messages
• Transport layer FB– general purpose feedback information, i.e., information
independent of the particular codec or the application– Generic NACK
• Payload-specific FB– specific to a certain payload type and will be generated and acted
upon at the codec layer– Picture Loss Indication (PLI)– Slice Loss Indication (SLI)– Reference Picture Selection Indication (RPSI)
• Application layer FB– convey feedback from the receiver’s to the sender’s application
Common Packet Format for Feedback Messages
Generic NACK
• indicate the loss of one or more RTP packets
• Packet ID (PID)– RTP sequence number of the lost packet
• bitmask of following lost packets (BLP)– any of the 16 RTP packets immediately following
the RTP packet indicated by the PID
Payload-Specific Feedback Messages
• Picture Loss Indication (PLI)– a decoder informs the encoder about the loss of an
undefined amount of coded video data belonging to one or more pictures
• Slice Loss Indication (SLI)– a decoder can inform an encoder that it has detected the
loss or corruption of one or several consecutive macroblock(s) in scan order
• Reference Picture Selection Indication (RPSI)– an encoder has learned about a loss of encoder-decoder
synchronicity
Application Layer Feedback Messages
• transport application-defined data directly from the receiver’s to the sender’s application
• the application MUST be able to identify the message payload
Early Feedback and Congestion Control
• The way to react to the feedback received depends on the application using the feedback mechanisms
• Common requirement for (TCP-friendly) congestion control on the media stream
• RTCP feedback itself is insufficient for congestion control purposes as it is likely to operate at much slower timescales than other transport layer feedback mechanisms (that usually operate in the order of RTT)
• Therefore, additional mechanisms are required to perform proper congestion control
rfc5104 - Codec Control Messages in the RTP Audio-Visual Profile with Feedback (AVPF)
