Upload
rneto11
View
323
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Video streaming on e-lab
Citation preview
R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
e-labInstituto Superior TécnicoLisboa, Portugalhttp://elab.ist.utl.pt
Video Streaming
Rafael Bagagem [email protected]
2 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Streaming multimediaStreaming multimedia
● Multimedia that is constantly received by, and normally displayed to, the end-user
● A media stream can be on demand or live● Usually applied to media that are distributed
over telecommunications networks— Radio and TV are inherently streaming
● Real world examples— Youtube, on-line TV and Radios, etc.
3 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
OSI ModelOSI Model
● A set of seven layers that define the different stages that data must go through to travel from one device to another over a network
4 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
OSI Model – layers and OSI Model – layers and instancesinstances
● Layer – a collection of conceptually similar functions that provide services to the layer above and receives service from the layer below.
● Instance – on each layer an instance provides services to the instances at the layer above and requests service from the layer below; conceptually two instances at one layer are connected by a horizontal protocol connection on that layer.
5 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
OSI Model – layers OSI Model – layers descriptiondescription
● Physical Layer – electrical and physical specifications (layout of pins, voltages, cable specifications, hubs, network adapters …
● Data Link Layer – functional and procedural means to transfer data; detect/correct errors on Physical.
● Network Layer – functional and procedural means of transferring variable length data via one or more networks; performs network routing functions; perform fragmentation/reassembly, and report delivery errors; maintain the QoS requested by the Transport.
● Transport Layer – provides transparent transfer of data between end users, providing reliable data transfer services to upper layers; controls reliability of a given link through flow control, segmentation and de-segmentation, and error control; keep track of the segments and retransmit those that fail.
6 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
OSI Model – layers OSI Model – layers descriptiondescription
● Session Layer – controls the dialogues between computers; establishes, manages and terminates the connections between the local and remote application; provides for full-duplex, half-duplex, or simplex operation, and establishes check-pointing, adjournment, termination, and restart procedures.
● Presentation Layer – independence from differences in data representation (e.g., encryption) by translating from application to network format, and vice versa; works to transform data into the form that the application layer can accept; formats and encrypts data to be sent across a network, providing freedom from compatibility problems.
● Application layer – interacts with software applications that implement a communicating component; functions include identifying communications partners and resource availability, and communications synchronization.
7 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Transport Layer (I)Transport Layer (I)
● Most common protocols for media broadcast are UDP and TCP
● User Datagram Protocol (UDP)
— Unreliable: no concept of acknowledgement, retransmission and time-out
— Unordered - the order in which they arrive cannot be predicted.
— Lightweight - Small transport layer designed on top of IP.
— Datagrams - Packets are sent individually and are guaranteed to be whole if they arrive.
8 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Transport Layer (II) Transport Layer (II)
● Transmission Control Protocol (TCP)
— Reliable: manages message acknowledgement, retransmission and time-out
— Ordered - the order is well known.
— Heavyweight - Large transport protocol designed on top of IP
● TCP requires three packets just to set up a socket, before any actual data can be sent....
— Streaming - nothing distinguishing where one packet ends and another begins. Packets may be split or merged into bigger or smaller data streams arbitrarily.
9 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Streaming Media Streaming Media ProtocolsProtocols
● MMS – Microsoft Media Services
— Deprecated in favour of RTSP
— UDP or TCP● RTP/RTSP – Real Time Streaming Protocol
— Open, widely used● PNM/PNA
— Real Audio/Networks proprietary● RTMP – Real Time Messaging Protocol
— Proprietary protocol developed by Adobe Systems
10 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Broadcasting raw dataBroadcasting raw data
● The number of bytes produced in one second is given by:— Image width x Image height x Colours (typically
RGB=3 at 8 bits) x number of frames per second (FPS)
— 10FPS with 320 x 240 RGB image:
— 2304000 bytes/s ~ 2.3MB/s ~ 18432000 bits/s ~ 18.4Mbits/s
11 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Problems with this Problems with this approachapproach
● Typically a user has at home a bandwidth of something between 2 and 24 Mbits/s
● If 10 users are connected the bandwidth on the server grows to 180Mbits/s!!!
● Not so good solutions— lower the image size
— Use only one color (gray scale)
12 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
SolutionSolution
● Compress the image● Two approaches
— Compress each image
— Compress over time● This is clearly the most clever solution since most of the
time the images don't change that much!
● CODECs are specialized algorithms to perform this work
13 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Frame compressionFrame compression
● Compress each frame:
● Compress over time:— The difference between two the successive frames
14 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Video compressionVideo compression
● Lossless— Ideal but not feasible for video broadcast. The
bandwidth is still to high
— Examples are: PNG, JPEG 2000, ZIP
● Lossy— Define the best relation between quality loss and
data reduction
— Find a format which allows to maximize the previous point
15 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Lossy compressionLossy compression
● The big trick:— Remove information from video that is not important
for human perception in order to achieve very high compression rates while still keeping very good visual quality.
● Some video formats allow ratios of 200:1 !
16 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Common Video Coding Common Video Coding FormatsFormats
● H.261● H.263● H.264
— x264
● MPEG-4 AVC (Advanced Video Coding)
— Xvid, FFmpeg, DivX, x264...
● WMV● RealVideo
17 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Why x264?Why x264?
● Open source● H.264/MPEG-4 AVC video CODEC. Lots of
advanced encoding features● 200:1 ratios and more...● Available for free in every operating system● Supported by most popular video players:
— Quicktime, VLC, Windows Media Player 12, mplayer
18 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Video sizesVideo sizes
● VGA (Video Graphics Array)● QQVGA 160 × 120 quarter of a quarter of VGA
● HQVGA 240 × 160 half of a quarter VGA
● QVGA 320 × 240 e-lab videos (quarter of VGA)
● VGA 640 × 480 VGA
● XGA 1024 × 768 Extended Graphics Array
19 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Best bit ratesBest bit rates
● As discussed before, the output of the CODEC must maximize the quality of the image at the lowest bandwidth
● 8 kbit/s — telephone quality (using speech codecs)
● 120kbit/s — elab videos● 5 Mbit/s — DVD quality● 15 Mbit/s — HDTV quality
20 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Multicast vs UnicastMulticast vs Unicast
● Multicast— Streams the same media to all the receivers
— It is the perfect solution... but NO Internet Provider will allow multicast on its networks
— It only works in private networks
● Unicast— Multiple connections for the same stream
— The only solution
21 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Streaming serversStreaming servers
● Microsoft Media Services
— Not free
— In the past only support the MMS protocol (now uses RTSP)● Adobe Flash Media Streaming Server 3
— Not free● Darwin Streaming Server (DSS)
— Open source
— Shares the same code base as QuickTime Streaming Server
— Streams MPEG-4
22 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
e-lab structure diagram e-lab structure diagram Video Streaming ServerVideo Streaming Server
23 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
DSS – ReflectorDSS – Reflector
● Allows to deliver live broadcast to clients● Client connects to DSS which then reflects the
connection to the desired broadcast● Separate program sends RTP packets to DSS
with the video● UDP packets are sent from the source● DSS makes the bridge between the
broadcaster and the client
24 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Session Description Session Description Protocol (SDP)Protocol (SDP)
● DSS stores files, describing each broadcaster, saved with the SDP format
● Most important information:
— IP of the source
— Port of the source
— Media type (payload)
— Bit rate
— Title● There are programs which automatically generate these files
25 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
SDP ExampleSDP Example
● v=0
● o=- 1204476969191332 1204476969191334 IN IP4 192.168.0.123
● s=GAMMA
● b=RR:0
● t=0 0
● a=tool:vlc 1.1.13
● a=type:broadcast
● m=video 2026 RTP/AVP 96
● c=IN IP4 192.168.0.2
● b=AS:80
(red lines can be optional)
26 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Connection flowConnection flow
● Client connects to DSS with an URL like:
— rtsp://elabmc.ist.utl.pt/gamma.sdp● DSS parses the SDP file and connects it to the desired
source (typically an UDP port)
● On the previous example the video is being sent to the port 2026 of the 192.168.0.2 from 192.168.0.123
● Client also uses the SDP file to check the media type (in the previous case 96=MPEG4)
27 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Video broadcastVideo broadcast
● In order to send the video to the streaming server one needs— Cameras
— Drivers
— Generic API to access the images from the cameras
— Encoder
— Broadcaster
28 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
CamerasCameras
● In the case of remote laboratories, commercially available webcams are the perfect solution
● Problem is that a lot of webcams only work under Windows
— Imply encoding under Windows
— Costs money
— Highly unreliable● Some webcams are supported quite well under Linux
29 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Video 4 Linux (V4L)Video 4 Linux (V4L)
● Video capture API for Linux;● Unifies the way programs access the cameras● Closely integrated with the Linux kernel● V4L is in its second version (V4L2)
— Include a compatibility mode for V4L1
— Better use V4L2 native devices/drivers
30 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Linux webcamsLinux webcams
● Look for cameras which already have a driver in the kernel tree
● Camera drivers must support native V4L2● All the cameras supported by the PWC (Phillips
USB Webcam Driver for Linux) project are a good choice— Philips
— Logitech Quickcams
31 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Video encodingVideo encoding
● The ideal program grabs the video using V4L2● Encodes using MPEG-4● Sends the data using RTP to the streaming
server● Automatically writes an SDP file which can be
placed on the DSS● All these features are present on the VLC
project
32 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
VLCVLC
● Open-source● VLC live streaming
— audio/video capture utility that can capture and encode audio and video in real-time
— results can be written to either an .mp4 file, transmitted onto the network via either unicast or multicast, or both simultaneously
33 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
VLC internalsVLC internals
● Download source from
— http://www.videolan.org/vlc/
— Configure, make and install
● Gentoo Linux OS:
— emerge vlc
— USE=”X ffmpeg libv4l2 live rtsp stream v4l2 x264”
● Each broadcaster uses a specific configuration command
● The program is launched with the following syntax:
— vlc + “configurations” (with GUI)
— cvlc + “configurations” (without GUI)
34 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
VLC configurations VLC configurations
● Entire commandsu elab --command "cvlc -vvv v4l2:///dev/video1 :input-slave=alsa:// :v4l2-standard=0 :file-caching=300 \--sout '#transcode{vcodec=h264,vb=80,fps=10,scale=1,width=320,height=240,acodec=none}'\':rtp{proto=udp,dst=192.168.0.2,port-video=20002,ttl=127,name=SCUBA,sdp=file:///home/elab/webcam_streaming/sdps/scuba.sdp}' \ --no-sout-audio" &
● Parts of the command— Open the webcam:
● cvlc -vvv v4l2:///dev/video1 :input-slave=alsa:// :v4l2-standard=0 :file-caching=300
— Stream output (transcoding):
● '#transcode{vcodec=h264,vb=80,fps=10,scale=1,width=320,height=240,acodec=none}'
— RTP and SDP configurations:
● ':rtp{proto=udp,dst=192.168.0.2,port-video=20002,ttl=127,name=SCUBA, sdp=file:///home/elab/webcam_streaming/sdps/scuba.sdp}'
— No audio:
● --no-sout-audio
35 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
Launching VLC atLaunching VLC ate-lab computer clustere-lab computer cluster● Each broadcaster have a script file with VLC
commands corresponding to the respective webcams— /etc/local.d/vlc.start
● Information about the streaming port number is located in a web-page:— elab1.ist.utl.pt
● All broadcaster execute the script file located in their /etc/local.d directory by executing local daemon:— /etc/local.d/vlc.start which is executed by:
— /etc/init.d/local start
36 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
RemarksRemarks
● This set-up allows to have a complete, open-source and free video broadcasting solution
● It can be easily adapted to broadcast other kind of sources (stored files, movies, music, etc.)
● Required software— Linux installation
— Darwin Streaming Server
— VLC
37 R. B. Henriques | Lisbon, 23 March 2012 | Video Streaming
The final pictureThe final picture