2016 Streaming Media West: Transitioning from Flash to HTML5

  • Published on
    19-Jan-2017

  • View
    266

  • Download
    1

Transcript

<p>PowerPoint Presentation</p> <p>Choosing an HTML5 player An overview of how media engines work &amp; benchmark of open-source frameworksStreaming Media West Track DTuesday, May 10, 2016 1:45 to 2:30 pm</p> <p>Pioneers in hybrid video delivery systems to accompany exponential growth in OTT traffic</p> <p>Experts in HTML5 and consultants in the transition from Flash</p> <p>Infinite scale, limitless delivery.</p> <p>Streamroot: Who are we?</p> <p>And who are you?Infinite scale, limitless delivery.</p> <p>HTML5 basicsA. Why make the transitionB. Formats, APIs, standardization</p> <p>Delving in to the migrationA. Formats &amp; EncodingB. PlayersC. DRMs, ads and business featuresD. HTTPs</p> <p>Choosing a playerA. Modern player architectureB. Considerations &amp; what to look forC. Benchmark of open-source and proprietary options What well be talking about today.</p> <p>Infinite scale, limitless delivery.</p> <p>Examples: hls.js &amp; dash.jsDebugging, subtitles, encryption, ads, analytics, Streamroot, ABR</p> <p>Going from POC to productionA. What could go wrongB. What happens after I do a POC? Workflow, fallbacks, etc.C. Smoothly transitioning into production: QoS metrics, AB testingD. Useful tools &amp; links </p> <p>What well be talking about today.</p> <p>Infinite scale, limitless delivery.</p> <p>Infinite scale, limitless delivery.I. Its time to switch to HTML5.</p> <p>Infinite scale, limitless delivery.I. HTML5 BasicsAPIsMedia Source Extensions</p> <p>Makes HTTP adaptive streaming possible in HMTL5</p> <p>Use JavaScript to build streams and inject data into the video tags buffer</p> <p>Source: W3C specs</p> <p>Infinite scale, limitless delivery.I. HTML5 BasicsAPIsEncrypted Media Extensions</p> <p>Allows for DRMs in HMTL5</p> <p>Provides a way to interact with content protection systems</p> <p>Plugin-free</p> <p>Common Encryption (CENC): standardized key and encryption methods - multiple DRMs for same file</p> <p>Source: W3C specs</p> <p>Infinite scale, limitless delivery.</p> <p>I. HTML5 BasicsAPIsSource: Bitmovin</p> <p>Infinite scale, limitless delivery.I. HTML5 BasicsFormats available</p> <p>HDS</p> <p>Infinite scale, limitless delivery.I. HTML5 BasicsFormats available</p> <p>Source: Encoding.com</p> <p>Infinite scale, limitless delivery.I. HTML5 BasicsStandardization</p> <p>CMAF!</p> <p>Infinite scale, limitless delivery.II. Delving into the migration What are you using today?</p> <p>Infinite scale, limitless delivery.II. Delving into the migrationFormats &amp; EncodingSignificant changes to the encoding side that can take some time</p> <p>Encode to MP4 and then repackage as necessary for target platforms</p> <p>Lots of packaging solutions out there: Wowza, Unified Streaming Platform, etc.</p> <p>Infinite scale, limitless delivery.II. Delving into the migrationPlayersMore flexible architecture, with separation between UX and media engine logic </p> <p>When choosing: look at use case and feature requirements</p> <p> Section III.</p> <p>Infinite scale, limitless delivery.II. Delving into the migrationBusiness featuresDRMs: switch from token authorization to HMTL5 DRM</p> <p>Ads: rewrite the ad logic in HTML5</p> <p>Business logic: make sure 3rd party plugins are also compatible with HTML5</p> <p>Infinite scale, limitless delivery.II. Delving into the migrationHTTPSBeware!</p> <p>Playing HTTP streams even in an HTTPS environment is not an option in HTML5.</p> <p>Changing to HTTPs can be expensive and long, depending on your CDN.</p> <p>Infinite scale, limitless delivery.Its like...III. Player architectureModern player architecture</p> <p> your favorite burger.</p> <p>Infinite scale, limitless delivery.</p> <p>Its like...III. Player architectureModern player architecture</p> <p>Infinite scale, limitless delivery.UIIII. Player architecture </p> <p>Infinite scale, limitless delivery.</p> <p>SkinUIIII. Player architecture </p> <p>Infinite scale, limitless delivery.</p> <p>SkinAdsAuthenticationUIIII. Player architecture </p> <p>Infinite scale, limitless delivery.</p> <p>SkinAdsPlaylistsAuthenticationUISocial sharingIII. Player architecture </p> <p>Infinite scale, limitless delivery.</p> <p>SkinAdsPlaylistsAuthenticationUIMedia Engine(s)Social sharingIII. Player architecture </p> <p>Infinite scale, limitless delivery.</p> <p>SkinAdsPlaylistsAuthenticationMedia Engine(s)UIMedia Engine(s)Social sharingIII. Player architecture </p> <p>Infinite scale, limitless delivery.</p> <p>SkinAdsPlaylistsAuthenticationSocial sharingMedia Engine(s)UIMedia Engine(s)Playback&amp; DRMIII. Player architecture </p> <p>Infinite scale, limitless delivery.</p> <p>SkinAdsPlaylistsAuthenticationContent decryptionmoduleMedia Engine(s)</p> <p>DRM ManagerUIMedia Engine(s)Playback&amp; DRMSocial sharingIII. Player architecture </p> <p>Infinite scale, limitless delivery.</p> <p>SkinAdsPlaylistsDRM ManagerDecoder / RendererAuthenticationContent decryptionmoduleMedia Engine(s)UIMedia Engine(s)Playback&amp; DRMSocial sharingIII. Player architecture </p> <p>Infinite scale, limitless delivery.1. Skin - the graphic design of your player</p> <p>III. Player architectureUser Interface</p> <p>@charlo - Eurosport video player example that we used in blog</p> <p>Infinite scale, limitless delivery.1. Skin - the graphic design of your player</p> <p>2. UI logic - features defining all interaction with the user on top of video playback</p> <p>Lots of plugins available, video.js for example</p> <p>III. Player architectureUser Interface</p> <p>@charlo - Eurosport video player example that we used in blog</p> <p>Infinite scale, limitless delivery.3. Business logicAuthenticationPaymentAds</p> <p>*Configuration / device detection logic!III. Player architectureUser Interface</p> <p>@charlo - Eurosport video player example that we used in blog</p> <p>Infinite scale, limitless delivery.</p> <p>Sample UI workflow with authentication, channels and pre-roll advertisementIII. Player architectureUser Interface</p> <p>Authentication</p> <p>Payment</p> <p>Channels / Playlists</p> <p>Configuration logic / device detection</p> <p>A/B testing logic</p> <p>Ads</p> <p>Infinite scale, limitless delivery.Easily customizable, with many plugins available for use or inspiration.</p> <p>Add features as plugins/modules to the core UI base.</p> <p>Create a unified user experience across browsers, even if the media engine behind it may vary from device to device.Check out tools such as React native, HaxeIII. Player architectureUser Interface</p> <p>Infinite scale, limitless delivery.III. Player architectureMedia Engine</p> <p>Manifest parser &amp; interpreter</p> <p>Downloader</p> <p>Streaming engine</p> <p>Quality metrics estimators</p> <p>ABR controller</p> <p>DRM manager (optional)</p> <p>Transmuxer (optional)</p> <p>Infinite scale, limitless delivery.III. Player architectureMedia Engine</p> <p>Manifest parser &amp; interpreter</p> <p>Downloader</p> <p>Streaming engine</p> <p>Quality metrics estimators</p> <p>ABR controller</p> <p>DRM manager (optional)</p> <p>Transmuxer (optional)</p> <p>Infinite scale, limitless delivery.III. Player architectureMedia Engine</p> <p>Manifest parser &amp; interpreter</p> <p>Downloader</p> <p>Streaming engine</p> <p>Quality metrics estimators</p> <p>ABR controller</p> <p>DRM manager (optional)</p> <p>Transmuxer (optional)</p> <p>Infinite scale, limitless delivery.III. Player architectureMedia Engine</p> <p>Manifest parser &amp; interpreter</p> <p>Downloader</p> <p>Streaming engine</p> <p>Quality metrics estimators</p> <p>ABR controller</p> <p>DRM manager (optional)</p> <p>Transmuxer (optional)</p> <p>Infinite scale, limitless delivery.III. Player architectureMedia Engine</p> <p>Manifest parser &amp; interpreter</p> <p>Downloader</p> <p>Streaming engine</p> <p>Quality metrics estimators</p> <p>ABR controller</p> <p>DRM manager (optional)</p> <p>Transmuxer (optional)</p> <p>Infinite scale, limitless delivery.III. Player architectureMedia Engine</p> <p>Manifest parser &amp; interpreter</p> <p>Downloader</p> <p>Streaming engine</p> <p>Quality metrics estimators</p> <p>ABR controller</p> <p>DRM manager (optional)</p> <p>Transmuxer (optional)</p> <p>Infinite scale, limitless delivery.Extremely important yet too often neglected.</p> <p>Most often necessary to have several media engines to reach your audience.</p> <p>III. Player architectureMedia Engine</p> <p>@charlo shma. (ill have to find it)</p> <p>Infinite scale, limitless delivery.</p> <p>III. Player architectureDecoder &amp; DRM manager</p> <p>Buffer ratio1. General criteria device, format and codec dependencies</p> <p>Infinite scale, limitless delivery.III. What to look for </p> <p>2. Custom featuresDRMs DVR subtitlesQoS, etc.</p> <p>3. Perfs &amp; Quality startup time ABRerror strategies4. Street cred, robustness, ease of use</p> <p>Light-touch dev: Simplicity and stablity of the media engineAssess extensibility &amp; ease of incorporating special featuresMedia engine customizationEvents exposedDebugging</p> <p>Infinite scale, limitless delivery.</p> <p>Digging deeper: Robust yet flexible core designTests and testing coveragecommunity &amp; supportIII. What to look for </p> <p>Buffer ratio</p> <p>Infinite scale, limitless delivery.III. Whats out there?</p> <p>Disclaimer!</p> <p>What follows are all GOOD options - probably the best out there. Weve tried to be solely objective based on our research and experience.Features, support and upkeep are always changing.</p> <p>Buffer ratio</p> <p>Infinite scale, limitless delivery.PROs</p> <p>Supported by DASH-IFPushed &amp; maintained by Akamai + tier-1s (BBC)Highest visibility, big communityLots of features and use-cases handledWidely used in production Huge test suite + online test page</p> <p>CONs</p> <p>A little bit of technical debt and complexityNon-trivial API and customization configH264/AAC support only</p> <p>III. Whats out there?</p> <p>Infinite scale, limitless delivery.PROs</p> <p>Built by an entire at Google: solid &amp; smartPerformances and robustness, quick to improveSimple to get started, good tutorialsOnly one supporting WebM, VP8, VP9, open audio codecs Good support on github and Google groups</p> <p>CONs</p> <p>Google-centricToday lacks some features for large broadcastersStricter PR &amp; features policyNo ES6 supportFewer OVP and open-source all-in-one integrations</p> <p>III. Whats out there?</p> <p>Buffer ratio</p> <p>Infinite scale, limitless delivery.PROs</p> <p>Lead by Dailymotion, built from scratch by author of Flashls Enormous traction and visibilityClear architecture design, easily extendableGood robustness and debug demo, responsive supportWidely used in prod by all-in-one players + tier-1s</p> <p>CONsNot all HLS features supported yetSome restrictions from the transmuxing &amp; HLS: no DRMs today, only AES128</p> <p>III. Whats out there?</p> <p>Buffer ratio</p> <p>Infinite scale, limitless delivery.PROs</p> <p>Seamless Flash fallback with a MediaSource polyfillDe facto solution for HLS with VideoJSLarge community of users (Brightcove + Videojs)In production with Brightcove with a wide range of customers</p> <p>CONsVideojs plugin: not usable without videojsLack of public debug tools or pagesLearning curve on providers &amp; tech behind it</p> <p>III. Whats out there?</p> <p>Buffer ratioBut there are also a lot of off-the-shelf options.</p> <p>Infinite scale, limitless delivery.</p> <p>III. Whats out there?</p> <p>All available online on github:</p> <p>https://github.com/streamroot/benchmarking-player</p> <p>Contains:Media engines codeExamples</p> <p>Infinite scale, limitless delivery.IV. Examples</p> <p>MSE example</p> <p>https://github.com/streamroot/benchmarking-player/blob/master/samples/mse.html </p> <p>Infinite scale, limitless delivery.IV. Examples</p> <p>https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-AES.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-subtitles.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-ads.htmlhttps://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-streamroot.html</p> <p>Online debug demo: debug http://dailymotion.github.io/hls.js/demo/ Videojs Ads plugin: https://github.com/videojs/videojs-contrib-adsAds state diagram: https://raw.githubusercontent.com/videojs/videojs-contrib-ads/master/ad-states.png Hls.js Events: https://github.com/dailymotion/hls.js/blob/master/API.md#runtime-events Video Events simulator: http://docs.youbora.com/video-event-simulator/ </p> <p>Streamroot Clappr integration: https://github.com/streamroot/clappr-p2phls-plugin Strearmoot Videojs integration: https://github.com/streamroot/videojs5-hlsjs-p2p-source-handler </p> <p>Infinite scale, limitless delivery.IV. Examples</p> <p>https://github.com/dailymotion/hls.js/blob/master/src/controller/cap-level-controller.js#L68</p> <p>Checks the max CapLevel corresponding to current player size</p> <p>Frequency: every 1000 ms</p> <p>IV. Examples: hls.js CapLevel</p> <p>Infinite scale, limitless delivery.</p> <p>Checks the max CapLevel corresponding to current player sizeEvery 1000ms.You can also add up manual level caps on initialization.If the cap level is bigger that the last one (which means the player size has grown, like in Fullscreen for exemple), then you flush the current buffer and ask for a new quality right away (force the buffer)</p> <p>https://github.com/dailymotion/hls.js/blob/master/src/controller/fps-controller.js#L33 </p> <p>Calculates the dropped frames per second ratio. </p> <p>If &gt; 0.2, bans the level forever goes into restricted capping levels</p> <p>fpsDroppedMonitoringThresholdfpsDroppedMonitoringPeriod </p> <p>IV. Examples: hls.js dropped frames</p> <p>Infinite scale, limitless delivery.</p> <p>Calculates the dropped frames per second ratio. If it is &gt; 0.2, bans the level for ever =&gt; goes into restricated levelsNot activated in production!fpsDroppedMonitoringThresholdfpsDroppedMonitoringPeriod </p> <p>https://github.com/dailymotion/hls.js/blob/master/src/controller/stream-controller.js#L131</p> <p>First segment is loaded from the first level in the playlist, then continues with normal ABR rule.</p> <p>IV. Examples: hls.js startup strat</p> <p>Infinite scale, limitless delivery.</p> <p>First segment always from the lowest quality, then it continues with normal rule (very simple simple rule in practice!)Another optimization is just to load this level (and playlist), and dont wait for the other levels to have been loaded </p> <p>https://github.com/dailymotion/hls.js/blob/master/src/controller/abr-controller.js</p> <p>Before: Simple algorithm,inspired by Androids AVControllers ABR algo</p> <p>Now: EWMA smoothing</p> <p>Infinite scale, limitless delivery.IV. Examples: hls.js bandwidth est.</p> <p>Simple algorithm, </p> <p>https://github.com/dailymotion/hls.js/blob/master/src/controller/abr-controller.js#L51 </p> <p>Infinite scale, limitless delivery.IV. Examples: hls.js abort rule</p> <p>One of the most important ones hereWhat happens if you started a request and then BW drops ? Especially important when you ahve long fragments, this can very easily lead to a buffer underrun! After Half of the needed time, compare the estimate time of arrival to time of buffer underrun. And then see if there is another level that could solve the issue?</p> <p>STRONG POINTSCOULD BE IMPROVEDVery simple and understandableStartup time constraint could be improved to get the lowest level firstHandles CPU &amp; player size constraintsConservative BW adjustment to avoid oscillation, with EWMA smoothingSound emergency abort mechanism</p> <p>Infinite scale, limitless delivery.IV. Examples: hls.js sumup</p> <p>Pros:Simple implementation, taking into account a lot of different paramsWorks as good as the other implementation at Dailymotion! (alshls, android, iPhone etc)Cons:Still...</p>