End-to-End Content Monitoring for OTT Workflows– CDN Providers – OTT Equipment Providers • Ensures superior QoS and QoE for Live and VOD OTT services • Checks for: – ABR

WWW.INTERRASYSTEMS.COM

Viktor Till

Application Engineer

[email protected]

End-to-End Content Monitoring for OTT Workflows

NETS Fokus 2018-01


Content

• Introduction

• Streaming Media Market

• End-to-End QC & Monitoring

• Interra Systems’ Products for Streaming Media Workflows

• VEGA - Media Analyzers

• BATON - File Based QC

• ORION MPEG-2 TS Monitoring

• ORION-OTT - Content Monitor

• ORION-OTT - Origin Performance Monitor

• ORION-OTT - Edge Performance Monitor

• ORION Central Manager

© Interra Systems2


Introduction


Interra Systems - Overview

4

• Interra Systems was founded in 2000

• Industry leader in end-to-end unified quality control (QC), monitoring and analysis solutions for the digital media industry

• Headquartered in the USA; Engineering and R&D Centers in India

• Over 30 channels partners across the globe

• Global, 24/7 customer support for all geographies


Streaming Media Market


Streaming Media Market

© Interra Systems6

• The global video streaming market is growing

• Millennials watching content on their mobile devices and computers

• In the US, 86% of smartphone users watch video content on their phones

• Broadcasters need to brace up for the changing viewing styles by embracing Over-The-Top (OTT) workflows

• New technologies enable us to deliver high quality OTT services

• All of these brings new challenges

• The right set of Quality Control tools are a must to ensure that you stay ahead in the OTT race


© Interra Systems

Streaming Media Challenges

• New technologies

• Dramatic increase in content size

• New standards and formats

• Variety of device types

• Changing video production and delivery approaches

• Transition from a linear workflow infrastructure to OTT

Content QC & monitoring is a must for flawless video Delivery


End-to-End QC & Monitoring


Streaming Media Workflow Example

Video Delivery

Live

Origin ServerPackager /

DRM

Live

VOD

Transcoder

Transcoder

Edge ServersReplica Servers

CDN

9

VOD Repository


© Interra Systems

Why End-to-End QC and Monitoring

• Built-in analytic capabilities of given components of the workflow are important and useful to handle specific issues, but do not address all the aspects of monitoring

• Dedicated QC and monitoring system can cover the whole workflow, keeping focus on overall quality and user experience

• Quality expectations are there at all points of the workflow, but experience is affected by issues raised in previous stages too

• Almost impossible to do all quality checks real time, so proper content preparation is important, thus different type of QC and monitoring features need to complete each others too

• An independent QC and monitoring solution from quality experts gives reliability, also ensuring long-term support and maintenance


End-to-End QC & Monitoring

Video Delivery

Live


DRM

Live

VOD

Transcoder

Transcoder


CDN

VOD Source

QC

VOD Out

QC

LIVE Source

Monitoring

LIVE Out

Monitoring

Active OS

Monitoring

Active & Passive

Monitoring

Passive OS

MonitoringPassive Client

Monitoring

11


Interra Systems’s Products

for Streaming Media Workflows


© Interra Systems13

• Enterprise-class, file-based QC

• Over 350 customers world wide

• Advantages:

– Most formats supported

– Most comprehensive quality checks

– Enterprise scalability

• Baton+

– Trend and Analytics for Intelligent, data managed file-based workflows

Interra Systems’ Products

• Enterprise-class, real-time

monitoring of content for

issues in:

– Delivery compliance

– Audio/video quality

– Digital Program / Ad

insertion

– Loudness

– And more…

Check Monitor

• Media Analyzers for standards

compliance of all popular

media/DTV formats

• Includes tools for:

– Buffer analysis, video

comparison, YUV Diff, and

more

• Vega HEVC

• Vega ABR

Debug



End-to-End QC & Monitoring for Linear & OTT Workflows

ORION Central Manager (OCM)

Video Delivery

BATON

ORION ORION ORION-OTT ORION-OTT

BATON ABR

Live Transcoder

Transcoder



DRM

Live

VOD

CDN

ORION-OTT

Comprehensive QC for VOD Baseband quality checks Compression artifacts Check for metadata consistency QC for large volume of content is best addressed in off-line, file-based

environment Ensure each video is as per corporate guidelines for format and quality Doing QC at delivery stage is inefficient, impractical, and risky

Problems not addressed here carry through the delivery chain

Checking content delivery related issuesMonitor Live and VOD streamsError alerts for each service in real time HTTP checks – download time, latency etc.Content Availability, Manifest ValidationDRM support – Without this you are only addressing QoS issuesPerformance, scalability are critical factors here

Thorough ABR and QoE impacting checks at the Delivery stage for best of breed System


Vega™ Media AnalyzersAnalyze . Debug . Refine . Optimize


VEGA™ - Media Analyzers for Standards Compliance & Debug


The most trusted platform for standards compliance and debugging

• Offers frame by frame analysis with error reports

• Provides in-depth analysis of ABR formats and streams and ability to report the

minutest violations

• Provides video comparison and quality checks for HEVC/4K/UHD streams

• Provides multi-core support for superior performance


Enterprise Class Software-based QC tool

TM

BATON - File Based QC


• Performs the most comprehensive quality checks in the industry, utilizing enhanced audio/video processing, computer vision and machine learning technics

• Offers a hybrid QC platform with a combination of automated & manual QC checks on content

• Provides high performance, enterprise wide scalability

• Integrates with leading MAM / DAM / Transcode / Workflow solutions

• Supports Cloud QC

• Provides sophisticated data analytics, review and content correction

• Direct HTTP content location and DRM support for OTT

BATON® - Next-Generation Hybrid QC Solution for File-based Workflow

a



Most comprehensive quality checks in the industry

• Baseband QC Checks

– Video signal levels, RGB color gamut, color bleeding, blotches, blur, defective pixels, black frames, color bars, chromaphase error, loudness, audio noise, etc.

• Compression Artifacts and In-depth Audio / Video Quality Checks

– Blockiness, Pixelation, Moire pattern, Ringing artifacts, language detection, burnt-in subtitle detection, etc.

• File Integrity & Standards Compliance

– Ensure that the file is not corrupt and has been encoded as per the standard to ensure that the downstream tools are able to play it without issues, including ABR compliances

– Very important in the OTT context (multiple devices, form factors / players from multitude of vendors)

• BATON can perform the following additional checks for ABR content:

– Checks to ensure each segment starts with an independent frame

– Checks to ensure that all variants of the content are properly aligned in terms of number of segments, segment duration, total duration and content structure

– Ensure consistency between metadata and actual content properties

BATON QC Overview



Software-based Monitoring Solution For Linear and OTT Broadcast Workflows

TM

ORION – Content Monitoring Suite


ORION Content Monitoring Suite of Products

The ORION Content Monitoring Suite is a complete package catering to various requirements for monitoring linear and OTT broadcast workflows.

The Suite consists of –

• ORION: For monitoring linear broadcast

• ORION - OTT : For monitoring OTT broadcast

• OCM : For enterprise-wide monitoring




Linear Broadcast OTT Delivery Linear and OTT

ORION – Software-based Real-time Content Monitor

ORION- OTT – Software-based Over The Top Content Monitor

OCM – Software-based Enterprise Level Centralized Monitoring Solution

• Primary users:– Broadcasters

– Telcos

– Video Headends

– Video Equipment Providers

• Ensures superior QoS and QoE for Linear Channels

• Checks for:– Comprehensive IP and TS

Monitoring

– TR101290

– Audio-Video Quality

– Ad-Insertion Monitoring

– Closed Captions & CALM Compliance

• Primary users:– OTT Service Providers

– CDN Providers

– OTT Equipment Providers

• Ensures superior QoS and QoE for Live and VOD OTT services

• Checks for:– ABR Package Integrity

– Audio-Video Quality

– Server and Network Performance

– HTTP Responses

• Primary users:– OTT Users

– Orion Users

• Allows viewing QoS and QoE for Linear Channels as well as Live and VOD OTT services at multiple points in the workflow

ORION – Content Monitoring Suite: Highlights


ORION – Software-based Real-time Content Monitor


Enabling video delivery with superior QoS and QoE

Encoded

Content

IP / ASI / SDI

MPEG-2 Transport] • Comprehensive IP & TS monitoring• Audio-video quality checks• HEVC 4K monitoring with QoE• Ad Insertion monitoring• Closed Captions & CALM compliance• Customizable alerts & reports, Historic logs

Local Content

VOD

Troubleshoot Most Formats Closed Caption CALM Compliance Video Quality Ad Insertion

E n s u r e c o n t ent q u al i t y a n d i n t e g r i t y f r o m a u n i f i ed m o n i t or i ng p l a t f or m


Monitoring Encoder Boundary Points

• The Encoder Boundary Point (EBP) is a signaling mechanism for taking continuous streams conditioned for adaptive streaming and creating discrete chunks of decodable content.

• Using Orion, services can be grouped representing different bitrate variants of an ABR asset, then EBP information can be compared across services within an ABR group.



Formats and Protocols Supported


Formats Protocols Supported

Interface IP (v4), ASI, SDI

Protocols MPEG-2 Transport, DVB, ISDB, and ATSC compliance

Audio Formats MPEG-1/MPEG-2 Audio, AAC/AAC+ Audio, Dolby AC-3, EAC-3 Audio

Video Formats SD/HD/UHD Video, HEVC, H.264/MPEG-4 Part 10 (AVC), MPEG2 Video

Embedded Data Closed Captioning (EIA-608, EIA-708, SCTE-20, DIVICOM, NADBS), DVB Subtitle, Teletext, SCTE-27, SCTE-35, XDS, Text Services

Network Transmission Protocol

UDP/RTP, Multicast (IGMP v1/2/3)


ORION-OTT Introduction


Software-based Over The Top Monitoring Solution

to help you deliver streaming video with superior QoS and QoE

ORION-OTT | Content Monitor

ORION-OTT | Origin Performance Monitor

ORION-OTT | Edge Performance Monitor


• 24x7 Proactive content monitoring required for early fault detection and resolution• Catch issues before your viewers start reporting problems

• For LIVE services, content monitoring at Origin is a must• Real-time environment, no offline QC possible• Cannot handoff content for delivery without checking

– Content availability and ABR compliance– DRM protection– Audio/Video quality

27

Why is Active Monitoring required ?


• Active monitoring does not provide any information regarding real-user sessions

• Passive monitoring at end-devices is required to understand issues during playback at client devices

• Together, Active and Passive monitoring methods complement each other to • Provide an end-to-end network visibility• Determine fault sources in the network

28

Active & Passive Monitoring – complement each other


ORION-OTT | Content Monitor




DRM

Live

VOD

Video Delivery

CDN

Content Monitor

• Deployed at Origin Server/CDN• Created for OTT service providers, CDN providers, OTT equipment providers• Extensive set of Checks

ABR Package Integrity Audio/Video Quality

Active Probe for Content Monitoring


• Package Integrity and ABR compliance

– URL validation, Manifest validation

– ABR Compliance• Start time and duration consistency across segments• Segment metadata mismatch• First frame should be a Key Frame• And more…

• Encryption

– Supports AES-128, CENC (MPD, HLS, HSS), SAMPLE-AES

– DRMs – Widevine, PlayReady, Fairplay, Verimatrix, BuyDRM, Content Key mapping for other DRMs

• Monitoring Content Quality (QoE)– Video Quality (Blockiness, Black frames, Freeze frames)– Audio Quality (Loudness (CALM, ITU, EBU), Level, Silence)

– Conformance checking– Closed Caption/Subtitle Checks

30

ORION-OTT | Content Monitor - Supported Checks


• In-depth monitoring of Live and VOD OTT services

• Intuitive Web-based User interface using intuitive dashboard and drill-down graphs

• Alarms and Triggers for Actions and Debug– Provides error categorization according to severity levels – Major, Minor, Critical

– Provides different alerting mechanisms – E-mail, SNMP; Real-time alarms for quick decision making

– Triggered and manual recordings for profiles and LIVE content

• Rich Reports and Trend Analysis– Provides graphical representation of content delivery quality

– Provides drill-down monitoring views for assets and services under test

– Provides alerts and data analytics for all tests run over last 60 days

– Global Live Alerts page for LIVE Services

• REST APIs for third-party integrations


ORION-OTT | Content Monitor - Features



ORION-OTT | Content Monitor – Feature Examples


Formats and Protocols Supported


Formats Protocols SupportedPackages Apple HLS, MPEG-DASH, HSS, Adobe HDS

Wrappers MP4, MPEG-2 TS

Audio Formats MPEG, AAC, Dolby AC-3 and EAC-3

Video Formats H.264 / AVC, H.265 / HEVC

Embedded Data Formats Closed Captioning (EIA-608, EIA-708, SCTE-20, DIVICOM, NADBS), DVB Subtitle, WebVTT, TTML, SCTE35, ID3 Metadata, External XML Subtitles

Network Transmission Protocol HTTP / HTTPS, RTMP

Checks Content Integrity Checks, Download Failure and Delays, HTTP Errors, Audio/Video Quality Checks

DRM/Encryption All Encryption schemes (CENC, AES-128, SAMPLE-AES) supported by Google Widevine, Microsoft PlayReady, Apple FairPlay, Verimatrix and BuyDRM




ORION-OTT | Origin Performance Monitor


DRM

Live

VOD

Video Delivery

CDN

Origin Performance Monitor

• Deployed at Origin Server

• Monitors performance based on:

Average response times and bandwidth served

Number of requests served, total bytes served

Server availability - Requests served per requests received

HTTP response codes

• Performs manifest conformance checks

Passive Probe for Origin Server Performance Monitoring




ORION-OTT | Edge Performance Monitor


DRM

Live

VOD

Video Delivery

CDN

Edge Performance Monitor

• Monitors QoS & QoE to understand recurrent issues, their causes and the impact of anomalies on your customer base in real time

• Analytics & business intelligence; Collect user data, derive data consumption trends• Monitors real client user experience: Startup time, buffering ratio, number of stalls, play failures• Data collected from the end-devices can be correlated with active monitoring results using OCM

Passive Probe for Client Device Monitoring (powered by NPAW YOUBORA)


36

How does Youbora’s End-Device Monitoring work?


– Buffer Ratio• Time spent in video buffering vs Play time

– Join Time• Time taken to display first frame of the requested video

– Interruptions• Number of times video stalled

– Play Failures• Could not play video

– Happiness Score• An overall score (1-10) based on all QoE metrics

37

What QoE Metrics are we using ?

WWW.INTERRASYSTEMS.COM 38

CHURN RATE INCREASES BY 50%

PLAYTIME INCREASES BY 2.3X

PLAY ERRORS AT 2% BUFFER RATIO > 2% BUFFER RATIO < 1%

User stickiness of QoE monitored services is 34% higher than non-monitored services

AUDIENCE DROP BY 40%

Impact of QoE on Audience Behaviour

CHURN RATE INCREASES BY 30%

JOIN TIME > 20 SEC.

Source: NPAW data

QU

AL

ITY

BE

NC

HM

AR

K


Transcoders

ORION-OTT Integration – Technology Partners

DRM

Publishing

Analytics

Cloud

ORION

OTT

Google Widevine Aspera

AWS

Azure

Google

Harmonic

AWS Elemental

NPAW

Microsoft PlayReady

Apple FairPlay

Verimatrix

BuyDRM

Wowza

Packagers


Transcoder

Transcoder



DRM

Live

VOD


ORION Central Manager (OCM)

ORION Central Manager

Video Delivery

BATON

ORION ORION

BATON ABR

Live

CDN

ORION-OTTContent Monitor

ORION-OTTOrigin Performance Monitor

ORION-OTTEdge Performance Monitor

• Enterprise-wide visibility

• Single point Control and Monitoring for multiple Linear and OTT probes

• Workflow monitoring for locating error source

• Distributed architecture for flexibility and efficient bandwidth management

Centralized Management


ORION Enterprise Scalability

• Easily scalable for thousands of programs

• Central Manager for enterprise-wide

visibility and multiple ORION

deployments

– Locate error source, Compare programs,

and feeds

• Distributed architecture for flexibility and

efficient bandwidth management

• Reliable and 24/7 monitoring



Quality Matters. Now, More Than Ever.

Reach us at:

Interra Systems, Inc.1601 S. De Anza Boulevard, Suite 212

Cupertino, CA 95014, USA

www.interrasystems.com

Documents

End-to-End Content Monitoring for OTT Workflows– CDN Providers – OTT Equipment Providers • Ensures superior QoS and QoE for Live and VOD OTT services • Checks for: – ABR