© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Michael Raposa, Principal Consultant, AWS

Brinton Miller, SVP, Discovery Communications

Dave Duvall, SVP, Discovery Communications

November 2016

MAE303

Discovery Channel’s Broadcast Workflows and Channel

Origination on AWS

What to expect from the session

M&E on AWS

AWS Professional Services overview

Broadcast on AWS

Discovery’s journey to AWS

AWS adoption

Content ingest & processing

Broadcast playout

Playout & Distribution

Visual Effects & Editing Analytics

DAM & Archive

Digital Supply Chain

Publishing

OTT

Acquisition

AWS Media Workloads

What is broadcast playout?

• Transmission of content

• Typically over satellite (broadcast)

• From broadcaster to affiliate (and ultimately

to the audience, e.g., set-top box)

• Largest generator of company

revenue

AWS Professional Services:

Accelerating the enterprise

IT transformation

AWS Professional ServicesPartnering in your journey

Technical

specialists

Specialty practices for

AWS skills transfer,

security, infrastructure

architecture,

application

optimization, analytics,

big data, and

operational integration

Advisory

services

Portfolio strategy and

planning, cost/benefit

modeling, governance,

change management

and risk management

as it relates to

implementing the AWS

platform

Collaboration

Working together with

you and APN Premier

Partners you already

trust to provide you

with access to all

resources needed to

realize breakthrough

results

Proven process

Best practices and

patterns to help your

teams get the

foundation right, deploy

and migrate workloads,

and create a modern IT

operating model to

support your business

Focus on every aspect of cloud adoption

Prescriptive guidance and implementation

assistance across the stack:

• IT transformation leads focus on the business

aspects of cloud adoption

• Application architects help developers build

AWS-aware applications

• Big data/analytics specialists help you design

and deploy your analytics platform

• Infrastructure architects take your unique

landscape of application, network, and security

needs to the cloud

• Security leads provide well-proven designs to

meet your compliance needs

• Operational integration specialists help you

successfully adopt DevOps practices

Application

Optimization

Big Data & Analytics

Infrastructure

ArchitectureS

ecu

rity

, R

isk &

Co

mp

lian

ce

Op

era

tio

nal

Inte

gra

tio

n

IT Transformation

Broadcast on AWS

Broadcast on AWS overview

Pre-rendered content

• Video

• Graphics

• Overlays

• Ads

• Etc.

Storage

Scheduler

GPU Playout

Live assembly of content

• Download all the pre-rendered pieces based on schedule

• Live assembles the pieces

• Outputs the stream

Injects schedule

into playout

Downloads

content

On-premises DC

50 mbps transmission

Affiliate

Users Users Users

Affiliate Affiliate

Broadcast challenges on AWS

• Live transcoding

• Designed for 99.999% availability

• Migration trifecta

• Hardware to software

• Bare metal to hypervisor

• On-premises to cloud

• Massive scale

• TCO

• Fixed bitrate output – NOT OTT ABR

NFS storage on AWS

• On-premises video assets stored on NFS

• Minimize changes to AWS

• EFS (Preview) performance testing

• 6.1 GBps read & 3.1 GBps write

• 375 clients

• “Three AZ replication is not cost effective for our

high-availability architecture.”

–DiscoveryAmazon EFS

NFS storage on AWS

• Try Intel’s Lustre File System

• Lustre performance testing

• 6.1 GBps read & 3.1 GBps write

• 375 clients

• 0.0 dropped frames per 1M

• “We prefer a managed storage solution.”

–Discovery

NFS S3 storage on AWS

• Switch to S3

• New file system interface

• Refactoring

• S3 performance testing

• 19 GBps read & 20 GBps write

• 500 clients

Amazon

S3

NFS S3 storage on AWS

• S3 is TOO FAST!

• Afraid that it will conflict with 50 mbps out

• Wrote S3 rate limiter

• Control download/upload speed to S3

• Token bucket algorithm

• Multi-threaded, byte-range request for high

throughput

• GitHub Gist: http://bit.ly/2e1Nxf1

Amazon

S3

NFS S3 storage on AWS

• We need to replicate content across regions

• S3 cross-region replication – automated, fast, &

reliable

• Is it fast enough? Let’s test it:

• Single file xFer US-East to/from Dublin: 31 MBps

• Aggregate throughput (multiple xFers): 103

MBps

• 2,993 xFers with 0 failures

Amazon

S3

Clustering MySQL for Auto Scaling

• Using MySQL with Galera for multi-master

replication

• How do use Auto Scaling where

“instances” have different roles?

• Master

• First node in Auto Scaling group

• Starts a cluster

• Slave

• Second+ node in Auto Scaling group

• Joins an existing cluster

• Common pattern with other software, e.g.

RabbitMQ

Clustering MySQL for Auto Scaling

First

instance

in Auto

Scaling

group

Get instance ID

from instance

metadata

Using instance ID

determine Auto

Scaling group

membership

AWS API

Promote to master

Second+

instance

in Auto

Scaling

group

Get instance ID

from instance

metadata

Using instance ID

determine Auto

Scaling group

membership

Get IP of other

cluster nodes in

Auto Scaling group

Contact master &

join the cluster

curl http://instance-data/latest/meta-data/instance-id

Extracting technical metadata from S3 object

• Video assets being uploaded to an S3

bucket

• How do you quickly get the technical

metadata about the asset, e.g., bit rate,

audio codec, aspect ratio, etc.

• Use Lambda and Mediainfo

• Blog: Extracting Video Metadata Using

Lambda and Mediainfo

• Extracting Video Metadata using

Lambda and Mediainfo

S3 bucket

AWS

Lambda

Cross-account auditing

• Third party needs access to your AWS

account

• Ex: managed service provider

• You want to limit and audit access

• Solution: cross-account IAM roles

• But the IAM role looks “shared” – How

do I audit access?

3rd Party Account

Cross-account auditing

• Use CloudWatch Logs to audit

• Blog: How to Audit Cross-Account

Roles Using AWS CloudTrail and

Amazon CloudWatch Events

• NEW! - Shared EventIDs

• AWS CloudTrail Now Tracks

Cross-Account Activity to Its Origin

3rd Party Account

22

Video removed for uploading; re-insert in

Speaker Ready Room

23

The big opportunity…

Transition Discovery’s supply chain

and business systems to a scalable

and flexible infrastructure to improve

speed to market, increase quality of

services, and reduce costs

Storage and infrastructure resources providing an architectural platform

Content distribution

Data mgmt

Media mgmt

Business systems

24

Global scale

• 6 data centers

around the globe• Sterling, VA

• Silver Spring, MD

• Miami, FL

• London UK (2)

• Paris

• Singapore

25

Why cloud? Why now?

Challenging information

Security threats

Growing business and commercial

opportunities, new platforms

Changing technology environment

Global company with

global supply chain

26

Evaluation process

Educate & evaluate Dogfood/foundational

Cloud Tiger team

AWS architecture workshop

Financial case & scale

Foundational design

Cloud native app build

Hybrid app build/lift and shift

Financial analysis

Migration Factory

Media vendor research Monitoring & compliance

27

Our view of the world: What can we move?

Linear delivery

StorageDelivery encoding

Post/ custom Playout

Transcode

Incoming content

Tech eval

Yellow light:• Industry newly defined• Discovery in POC

Green light: • Industry-proven• Discovery tested

Red light:• Industry to be defined• Discovery untested

HR systems

DR/data center

Finance systems

Collabora-tion tools

Ad sales systems

Affiliate systems

Office and portal tools

Content systems

MEDIA TECH CORP IT

Key

28

What workflows to move first?

• Corporate IT migration will be a combination of lift and shift and refactoring apps. Focus is 80%+ over the next year

• Content pipeline has already moved.

• Cloud delivery has allowed us to normalize content delivery globally.

• Additional media workflows required content library in AWS.

• Linear playout starts moving now.

• 150 channels over the next 12 months.

• Required significant development by DCI and our vendors.

• Post Production – Soon.

• Content volume is a challenge.

• Software licensing is a challenge.

• B2B distribution (reduction of satellite footprint).

• Vendors are not motivated to change.

• POCs start in March.

29

Network design for media

Bandwidth & interconnection

Security & inspection

Timing/latency

Resiliency/redundancy

Operational needs

Risks

30

Network design for media

Bandwidth and interconnection

Redundant colocation Direct Connect sites

Playout traversing back to on-premises uplink

facilities

Timing and latency

PTP is a must for real-time video traffic

Account for latency to resync primary and

backup streams for playout

31

Network design for media

Resiliency/redundancy

Redundant DirectConnect sites (Equinix,

CoreSite)

Juniper carrier grade routing design

(MPLS, Advanced QoS)

Security and inspection

VRFs used to isolate video traffic, SGs

used for service segmentation

Service-based routing - only subset of

traffic inspected by layer 7 firewall

Project On Ramp:

Moving our media supply chain

33

Traditional delivery pipeline

Tape-based, labor-intensive

Delivery to air in weeks/months

34

Cloud content factory

• Data-driven content

workflows

• Builds redundant

content repository in the

cloud

• Scale out infrastructure

eliminates needs to build

to ceiling

• Allows us to make

workflow decisions

based on the economics

of manufacturing in real

time

35

Media Supply Chain

Inventory systems

and asset managementInfrastructure and resource

management

Video

processing:

file check, AV,

QC, Transcode

Antivirus

Content and status messaging to

regional MAMs

File delivery to S3

Media storage/cloud archive

Management

layer

File delivery

Storage

Linear playout

37

Legacy hardware

NVISIONROUTER

OMNEON SD/HD PORTS

HY/BHB MAIN

S&W QU ASARUPCONV

EVERTZDWN

MIXER

EVERTZDWN

MIXER

MIRANDAIMAGESTORE

IS750

INTUITION XG

EVERTZGPI

EMBEDDER7721DE4H

EVERTZ 9084HD

XD SVCHIP

INSERTER

EVERTZ 9084HD

XD SVCHIP

INSERTER

OMNEONDELAY

SERVERADVANCED

FEED

OMNEONDELAY

SERVERADVANCED

FEED

EVERTZ9084HDCC ENC

EVERTZ9084HDCC ENC

MIRANDAIMAGESTORE

IS750

INTUITION XG

MIRANDAIMAGESTORE

IS750

INTUITION XG

MIRANDAIMAGESTORE

IS750

INTUITION XG

EVERTZGPI

EMBEDDER7721DE4H

EVERTZGPI

EMBEDDER7721DE4H

EVERTZGPI

EMBEDDER7721DE4H

S&WQU ASARUPCONV

OMNEON SD/HDPORTS

HY/BHY BACKUP

S&WQU ASARUPCONV

EVERTZDWN

MIXER

EVERTZDWN

MIXER

S&WQU ASARUPCONV

HD PGM PVW

PGMPVW

LIN EAR ACOU STICAERO.2000UPMIXER

(FUTURE)ARBITRON

ARBITRON

EVERTZFRAMES YNC

SCTE104VANC EMB

HD ADV CC HD ADV

PGM REC

HD ADV PGM

HD RLTPGM

EVERTZGPI

DE-EMB7721DD4H

ARBITRON WHAK-IT

ROSSNIELSEN

ENCODERNEW-3GA

EVERTZGPI

DE-EMB7721DD4H

RLT PGM

ADV PGMREC

ADVPGM

HD PGM OUT

ADV CC

7800AD8-3G+DMX

EVER

TZ 1

2x2

R

OU

TER

IN 01

IN 02

IN 03

IN 04

IN 05

OUT 1

OUT 2

IN 02

IN 04

IN 05

IN 01

IN 03

EVER

TZ 1

2x2

R

OU

TER

OUT 1

OUT 2

4 x 1 BYPASS

IN 01

IN 02

IN 03FROM ADV

T O W E ST

IN PUT PATCH

20

22

21

1 91816

PGM OUT

Marsha lMonitor

TO ADV MASSLOGGER

TO ADV MASSLOGGER

TO QC1 PATCH

TO QC1 PATCH

EVERTZAUDIO

DE-EMBEDDER

DTMF DECODER

TO WEST INPUT PATCH

ROUTER

ROUTER

1

7

8

10

11

9

14

54

19

18

15 16

4 5

10

7

8

9

11

1 4

1

17

TO MOTOROLA

TO MOTOROLA

TO DEM

20

22

21

TO MOTOROLA

TO MOTOROLA

EAST COASTSIMULCASTAERO2000

SIGNAL FLOW3/10/14

SD eq uipment

RLT

ADV

MVP view

TOC MVP only

HD equipment

AUDIO

• Drawing to the right

represents a single

channel/air chain

• Each of the 64 icons

represents a device

• Represents 100K in cap

investment

• Takes months to deploy

• Support complexity

• Exiting equipment is end of

life and end of support

38

Why playout in public cloud?

• Motivation to go fast was burning technology and real estate issues in a few facilities around the globe

• How do we support the fast changing world of Discovery?

• Stand up 6 feeds for 2.5 weeks every other year in support of the Olympics

• Support for new digital products

39

Challenges getting there

• Traditional media vendors

• Most have not embraced software only solutions.

• Those that have still focused on “private cloud”

• Industry challenges

• Measurement tech slow to port to software.

• Economic challenges

• Shift from capex to opex

• Must go all-in for a great financial story

• Public cloud architecture challenges

• Networking – egress, multicast support, storage, SLAs or lack of

• Freaks the hell out of the staff

40

Move key functions to software

Video

Server

Video

Mixer

Graphics

Insert

V-chip(Parental

ratings)

Delay

Server

Video

Switcher

GPI Box(MVPD

commercial

insertion)

Audio

Mixer

Nielsenwatermark

CC/Subtitle

EncodeDistribution

EncoderStat

Mux

Sat

Uplink

Hardware Layer

Software Layer

Distribution Infrastructure Remains Hardware

Housed in Discovery Data Centers

Video

Server

Video

Mixer

Graphics

Insert

V-chip(Parental

ratings)

Delay

Server

Video

Switcher

GPI Box(MVPD

commercial

insertion)

Audio

Mixer

Nielsenwatermark

CC/Subtitle

Encode

41

Move to cloud

Vide

o

Serv

erVideo

Mixer

Grap

hics

InsertGPI

Box(MVPD

commercial

insertion)

V-

chip

Dela

y

Serv

erVideo

Switc

her Audi

o

Mixe

rNiels

enwater

markCC

Enc

US – East

CoastEU - Ireland

Vide

o

Serv

erVideo

Mixer

Grap

hics

InsertGPI

Box(MVPD

commercial

insertion)

V-

chip

Dela

y

Serv

erVideo

Switc

her Audi

o

Mixe

rNiels

enwater

markCC

Enc

Distribution

EncoderStat

Mux

Sat

Uplink

42

COLO/TELCO/VRF

“Air chain”

US EAST REGION

AZ #1 AZ #2 AZ #3

EU WEST REGION

AZ #1 AZ #2 AZ #3

Unicast to Multicast

Conversion

Unicast to Multicast

Conversion

IP To Video

Conversion

IP To Video

Conversion

2X1Distribution

EncodingStat Mux

US Region

EU Region

43

COLO/TELCO/VRF

Broadcast reliability

AZ #1 AZ #2 AZ #3 AZ #1 AZ #2 AZ #3

Unicast to Multicast

Conversion

Unicast to Multicast

Conversion

IP To Video

Conversion

IP To Video

Conversion

2X1Distribution

EncodingStat Mux

US Region

EU Region

US EAST REGION EU WEST REGION

44

COLO/TELCO/VRF

Broadcast reliability

US EAST REGION

AZ #1 AZ #2 AZ #3

EU WEST REGION

AZ #1 AZ #2 AZ #3

Unicast to Multicast

Conversion

Unicast to Multicast

Conversion

IP To Video

Conversion

IP To Video

Conversion

2X1Distribution

EncodingStat Mux

US Region

EU Region

AZ #4

45

COLO/TELCO/VRF

Broadcast reliability

AZ #1 AZ #2 AZ #3 AZ #1 AZ #2 AZ #3

Unicast to Multicast

Conversion

Unicast to Multicast

Conversion

IP To Video

Conversion

IP To Video

Conversion

2X1Distribution

EncodingStat Mux

US Region

EU Region

US EAST REGION EU WEST REGION

46

COLO/TELCO/VRF

Broadcast reliability

AZ #1 AZ #2 AZ #3

Unicast to Multicast

Conversion

Unicast to Multicast

Conversion

IP To Video

Conversion

IP To Video

Conversion

2X1Distribution

EncodingStat Mux

US Region

EU Region

EU WEST REGION

AZ #1 AZ #2 AZ #3

US EAST REGION

47

COLO/TELCO/VRF

Broadcast reliability

AZ #1 AZ #2 AZ #3

Unicast to Multicast

Conversion

Unicast to Multicast

Conversion

IP To Video

Conversion

IP To Video

Conversion

2X1Distribution

EncodingStat Mux

US Region

EU Region

EU WEST REGIONUS WEST REGION

AZ #1 AZ #2 AZ #3

48

Monitoring

What’s next

5050

This is just European distribution…….

51

• B2B distribution

• Statistical Multiplexing as a Service?

• MVPD support for come meet me approach? Cloud or colo?

• Similar but different egress issues

• Encryption: Do we need industry-specific encryption tech?

Thank you!

Remember to complete

your evaluations!

Industry Pre-Day: Monday 11/28

• MAE304 - High Performance Cinematic Production in the Cloud

• MAE302 - Turner's cloud native media supply chain for TNT, TBS, Adult Swim, Cartoon Network, CNN

• MAE301 - Accelerating the Transition to Broadcast and OTT Infrastructure in the Cloud

Regular Conference: Tuesday – Thursday 11/29 – 12/1

• ARC406 - Encoding Artifacts to Emmy Awards: Taking on Terabyte-Scale, 1-Gbps, 4K Video Processing in the Cloud

• CMP312 - Powering the Next Generation of Virtual Reality with Verizon

• CTD305 - Media Delivery from the Cloud: Integrated AWS Solutions for Premium Over the Top (OTT) Content

• STG302 - Deep Dive on Amazon Glacier

Related Sessions