52
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Scalable Media Workflows in the Cloud Bhavik Vyas - AWS Stephane Houet - EVS Media &Entertainment Product Manager Partner Ecosystem Manager C-Cast

Scalable Media Workflows in the Cloud

Embed Size (px)

DESCRIPTION

This session will cover the approaches for a cloud-based workflow: media ingest, storage, processing and delivery scenarios on the AWS cloud. We will cover solutions for high speed file transfer, cloud-based transcoding, tiered storage, content processing, application deployment and global low-latency delivery, as well as the orchestration and management of the entire media workflow.

Citation preview

Page 1: Scalable Media Workflows in the Cloud

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

Scalable Media Workflows in the Cloud

Bhavik Vyas - AWS Stephane Houet - EVS

Media &Entertainment Product Manager

Partner Ecosystem Manager C-Cast

Page 2: Scalable Media Workflows in the Cloud

Media Workflows are Complex

Featurettes

Interviews

2D Movie

3D Movie

Archive Materials

Stills

Networks

Theatrical

DVD/BD

Online

Mobile Apps

Archive

MSOs

Process

Store

Deliver

Page 3: Scalable Media Workflows in the Cloud

Tools Need to Scale with Media …

Page 4: Scalable Media Workflows in the Cloud

Elasticity

Time tomarket

UndifferentiatedHeavy lifting

Pay as you go

Managed IT Services

Secure

High AvailabilityGlobal Scale

Page 5: Scalable Media Workflows in the Cloud

A Typical Media WorkflowIn

gest

Inde

x

Pro

cess

Pac

kage

Pro

tect

QC

Aut

h.

Tra

ck

Del

iver

Orchestrate

Store

Page 6: Scalable Media Workflows in the Cloud

Where AWS Fits

Amazon Web Services

Inge

st

Inde

x

Pro

cess

Pac

kage

Pro

tect

QC

Aut

h.

Tra

ck

Pla

ybac

k

Media Asset Management

Analytics and Monetization

Page 7: Scalable Media Workflows in the Cloud

Mapping the Workflow to AWS Services

AWS Direct Connect

Elastic LoadBalancing

AWS Import/ Export

Amazon S3

AWS Storage Gateway

Amazon Glacier

Amazon EBS

CloudFront

Amazon CloudSearch

Amazon SNS

Amazon SQS

Amazon SWF

Amazon Elastic

Transcoder

Amazon EC2 Amazon EMR

DynamoDB

Amazon VPC

Amazon RDS

Amazon Redshift

ElastiCache

AMI

Route 53

Ingest Store DeliverProcess/ Manage

Amazon EC2

Amazon EC2

Amazon Workspaces

Amazon Kinesis

Elastic LoadBalancing

Elastic Beanstalk CloudWatch CloudFormation

Page 8: Scalable Media Workflows in the Cloud

AWS SDKs

Java Python (boto) PHP .NET Ruby Node.js

iOS Android AWS Toolkit for Visual

Studio

AWS Toolkit for Eclipse

AWS Tools for Windows

PowerShell

AWS CLI

JavaScript

Page 9: Scalable Media Workflows in the Cloud

INGEST STORE MANAGE SECUREPROCESS

CREATEMONETIZE

INTEGRATEDELIVER

AWS Media Partners: Powering the Workflow

Page 10: Scalable Media Workflows in the Cloud

Media S/W in the AWS MarketplaceAWS Marketplace: Online Software Store

• Find, research, try & buy software• 1-Click launch – application running minutes• S/W Billing fully integrated into AWS account• 1700 products across 25 categories

Developer Tool Categories Include• Media Servers• Transcoding• Media Publishing• Digital Asset Management• Image Resizing

Page 11: Scalable Media Workflows in the Cloud

Let’s Dive Deeper.

Page 12: Scalable Media Workflows in the Cloud

Media Ingest for AWS

AWS Import/ Export

Amazon S3(multi-part Upload)

AWS Storage Gateway Service

AWS Direct Connect

Page 13: Scalable Media Workflows in the Cloud

Media Storage on AWS

AWS Cloud

Amazon Glacier

Gateway Appliance/ AWS Storage Gateway

Amazon S3

Block File

On-premises Data Center

Archive Backup Disaster Recovery

Amazon EBS

Page 14: Scalable Media Workflows in the Cloud

Choosing the Right Storage

>Durability >>Scale >>>Cost

Object Storage - http(s)

Amazon S3

Amazon Glacier

EBS

Elastic Block Storage

Object Storage - http(s)Block (CIFS/NFS)

Page 15: Scalable Media Workflows in the Cloud

EBS Options:• Three types:

– General Purpose (SSD) volumes• Ideal for a broad range of use cases such as boot volumes, small and

medium size databases, and development and test 

– Provisioned IOPS (SSD) volumes• You specify the I/O performance, up to 4000 IOPS per volume

– Magnetic volumes• Ideal for workloads where data is accessed infrequently, and

applications where the lowest storage cost is important. 

• Encrypted Amazon EBS volumes to meet a wide range of data-at-rest encryption requirements

EBS

Elastic Block Storage

Block (CIFS/NFS)

Page 16: Scalable Media Workflows in the Cloud

Reducing Object Storage Costs

Amazon S3

Amazon Glacier

Bucket lifecycle

Policies

Single API access (regardless of the

storage class)

> Durability

>> Scale

>>> Cost Amazon S3 (RRS)

Bucket lifecycle

Policies

<$0.03/GB

<$0.024/GB

<$0.01/GB

Page 17: Scalable Media Workflows in the Cloud

S3 & Working with Large Files

S3 Bucket

S3 SDK

AWS S3

S3 multi-part Upload

Files

corporate data center S3 Object <= 5TB

Page 18: Scalable Media Workflows in the Cloud

S3 Encryption Options

1. Sever-Side Encryption (SSE)– Provides an integrated solution where Amazon handles key

management

2. SSE-Customer (SSE-C)– Perform the encryption and decryption of your objects while retaining

control of the keys used to encrypt objects.

3. Client library (e.g. Amazon S3 Encryption Client)– Objects encrypted before they are sent to Amazon S3 for storage.

Page 19: Scalable Media Workflows in the Cloud

Media Processing

Amazon S3

Instance Store

EC2 Instance

AMI

EBS

Page 20: Scalable Media Workflows in the Cloud

Ensure Flexibility: EBS Backed Instances

AMIEBS

Instance Store

EC2 Instance

SIZE INSTANCE BY APPLICATION NEED

Page 21: Scalable Media Workflows in the Cloud

Size AWS Instance Type to Application

EC2 Instance Type

CPU’smemory

netw

ork

I/O

loca

l sto

rag

e

Page 22: Scalable Media Workflows in the Cloud

Scale Capacity Up & Down as Needed

Elastic

CLOUD capacity

Traditional

IT capacity

Your needs

Time

Ca

pa

city

Page 23: Scalable Media Workflows in the Cloud

Use What You Need, When You Need To

Capa

city

Time

Launch of new device

1000’s of transcoders

Page 24: Scalable Media Workflows in the Cloud

Amazon EC2 Instance Types

2006 2007 2008 2009 2010 2011 2012 2013m1.small

m1.xlarge

m1.large

m1.small

m2.2xlarge

m2.4xlarge

c1.medium

c1.xlarge

m1.xlarge

m1.large

m1.small

cc2.8xlarge

cc1.4xlarge

cg1.4xlarge

t1.micro

m2.xlarge

m2.2xlarge

m2.4xlarge

c1.medium

c1.xlarge

m1.xlarge

m1.large

m1.small

hs1.8xlarge

m3.xlarge

m3.2xlarge

hi1.4xlarge

m1.medium

cc2.8xlarge

cc1.4xlarge

cg1.4xlarge

t1.micro

m2.xlarge

m2.2xlarge

m2.4xlarge

c1.medium

c1.xlarge

m1.xlarge

m1.large

m1.small

cc1.4xlarge

cg1.4xlarge

t1.micro

m2.xlarge

m2.2xlarge

m2.4xlarge

c1.medium

c1.xlarge

m1.xlarge

m1.large

m1.small

c3.large

c3.xlarge

c3.2xlarge

c3.4xlarge

c3.8xlarge

i2.large

i2.xlarge

i2.2xlarge

i2.4xlarge

i2.8xlarge

g2.2xlarge

cr1.8xlarge

hs1.8xlarge

m3.xlarge

m3.2xlarge

hi1.4xlarge

m1.medium

cc2.8xlarge

cc1.4xlarge

cg1.4xlarge

t1.micro

m2.xlarge

m2.2xlarge

m2.4xlarge

c1.medium

c1.xlarge

m1.xlarge

m1.large

m1.small

c1.medium

c1.xlarge

m1.xlarge

m1.large

m1.small

new

existing Entry into GPU space

G2 Instances

Page 25: Scalable Media Workflows in the Cloud

2014 EC2 Instance Highlights• HI1 – 120,000 random read IOPS on SSDs• I2 – Next generation I/O optimized (350,000+ random read IOPS)• M3 – Next generation general purpose• HS1 – 48 TB of storage per instance• CR1 – 244 GiB of RAM &Intel Xeon E5 2670 (Sandy Bridge) CPUs • C3 – 2.8 GHz Intel Xeon E5-2680v2 (Ivy Bridge) CPUs (SSD)• R3 – Next generation memory optimized

Page 26: Scalable Media Workflows in the Cloud

EC2 GPU Instances (G2)

1 NVIDIA GK104 GPU (Kepler / GRID)

2.6 GHz Sandy Bridge CPU w/ Turbo enabled

8 vCPUs, 15 GiB of RAM

60GB SSD storage

Frame Capture and Encoding APIs

g2.2xlarge

Page 27: Scalable Media Workflows in the Cloud

Unix/Linux instances start at $0.02/hour

Pay as you go for compute power

Low cost and flexibility

Pay only for what you use, no up-front commitments or long-term contracts

Use Cases:

Applications with short term, spiky, or unpredictable workloads;

Application development or testing

On-demand instances

1- or 3-year terms

Pay low up-front fee, receive significant hourly discount

Low Cost / Predictability

Helps ensure compute capacity is available when needed

Use Cases:

Applications with steady state or predictable usage

Applications that require reserved capacity, including disaster recovery

Reserved instances

Bid on unused EC2 capacity

Spot Price based on supply/demand, determined automatically

Cost / Large Scale, dynamic workload handling

Use Cases:

Applications with flexible start and end times

Applications only feasible at very low compute prices

Spot instances

EC2 Instance Options

Page 28: Scalable Media Workflows in the Cloud

Managed Database ServicesAmazon

RDSAmazon

DynamoDB

Managed relational database service

Managed NoSQL database service

Amazon ElastiCache

In-Memory Caching Service

Page 29: Scalable Media Workflows in the Cloud

Workflow Building Blocks

Amazon Simple Queue Service

(SQS)

Amazon SWF

Amazon Simple Email Service

(SES)

Amazon Simple Notification

Service (SNS)

Amazon CloudSearch

CloudWatch

AWS CloudTrail

Amazon ElastiCache

Amazon Elastic

MapReduce

Elastic Load

Balancing

Amazon Relational Database

Service (RDS)Process

Store

Deliver

Page 30: Scalable Media Workflows in the Cloud

AWS Media Delivery

Amazon S3

Processed Media Files HTTP/HTTPS

Streaming or Distribution

Amazon CloudFront

Page 31: Scalable Media Workflows in the Cloud

ABR VOD & Live Streaming

Page 32: Scalable Media Workflows in the Cloud

Content: Consumption & Reporting

Amazon CloudFront

Access Logs

Amazon S3

EMR

Amazon Redshift

BusinessSmarts

Page 33: Scalable Media Workflows in the Cloud

Mapping the Workflow to AWS Services

AWS Direct Connect

Elastic LoadBalancing

AWS Import/ Export

Amazon S3

AWS Storage Gateway

Amazon Glacier

Amazon EBS

CloudFront

Amazon CloudSearch

Amazon SNS

Amazon SQS

Amazon SWF

Amazon Elastic Transcoder

Amazon EC2 Amazon EMR

DynamoDB

Amazon VPC

Amazon RDS

Amazon Redshift

ElastiCache

AMI

Route 53

Ingest Store DeliverProcess/ Manage

Amazon EC2

Amazon EC2

Amazon Workspaces

Amazon Kinesis

Elastic LoadBalancing

Elastic Beanstalk CloudWatch

AWS CloudFormation

Page 34: Scalable Media Workflows in the Cloud

INGEST STORE MANAGE SECUREPROCESS

CREATEMONETIZE

INTEGRATEDELIVER

AWS Media Partners: Powering the Workflow

Page 35: Scalable Media Workflows in the Cloud

Media Workflows in the Cloud:Customer Examples

Page 36: Scalable Media Workflows in the Cloud

in the Cloud

AKA “Video Factory”

Page 37: Scalable Media Workflows in the Cloud

Video Factory – Workflow

SDI Broadcast Video Feed

x 24

Playout Data Feed

Broadcast Encoder

Live Ingest Logic

Amazon Elastic Transcoder

ElementalCloud

DRM

QC

Editorial Clipping

MAM

Amazon S3Mezzanine

Time AddressableMedia Store

Amazon S3Distribution Renditions

RTPChunker

Transcode Abstraction

Layer

Mezzanine

Playout Video

Transcoded Video

Metadata

SMPTE Timecode

Mezzanine Video Capture

Page 38: Scalable Media Workflows in the Cloud

• The UK’s biggest video & audio on-demand service– And it’s free!

• Over 7 million requests every day– ~2% of overall consumption of BBC output

• Over 500 unique hours of content every week– Available immediately after broadcast, for at least 7 days

• Available on over 1000 devices including– PC, iOS, Android, Windows Phone, Smart TVs, Cable Boxes…

• Both streaming and download (iOS, Android, PC)

• > 20 million app downloads

Sources: BBC iPlayer Performance Pack August 2013http://www.bbc.co.uk/blogs/internet/posts/Video-Factory

Page 39: Scalable Media Workflows in the Cloud

www.evs.com

POWERING MULTIMEDIA CONTENT DELIVERYAT 2014 FIFA WORLD CUP BRAZIL™

Page 40: Scalable Media Workflows in the Cloud

EVSSPORTS

June 12 - July 13 2014- 12 Stadiums

- 64 Games

IBC1 in Rio de Janeiro

EVS appointed for multilateral production- On site2 live production- IBC file-based video management- Multimedia production and distribution

FIFA WORLD CUP 2014

Notes:1. The International Broadcast Centre (IBC) is a temporary hub for broadcasters during major sport

events.2. Remote broadcast television studio where the event is produced and then beamed to

broadcasters via satellite.

Page 41: Scalable Media Workflows in the Cloud

EVS ENABLED A NEVER-SEEN-BEFORE MULTIMEDIA EXPERIENCE

6 Live Streams

HLS streaming of 6 HD streams to tablets & mobiles per match

24 Replay cameras

On demand replay of selected events from any of the 20+ cameras on the field.

+4000 VoD elements

On demand multimedia exclusive edits.

Page 42: Scalable Media Workflows in the Cloud

EVSSPORTS

Page 43: Scalable Media Workflows in the Cloud
Page 44: Scalable Media Workflows in the Cloud

Max of 12 streams at 10Mbps each.

Max of 36 streams at 3,5-10Mbps

9 HLS encoding, from 0.4 to 3.5Mbps + 2 thumbs 1 RTMP

EVS C-CAST LIVE STREAMING WORKFLOW

Page 45: Scalable Media Workflows in the Cloud

EVS C-CAST VODWORKFLOW

9 HLS encoding, from 0.4 to 3.5Mbps + 2 thumbs1 RTMP

Page 46: Scalable Media Workflows in the Cloud

EVSSPORTS

Cloud PlatformLinking mobile trucks, studios and viewersContent selection and approval processMetadata editing stationAutomatic translations in 20+ languagesAutomatic HD backup on topWorkflow engine, including- VoD file transfers and transcoding management- Live streaming activation and transcoding management

Output: API mode or white label embedded

EVS C-CAST MANAGES ALL FROM PRODUCTION TO VIEWERS

Page 47: Scalable Media Workflows in the Cloud

MULTIMEDIA PRODUCTIONHBS OVERVIEW MAP

Page 48: Scalable Media Workflows in the Cloud

MULTIMEDIA PRODUCTION IN THE AWS CLOUD

Aspera for transfers into S3Store and deliver data using S3Create image thumbnails (EC2, SQS & S3)Transcode VoD files using ZencoderTranscode Live using Elemental Cloud- Up to 128 concurrent live streams encoded

Operator GUI for Rio and international sites- S3, Cloudfront, EC2 and RDS

Manage all workflows at IBC and Venue- EC2, SQS & RDS

1Gbps Direct Connect from Rio to Dublin

Page 49: Scalable Media Workflows in the Cloud

EVSSPORTS

6 MULTILATERAL STREAMS

17 INDIVIDUAL MRL STREAM

4 INDIVIDUAL COMMENTARY

243 DIFFERENT LIVE STREAMS /GAME

2,799,360 MINUTES OF ENCODED STREAMS

LIVE STREAMING

Page 50: Scalable Media Workflows in the Cloud

MULTI-ANGLE CLIPS

On average 60 CLIPS RECORDED PER MATCH, overall

UP TO 20,000 MULTILATERAL CLIPS throughout the tournament

Roughly a third of clips from multicam with 14 DIFFERENT ANGLES on average

Page 51: Scalable Media Workflows in the Cloud

COMBINED DATA FOR FIFA WORLD CUP WEB PLAYER AND APP AFTER 64 MATCHES FOR ALL MARKETS

TOTAL HOURS WATCHED:

>15 MILLION

UNIQUE USERS: >35 MILLION

Available in more than 10 TERRITORIES

Peak ACTIVE CONCURRENT USERS

500,000

Page 52: Scalable Media Workflows in the Cloud

FIN & THANK YOU!

Hall #3.B16 

Hall #7.G30 

Hall #4.B75 

Hall #8.B90

Hall #5.B20