Upload
amazon-web-services
View
514
Download
0
Embed Size (px)
Citation preview
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Michael Koetter | VP Digital Media System Turner
Usman Shakeel | WW Tech Lead (M&E) Amazon Web Services
11/28/2016
MAE302
Turner’s Cloud Native Content Supply Chain
for TNT, TBS, Adult Swim, Cartoon Network, CNN
Playout & Distribution
Visual Effects & Editing Analytics
DAM & Archive
Digital Supply Chain
Publishing
OTT
Acquisition
AWS Media Segments
Key Evolutions in our Industry
The Newcomers (Netflix, Amazon) effect
Direct B2C (OTT, Digital)
Know thy consumer and create the content they dig
Cost Crunch on production/post-production
Content has gravity and getting heavier Annual Expenditure on Programming ($B)
“Millennials spend nearly 50% of their time watching movies
and TV shows on devices other than a TV, primarily
laptops/desktops, followed by smartphones and tablets”
Source: Deloitte Digital Democracy Survey 2016
Millions Netflix Subscribers
The New Normal
Media – ALL IN – to the Cloud
Where do I begin?
Media – ALL IN – to the Cloud
De-Coupling
Micro-servicesBuzzword?
Specific scope
Interoperable services
Allow rapid innovation
An inside-out view
Playout & Distribution
Production/Post Production B2B/Cinematic Delivery
DAM & Archive
Digital Supply Chain
Publishing
OTT
Content Acquisition
Ingestion
Manual QC
Auto-QC
Metadata Extraction
Transcoding
Delivery
Packaging
Clips/Edits
Complex workflow/State management
Bursty sub-workflow tasks (transcoding/auto-QC, Packaging)
Elastic Licensing
Content movement between workflow tasks (a common repository)
Different delivery formats for each affiliate and downstream
Manual tasks – (content movement or interactivity)
CMS/DAM
Normalization
Normalization
All-In Cloud-Enabled Domino Effect
Playout & DistributionLive TV
Production/Post
ProductionEditorial
Dailies
Post
Social Media
Marketing/Promo
B2B/Cinematic DeliveryDPP, IMF, NABA
DAM & ArchiveActive Archive
Golden Copy
Preservation
Digital Supply Chain
Publishing
OTTLive-To-VOD
AVOD, SVOD, TVOD
Catchup TV
Cloud PVR
Content AcquisitionLive
File
Ingestion
Manual QC
Auto-QC
Metadata Extraction
Transcoding
Delivery
Packaging
Clips/Edits
CMS/DAM
ETC DI Workflow for Major Studios
An Industry View of Media Supply Chain
Complex workflow
State management
Bursty workflow
tasks (transcoding/auto-
QC, Packaging)
Elastic Licensing
Content movement
Delivery formats for
each affiliate and
downstream
Manual tasks(content movement or
interactivity)
Media Supply Chain (simplified)
Quality Check Create, edit, &
collaborate in the cloud
Distribute content to
playout/OTT solution
Import mastered
content, along with
metadata into S3
Macro Level Ideas/Highlights
• Amazon EC2 New Instances
• Cloud Storage
• Amazon S3, Amazon Glacier (new retrieval tiers)
• Amazon EFS
• Amazon EBS new Volume types and performance
• NLE in the Cloud (Remote Application Streaming)
• Graphics Amazon WorkSpaces
• Microservices/Serverless
• AWS Lambda
• Amazon API Gateway
Media Supply Chain at Turner
Michael KoetterVP Digital Media Systems @ Turner
Stuff for which I’m responsible:
• Content Creation Systems
• Content Supply Chain Systems
• Technology Infrastructure
Roll Video…
OVERALL OBJECTIVE.
We are leading the media industry’s transformation through our approach to technology, data science and innovation.
14
SUPPLY CHAIN MISSION
Connecting CONTENT
PRODUCERS with content monetization PLATFORMS to
drive Turner’s business across all brands, networks
and products.
ETC DI Workflow for Major Studios
An Industry View
D2C
ES
TS
VO
D
`
`
CO
NN
EC
TE
D
DE
VIC
ES
PA
Y T
VB
RO
AD
BA
ND
Linear Network
Origination
Downstream
Processing
Content
Suppliers
SchedulingRightsTitles Ordering
Turner Business Systems
Cloud Media Processing
Turner’s Content Supply Chain
Content Receiving26,000 Items Processed (Jan – Nov 2016)
Direct to Consumer~ 1400 movie initial load
SVOD Initial Load(deal dependent)
Promo Production~ 1000 items per program per agency
Flexibility Analytics Elasticity
SUPPLY CHAIN REQUIREMENTS
Emerging challenges are putting pressure on our traditional processing model.
Flexible Process Engine
Media Micro-Services
Supply Chain Components
Upload Auto QCInput
ProcessingMaster Version
Output
ProcessingDeliver
Cloud Library
Work Order Management (Process Engine)
Content tracking and
flow control.
Integration API for
ODT & Playout
Automation.
Final content
delivery to
distribution systems
Output sequence
conforming, trigger
insertion and
transcode.
Creation of golden library
master, preservation in
library and creation of
versions for specific
platform needsNormalization of
incoming
distribution masters
Automated quality
check of incoming
distribution masters
Content submission
to CSC
Ingest of material into
library’s normalized
media data model
Selection of material
from master asset and
creation of new and
unique sequences.
25
Order Core
Fle
xib
le P
roces
s E
ng
ine
• Fully abstracted
• Instance-less
Order Types
• Rapid Development
• Simplified Environment
• Helper Classes
Order Types
26
Order Core
State
MachineResource
AllocationRules Engine
Order
Scaffold
Generator
HTML
Template
Document
DB
Form Data
UI Shared
WidgetsFle
xib
le P
roces
s E
ng
ine
WHY USE AN INSTANCE-LESS
DESIGN?
to implementSimple
Agile
Available
Scalable
Zero effort to administer
on demand
without ”clustering”
coding practices
Instance-less
Fle
xib
le P
rocess E
ng
ine
Fle
xib
le P
rocess E
ng
ine
Fle
xib
le P
rocess E
ng
ine
AWS
S3
Rules Engine
incl.Lambda Cron
State Management
CSC Work Order Architecture Version 1.3Chris Hinton, 8th Nov 2016
SMTP
AWS
DynamoDB
Client / Chrome
Quark / Topbar
Work Orders App
List / Search / Filter
Work Order Item
CRUD &
Business Logic
JS Lambda
AW
S A
PI G
ate
wa
y
S3 H
TT
PS
Conversations
forms &
utils JS
Other AWS
Document Management
document templates
Audit Log
KPI & Cost
Turner Quark &
Content Apps
Client JS
Turner
Network
Public
Internet
SNS
SDVI
External
Orchestrator
PSP
Fle
xib
le P
rocess E
ng
ine
Media Micro-Services
Upload Auto QCInput
ProcessingMaster Version
Output
ProcessingDeliver
Cloud Library
Work Order Management
Dynamically created infrastructure
Usage based licensing
Media-aware analytics and
optimization
Fine grained media processing as a
micro-service
Media Micro-Services
Media Micro-Services
Content Receiving Workflow
Input files
in S3
S3 Multi-Part
Client
S3 Transfer
Acceleration
EC2 instances
S3 Notification
Trigger a Lambda Function to
update portal and job queue
File Transmission
complete
SDVI
Amazon SNS
Integration
Supplier 1
Supplier 3
S3 Life Cycle Policies for purging temp files
S3 as input/output to different tasks
Supplier 2
Order
fulfillment State
Web Portal
Upload Content
File to S3
Save Order
Amazon Dynamo DB
Order Portal
REST
Client API
Process Rules,
Update State
Turner’s Media Supply Chain in the AWS Cloud
AWS cloud – US East 1
Turner Campus - Atlanta
Workorders
Content
Amazon API
Gateway
AWS
Lambda
Turner VPC
SDVI VPC
ProxyAmazon
Elasticsearch
AWS SNS
Content
Distributors
Turner
Employee
Legacy
Infrastructure
DynamoDB
Macro Implications
Cloud Content Security Strategy
(Considerations)
39
• Traditional on prem content security has been perimeter
based
• Perimeters can be breached
• Risk domains are very large
• Logging is not always robust
• Cloud security affordances enabled Turner’s Information
Security Office to take a favorable position for change
Macro
Im
pli
cati
on
s
Cloud Content Security Strategy
40
• Separate AWS account for Content storage with limited
employee access (and no compute)
• S3 buckets with Bucket Policies limiting Get and Puts
based on Amazon VPC Endpoints (Turner and SDVI)
• Time-limited tokens for SDVI & other vendors to access
pre-air & master quality content
• Bucket Access logging into AWS CloudTrail S3 Bucket
• On-Prem Low-Res Proxy Streaming system limits access
to Turner's network for better cost & performance.
Macro
Im
pli
cati
on
s
Change Control
Continuous Delivery
41
• Media industry is traditionally averse to change.
• ”Set it and forget it” is held as an ideal
• Changes occur infrequently & at the most low impact
times… yet they are in reality always changing.
• Systems have been architected around these principles
• If the business wants agility and flexibility, we need to
change our delivery model
Macro
Im
pli
cati
on
s
Change Control
Continuous Delivery
42
Macro
Im
pli
cati
on
s
Automated
Daily Release
Process
Faster delivery
of value to the
business
Release in
smaller
increments,
control risk
Reduced “Bit
Rot” &
Improved
Developer
Productivity
Change Control
Continuous Delivery
43
Macro
Im
pli
cati
on
s
On Prem
(UI)
Stateful Infrastructure
(EC2, S3, AWS IAM, Dynamo DB)
Stateless Infrastructure
(Lambda, API gateway)
What? How? When?
Challenges Overcome
Ch
alle
ng
es
Ov
erc
om
eJavascript development and
realtime debugging
Out of Order Messages
Tagging Lambda for dynamic
rule-sets
15 second message read-filter
allows all arriving messages to
buffer.
Had to create an EC2 Instance to
proxy requests between lambda &
on-premises
Automated run-time selection
within code
• More Workflows
• Cloud Master Library
• Cloud Editing & Versioning
• Cloud Based Channel Origination
Playout & Distribution
Visual Effects & Editing Analytics
DAM & Archive
Digital Supply Chain
Publishing
OTT
Acquisition
Wh
at’
s N
ext
WANT TO WORK ON
THIS?
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ChrisHintonVP, Product DevelopmentTurner BEST
ChrisFarrisEngineering Manager,Turner BEST
BrigitteMabryMgr, Content LogisticsTurner NATO
Thanks to:
Questions?
Remember to complete
your evaluations!
Related Sessions
Industry Pre-Day: Monday 11/28• MAE303 - Discovery Channel's Broadcast Workflows and Channel Origination on AWS
• MAE304 - High Performance Cinematic Production in the Cloud
• MAE301 - Accelerating the Transition to Broadcast and OTT Infrastructure in the Cloud
Regular Conference: Tuesday – Thursday 11/29 – 12/1• ARC406 - Encoding Artifacts to Emmy Awards: Taking on Terabyte-Scale, 1-Gbps, 4K Video
Processing in the Cloud
• CMP312 - Powering the Next Generation of Virtual Reality with Verizon
• CTD305 - Media Delivery from the Cloud: Integrated AWS Solutions for Premium Over the
Top (OTT) Content
• STG302 - Deep Dive on Amazon Glacier