Seamless service migration with AWS Enterprise Support

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

官順暉 (Drake), KKStream / KKBOX

May 20th, 2016

Seamless service migration with AWS Enterprise Support

Who am I? Who are We?

• Technical Director, KKStream. • Committee member, PyCon TW

• Spin off from KKBOX on March. • Media streaming solutions.

Proclaim #1

“I would like to talk about our journey, starting from migration of video streaming services from on-premises to cloud. What we experienced and learnt with AWS Enterprise Support team would be shared, and then our future strategy with AWS.

No any deep-dives, or even codes gets mentioned on this stage.”

Proclaim #2

“What I’m going to share is our stories in some context. They might not fit to your business challenges, and there might be even something subtly wrong.

Listen carefully with doubt is necessary; adapt them to your challenges with twist is suggested.”

Let’s start from the result

Videopass spec

• Video-on-demand streaming service in Japan. • Modern adaptive streaming: HLS, DASH, MSS. • Multi-platform: Android, iOS, Web, STB and TV. • Seamless migration w/o down time from the previous not-

that-ok platform.

Videopass status

• More than 1M paid subscriptions (Japan only) • 60,000 videos encoded • 3,000 films encoded in one day (peak) • Couple of million video-hours are watched • EC2 instances: 4,000 cores (peak) • Request per month: 3 billion (peak)

Embraced AWS services

• EC2 • S3 • RDS • VPC • ELB • Route 53 • CloudWatch

• AWS Config • Elasticsearch • ElastiCache • SQS

• SNS

• SES

• SWF

• Lambda

• API Gateway

• DynamoDB • EMR

• Redshift

All on cloud from day oneApril 2014

Content Pipeline

• meta, images, videos, and others, a variety of contents we are dealing with.

• Different and even evolving workflows, as DAGs, comprise the core backbone.

• It should be robust and accountable as we just like put 1.5 (video) engineer.

Encoding System

• Amazon Simple Workflow (SWF) • It is accountable, extendable, error-proofing • We proposed priority queuing feature with others • Raising soft limits is easy

• Orchestrating the execution of workflows for media streaming service and even more

• PyCon APAC 2015 • 轉轉轉好運旺來⼀一起來之雲端轉檔⼤大作戰！

• PyCon Taiwan 2016

Counting our core pipeline technology on AWS can be robust and secured, even though it (ex, SWF) is not that mentioned here.

Lesson #1:

D-Day, Videopass migrationJuly 2015

Todo list

• Seamless migration to 1M users. • Migration is split into 2 or 3 phases. • Service status is watched and reported in minutes. • 4+2 vendors to cooperate through API, data exchange

and discussion. (Taipei, Tokyo, London) • Both on-site and remote engineers, operators and even

SQA are standby. (Taipei, Tokyo)

Live status

Everyone was caring about the live status, which meansno one was thinking about others!

AWS IEM

• One AWS SA noticed something risky about RDS!

• He continuously and clearly reported his discovery and discussed with us.

• Obviously, it turns out he is much familiar with underneath services than us!

On d-day, all people is easier concentrating on instant status, while AWS IEM team, as a team player with us, help predict the future and prevent disasters.

Lesson #2:

AWS re:Invent 2015October 2015

After AWS re:Invent, We Thought

• Cost optimization:- development faster, then deployment- quit/change even faster if necessary

• Brainless scalability (PaaS):- serverless microservices- Lambda + API Gateway

• Deeper support:- continuous mutual cooperation- more KKBOX staffs to AWS re:Invent!

https://aws.amazon.com/solutions/case-studies/kkbox/

We could leverage AWS resources to not just evangelize technology and mindset, but also cultivate our next-gen platform and even broaden our exposure.

Lesson #3:

Moving on to the nextJanuary 2016

What is the next-gen streaming big thing?

• Much complicated business logics • FVOD, SVOD, TVOD, AVOD, bundle sale, …

• Higher demand of streaming experience • Viewing quality, picking w/o searching, personalization, …

• Different but rich content • Live music, linear TV, enterprise education, …

• Insane operation situation • Massive ingestion, dynamic layout, auto-subtitle, auto-

screenshot, …

Our strategy for the next

• We just like to have 1.5 AWS expert if possible, or 1 expert with some intermediates.

• We would focus on core part of next-gen services while infrastructure issues are delegated to someone trustful, like AWS.

Serverless APIs on AWS!

Lots of trickys in AWS services

• We might be one of pioneering teams here trying to build fully serverless APIs through API Gateway, Lambda, …

• DynamoDB is really expensive if you use it in this way … • Lambda has cold start issue such that … • Not just CPU, but also network and disk performance are

all positively related to memory in Lambda … • Don’t use CodeCommit or CodePipeline because …

We realized that we won’t be that expertized in some AWS tricks, but we can aggressively start some plan because AWS ES team would fully and honestly support us.

Lesson #4:

And we want more from AWSMarch 2016

“How about putting # of AWS support cases into our KPI?” – interesting feedback

Then, all our team, including non-developers, video content quality assurers in Kaohsiung, has access to AWS Support Center!

Treating AWS as a team player in KKStream can strongly increase our expertise, team size, and even surprising feedback. It’s like we hired the whole AWS ES team in!

Lesson #5:

Overall, the takeaways

Takeaways

• Before evaluating new AWS services, ask them directly. They can always feedback something inspiring, trustful, and accountable.

• AWS Support could be treated like part of your team, instead of just tech Q&A vendor or consultant.

• Thinking about what your team like to be and focus on that, then leveraging AWS resources for other parts.

Have I mentioned that we are hiring?

Thank you! @drakeguan