Cloud First: New Architecture for New Infrastructure

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

KJ Wu (吳貴融), Solutions Architect, AWS

Leo Wang, Staff Engineer, Trend Micro

2016/05/20

Cloud First: New Architecture for

New InfrastructureCustomer Sharing: Migration Story of Trend Micro File

Reputation Service

Agenda

• What is your Cloud Journey?

• Cloud well-architected patterns

• Customer Sharing

• Migration Story of Trend Micro File Reputation Service

The New Normal

Cloud First

Object Oriented

Programming

Client Server

Architecture

Service-oriented

Architecture

(SOA)

The journey we’re seeing with AWS customers

Dev & Test True Production Mission Critical All-in

Build production apps

Migrate production apps

Marketing

Build mission-critical apps

Migrate mission-critical apps

Development and

test environments

Corporate standard

This is not your focus

And focus on your core mission

Lower the time spent

on infrastructure

Dedicate more

resources to

innovation

Concentrate on

new business

initiatives

“Our goal is to move at the speed of business. Our customers’ needs change

constantly, and we need to be able to adapt to that.”

Keith Homewood – Cloud Product Owner, Nordstrom

Pillars of Well-Architected Framework in Cloud

Security Reliability Performance

Efficiency

Cost

Optimization

Security

The ability to protect information, systems, and assets

while delivering business value through risk assessments

and mitigation strategies.

• Data protection

• Privilege management

• Infrastructure protection

• Detective controls

Well-Architected Cloud Patterns

Build Security into every layer

• Encrypt data in transit and

rest between application tiers

• Enforce principle of least

privilege across every service

• Protect your critical resources

from application-layer and

network attacks

Amazon

VPC

AWS Direct

Connect

AWS IAM AWS KMSAWS WAF

AWS

CloudTrail

Reliability

The ability of a system to recover from infrastructure or

service failures, dynamically acquire computing resources

to meet demand, and mitigate disruptions such as

misconfigurations or transient network issues.

• Foundations

• Change management

• Failure management


Design for failure and nothing will fail

App

Server

Database

Server

• Applications should continue to

function even if the underlying

application component fails,

communication is lost or physical

hardware fails, is

removed/replaced.

• Auto-healing & Monitoring

It also means High Availability

Amazon

RDSRDS DB

instance

read

replica

Amazon

CloudWatch

Multi-AZ


Leverage global infrastructure

(Multi-AZ and Multi-Region)

Web Instance

RDS DB Instance

Active (Multi-AZ)

Availability Zone #1 Availability Zone #2

Web Instance

RDS DB Instance

Standby (Multi-AZ)

ELB

Balancer

User

Amazon

Route 53

Oregon Region

Tokyo Region


Loose coupling sets you free

• Design architectures with

independent components

• Design every component as a

black box

• Load balance clusters

Amazon

SQSElastic Load

Balancing


Become Stateless

• Don’t store state in server

• Leverage services to hold

state information

• Application functions

regardless of which

application node processes

the request

Amazon

DynamoDB

Amazon

ElastiCache

Memcached

Redis

Performance Efficiency

The ability to use computing resources efficiently to meet

system requirements, and to maintain that efficiency as

demand changes and technologies evolve.

• Compute

• Storage

• Network


Embrace Elasticity & Automate

• Auto Scaling sets you free on

guessing your application

capacity need

• Automate installation and

configuration of environment

• Automate with CI/CD toolsAuto Scaling

AWS

CodeCommit

AWS

CodeDeploy

AWS

CodePipelineAWS

OpsWorks


Leverage different storage options

• RDBMS only? Do you need

data warehouse?

• Key-value data only?

• Need a scalable object

storage?

• Read/write ratio of your data

request?

Amazon

Glacier

Amazon

S3

Amazon

DynamoDB

Amazon

ElastiCacheAmazon

RDS


Think Parallel

Hour 1 Hour 2 Hour 3 Hour 4

• One Server working for Four

hours costs the same as Four

servers working for One hour

• Combine with elasticity to

increase capacity when you

need it most

• MicroServices architecture

Amazon ECS Amazon ECR

Cost Optimization

The ability to avoid or eliminate unneeded cost or

suboptimal resources.

• Matched supply and demand

• Cost-effective resources

• Expenditure awareness

• Optimizing over time

Cost Optimization – Common Practices on AWS

Monitoring to

collect and

track metrics

Consolidated

Billing &

Detailed

Billing Reports

Tag your

resources

Design

Architecture

with the right

services

On-Demand,

Reserved and

Spot Instances

Cloud Architecture KEEPS Innovating….

Serverless Architecture

No servers by managed services

SOA design & stateless function

Event-driven

Integration with the third

party services

Shift focus to Richer user experience

The real example of Serverless architecture

My Customer Service

Motivation for listening to

social media

Twitter API Amazon

Kinesis

AWS

Lambda

Amazon

Machine Learning

Amazon

SNS

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Leo Wang – FRS Infra.

Staff Engineer

5/26/2016

Trend Micro FRS Infra.

Migration Experience Sharing

Agenda

• Introcution

• Why AWS?

• Migration Experience

• Data Migration Experience

• Application Migration Experience

Enabling a Smart Protection

Strategy

Inspired to Stay a Step Ahead

Global Threat Intelligence from

the Smart Protection Network

Backed by Global Research

and Support

Fast Facts

Founded: 1988, United States

Headquarters: Tokyo, Japan

Number of Employees: 5,258

File Reputation

Files

Process

Solution

Personal Profile

• Leo Wang

• Trend Micro File Reputation Service Team

• Staff Engineer

WHY AWS?

Data Grows and Grows

0

200

400

600

800

1000

1200

1400

Total Storage Needed

2017 Q1

2019 Q4

HA?

DR?

Stable?

Easy Extend?

Data Temperature?

200%

Growth

We Need Scaling

0

500000

1000000

1500000

2000000

2500000

Daily Sourcing Samples

Max

Capacity

We Need Flexibility

Request / Time Effort On-Premise AWS

A Machine 10~30 Days 5 Minutes

PoC Environment 1~3 Months 10 Minutes

DR Site 6~13 Months 30 Minutes

DATA

MIGRATION

EXPERIENCE

AWS cloudcorporate data center

File

Metadata

AWS Data Migration Solution

DO BY OUR OWN WAY

Amazon

S3

Amazon

Glacier

AWS Import/

Export Snowball

import/

export

AWS Storage

Gateway

Amazon

RDSAWS Database

Migration Service

Service to Service

corporate data center

File

Metadata

Storage Service on top of AWS

S3

Glacier

Dynamo

Hot File

Storage

Cold File

Storage

Metadata

Database

Common

Storage

Webservice

Availability Zone A

Availability Zone B

Internet

Gateway

ELB

Service to Service

Common

Storage

Webservicecorporate data center


APP / Service

On AWS

File

Metadata

APPLICATION

MIGRATION

EXPERIENCE

Fully Leverage Cloud

Highly Scalability

Highly Flexibility

Re-Design

And

Re-Write

EVERYTHING!

On-Premise Design

Analytic

Service


Worker

Worker

Worker

Limited

Queue

Size

Limited

Number

of

Machines

Throttling

to Prevent

Overloaded

Limited

Machine

Spec

Flexible and Scalable

Analytic

Service


Worker

Worker

Worker

Throttling

to Prevent

Exceed

Budget

Plan

Dynamic

Machine

Spec

Unlimited

Queue

Size

Auto-

Scaling

RD’s Choice

Analytic

Service


Worker

Worker

Worker

Machine Spec

# of

Machines

# of Clusters

Buffer

Size

Size of Historical

Data

HOW TO MAKE DECISION?

COST CONSCIOUS DESIGN

On-Premise

Server $500 / monthRack+Power $130 / monthNetwork $100 / month * Mbps

AWS

All Prod STG,DEV, DR EC2 RDS DataTransfer S3 ElastiCache Glacier

System A6,000 5,000 1,000 3,000 2500 100 - 400 -

System B7,000 6,500 500 2,000 - 500 1500 - 3,000

System C1,000 400 600 900 50 10 - 40 -

View of Cost

Analytic

Service


Worker

Worker

Worker

$3000

$30

$10

$600

2M

Requests

Change of Mindset

RD’s View:

• Every resource’s cost is clear

• Cost defines design

• Manager’s View:

– Every system’s/request’s cost is clear

– Cost defines business scope

Experiences from

• Enterprise Session

• 1:00 – 1:40pm

FRS Migration Experience

• Deep Drive on AWS session

• 3:30 – 4:10pm

DevOps at FRS TrendMicro

• IoT & Big Data session

• 3:30 – 4:10pm

Analytic Engine - A common Big Data computation service on the AWS

Thank You

Technology

Cloud First: New Architecture for New Infrastructure