Edge and Hybrid Storage - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS_Summit_2018/June… · •Low-cost: Simple licensing, predictable storage costs, and reduced

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Everett Dolgner

Senior Storage BD Manager, Global Platform BD - Today Storage Specialist, Solutions Architect EMEA – Starting 2 July 2018

Edge and Hybrid Storage


Data movement

Onl

ine

Off

line

Data security and management

Complete Set of Data Building Blocks

Amazon EFS

Amazon EBS

AWS Snow family

AWS Storage Gateway Family

AWS Direct Connect

Amazon EFS File Sync

Amazon S3Transfer Acceleration

Storage Partners

Amazon Kinesis Data Streams

Amazon Kinesis Video Streams

Amazon S3

Amazon Glacier

AWS KMS

AWS IAM

AWS CloudWatch

AWS CloudTrail

AWS CloudFormation

AWS Lambda

Amazon Macie

AWS QuickSight


Storage Gateway Hybrid Storage SolutionsUse standard storage protocols to access AWS storage

AWS StorageGateway

Amazon EBS snapshots

Amazon S3 & S3 –infrequent access

Amazon Glacier

AWS Identity and Access Management (IAM)

Amazon CloudWatch

Files

Volumes

TapesO

n-pr

emis

es

AW

S Cloud

AWS Key Management Service (KMS)

AWS CloudTrail


A Few of Our Customers


AWS Storage Gateway Amazon S3 Standard & Infrequent Access

Files(NFS v3 / v4)

Volumes(iSCSI)

Tapes(iSCSI VTL)

AWS Storage Gateway

Data stored in Amazon S3 with• Native S3 management (File)• Snapshots to EBS (Volume)• Archive to Amazon Glacier (Tape)

Service uses an appliance to provide• Protocol conversion and device emulation• Caching (read-through / write-back)• Optimized data transfer

HTTPS

Amazon EC2

Amazon Glacier



Tape Gateway


Tape Gateway

• Easy cloud adoption: Virtual tape storage in S3 & Glacier accessed via gateway• Secure & efficient: Data encrypted and compressed in-transit and at-rest• Scalable: Unlimited archive capacity; up to 1 PB total tape storage per gateway• Dependable: 3-5-hour tape retrieval from Amazon Glacier with 99.999999999% data durability• Low-cost: Simple licensing, predictable storage costs, and reduced management time

Media ChangerVirtual Tapes

stored inAmazon S3

Archived Tapes in Amazon Glacier

BackupServer

iSCSI

TapeGateway

Tape Gateway Appliance (VM)

Tape Drive

HTTPS

Customer Premises Region

Virtual tape storage in Amazon S3 and Amazon Glacier with VTL management

Local Cache


Drop-in replacement for tape backups• Works with leading common backup applications• Emulates a tape library • Supports existing backup workflows


Backup, archive, and disaster recoveryCost-effective storage in AWS with local or cloud restore

“Tapes are a headache, prone with hardware failures, offsite storage costs, and constant

maintenance needs. Storage Gateway provided the most cost-effective and simple alternative. We even got disaster recovery by

using a bi-coastal data center.”


Your tape archive today

BackupServer

Customer Premises

Tape Library

Offsite Vault

Backup Disk

Appliance


Move your tape archive

Media ChangerVirtual Tapes

stored inAmazon S3

Archived Tapes stored in

Amazon Glacier

iSCSI

StorageGatewayService

Tape Gateway VM

Tape Drive

HTTPS

RegionBackupServer

Customer Premises

Tape Library

Offsite Vault

• Deploy Storage Gateway VTL as an archive target• Old archives can expire, or be recalled and written to SGW• Decommission Tape Library when all tapes are expired

Backup Disk

Appliance


Volume Gateway


Volume Gateway• Cloud-backed block storage that can be presented on-premises• Tier snapshots or whole volumes to the cloud to reduce SAN/NAS mgt.• Flexible recoveries in-cloud or on-prem. with snapshots and clones• Common use cases: Tiered storage for Windows file servers, backup and

restore, disaster recovery, data migrations

Application Server


Storage GatewayService

Volume Gateway appliance

Volume Stored in

Amazon S3

HTTPSiSCSI



Volume Gateway – Stored Mode

Using Stored Volumes a full copy of the volume is stored on-premises and in the cloud

Application Server




Volume Stored in

Amazon S3

HTTPSiSCSI


100% of volume stored in AWS & on-prem


Volume Gateway – Cached Mode

Using Cached Volumes a full copy of the volume is stored in AWS and a portion (hot data) is stored on-premises

Application Server



Volume Stored in

Amazon S3

HTTPSiSCSI


100% of volume stored in AWS & on-prem

Volume Gateway VM

virtual volume cache frequent accessed blocks (reduce onsite storage)


High availability Windows Server storageLocal HA with remote DR

us-west-1Data Center

WindowsClients Storage

Gateway VMMicrosoft DFS

HTTPSiSCSI

HTTPSiSCSI

SMB

Storage Gateway Service

us-east-1

us-west-1

us-east-1

Microsoft DFS

HTTPSiSCSI



Flexible storageEasily expand storage capacity on-demand

“Storage Gateway allows us to store backups offsite, create online archives of big files, and easily expand our file servers. With cached volumes, we have the

most used files local, and all of the files are stored safely in AWS. We use Storage Gateway whenever we need

a quick way to have big storage available.”


Volume Gateway + EBS Snapshots


Volume Gateway – EBS Snapshots

Snapshots are point-in-time backups of a volumeCreated on-demand or on a configurable scheduleCan be used to create an EBS or Storage Gateway volume

Application Server Amazon EBS

snapshots



Volume Stored in

Amazon S3

HTTPSiSCSI



Recovery with Volume Gateway


Recovery with EBS Snapshots

AppServer



Storage Gateway appliance

iSCSI volumeA B

A B

C

C

volume

D

D

snap1 snap2 snap3

A B C D

A B CvolumeA B C

AZEBS

EC2A BC D


File Gateway


File GatewayStore files as objects in Amazon S3; Access via S3 API or file protocols

• Storage in your S3 buckets with 1:1 mapping from files-to-objects• File metadata stored in object metadata• Full control over how your data is accessed and stored• Use cases: Hybrid cloud data processing (Big Data/Data Lakes), data

distribution, e.g. for content or media, backup & recovery, etc.

Customer Premises

Glacier

HTTPSNFS

v3 / v4.1

Application Server

Objects in your S3 bucket

Storage Gateway appliance


s3://{bucket name}/{folder1}/{file1}s3://{bucket name}/{folder1}/{file2}s3://{bucket name}/{folder1}/{file3}s3://{bucket name}/{folder1}/{file4}

s3://{bucket name}/{folder2}/{file1}s3://{bucket name}/{folder2}/{file2}

File GatewayOne-to-one mapping from files-to-objects

nfs://{IP}/{share name}/{folder1}/{file1}nfs://{IP}/{share name}/{folder1}/{file2}nfs://{IP}/{share name}/{folder1}/{file3}nfs://{IP}/{share name}/{folder1}/{file4}

nfs://{IP}/{share name}/{folder2}/{file1}nfs://{IP}/{share name}/{folder2}/{file2}

File Gateway appliance

NFS client Objects in your S3 bucket


AmazonAthena

Amazon QuickSight

Amazon EMR

AWSLambda

Amazon Rekognition

File GatewayMetadata and workload automation

Metadata and objectinventory

Data cache

Fileshare

File Gateway VM

Amazon CloudWatch

Events


NotifyWhenUploadedAPI




Data cache

Fileshare

File Gateway VM


ls /foo/*




Data cache

Fileshare

File Gateway VM

RefreshCacheAPI

Amazon CloudWatch

Events



Content distribution with File Gateway

Miami regional office

New York regional officeLos Angeles headquarters

Amazon S3 bucket

NFS client (writer)

File Gateway VM

File Gateway VM

NFS client (read-only)

NFS client (read-only)

File Gateway VM

Request upload

notification

1

Amazon CloudWatch

2

3

Client writes files and calls NotifyWhenUploaded API…1 When files are uploaded, the gateway sends a CloudWatch Event…

2

Event triggers RefreshCache API on gateways in the remote offices

3 When RefreshCache completes, the gateway sends a CoudWatch Event notification

4


Enabling cloud workloadsMove data to AWS storage for big data, cloud bursting, or migration

“Storage Gateway has the promise to transform the way we move data into the cloud. The NFS interface lets us easily integrate data files from analytical instruments, and the transparent S3 storage

lets us easily connect our cloud-based applications and leverage the powerful storage capabilities of S3.

With Storage Gateway, we can now unleash the full power of AWS on our instrument data.”


• Seamlessly connect on-premises applications with AWS Cloud storage; No changes to existing applications

• Local cache for low-latency access to data stored durably in AWS

• Supports use cases throughout your cloud journey: Backup & DR, tiered storage, hybrid processing workflows, migration

• Login to your AWS account, download the VM (or launch an EC2 instance), deploy new on-premises storage in minutes

AWS Storage Gateway


Documents

Edge and Hybrid Storage - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS_Summit_2018/June… · •Low-cost: Simple licensing, predictable storage costs, and reduced