27
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Anil Rebello AWS Service Catalog Business Development - EMEA, Amazon Web Services Dinah Barrett AWS Service Catalog Specialist Solution Architect, Amazon Web Services Enabling Self-Service for Data Scientists with AWS Service Catalog

Enabling Self-Service for Data Scientists with AWS …aws-de-media.s3.amazonaws.com/images/AWS_Summit... · Amazon EC2 Systems Manager AWS CloudTrail AWS Config AWS Managed Services

Embed Size (px)

Citation preview

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Anil Rebello

AWS Service Catalog Business Development - EMEA, Amazon Web Services

Dinah Barrett

AWS Service Catalog Specialist Solution Architect, Amazon Web Services

Enabling Self-Service for Data Scientists with AWS Service Catalog

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

What do we want to gain?

Ease of Use Agility Governance Scale

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

How to get self-service?

Standardize Enforce Policy Integrate Automate

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

AWS Service Catalog

ControlStandardization

Governance

Agility Self-Service

Time to Market

Organizations Developers

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

AWS Service Catalog: Terminology

ConstraintRestriction on the ways that specific AWS resources can be deployed for a product, e.g., template constraints to allow only certain EC2 instance sizes

ProductAn IT service (VPC, web server, n-tier environment, database) that you want to make available for deployment on AWS

Provisioned ProductAn AWS Service Catalog product is launched through an AWS CloudFormation process, and the collection of launched services is called a Provisioned Product

PortfolioA collection of products, together with configuration information, launch controls, and administrator-controlled access

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Key Benefits

Enforce Consistency and Compliance

Standardize

Limit Access

Enforce Tagging

Guardrail Resources

Developer Autonomy

Automate Deployments

Single-pane for Provisioning

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

The self-service management framework

CurateGovern

Quick startAWS SERVICES

Develop

Launch

Operate

PublishCopy & Customize

Modernize and Migrate

AWS DMSAWS Migration Hub AWS SMSAWS Application Discovery Service

from

AWSCloudFormation

AWS CodeCommit

AWS CodePipeline

Amazon EC2 Systems Manager

AWSCloudTrail

AWSConfig

AWS Managed Services

Publish Amazon ECS

Amazon Lambda

Elastic Load Balancing

Migrate

Operate

AWS Systems Manager

Design and Deploy

Amazon RDS

Amazon EC2

AMI & CloudFormation

Elastic Load Balancing

Amazon ECS

Amazon RDS

Amazon Lambda

Manage and Govern

Users

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

ITSM Integration

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

AWS Service Catalog

ITOM / ITSM / Global Service Catalog

AWS Provisioned Resources

ITOM / ITSM / Global Service Catalog AWS Marketplace

AWS Provisioning

Public APIs

Public APIs End User

BMC in betaDigital Workplace

Integration with AWS Service Catalog

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Creates portfolio and assigns products to portfolio

Authors template

Administrator experience

1

Adds constraints, grants access and adds tags

2 Creates product

ProductX

Versions

Portfolio B

Portfolio A

• Users and Roles• Constraints • Tags

Service Catalog

AWS Service Catalog Administrator

4

3

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

ITSM Administrator

Assign Roles to Users

ServiceNow Workflows

ITSM Admin

Synchronizes Latest Updates

AWS Service Catalog API

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Outputs

User experience

Browse Products

3

2

1

Portfolio

Users

Select Product, configure

parametersRequest Launch

Deploy

4

Service RequestWorkflow - RITM

Maps to SC Portfolio

Launch Product

Provisioned Product

Configuration Item

AWS Service Catalog API

AWS Service Catalog

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Enabling Self-Service for Data Scientists

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

AWS has 100+ services

USERS

How do I logically identify my project assets?

There are too many steps. Can we not automate this?

I’d like my team to use these services consistently

Tired of manually creating monthly dashboards

Not sure of the best way to represent the data visually

How do I make this easy?

I only need to use services for Analytics. I am lost in the console..

What are these security groups? Should I care? Is there a policy I can use?

How many services do I need to learn?

Isn’t there a way to create a product bundle for my project?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Solution approach - step 1

The Project (Portfolio) Layout

Active Directory

Group

IAM role

Portfolio

Principals

Launch Launch

Launch

LaunchLaunch

Tags• 1:1 Portfolio (Project) to an IAM role: “Portfolio Role”• 1:1 Portfolio Role to an IAM Policy: “Portfolio Policy”• Full access between all AWS resources created through Portfolio

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Solution approach - step 2

Scale Infrastructure as Code

Minimize Human Error

Project Onboarding

Automate Portfolio Creation

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Solution approach - step 3

Least Privilege Automate Block Lateral Attacks

Customer Experience

Dynamically Update IAM Policies

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

IAM Bound Architecture

Data Scientist

S3

SQS

Lambda

Lambda Custom Resource

EC2

Get ARN for S3 bucket

Create IAM Portfolio role

Leverages Portfolio role

Update IAM Role Policy

Lambda Custom Resource

Dynamically update Portfolio Policy and assign Portfolio Role to new resources

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Tags as metadata glue

Portfolio

I

Tags will be used by Lambda function to identify the project, its Role,

and its Policy

Provisioned Product (Stack)

PolicyRoleId

Id

Tags Enforced

PolicyRole

Id

Id

Id

Id

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Logging and tracking

S3

SQS

Lambda

EC2

Lambda Custom ResourceGet ARN for

S3 bucket

Update IAM Role Policy

Data Scientist

Id

Id

Id

Id

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Automated AMI Build and Validation workflow.

OS AMI

New Instance

PlaybookUpdates

Base AMI

New Instance

Validator

SNS

Parameter Store

Service Catalog

TeamTeam

Updates

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Solution

• Step 1: 1:1 portfolio to project mapping• Enforce tagging• Align security controls and resource allocation

• Step 2: Automate product and portfolio creation

• Step 3: Dynamically update IAM policies as new resources are provisioned

RolePolicy

Id

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

ServiceNow / AWS Service CatalogDemo

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

ServiceNow and Service Catalog

• AWS Service Catalog• Cloud Management• ServiceNow Integration

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Key Takeaways

ü Start small, iterate quickly, and build incrementally – expand product catalog in steps ( ML ,RDS, Amazon EMR ,Marketplace Integration)

ü Align Security and Resource Associations through Portfolios on a per-project basis

ü Automate security processes with AWS Lambda to reduce turnaround time

ü IAM bound your applications through Automation with Tags and Tag Enforcement through AWS Service Catalog

ü Create an easy click-and-go experience for users

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

AWS Service Catalog 2018

Ease of Use&

Self Served

Secure & Aligned

Dynamically

Track Changes

Automation &

Tagging

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Thank you