Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
© 2019 Snowflake Inc. All Rights Reserved
MIGRATING TO SNOWFLAKE
BLAST OUT OF THE STARTING GATE
JOSHUA SKARPHOL
CONSULTING MANAGER
SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved
AGENDA
2
My Migration Story
Migration Basics
Migration Strategy
How Snowflake Helps Our Customers Migrate
© 2019 Snowflake Inc. All Rights Reserved
MY MIGRATION STORY
© 2019 Snowflake Inc. All Rights Reserved
MY MIGRATION STORY BEGAN AS A TERADATA ARCHITECT AT A FORTUNE 100 COMPANY
4
© 2019 Snowflake Inc. All Rights Reserved
CLOUD FIRST INITIATIVE
5
© 2019 Snowflake Inc. All Rights Reserved
CAPACITY CHALLENGES
6
Added 2 more
unplanned for
production nodes -
Summer 2015
Teradata on
AWS POC -
Summer 2016
Teradata floor
sweep -
Summer 2014
Teradata offload
to Hadoop POC -
Winter 2015
© 2019 Snowflake Inc. All Rights Reserved
• The floor sweep in Summer 2014 was
projected to meet the Teradata capacity
needs for the next 3 years
• In Summer 2015, the new CIO signed
for an unplanned Teradata capacity
purchase on his second day on the job
• The CIO delivered the message that he
wasn’t going to sign for any more
unplanned Teradata capacity purchases
7
COST
© 2019 Snowflake Inc. All Rights Reserved
SNOWFLAKE EVALUATION PROCESS
8
POC
End-to-end data processing
from source to target for 3
Dev/QA/Prod environments of
the Teradata tables used for an
existing analytics application
Scope
Conducted a bake off
between 4 GSIs to see
their migration
capabilities in action
ApproachValidated Teradata
benchmark of 99
concurrent queries could
run on Snowflake in
less than 2 minutes
Pilot
© 2019 Snowflake Inc. All Rights Reserved
• Snowflake was selected as the going
forward Enterprise Data Warehouse
• The pilot proved doing a full Teradata to
Snowflake migration was achievable
• POS Migration - Point of Sale workload
that consumed 30% of the production
Teradata server was successfully
offloaded to Snowflake
9
EVALUATION OUTCOMES
30%Teradata Production
© 2019 Snowflake Inc. All Rights Reserved
BENEFITS OF MIGRATION
10
Support teams of all sizes and provide a
chargeback model for system usage
Data consolidated across the company
onto a single platform
Business units scale to meet their own needs
without impacting or being impacted by
other business units
Provide capacity on demand instead of
being capacity constrained
© 2019 Snowflake Inc. All Rights Reserved
• Establish security model and resource
monitors at the beginning of the
migration
• Bring in a Snowflake Technical Account
Manager to help get teams up and
running more quickly and share
Snowflake knowledge across the
company
• Communicate best practices and wins
more proactively
11
LESSONS LEARNED
© 2019 Snowflake Inc. All Rights Reserved
MIGRATION BASICS
© 2019 Snowflake Inc. All Rights Reserved 13
How much does it
cost?
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved 14
How much does it
cost?
How long will it take?
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved 15
Have other customers
done what we’re
trying to do?
How much does it
cost?
How long will it take?
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved 16
Have other customers
done what we’re
trying to do?
How much does it
cost?
Costs depend on the size and
scope of what is migrated from the
existing solution to Snowflake
How long will it take?
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved 17
Duration depends on the size and
scope of what is migrated from the
existing solution to Snowflake
Have other customers
done what we’re
trying to do?
How much does it
cost?
Costs depend on the size and
scope of what is migrated from the
existing solution to Snowflake
How long will it take?
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved 18
Duration depends on the size and
scope of what is migrated from the
existing solution to Snowflake
Have other customers
done what we’re
trying to do?
How much does it
cost?
Costs depend on the size and
scope of what is migrated from the
existing solution to Snowflake
How long will it take?
Customers have successfully migrated
from other data warehouse platforms to
Snowflake, but each customer is different
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
19
The answer is…it depends!
Duration depends on the size and
scope of what is migrated from
the existing solution to Snowflake
Have other customers done
what we’re trying to do?
How much does it cost?
Costs depend on the size and
scope of what is migrated from the
existing solution to Snowflake
How long will it take?
Customers have successfully migrated
from other data warehouse platforms to
Snowflake, but each customer is different
© 2019 Snowflake Inc. All Rights Reserved
MIGRATION STRATEGY
© 2019 Snowflake Inc. All Rights Reserved
THE NEED FOR MIGRATION STRATEGY
21
Migration PlanningA successful migration begins with planning that is led by Snowflake PS or a SI partner to accurately scope the migration.
Code ConversionDDL, procedures, and scripts must be translated to be valid in Snowflake. Code Conversion effort is typically the largest unknown in scoping migrations.
Data MigrationData migration is the movement of an initial dataset from the legacy system to Snowflake. Effort in this phase varies based upon the number of tables and the method for moving the data.
Data IngestionData ingestion is the movement of data changes from the source system to Snowflake. Effort in this phase varies based upon the number of data sources and the method for moving the data.
Data ValidationSide by side comparison of the legacy data results and Snowflake results. Verifies data was moved successfully and that any translated logic is giving the intended results.
Reporting & Analytics Tools
Integrations for reporting and analytics tools must be modified and reconnected. This can be very involved for some systems where objects must be re-architected.
Security Implementation
Evaluate legacy security requirements, architect and implement the new security plan including SSO, roles, and data level security.
© 2019 Snowflake Inc. All Rights Reserved
3 QUESTIONS THAT SHOULD BE ASKED ABOUT MIGRATION TO SNOWFLAKE
22
What support is available to
help with the migration to
Snowflake?
What really needs to
be migrated to
Snowflake?
What are the common
characteristics of successful
migrations to Snowflake?
© 2019 Snowflake Inc. All Rights Reserved
• List databases and database objects to
migrate
• List processes and tools that populate
and pull data from the existing data
warehouse
• List security roles, users and
permissions
• Documentation of the existing data
warehouse solution into an as-is
architecture diagram
23
WHAT NEEDS TO BE MIGRATED –
DOCUMENT THE EXISTING SOLUTION
© 2019 Snowflake Inc. All Rights Reserved
• Categorize:
• processes to migrate as-is
• processes that need re-engineering
• processes that need fixing
• processes that are no longer needed
• Draft migration deliverables
• Create to-be architecture diagram
24
WHAT NEEDS TO BE MIGRATED –
ESTABLISH A MIGRATION APPROACH
© 2019 Snowflake Inc. All Rights Reserved 25
WHAT NEEDS TO BE MIGRATED –
MODERNIZATION CONSIDERATIONS
• Migrating to Snowflake provides an
opportunity to reevaluate the design of
the data warehouse solution
• Consider the tradeoffs between
refactoring parts of the previous data
warehouse solution and the
downstream impacts of that refactoring
• Scalability limitations that existed in the
previous data warehouse solution are
mitigated in Snowflake which requires a
new approach to capacity utilization
© 2019 Snowflake Inc. All Rights Reserved
• List data sets to migrate first
• Create method for identifying process
dependencies for data sets
• Document process dependencies for
data sets
26
WHAT NEEDS TO BE MIGRATED –
PRIORITIZE DATA SETS FOR MIGRATION
© 2019 Snowflake Inc. All Rights Reserved
• Completing a migration requires time,
money and energy, so you need a
compelling reason to migrate
• Having a compelling reason to migrate
keeps you moving forward if challenges
arise with the migration
• Completing the migration brings a
sense of accomplishment that
something important has been achieved
27
CHARACTERISTICS OF SUCCESSFUL MIGRATIONS –
MOTIVATION
© 2019 Snowflake Inc. All Rights Reserved
• Determine the commitment of decision
makers to providing the necessary
funding
• Document budget allocated for the
migration project
• Work with the Snowflake sales team to
estimate the costs to run Snowflake and
the Snowflake Professional Services
team to estimate the cost of the
migration
28
CHARACTERISTICS OF SUCCESSFUL MIGRATIONS –
FINANCIAL COMMITMENT
© 2019 Snowflake Inc. All Rights Reserved
• Need automated repeatable processes
to minimize the amount of time required
to migrate to Snowflake
• Snowflake has solution partners with
migration tooling to help our customers
with their migrations
29
CHARACTERISTICS OF SUCCESSFUL MIGRATIONS –
MIGRATION TOOLING
© 2019 Snowflake Inc. All Rights Reserved
SUPPORT FOR MIGRATING TO SNOWFLAKE
30
Our Snowflake Solutions Partners
have experience migrating
customers from other Data
Warehouse platforms to Snowflake
Professional Services has a scalable,
repeatable methodology and tools to
help our customers with their migration
to Snowflake
Solutions PartnersDocumentation
Migration Reference Manuals
available for:
Teradata, Oracle, and Netezza https://resources.snowflake.com/
Snowflake Migration Team
© 2019 Snowflake Inc. All Rights Reserved
HOW SNOWFLAKE HELPS OUR
CUSTOMERS MIGRATE
© 2019 Snowflake Inc. All Rights Reserved
IS THERE A MIGRATION EASY BUTTON?
32
© 2019 Snowflake Inc. All Rights Reserved 33
THERE IS A MIGRATION EASIER BUTTON
© 2019 Snowflake Inc. All Rights Reserved
SNOWFLAKE CONVERSION SERVICES
34
We are looking for customer sponsors to develop additional sources!
Tables
Views
DML Statements
User Defined Functions
Stored Procedures
BTEQ
Tables
Views
DML Statements
User Defined Functions
Stored Procedures
© 2019 Snowflake Inc. All Rights Reserved
MIGRATION READINESS ENGAGEMENT
Delivery of a migration plan to a customer
35
• 1 or 2 week, depending on
code conversion
approach, onsite paid
Professional Services
engagement to provide a
detailed migration plan for
customers across all
phases of the migration
• Cost - $20K (1 week) or
Custom SOW (2 weeks)
What it is:
• Completed migration
questionnaire
• Customer DDL and DML
scripts
• Customer subject matter
experts for the 1 or 2 week
on-site engagement
What is Required:
• Customer receives a full migration plan including cost
estimation, migration timeline, resource plan, and
detailed list of migration steps for code conversion,
data migration, data ingestion, data validation, and
reporting & analytics tools
• Customer determines if the migration plan is based
on a SI, Snowflake Professional Services, or a
combination of both executing the migration
Output & Value
© 2019 Snowflake Inc. All Rights Reserved
SUMMARY
36
• The cost and timeline for migrating to Snowflake
varies based on each customer’s requirements
• Migrating to Snowflake is most successful when
using a planned-out migration strategy that
leverages as much automation as possible
• Snowflake has migration services to help our
customers migrate successfully
© 2019 Snowflake Inc. All Rights Reserved
Thank You