Upload
vutruc
View
240
Download
1
Embed Size (px)
Citation preview
The presentation is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
40% DATA
GROWTH PER YEAR
5.3BN MOBILE SUBS IN
2010 (78% PENETRATION)
13K iPHONE APPS DOWNLOADED PER MINUTE
370K CALL MINUTES EVERY 60 SECONDS
$1TR BY 2014
$700BN IN 2011
2.1BN USERS
8X DATA GROWTH IN 5 YRS
70+ NEW DOMAINS EVERY 60 SECONDS
100K TWEETS PER MINUTE
750M USERS
20M APPS PER DAY
600 NEW VIDEOS
EVERY 60 SECONDS
EXTREME WRITE SCALABILITY REAL TIME USER EXPERIENCE
ROCK SOLID RELIABILITY ELIMNATE BARRIERS TO ENTRY
Driving new Database Requirements
EXTREME WRITE SCALABILITY REAL TIME USER EXPERIENCE
ROCK SOLID RELIABILITY ELIMNATE BARRIERS TO ENTRY
No Trade-Offs
Transactional Integrity
Complex Queries
Standards & Skillsets
No Trade-Offs: Cellular Network
HLR / HSS
Billing, AuC, VLR
AuC, Call
Routing, Billing
Location
Updates
Pre & Post Paid
• Massive volumes of write traffic
• <3ms database response
• Downtime & lost transactions = lost $
MySQL Cluster in Action: http://bit.ly/oRI5tF
No Trade-Offs: eCommerce
• Integrated Service Provider
platform
• eCommerce
• Payment processing
• Fulfillment
• Supports 1k+ manufacturers &
18k retail partners
• Requirements
• Scaling, On-Demand
• HA: failures & on-line upgrades
• High batch & real time loads
• Low TCO: capex and opex
http://mysql.com/customers/view/?id=1080
No Trade-Offs: Flight Control
• US Navy aircraft carriers
• Consolidated flight operations
management system
• Maintenance records
• Fuel loads
• Weather conditions
• Flight deck plans
• Requirements
• No Single Points of Failure
• Complete redundancy
• Small footprint, harsh environment
• 4 x MySQL Cluster nodes,
Linux and Windows
MySQL User Conference Session: http://bit.ly/ogeid3
ACID Compliant Relational Database • SQL & NoSQL interfaces
Write-Scalable & Real-Time • Distributed, multi-master, auto-sharding, optimized in-memory structures & indices
99.999% Availability • Shared-nothing, integrated clustering & sub-second recovery, local & geographic
replication, on-line operations
Low Barriers to Entry • Open-source, elastic, multiple APIs, management tools, commodity hardware
MySQL Cluster Architecture
Data Nodes
Node Group 1
F1
F3
F3
F1
No
de
1
No
de
2
Node Group 2
F2
F4
F4
F2
No
de
3
No
de
4
Application Nodes
Cluster
Mgmt
Cluster
Mgmt
REST
JPA
MySQL Cluster - Extreme Resilience
Data Nodes
Node Group 1
F1
F3
F3
F1
No
de
1
No
de
2
Node Group 2
F2
F4
F4
F2
No
de
3
No
de
4
Application Nodes
Cluster
Mgmt
Cluster
Mgmt
REST JPA
Handling Scheduled Maintenance On-Line Operations
• Scale the cluster (add nodes on-line)
• Repartition tables
• Upgrade / patch servers & OS
• Upgrade / patch MySQL Cluster
• Back-Up
• Evolve the schema on-line, in real-time
Auto-Sharding with High Availability
• Transparent to the application
• No need for application-layer sharding logic
• Partitioning based on hashing all or part of the primary key
• Each node stores primary fragment for 1 partition and back-up fragment for another
• Transparency maintained during failover, upgrades and scale-out
• No need to limit application to single-shard transactions
Best Practice : Primary Keys
• ALWAYS DEFINE A PRIMARY KEY ON THE TABLE!
• A hidden PRIMARY KEY is added if no PK is specified. BUT..
• .. NOT recommended
• The hidden primary key is for example not replicated (between Clusters)!!
• There are problems in this area, so avoid the problems!
• So always, at least have id BIGINT AUTO_INCREMENT PRIMARY KEY
• Even if you don't “need” it for you applications
Scale-Out on Commodity Hardware
• 8 Intel servers • Dual-6-core CPUs @2.93 GHz
• 24GB RAM
• 2 Data Nodes per server
• flexAsync benchmark • 16 parallel threads, each issuing 256
simultaneous transactions
• Read / Write 100KB attribute
• Access via NDB API
4.3M
QPS
Copyright 2011 Oracle Corporation 18
MySQL Cluster 7.2.1 DMR
• 70x Higher Complex Query Performance: Adaptive Query Localization
• Native memcached API
• MySQL 5.5 Server Integration
Enabling Next Generation Web Services
• Multi-Site Clustering
• Simplified Active / Active Replication
Enhancing Cross Data Center Scalability
• Consolidated Privileges
Simplifying Provisioning & Administration
Adaptive Query Localization Scaling Distributed Joins
• Perform Complex Queries
across Shards • JOINs pushed down to data nodes
• Executed in parallel
• Returns single result set to MySQL
• Opens Up New Use-Cases
• Real-time analytics
• Recommendations engines
• Analyze click-streams
mysqld
Data Nodes
mysqld
A
Q
L
The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion.
October 3rd, 2011
Data Nodes
70x More
Performance
DON’T COMPROMISE
FUNCTIONALITY TO SCALE-OUT !!
MySQL Cluster 7.2 AQL Test Query Web-Based Content Management System
Copyright 2011 Oracle Corporation 21
Data
Node1
Data
Node2
MySQL
Server
MySQL Cluster 7.2 70x Speedup! (DM)
• 70x Speedup for real
customer JOIN across
11 tables
• Must Analyze tables for
best results
mysql> ANALYZE
TABLE <tab-name>;
7.2 = 1.26s
7.1 = 87.23s
7.2
analyzed
= 65.3s
The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion.
October 3rd, 2011
Memcached Key-Value API
• Persistent, Scalable, HA
Back-End to memcached • No application changes: re-
uses standard memcached
clients & libraries
• Consolidate Caching &
Database Tiers • Eliminate cache invalidation
• Simpler re-use of data across
services
• Improved service levels
• Flexible Deployment • Schema or Schema-less
storage
The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion.
October 3rd, 2011
New
NoSQL
Access
API Flexibility: SQL & NoSQL Combined
• SQL: Complex, relational queries
• HTTP / memcached: Key-Value web services
• Java: Enterprise Apps
• NDB API: Real-time services
Mix
&
Match
Data Nodes
NDB API
Clients
Native memcached HTTP/REST
JDBC / ODBC
PHP / PERL
Python / Ruby
The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion.
October 3rd, 2011
MySQL 5.5 Server Integration
• Exploit Latest Server / Storage
Engine Developments within a
Single Application • InnoDB: Foreign Keys, XA Transactions,
Large Rows
• MySQL Cluster: HA, High Write Rates, Real-
Time
• Reduces Complexity, Simplifies
DevOps
• MySQL 5.5 • 3x higher performance
• Improved partitioning, diagnostics, availability,
etc.
The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion.
October 3rd, 2011
Enhanced Cross Data Center Scalability Geographic Replication
• Replicating complete
clusters across data
centers
• DR & data locality
• No passive resources
• Simplified Active /
Active Replication
• Eliminates requirement
for application & schema
changes
• Transaction-level
rollback
Geographic
Replication
The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion.
October 3rd, 2011
Enhanced Cross Data Center Scalability Multi-Site Clustering
• Split data nodes & node
groups across data
centers
• Synchronous replication
and auto-failover between
sites
• Improved heartbeating to
handle network partitions
• Extends Disaster
Recovery Options
• Active/Active with no
need for conflict
handling
Data Node 1
Synchronous
Replication
Data Node 3
Data Node 2
Data Node 4
The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion.
October 3rd, 2011
The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion.
October 3rd, 2011
Simplified Provisioning & Maintenance User Privilege Consolidation
Monitoring &
Recovery
High
Availability
Operation
Automated
Management
Reducing TCO and creating a more agile, highly
available database environment
MySQL Cluster Manager
Copyright 2011 Oracle Corporation 33
How Does MySQL Cluster Manager Help?
Example: Initiating upgrade from MySQL Cluster 6.3 to 7.1
• 1 x preliminary check of cluster state
• 8 x ssh commands per server
• 8 x per-process stop commands
• 4 x scp of configuration files (2 x mgmd & 2 x
mysqld)
• 8 x per-process start commands
• 8 x checks for started and re-joined processes
• 8 x process completion verifications
• 1 x verify completion of the whole cluster.
• Excludes manual editing of each configuration
file.
Total: 46 commands -
2.5 hours of attended operation
Before MySQL Cluster Manager With MySQL Cluster Manager
upgrade cluster --package=7.1 mycluster;
Total: 1 Command -
Unattended Operation
• Results • Reduces the overhead and complexity
of managing database clusters
• Reduces the risk of downtime resulting from administrator error
• Automates best practices in database cluster management
Copyright 2011 Oracle Corporation 34
Creating & running your first Cluster
- the “manual” way (without MCM)
• Up & running in 10-15 minutes using Quick Start guides from
http://dev.mysql.com/downloads/cluster/ • Versions for Linux, Windows & Solaris
Copyright 2011 Oracle Corporation 35
Download & Extract
• edelivery.oracle.com
• www.mysql.com
• dev.mysql.com
Configure
• Cluster-wide “config.ini”
• Per-mysqld “my.cnf”
Start processes
• Management Nodes
• Data Nodes
• MySQL Servers
New in MySQL Cluster Manager 1.1.2 Bootstrap single host Cluster
1. Download MCM from edelivery.oracle.com:
• Package including Cluster
2. Unzip
3. Run agent, define, create & start Cluster!
$> bin\mcmd –bootstrap MySQL Cluster Manager 1.1.2 started
Connect to MySQL Cluster Manager by running "D:\Andrew\Documents\MySQL\mcm\bin\mcm" -a NOVA:1862
Configuring default cluster 'mycluster'...
Starting default cluster 'mycluster'...
Cluster 'mycluster' started successfully
ndb_mgmd NOVA:1186
ndbd NOVA
ndbd NOVA
mysqld NOVA:3306
mysqld NOVA:3307
ndbapi *
Connect to the database by running "D:\Andrew\Documents\MySQL\mcm\cluster\bin\mysql" -h NOVA -P 3306
-u root
4. Connect to Cluster & start using database
To bootstrap with Cluster 7.2 replace contents of mcm/cluster directory
http://www.clusterdb.com/mysql-cluster/mysql-cluster-manager-1-1-2-creating-a-cluster-is-now-trivial
Copyright 2011 Oracle Corporation 36
MySQL Cluster in Action Web Reference Architectures
Session Management eCommerce
MySQL Master
Content Management
Slave 1
MySQL Master
Analytics Slave 2 Slave 3 Slave 4 Slave 5
Slave 6 Slave 7 Slave 8 Slave 9 Slave 10
Slave N
Slave 2
Node Group 2
F2
F4
Nod
e 3
N
od
e 4
F2
F4
Node Group 1
F1
F3
Nod
e 3
N
od
e 4
F1
F3
MySQL Cluster Data Nodes
MySQL Servers
Node Group 2
F2
F4
Nod
e 3
N
od
e 4
F2
F4
Node Group 1
F1
F3
Nod
e 3
N
od
e 4
F1
F3
MySQL Cluster Data Nodes
MySQL Servers
Slave 1 Slave 3
Data
Refinery Memcache / Application Servers
Distributed
Storage
XOR • 4 x Data Nodes: 6k
page hits per second
• Each page hit
generating 8 – 12
database operations
Whitepaper: http://www.mysql.com/why-mysql/white-papers/mysql_wp_high-availability_webrefarchs.php
Getting Started
Learn More
Evaluate MySQL Cluster 7.2 Bootstrap a Cluster!
Scaling Web Databases Guide www.mysql.com/cluster/
Download Today http://dev.mysql.com/downloads/cluster/
Download, No Obligation https://edelivery.oracle.com/
Copyright 2011 Oracle Corporation 47
Summary
Scale Web Services with
Carrier-Grade Availability
Don’t Trade Functionality for Scale
Evaluate the Development Milestone Today
Copyright 2011 Oracle Corporation 48
Data Node 1
F1
F2
Data Node2
F2
F1
Application Nodes
Data Nodes
Cluster
Mgmt
Scale the Cluster Capacity & Performance
Data Nodes
Node Group 1
F1
F3
F3
F1
No
de
1
No
de
2
Node Group 2
F2
F4
F4
F2
No
de
3
No
de
4
Application Nodes
Cluster
Mgmt
Cluster
Mgmt
Scale the Cluster Capacity & Performance
On-Line Scaling & Maintenance
• Can also update schema on-line
• Upgrade hardware & software with no downtime
• Perform back-ups on-line
1. New node group added
2. Data is re-partitioned
3. Redundant data is deleted
4. Distribution is switched to share load with new node group