Upload
tugdual-grall
View
1.147
Download
3
Embed Size (px)
DESCRIPTION
Generic introduction to NoSQL, Couchbase presentation and Demonstration
Citation preview
Monday, June 17, 13
Introducing Couchbase Server 2.0
Tugdual Grall
Technical Evangelist
Monday, June 17, 13
About me...
• Tugdual “Tug” Grall Couchbase
• Technical Evangelist
eXo
• CTO
Oracle
• Developer/Product Manager
Mainly Java/SOA Developer in consulDng firms
• Web
@tgrall
hHp://blog.grallandco.com
tgrall
NantesJUG co-‐founder
Pet Project :
hHp://www.resultri.com
Monday, June 17, 13
RDBMS are not Enough?
Monday, June 17, 13
Growth is the New Reality
• Instagram gained nearly 1 million users overnight when then expanded to Android
Monday, June 17, 13
Draw Something Viral Growth
191715131197533/12826242220181614121082/6
Draw Something by OMGPOPDaily Ac)ve Users (millions)
21
2
4
6
8
10
12
14
16
Monday, June 17, 13
RDBMS is good for many thing, but hard to scale
RDBMS Scales UpGet a bigger, more complex server
Users
ApplicaNon Scales OutJust add more commodity web servers
Users
System CostApplicaDon Performance
RelaNonal Database
Web/App Server Tier
System CostApplicaDon Performance
Won’t scale beyond this point
How do you take this growth?
Monday, June 17, 13
Web/App Server Tier
Memcached Tier
MySQL Tier
Scaling out RDBMS
• Run Many SQL Servers
• Data could be shared
Done by the applicaDon code
• Caching for faster response Nme
Monday, June 17, 13
Scaling out flaWens the cost and performance curves
NoSQL Database Scales OutCost and performance mirrors app Ner
Users
NoSQL Distributed Data Store
Web/App Server Tier
ApplicaNon Scales OutJust add more commodity web servers
Users
System CostApplicaDon Performance
ApplicaDon Performance System Cost
NoSQL Technology Scales Out
Monday, June 17, 13
Building new database to answer the following requirementsNo schema required before inserDng dataNo schema change required to change data formatAuto-‐sharding without applicaDon parDcipaDonDistributed queriesIntegrated main memory cachingData synchronizaDon ( mulD-‐datacenter)
DynamoOctober 2007
CassandraAugust 2008
BigtableNovember 2006
VoldemortFebruary 2009
Very few organizaNons want to (fewer can) build and maintain database so[ware technology.But every organizaNon building interacNve web applicaNons needs this technology.
A New Technology?
Monday, June 17, 13
Lack of flexibility/rigid schemas
Inability to scale out data Performance challenges Cost All of these Other
49%
35%
29%
16%12% 11%
What Is Biggest Data Management Problem Driving Use of NoSQL in Coming Year?
Source: Couchbase Survey, December 2011, n = 1351.
Monday, June 17, 13
NoSQL CatalogKey-‐Value
Memcached
Membase
Redis
Data Structure Document Column Graph
MongoDB
Couchbase Cassandra
Cache
(mem
ory on
ly)
Database
(mem
ory/disk) Neo4j
HBase InfiniteGraph
Coherence
Monday, June 17, 13
Couchbase ServerNoSQL Document Database
Monday, June 17, 13
Couchbase Open Source Project
• Leading NoSQL database project focused on distributed database technology and surrounding ecosystem
• Supports both key-‐value and document-‐oriented use cases
• All components are available under the Apache 2.0 Public License
• Obtained as packaged soiware in both enterprise and community ediDons. Couchbase
Open Source Project
Monday, June 17, 13
Easy Scalability Consistent High Performance
Always On 24x365
Grow cluster without applicaDon changes, without downDme with a single click
Consistent sub-‐millisecond read and write response Dmes with consistent high throughput
No downDme for soiware upgrades, hardware maintenance, etc.
Flexible Data Model
JSON document model with no fixed schema.
Couchbase Server
Monday, June 17, 13
Flexible Data Model
• No need to worry about the database when changing your applicaDon• Records can have different structures, there is no fixed schema• Allows painless data model changes for rapid applicaDon development
{ “ID”: 1, “FIRST”: “DipN”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA”}
JSONJSON
JSON JSON
Monday, June 17, 13
AddiDonal Couchbase Server Features
Built-‐in clustering – All nodes equal
Data replicaDon with auto-‐failover
Zero-‐downDme maintenance
Built-‐in managed cached
Append-‐only storage layer
Online compacDon
Monitoring and admin API & UI
SDK for a variety of languages
Monday, June 17, 13
Market AdopDon – CustomersInternet Companies Enterprises
Monday, June 17, 13
Heartbeat
Process m
onito
r
Glob
al singleton supe
rviso
r
Confi
guraDo
n manager
on each node
Rebalance orchestrator
Nod
e he
alth m
onito
r
one per cluster
vBucket state and
replicaD
on m
anager
hFp
REST m
anagem
ent A
PI/W
eb UI
HTTP8091
Erlang port mapper4369
Distributed Erlang21100 -‐ 21199
Erlang/OTP
storage interface
Couchbase EP Engine
11210Memcapable 2.0
Moxi
11211Memcapable 1.0
Memcached
New Persistence Layer
8092Query API
Que
ry Engine
Data Manager Cluster Manager
Couchbase Server Architecture
Monday, June 17, 13
Heartbeat
Process m
onito
r
Glob
al singleton supe
rviso
r
Confi
guraDo
n manager
on each node
Rebalance orchestrator
Nod
e he
alth m
onito
r
one per cluster
vBucket state and
replicaD
on m
anager
hFp
REST m
anagem
ent A
PI/W
eb UI
HTTP8091
Erlang port mapper4369
Distributed Erlang21100 -‐ 21199
Erlang/OTP
storage interface
Couchbase EP Engine
11210Memcapable 2.0
Moxi
11211Memcapable 1.0
Memcached
New Persistence Layer
8092Query API
Que
ry Engine
Couchbase Server Architecture
Monday, June 17, 13
Couchbase OperaDons
Monday, June 17, 13
33 2
Single node -‐ Couchbase Write OperaDon
Managed Cache
Disk Que
ue
Disk
ReplicaDon Queue
App Server
Couchbase Server Node
Doc 1Doc 1
Doc 1
To other node
Monday, June 17, 13
COUCHBASE SERVER CLUSTER
Basic OperaDon
• Docs distributed evenly across servers
• Each server stores both acPve and replica docsOnly one server acDve at a Dme
• Client library provides app with simple interface to database
• Cluster map provides map to which server doc is onApp never needs to know
• App reads, writes, updates docs
•MulPple app servers can access same document at same Pme
User Configured Replica Count = 1
READ/WRITE/UPDATE
ACTIVE
SERVER 1
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc
SERVER 2
Doc 8
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
REPLICA
Doc 6
Doc 3
Doc 2
Doc
Doc
Doc
REPLICA
Doc 7
Doc 9
Doc 5
Doc
Doc
Doc
SERVER 3
Doc 6
Doc 5
Doc 2
Doc
Doc
DocDoc 9
APP SERVER 1
COUCHBASE Client LibraryCLUSTER MAP
COUCHBASE Client LibraryCLUSTER MAP
APP SERVER 2
Monday, June 17, 13
Client SDK
www.couchbase.com/develop
Clojure
Python
Ruby
libcouchbase
Go
Monday, June 17, 13
NPM
{ "name": "my-node-application", "version": "1.0.0", "private": true, "dependencies": { "express": "3.x", "couchbase": "0.0.12", "ejs": ">= 0.0.1" }}
Monday, June 17, 13
Connect to the cluster
var driver = require('couchbase'); dbConfiguration = { "hosts": ["localhost:8091"], "bucket": "ideas"}; driver.connect(dbConfiguration, function(err, cb) { if (err) { throw (err) } // your application code here }
Monday, June 17, 13
Insert Data
var meetup = {"type" : "meetup", "language" : "javascript"};cb.set("web5",meetup, function(err, meta) {}); var tmp = {"message" : "hello world!"};cb.set("tmp", tmp, {"expiry" : 5},function(err, meta) {});
Monday, June 17, 13
Insert / Delete Data
var meetup = {"type" : "meetup", "language" : "javascript"};cb.set("barcelonajs",meetup, function(err, meta) {});
var tmp = {"message" : "hello world!"};cb.set("tmp", tmp, {"expiry" : 5},function(err, meta) {});
cb.set("todelete", tmp, function(err, meta) {});cb.remove("todelete", function(err, meta) {});
Monday, June 17, 13
Retrieve the Data
cb.get("product:45", function(errs, doc, metas) { console.log("=== get the document ==="); console.log( doc );
});
var keys = new Array(); keys.push("product:1"); keys.push("product:45"); keys.push("product:65"); keys.push("product:80"); cb.get(keys, null, function(errs, docs, metas) { console.log("\n=== get List of documents ==="); console.log( docs );
});
Monday, June 17, 13
Retrieve the Data
What if I want all products or meetups?
key : barcelonajs{ "type": "meetup", "language": "javascript"}
key : product:10{ "type": "product", "name": "Product with id 10"}
Monday, June 17, 13
Calling a view from your app
var queryParams = { stale: false, key : "meetup"}; cb.view("my_views", "by_type", queryParams, function(err, view) { var keys = new Array(); for (var i = 0; i < view.length; i++) { keys.push(view[i].id); } cb.get(keys, null, function(errs, docs, metas) { console.log(docs); }); });
Monday, June 17, 13
Demo Time
Monday, June 17, 13
www.couchbase.com/download
Couchbase Server
Monday, June 17, 13
Q & A
Monday, June 17, 13
Thank [email protected]
@tgrall
Get Couchbase Server at hHp://www.couchbase.com/download
Monday, June 17, 13
Monday, June 17, 13