Upload
paradigma-digital
View
42
Download
0
Embed Size (px)
Citation preview
Introducción a Couchbase
Manuel HurtadoSolutions Engineer
What is Couchbase?
©2015 Couchbase Inc. 3
Couchbase Server Defined
§ Couchbase Server 4.X is the first NoSQL Database to enable you to develop with agility and operate at any scale.
Managed Cache Key-Value Store Document Database Embedded Database Sync Management
©2015 Couchbase Inc. 4
The Power of the Flexible JSON Schema
Ability to store data in multiple ways• Denormalized single document, as opposed to normalizing data across multiple table• Dynamic Schema to add new values when needed
©2015 Couchbase Inc. 5
Document Fundamentals -‐ Composition
§ Similar to primary keys in relational databases§ Documents are partitioned based on the
document ID§ ID based document lookup is extremely fast § Must be unique
§ JSON§ Binary -‐ integers, strings, booleans§ Common binary values include serialized objects,
compressed XML, compressed text, encrypted values
Document ID or Key
Value
§ CAS Value (unique identifier for concurrency)§ TTL§ Flags (optional client library metadata)§ Revision #
Metadata
Document
Document ID / Key
MetadataValue
©2015 Couchbase Inc. 6
Storing and Retrieving Documents
©2015 Couchbase Inc. 7
Couchbase Data Access
• Everything is layered on top of Key Value
• A Document store is a special case of Key-Value
• Views provide aggregation and real-time analytics through incremental map-reduce
• Global Secondary Indexes provide low latency/high throughput indexes
• N1QL is a language that provides a powerful and expressive way of accessing documents
©2015 Couchbase Inc. 8
Choose how to build your apps, and where to run themCouchbase supports a wide range of frameworks, languages, platforms, and infrastructure choices
©2015 Couchbase Inc. 9
Docker Support
§ Deployment Scenarios§ Single host, single container§ Single host multiple containers§ Multiple hosts, single container on each host§ Multiple hosts, multiple containers per host
§ https://hub.docker.com/_/couchbase/
©2015 Couchbase Inc. 10
Top Use Cases
Profile Management Personalization 360o Customer View Internet of Things
Content Management Catalog Real Time Big Data Digital Communication
Mobile Applications
High Availability Caching
©2015 Couchbase Inc. 11
World of Operational Databases
11
Graph Stores
Neo4j
RDBMS/Transactional Stores
Oracle, DB2, SQLServer, MySql, Postgres
Caching Stores
Coherence, Redis, Gemfire
Full-‐text Stores
Lucene, SOLR, ElasticSearch
Key/Value Stores
Hbase,Cassandra,
Riak, Dynamo
Document Stores
Marklogic, mongoDB, Cloudant, DocumentDB
Couchbase
Couchbase ArchitectureSingle Node
12
©2015 Couchbase Inc. 13
Couchbase Architecture
• Data Service – builds and maintains Distributed secondary indexes (MapReduce Views)
• Indexing Engine – builds and maintains Global Secondary Indexes
• Query Engine – plans, coordinates, and executes queries against either Global or Distributed indexes
• Cluster Manager – configuration, heartbeat, statistics, RESTful Management interface
©2015 Couchbase Inc. 14
Couchbase Read Operation
APPLICATION SERVER
MANAGED CACHE
DISK
DISKQUEUE
REPLICATIONQUEUE
DOC 1
GETDOC 1
DOC 1
©2015 Couchbase Inc. 15
APPLICATION SERVER
MANAGED CACHE
DISK
DISKQUEUE
REPLICATIONQUEUE
Write Operation
DOC 1
DOC 1DOC 1
©2015 Couchbase Inc. 16
Auto sharding – Bucket and vBuckets
vB
Data buckets
vB
1 ….. 1024
Virtual buckets
§ A bucket is a logical, unique key space§ Multiple buckets can exist within a single cluster of nodes
§ Each bucket has active and replica data sets (1, 2 or 3 extra copies)§ Each data set has 1024 Virtual Buckets (vBuckets)§ Each vBucket contains 1/1024th portion of the data set§ vBuckets do not have a fixed physical server location
§ Mapping between the vBuckets and physical servers is called the cluster map
§ Document IDs (keys) always get hashed to the same vbucket§ Couchbase SDK’s lookup the vbucket -‐> server mapping
©2015 Couchbase Inc. 17
Cluster Map
Couchbase Cluster
CLUSTER MAP
Hashing Algorithm
CRC32
vBucket2
vBucket3
vBucket4
vBucket5
vBucket6
vBucket7
vBucket1024
vBucket1
...
Couchbase SDK
©2015 Couchbase Inc. 18
Cluster Map
Couchbase Cluster
CLUSTER MAP
Hashing Algorithm
CRC32
vBucket2
vBucket3
vBucket4
vBucket5
vBucket6
vBucket7
vBucket1024
vBucket1
...
Couchbase SDK
©2015 Couchbase Inc. 19
Cluster Map – 2 nodes added
Couchbase Cluster
CLUSTER MAP
Hashing Algorithm
CRC32
vBucket2
vBucket3
vBucket4
vBucket5
vBucket6
vBucket7
vBucket1024
vBucket1
...
Couchbase SDK
©2015 Couchbase Inc. 20
ACTIVE ACTIVE ACTIVE
REPLICA REPLICA REPLICA
Couchbase Server 1 Couchbase Server 2 Couchbase Server 3
Basic Operation
SHARD5
SHARD2
SHARD9
SHARD SHARD SHARD
SHARD4
SHARD7
SHARD8
SHARD SHARD SHARD
SHARD1
SHARD3
SHARD6
SHARD SHARD SHARD
SHARD4
SHARD1
SHARD8
SHARD SHARD SHARD
SHARD6
SHARD3
SHARD2
SHARD SHARD SHARD
SHARD7
SHARD9
SHARD5
SHARD SHARD SHARD
©2015 Couchbase Inc. 21
Add Nodes to Cluster
ACTIVE ACTIVE ACTIVE
REPLICA REPLICA REPLICA
Couchbase Server 1 Couchbase Server 2 Couchbase Server 3
ACTIVE ACTIVE
REPLICA REPLICA
Couchbase Server 4 Couchbase Server 5
SHARD5
SHARD2
SHARD SHARD
SHARD4
SHARD SHARD
SHARD1
SHARD3
SHARD SHARD
SHARD4
SHARD1
SHARD8
SHARD SHARD SHARD
SHARD6
SHARD3
SHARD2
SHARD SHARD SHARD
SHARD7
SHARD9
SHARD5
SHARD SHARD SHARD
SHARD7
SHARD
SHARD6
SHARD
SHARD8
SHARD9
SHARD
READ/WRITE/UPDATE
©2015 Couchbase Inc. 22
Fail Over Node
ACTIVE ACTIVE ACTIVE
REPLICA REPLICA REPLICA
Couchbase Server 1 Couchbase Server 2 Couchbase Server 3
ACTIVE ACTIVE
REPLICA REPLICA
Couchbase Server 4 Couchbase Server 5
SHARD5
SHARD2
SHARD SHARD
SHARD4
SHARD SHARD
SHARD1
SHARD3
SHARD SHARD
SHARD4
SHARD1
SHARD8
SHARD SHARD
SHARDSHARD6
SHARD2
SHARD SHARD SHARD
SHARD7
SHARD9
SHARD5
SHARD SHARD
SHARD
SHARD7
SHARD
SHARD6
SHARDSHARD8
SHARD9
SHARD
SHARD3
SHARD1
SHARD3
SHARD
©2015 Couchbase Inc. 23
Modern Architecture – Multi-Dimensional Scaling
MDS is the architecture that enables independent scaling of data, query, and indexing workloads while being managed as
one cluster.
©2015 Couchbase Inc. 24
Modern Architecture – Multi-Dimensional Scaling
©2015 Couchbase Inc. 25
Modern Architecture – Multi-Dimensional Scaling
©2015 Couchbase Inc. 26
Demo: Install Couchbase
§ Install node 1§ Add node 2§ Rebalance§ Admin UI
beer-‐sample travel-‐sample
node 1192.168.56.101
node 2192.168.56.102
Cluster Madrid prod
Cross Datacenter Replication (XDCR)
27
©2015 Couchbase Inc. 28
XDCR: Cross Data Center Replication
§ Application can access both clusters (master – master)§ Scales out linearly§ Within a cluster operations are strongly consistent and you can read-‐your-‐own-‐write. Between clusters mutations are eventually consistent.
©2015 Couchbase Inc. 29
Memory-‐to-‐memory, topology aware replication
New York
San Francisco
SF Server Cluster
Couchbase Server 1
MEMORY DISK
Couchbase Server 2
MEMORY DISK
Couchbase Server 3
MEMORY DISK
NYC Server Cluster
Couchbase Server 1
MEMORY DISK
Couchbase Server 2
MEMORY DISK
Couchbase Server 3
MEMORY DISK
Couchbase Server 4
MEMORY DISK
USE CASES
§ Access by Localization§ Active-‐Active§ Active-‐Passive§ Disaster Recovery§ BI environment§ Test environment
©2015 Couchbase Inc. 30
Demo: XDCR: Replicate data between clusters
§ Install node 3 / Cluster backup§ Enable XDCR between clusters§ Replicate airlines
Developing an Application
31
©2015 Couchbase Inc. 32
Languages and Interfaces for Couchbase
§ Official SDKs– Java– .NET– Node.js– Python
§ For each of these we have– Full Document support– Interoperability– Common Programming Model
Others: Erlang, Perl, TCL, Clojure, Scala
– PHP– C– Go– Ruby
JDBC and ODBC
©2015 Couchbase Inc. 33
Interacting with Data
View
s
N1Q
LQuery
Key Value Store
JSON Documents
Document Modeling
©2015 Couchbase Inc. 35
Data Modeling
§ An ecommerce order in relational world
©2015 Couchbase Inc. 36
Data Modeling
§ Embed vs Refer
©2015 Couchbase Inc. 37
Data Modeling
§ Embed vs Refer
©2015 Couchbase Inc. 38
Reference Documents for Lookups
38
§ Lookup documents which reference the primary one§ PROS§ High Performance§ Extremely scalable§ Immediately consistent
§ CONS§ Needs to be maintained by the application
§ GSI is a viable alternative§ PROS§ No need for Referential integrity§ Good performance
§ CONS§ Not immediately consistent -‐ but can optional wait
for index updates (READ YOUR OWN WRITES)
©2015 Couchbase Inc. 39
Compound keys: example
u::1001
{
"name": "Matthew Revell",
"email": "[email protected]",
"address": "11-21 Paul Street",
"city": "London",
"postCode": "EC2A 4JU",
"telephone": "44-20-3837-9130",
"orders": [ 1, 9, 698, 32 ],
“productsViewed”: [8, 33, 99, 100]
}
©2015 Couchbase Inc. 40
Compound keys: example
u::1001
{
"name": "Matthew Revell",
"email": "[email protected]",
"address": "11-21 Paul Street",
"city": "London",
"postCode": "EC2A 4JU",
"telephone": "44-20-3837-9130",
"orders": [ 1, 9, 698, 32 ]
}
u::1001::productsviewed
{"productsList": [
8, 33, 99, 100]
}
©2015 Couchbase Inc. 41
Compound keys: example
u::1001
{
"name": "Matthew Revell",
"email": "[email protected]",
"address": "11-21 Paul Street",
"city": "London",
"postCode": "EC2A 4JU",
"telephone": "44-20-3837-9130",
"orders": [ 1, 9, 698, 32 ]
}
u::1001::productsviewed
{"productsList": [
8, 33, 99, 100]
}
p::8
{
id": 1,"name": "T-shirt","description": "Red Couchbase shirt","quantityInStock": 99,"image": "tshirt.jpg”
}
©2015 Couchbase Inc. 42
Compound keys: example
u::1001
{
"name": "Matthew Revell",
"email": "[email protected]",
"address": "11-21 Paul Street",
"city": "London",
"postCode": "EC2A 4JU",
"telephone": "44-20-3837-9130",
"orders": [ 1, 9, 698, 32 ]
}
u::1001::productsviewed
{"productsList": [
8, 33, 99, 100]
}
p::8
{
id": 1,"name": "T-shirt","description": "Red Couchbase shirt","quantityInStock": 99
}
p::8::img
“http://someurl.com/tshirt.jpg”
©2015 Couchbase Inc. 43
Demo: Code examples: CRUD
Sample code for:
§ Create § Read§ Update§ Delete
Couchbase Query (N1QL)N1QL Introduction
©2015 Couchbase Inc. 45
Why SQL for JSON?
JSON Documents Provide
Rich Structure(No assembly)
Structure Evolution(Flexible schema, easy to change)
SQL Provides
General query functionality
Query across relationships
Why N1QL?
Most developers already know SQL
No need for complex query frameworks
©2015 Couchbase Inc. 46
SELECT
§ Standard SELECT pipeline
– SELECT, FROM, WHERE, GROUP BY, ORDER BY, LIMIT, OFFSET
§ Queries across relationships
– JOINs
– Subqueries
– NEST — a JOIN that embeds child objects within their parent
– UNNEST — a JOIN that surfaces nested objects as top-‐level data
§ Aggregation
§ Set operators
– UNION, INTERSECT, EXCEPT
©2015 Couchbase Inc. 47
Indexing
§ CREATE / DROP INDEX
§ Two types of indexes– View indexes– GSI indexes (global secondary indexes)
§ Can index any data expression– Nested / complex expressions– Computed expressions
§ EXPLAIN
©2015 Couchbase Inc. 48
Data writes
§ UPDATE … WHERE …– Partial updates; deep updates
§ DELETE … WHERE …– Deeply nested conditions
§ INSERT … VALUES …; INSERT … SELECT …– Bulk insert; transfer and transformation
§ MERGE– INSERT or UPDATE; ETL support
©2015 Couchbase Inc. 49
Document-‐oriented Extensions
§ Nested
– Paths—user.profile.email, children[0], children[0:2]
– NEST, UNNEST
– Ranging—EVERY child.age > 5 FOR child IN children END
– Transformations—SELECT {“name”: first_name || last_name}
§ Non-‐uniform
– IS MISSING
– Type checking and conversion
§ Distributed– Direct lookup—USE KEYS
– Efficient joins—ON KEYS
©2015 Couchbase Inc. 50
Expressions
©2014 Couchbase, Inc. 50
Expressions
Literalsq Primitives [ 0, ‘hello’, TRUE ]
q NULL
Operators
q Arithmetic [ +, -‐, *, /, % ]
q Logical [ AND, OR, NOT ]
q Comparison [ <, <=, =, !=, >=, >, BETWEEN, IS NULL ]
q Pattern matching [ LIKE ]
q Conditional [ CASE ]
Scalar functions
q Numeric [ trigonometric, ROUND, TRUNC, … ]
q String [ UPPER, LOWER, TRIM, SUBSTR, … ]
q Date [ string and numeric dates, NOW, date arithmetic, … ]
Aggregate functions q MIN, MAX, SUM, AVG, COUNT [ DISTINCT ]
Subqueries q Subqueries are full expressions
©2015 Couchbase Inc. 51
Document-‐Oriented Expression Features
©2014 Couchbase, Inc. 51
Features
Ranging over collectionsWHERE ANY c IN children SATISFIES c.age > 10 END
WHERE EVERY r IN ratings SATISFIES r > 3 END
Mapping with filtering ARRAY c.name FOR c IN children WHEN c.age > 10 END
Deep traversal and updateWHEREANY node WITHIN request SATISFIES node.type = “xyz” END
UPDATE doc SET c.name = Title(c.name) FOR c WITHIN doc END
Dynamic ConstructionSELECT { “a”: expr1, “b”: expr2 } AS obj1, name FROM … // Dynamic object
SELECT [ a, b ] FROM … // Dynamic array
Nested sourcing SELECT * FROM a.b.c … // Select from nested objects
©2015 Couchbase Inc. 52
Demo: N1QL samples
N1QL Queries from
§ Web UI§ Command line cbq§ REST
Summary
53
©2015 Couchbase Inc. 54
Develop with Agility
With Couchbase Server, you can Develop Applications with Agility. Here’s why:
Easier, Faster Development Flexible Data Modeling Powerful Querying
SQL Integration & Migration Big Data Integration Mobile / IoT
©2015 Couchbase Inc. 55
Operate at Any Scale
With Couchbase Server, you can deploy, manage, and scale apps without limits. Here’s why:
Elastic Scalability Consistent High Performance
Always-on Availability
Multi-Data Center Deployment Simple, Powerful Administration
Enterprise Grade Security
Q&A
56
©2015 Couchbase Inc. 57
Additional Resources
57
§ Couchbase Labs: https://github.com/couchbaselabs
§ Sample Applications:§ https://github.com/couchbaselabs?utf8=%E2%9C%93&query=try§ https://github.com/couchbaselabs?utf8=%E2%9C%93&query=beer
§ General Docs: http://docs.couchbase.com
§ Query Portal: http://query.couchbase.com
§ Blog: http://blog.couchbase.com
§ Forum: http://forums.couchbase.com
©2015 Couchbase Inc. 58
beer-‐sample travel-‐sample
node 1192.168.56.101
node 2192.168.56.102
Cluster Madrid prod
airlines
node 3192.168.56.1013
Cluster Backup
XDCR
airlines
©2015 Couchbase Inc. 59
airlines
node 3192.168.56.1013
Cluster Backup