32
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1

Oracle NoSQL Database release 3.0 overview

Embed Size (px)

DESCRIPTION

Oracle NoSQL Database release 3.0 overview. An overview of the new features in NoSQL DB 3.0, including benefits, technical details and best practices.

Citation preview

Page 1: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1

Page 2: Oracle NoSQL Database release 3.0 overview

Oracle NoSQL Database Release 3.0 Summary

Page 3: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3

Agenda

Oracle NoSQL DB Overview

Oracle NoSQL DB Release 3.0

Best Practices

Page 4: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4

Big Data Architecture

Data Warehouse Data Reservoir +

Oracle Big Data Connectors

Oracle Data Integrator

Oracle

Advanced

Analytics

Oracle

Database

Oracle Spatial

& Graph

Oracle NoSQL

Database

Cloudera Hadoop

Oracle R Distribution

Oracle Industry

Models

Oracle GoldenGate

Oracle Data Integrator

Oracle Event Processing

Oracle Event Processing

Apache Flume

Oracle GoldenGate

Oracle Advanced

Analytics

Oracle Database

Oracle Spatial

& Graph

Oracle Industry

Models

Oracle Data Integrator

Oracle NoSQL Database

Where does NoSQL fit?

Page 5: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5

Simple Data Model

Distributed, Replicated data

Transparent load balancing

Elastic configuration

Simple administration

Enterprise-ready Integration

Commercial grade software and support

Characteristics

Oracle NoSQL Database Scalable, Highly Available, Key-Value Database

Application

Storage Nodes Datacenter B

Storage Nodes Datacenter A

Application

NoSQL DB Driver

Application

NoSQL DB Driver

Application

Page 6: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6

Features

Release

3.0

Oracle NoSQL Database Scalable, Highly Available, Key-Value Database

Application

Storage Nodes Datacenter B

Storage Nodes Datacenter A

Application

NoSQL DB Driver

Application

NoSQL DB Driver

Application

Key-value, JSON & RDF data

Large Object API

BASE & ACID Transactions

Data Center Support

Online Rolling Upgrade

Online Cluster Management

Table data model

Secondary Indices

Secondary Zones (Data Centers)

Security

Page 7: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7

Scalability Architecture – Applications View

Elastic Shards

(split, add, contract)

Store

Shard

M

Shard

M

R

Shard

M

R R

Application

NoSQL Driver

R R

R

Writes to elected

node

Reads from any

node in system

Page 8: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8

Oracle NoSQL Database

Predictability

Reliability & Support

Integration

When you need:

Web-Scale Transactions, Personalization

Sensor Data Management

Real-Time Event Processing

For Applications that do:

Page 9: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9

Agenda

Oracle NoSQL DB Overview

Oracle NoSQL DB Release 3.0

Best Practices

Page 10: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 10

Oracle NoSQL DB Release 3.0 Enterprise Ready

Ease of Adoption

Security

Business Continuity

Page 11: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 11

Oracle NoSQL DB Release 3.0

Table data model support

– Simplifies data modeling & leverages existing “table” expertise

Secondary indexing

– Allow indexing on any component of the record

– Huge performance benefit due to parallel, indexed data access

Data centers

– Metro-area zones for disaster recovery and business continuity

– Secondary zones for read-only workloads

Security

– Authentication and network-level encryption

Feature Summary

Page 12: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 12

Oracle NoSQL DB Release 3.0

Lower barrier to adoption, shorter time to market

Simplified application modeling

– Uses familiar table concepts

– Introduces strongly typed fields

– Easy to/from JSON with full type support and AVRO serialization

– Allows secondary indexing of non-key data

Sets foundation for SQL query access from Oracle DB

Previous Key/Value and JSON schema APIs still supported

Table Data Model Benefits

Page 13: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 13

Oracle NoSQL DB Release 3.0

Layered on top of distributed key-value model

Compatible with most Release 2.0 JSON schemas

Defines JSON schema automatically

Supports table evolution

New access API’s with table terminology

Retains flexible client access

Administrative CLI for schema creation and evolution

Table Data Model Characteristics

Page 14: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 14

Oracle NoSQL DB Release 3.0

Distributed Table records or “JSON Documents” – you choose

Records grouped locally (by shard-key)

ACID & BASE transactions

Automatic mapping of Major/Minor key structure

Enables future External Table improvements

Rich queries

Predicate push down

SQL query language

Table Data Model Advantages

Page 15: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 15

Oracle NoSQL DB Release 3.0

table create -name Users

add-field -name userid -type integer

add-field -name lname -type string

add-field -name fname -type string

add-field -name email -type string

primary-key -field userid

shard-key -field userid

exit

plan add-table -name Users -wait

Simple Table Example

Can be specified as a JSON string

Must be proper subset of primary-key

By default shard-key == primary-key

Page 16: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 16

Oracle NoSQL DB Release 3.0 Simple Table Example

userId lname fname email

Table

Shard Key

Users

Value Primary Key

Page 17: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 17

Oracle NoSQL DB Release 3.0

table create -name Users

add-field -name userid -type integer

add-field -name lname -type string

add-field -name fname -type string

add-field -name email -type string

primary-key -field userid

exit

plan add-table -name Users –wait

table create -name Users.Folders

add-field -name foldername -type string

add-field -name msgcount -type integer

add-field -name favorite –type boolean

-default 'F'

primary-key -field foldername

exit

plan add-table -name Users.Folders -wait

Nested Table Example

Page 18: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 18

Oracle NoSQL DB Release 3.0 Table Data Model – Nested Tables

UserId lname fname email

Table

Users

Users.

Folders

[Value] Primary Key

[Value]

UserId Foldername msgcount favorite

Primary Key Shard Key

Page 19: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 19

Oracle NoSQL DB Release 3.0

Broader low latency use case support

– Primary and Secondary Indexes

– Indexes automatically maintained

– Provides sorted results

Low resource cost with high performance

– Low cardinality matching, guaranteed consistent

– Shard-local indexing, low latency record access

– B-trees locally ordered, scanned in parallel, merged result set

– Improved index cache eviction

Secondary Index support

Page 20: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 20

Get() API specifies index

Equality or range searches

Low order multi-value indexes

– Composite index on values

– Parallel Index Scan

– Ordered results

– Indexable single element arrays

Oracle NoSQL DB Release 3.0 Secondary Indexes

Index on (lname, fname)

UserId fname lname email

Table

Users

Equality search on lname + fname

Equality search on lname and range on fname

Range or Equality search on lname by itself

Can’t search on just fname

Page 21: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 21

Oracle NoSQL DB Release 3.0

plan add-index –table Users -name Users_idx1 –field lname –field fname

1. Defines index on all shards

2. In parallel (per shard) starts table scan via primary key, populates

index

3. Updates get populated into index automatically

4. When all shards are complete, index is available for access

Simple Example

Page 22: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 22

Secondary Index Storage

Pkey A Value A

F1 F2 F3 V1 V2 V3

Pkey B Value B

F1 F2 F3 V1 V2 V3 Skey A Pkey A

V2 V3 F1 F2 F3

Skey B Pkey B

V2 V3 F1 F2 F3

Skey A Pkey A

V1 F1 F2 F3

Skey B Pkey B

V1 F1 F2 F3

Primary Records

Secondary Indexes Shard 1

Writes

Reads

Shard 2

Shard N

Index on (V1)

Index on (V2, V3)

Page 23: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 23

Oracle NoSQL DB Release 3.0

Metro-Local Quorum

– Low latency writes, HA

2nd’ary Read-Only Zones

– Analytic workloads

– Report generation

– Asynchronous replication

New Read Consistency: No Master

Topology Aware Client Driver

Provides business continuity and distributed workload management

Zones

DC1 DC2 DC3

Metropolitan Zones

Reports

Batch Analytics

Page 24: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 24

Oracle NoSQL DB Release 3.0

Protected Access to Data

– Enforcement is configurable

Authentication

– OS independent, password based user access

– Internal cluster components self authenticate (HA password distribution)

– Oracle Wallet integrated

– Admin utility to generate self-signed certificates

Wire level data encryption

– All client-server, server-server channels SSL encrypted

Security Overview

Page 25: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 25

Oracle NoSQL DB Release 3.0

Port Restrictions for client and intra server communications

SSL Certificates stored, protected and obfuscated at the server

Client configured login-file or truststore for SSL credential management

Authenticated client sessions have admin configurable time-out

Auth Exceptions: AuthRequired, AuthFailed, AccessDenied

Supports 2 roles: Admin & User

System Admin manages security via Admin CLI

Security Features

Page 26: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 26

Agenda

Oracle NoSQL DB Overview

Oracle NoSQL DB Release 3.0

Best Practices

Page 27: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 27

Choosing a Data Model

Tables

– Highest level abstraction, simple to model, familiar to developers

– System managed Secondary indices, Table evolution support

– Future: Security per table, Query Language

JSON

– Medium level abstraction, need to model keys (strings), appealing to JSON-centric applications

– Application managed Index Views, Schema evolution support

– Future: Limited security

Raw Key-Value

– Lowest level abstraction, need to model keys (strings), application serialized data, maximum

flexibility

– Application managed Index Views, record evolution and security – roll your own

Page 28: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 28

Metrics and Best Practices

No overhead, performs same as JSON schemas with AVRO

serialization

Use nested tables to encapsulate record types in a hierarchy

Use Arrays and Arrays of Records to store self contained sets

Record types can’t be indexed -> flatten into simple types if indexing is

required

Tables

Page 29: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 29

Metrics and Best Practices

Same rationale as an RDBMS

– Reduces time for lookups/range scans

– Increases overhead for updates

– Provides sorted results

– Add indexes when required

No optimizer -- application picks index to be used

Remember to calculate cache size requirement

New CLI option for get: -reportsize tells you the size of a key. Use output with

DBCacheSize

Indexes

Page 30: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 30

Metrics and Best Practices

Data Centers may add write latency, depending on durability policy and

inter-data center latency

Options for tuning:

– Consider reducing durability policy ACK requirement if latency is high

– Consider adding processing threads to use additional CPU

1. Allow Client Driver to perform load balancing

2. Use Secondary Zones for read-only, batch, analytical workloads

3. Use No-Master read consistency in the application

Data Centers/Zones

Page 31: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 31

Page 32: Oracle NoSQL Database release 3.0 overview

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 32