Oracle NoSQL

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Oracle NoSQL

J. Andrés AraújoPrincipal Sales ConsultantTechnology Sales ConsultingOctober 02, 2014

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 3

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Program Agenda

Introducing NoSQL

Technical Features Review

Summary

1

2

3


Program Agenda

Introducing NoSQL


Summary

1

2

3


The NoSQL Landscape

NoSQL

Columnar & Key/Value• Keyspaces, Tables & Records• Key-based access• Limited Transactions• Broad set of use cases

Document• Collections• Document-based access• JSON & XML• “Objects as documents” use cases

Graph• Interconnected graphs• Relatedness-based access• Properties and Graphs, RDF• Specific use cases

• Developer- centric APIs• Flexible schemas• Partitioned/sharded data

• Horizontally scalable• High Availability via Replication• Integrated with Hadoop


Oracle Database Offer

Embedded+Light weight

Web Scale-out & Real-time events+Schema-less+Relaxed consistency+Built-in availability

Web & Departmental +SQL and NoSQL+Basic security

Mission Critical OLTP and Data Warehousing+SQL, Transactions+High Availability and Security+Multitenant architecture

Oracle Enterprise ManagerOracle Enterprise Manager

Oracle Support


Choose the RIGHT storage option for the jobHadoop Distributed File System

(HDFS) Oracle NoSQL Database Oracle Database

File System Key-Value Database Relational Database

No inherent structure Simple data structure Complex data structures, rich SQL

High volume writes High volume random reads and writes High volume OLTP with 2-PC

Limited functionality, roll-your-own applications

Simple get/put high speed storage, flex configuration

Security, Backup/Restore, Data life cycle mgmt, XML, etc.

Batch Oriented Real-Time, web-scale specialized applications

General purpose SQL platform, multiple applications, ODBC, JDBC


Scalable, HA, Key-Value• Simple Key-Value Data Model• Horizontally Scalable• Highly Available • Simple administration• ACID Transactions at scale• Transparent load balancing• Elastic Configuration• Commercial grade software and

support

What is Oracle NoSQL Database


Big Data Architecture

18

Where Does NoSQL Fit?

Data WarehouseData Reservoir +

Oracle Big Data Connectors

Oracle DataIntegrator

Oracle Advanced Analytics

OracleDatabase

Oracle Spatial & Graph

Oracle NoSQL Database

Cloudera Hadoop

Oracle R Distribution

Oracle Industry Models

OracleGoldenGate


Oracle Event Processing

Oracle Event Processing

Apache Flume

OracleGoldenGate

Oracle Advanced Analytics

Oracle Database

Oracle Spatial & Graph

Oracle Industry Models


Oracle NoSQL Database


Oracle NoSQL Database – Commodity Servers

NoSQL DB Driver

Processing ApplicationData

Connector

Servers

Big Data Acquisition

Real Time Access


Oracle NoSQL Database – Engineered Systems

NoSQL DB Driver

Processing Application

HDFS, Hadoop, CDH

Map ReduceORCH - Stats

Map ReduceHive - Activities

Map ReducePig - Sessionize


Batch Processing

Big Data Appliance


Real Time Access

DataConnector


Program Agenda

Introducing NoSQL


Summary

1

2

3


Release3.0

Features• Key-value, JSON & RDF data• Large Object API• BASE & ACID Transactions• Data Center Support• Online Rolling Upgrade• Online Cluster Management• Table data model• Secondary Indices• Secondary Zones (Data Centers)• Security

Oracle NoSQL DatabaseScalable, Highly Available, Key-Value Database

Application

Storage NodesDatacenter B

Storage NodesDatacenter A

Application

NoSQL DB Driver

Application

NoSQL DB Driver

Application


Scalability ArchitectureApplications View

Elastic Shards (split, add, contract)

Store

Shard

M

Shard

M

R

Shard

M

R R

Application

NoSQL Driver

R R

R

Writes to elected node

Reads from any node in system


Features:Flexible Data Model

• Simple data model – key-value pair (major+minor-key paradigm)• Simple operations – read/insert/update/delete, RMW support• Scope of transaction – records within a major key, single API call• Unordered scan of all data (non-transactional)

Key-Value Pairs

userid

addresssubscriptions

email idphone #expiration date

Major key:

Minor key:

Value:

Strings

Byte Array

Value Options: Key-Value JSON RDF Triples Tables/Rows

picture

.jpg


Store JSON documents

{ "firstName": "John", "lastName": "Smith", “age": 25, “ address": { "streetAddress": "21 2nd Street", "city": "New York", "state": "NY", "postalCode": 10021 }, "phoneNumbers": [ { "type": "home", "number": "212 555-1234" }, { "type": "fax", "number": "646 555-4567" } ] }


What is Avro?

• Serialization protocol • Apache Open Source• Developed by Doug Cutting from Cloudera• C++ and Java language bindings• “Schemas” defined using JSON


Oracle NoSQL – Why AVRO?

• Compact, highly efficient serialization• Synergy with Hadoop• Supports serialization from and to JSON strings• Bindings from serialized formats to language constructs• Easy to use mechanism for schema evolution• Schema definition tracked with schema ID of data writer• Schema versions can be opaque to readers


Avro Schemas

• Defined using a JSON document• Supports records, enums, unions, maps, arrays, fixed• Supports field level aliasing• Supports a sort order for primitive and complex types• Supports schema evolution through reader/writer mechanism


Features:Fexible Data Model

• Benefits– Lower barrier to adoption, shorter time to market– Simplified application modeling– Uses familiar table concepts

• Features– Layered on top of distributed key-value model– Compatible with Release 2.0 JSON schemas– Supports table evolution, retains flexible client access

• Sets foundation for future capabilities

NoSQL DB Table Model (New 3.0)



table create -name Users

add-field -name userid -type integer

add-field -name lname -type string

add-field -name fname -type string

add-field -name email -type string

primary-key -field userid

shard-key -field userid

exit

plan add-table -name Users -wait

Simple Table Example (New 3.0)

Can be specified as a JSON string

Must be proper subset of primary-keyBy default shard-key == primary-key


Features:Fexible Data Model Simple Table Example (New 3.0)

userId lname fname email

Table

Shard Key

Users

ValuePrimary Key



table create -name Users

add-field -name userid -type integer

add-field -name lname -type string

add-field -name fname -type string

add-field -name email -type string

primary-key -field userid

exit

plan add-table -name Users –wait

table create -name Users.Folders

add-field -name foldername -type string

add-field -name msgcount -type integer

add-field -name favorite –type boolean

-default 'F'

primary-key -field foldername

exit

plan add-table -name Users.Folders -wait

Nested Table Example (New 3.0)


Features:Fexible Data Model Nested Table Example (New 3.0)

UserId lname fname email

TableUsers

Users.Folders

[Value]Primary Key

…

[Value]

UserId Foldername msgcount favorite

Primary KeyShard Key


Auto-Sharding• Provides linear scale-out of write

ops/sec• No need to develop sharding logic in

Application• Intelligent Driver – aware of hash

function and shard location• Hash function to map a key to a

partition• Each partition is routed to a single

shard

Features:Sharding

Shard 0

Rep Node

Master

Rep Node

Replica

Rep Node

Replica

Shard 11

Rep Node

Master

Rep Node

Replica

Rep Node

Replica

…

Partitions

Hash Fn


High Availability• Master-Slave replication

• Supports Heterogeneous platform hardware/OS/JVM

• Provides linear scale out for read ops/sec

Features:Replication

Shard 0

Rep Node Master

Rep Node Replica

Rep Node Replica


Automatic Failover• Automatic election of new Master

• Rejoining nodes automatically synchronize with the Master

• Isolated nodes can still service reads

• All nodes are symmetric

Features:Failover

Replication factor = 5

Rep Node Master

Rep Node Replica

Rep Node Replica

Rep Node Replica

Rep Node Replica

New Master


Automated Resource Planning• Storage nodes have indication of

“capacity”• System allocates replicas per storage

node• Intelligent Master/Replica load

balancing• Ensures distribution of replicas• Efficient use of system resources• Reduces operator-caused configuration

errors

Features:Smart Topology Management

Application

Smart Topology Driver


Smart topology• Increase Data Capacity

– Add more storage nodes– New shards automatically created

• Increase Data Throughput– More shards = better write throughput– More replicas/shard = better read throughput

Features:Elastic Cluster Expansion

NoSQL DB Driver

Application

Master

Replica

Replica

StorageNode StorageNode StorageNode

Shard-1

Master

Replica

Replica

Shard-2


Improved Performance• Supports heterogeneous storage

topology• Replicas move from over-utilized to

under-utilized storage nodes• Number of shards and replication

factor remain unchanged

Rebalance an Unbalanced StoreStorage Node 1 Storage Node 2 Storage Node 3

Represents a partition


Greater Flexibility• Configurable Durability per

operation• Configurable Consistency per

operation• ACID by default• Transaction scope is single API call• Records share same major key• Multiple operations supported

Features:Configurable CAP


Java, C, Jython, Jruby, etc.• Simple CRUD operations

– Minimize client/server round trips

• Iteration operations– Returns multiple results in a single API

call– Get full records or just the keys

• Multi and Sub-Key operations– Multiple operations and/or Multiple

records in a single transaction

Features:Easy to use API

JSON

putIfPresent

PUTputIfAbsent

multiGet

deleteAVROgetLOB


Parallel Streaming Interface• Efficient storage and retrieval of

large objects• Client side streaming interface for

low memory consumption• Server side splitting and

distribution of object chunks across nodes for better read/write latency

• Automatic partial LOB detection

Features:Large Objects Support

Large Object

NoS

QL

DB

Driv

er

Appl

icati

on Shard 2

Shard N

Shard 1


Centralized Management• Configuration via CLI commands• Web-based console option• Manages and Monitors

– Configuration changes– Load: Performance: Events:– Alerts: Failure, poor performance, etc.

• SNMP and JMX interfaces

Features:Simple Administration


Easy Prototype Development• Topology Planning• JSON Schema load• Put/Get/Delete data• Aggregates

– Average, Count, Sum

• File based data loading

Features:Data and Management CLI


Availability Zones• Flexible configuration• Metro-Local Quorum

– Low latency writes, HA

• Secondary Read-Only Zones– Analytic workloads– Oracle Reporting– Asynchronous replication

• Topology Aware Client Driver• Provides business continuity and

distributed workload management

Features:Data Center Support (New 3.0)

DC1 DC2 DC3

Metropolitan Zones

ReportsBatch Analytics


What’s the Big Deal• We did do it!• Admin commands available to

describe safe upgrade order• Scripted available hands-free

upgrade experience• Read/Write availability throughout

the upgrade process

Features:Online Rolling Upgrades

72 (24x3) 144 (48x3) 216 (72x3)0

2.5

5

7.5

10

12.5

15

17.5

Online Rolling Upgrade

Total Nodes (Shards x Rep. Factor)

Tim

e to

Upg

rade

(min

)

Ever tried to upgrade a 200 node system while it’s active?


Standard Protocol Support• SNMP (Enterprise Edition Only)

– MIBs in SNMPv2 format– Storage node SNMP agent – admin and rep

nodes– Performance metrics – Via polling– Service change events – Via traps or polling– Notifications (Traps) – Latency and throughput

limits

• Java JMX– Performance Metrics – Via Suscriptions– Service changes events – Via subscriptions– Notifications – Via subscriptions

Features:System Monitoring


No Data Loss!• Backup is done by creating a

snapshot of the store at each node

• Restore from the snapshot– Load utility to restore snapshot into

any topology– Direct Recovery to recover to the exact

topology of the snapshot

Backup & Restore

Oracle Confidential – Internal/Restricted/Highly Restricted 53Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Integration


NoSQL DB integration - RDF

RDF description

GeraldAlex

Steve

NoSQL

knows knows

has skill

has skill


NoSQL DB integration - Avro


NoSQL DB integration - Hadoop

HDFS

HDFS Files

REDUCE

MAP

MAP

MAP

MAP

MAP

MAP

REDUCE

REDUCE


NoSQL DB integration – Oracle RDBMS

Externaltable

SELECT name FROM unstruct_peopleSQL engine

nosql_streampreprocessor


NoSQL DB integration – Oracle Coherence

NoSQL DB DriverTB

Mem

ory


NoSQL DB integration – OEP

NoSQL DBDriver

NoSQL DBDriver


Program Agenda

Introducing NoSQL


Summary

1

2

3


PointFlexibility PointWeb-Scale

Transaction Processing

High velocity, volume, variety, low information density data

capture

Web browsing, Shopping Carts, CDR processing, Sensor data

capture

Web-Scale Personalization

Guaranteed low latency lookups for end-customers

Advertising, Product Recommendations, Online

Catalogs, Social Media, Profile Management, Personalization

Real-Time Event Processing

Real time events trigger rule that perform low latency lookups

Medical Monitoring, Factory Automation, Oil & Gas, Geo-

location

Oracle NoSQL Database Use Case Summary


NoSQL or Relational?

?

NoSQL DBMSSchema private to application

Collections of data objects

Basic Availability, Soft-state, Eventual consistency

Schema shared by multiple applications

Atomic, Consistent, Isolated, Durable

Inter-related collections of data objects

Relational DBMS


Key Value Store Document / Relational Store

Data Collections Integrated Documents and Relational

ACID & Eventual Consistency Joins and Transactions across Documents

Simple Key-Value and Table Model SQL, Reporting, Analytics

Sharded Trusted Technology Foundation: RAC, Multitenant, Security, Availability, In-Memory

Oracle for “NoSQL”

Oracle Enterprise Manager and Support

JSON


Oracle Database 12.1.0.2Flexible Schema development

JSON

JSONVia RESTful service

Via Native APIsData persisted in database

In JSON

SQL

Data analyzed via SQL

Oracle Database 12c

JSON


Oracle NoSQL DB Resources

• NoSQL DB Use Cases, White Papers, Data Sheets, Benchmarkshttp://www.oracle.com/technetwork/products/nosqldb/overview/index.html

• NoSQL DB Documentationhttp://www.oracle.com/technetwork/products/nosqldb/documentation/index.html

• NoSQL DB Downloadshttp://www.oracle.com/technetwork/products/nosqldb/downloads/index.html

• NoSQL DB OTN Forumhttp://forums.oracle.com/forums/forum.jspa?forumID=1388

NoSQL DB version 2.0 Featureshttp://bit.ly/UKn5Sc

• OU Training Classeshttp://bit.ly/V5qbmY

http://www.oracle.com/technetwork/products/nosqldb/overview/index.html

http://www.oracle.com/technetwork/products/nosqldb/documentation/index.html

http://www.oracle.com/technetwork/products/nosqldb/downloads/index.html

http://forums.oracle.com/forums/forum.jspa?forumID=1388

http://bit.ly/UKn5Sc

http://bit.ly/V5qbmY


Oracle NoSQL DB Resources

• Great Application Development article on OTN: http://www.oracle.com/technetwork/articles/bigdata/oracle-nosqldb-appdev-1891870.html

http://www.oracle.com/technetwork/articles/bigdata/oracle-nosqldb-appdev-1891870.html

75Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Q & A
