Latest trends in database management

Latest Trends in Database Management

by Prabath KumarasingheSenior Tech Lead at Ensiz PVT Ltd

MSc in Web Technology (University of Southampton, UK)Bachelor of Computer Science(University of Colombo, SL)

RDBMS

RDBMS Products

New trends in RDBMS

Content

1

2

3

NoSQL Databases

NoSQL Products

New trends in NoSQL Databases

Content contd ...

4

5

6

Cloud Databases

Cloud Products

New trends in Cloud Databases

Content contd ...

7

8

9

Graph Databases

Graph Products

New trends in Graph Databases

Content contd ...

10

11

12

Database Management Systems based on relational Model

Query language is SQL

● Firstly proposed by Edgar F. Codd in 1969.

● Contains tables with rows, columns and primary key.

● Maintain relationship between tables using foreign key.

● SQL was one of the first commercial languages for Edgar F. Codd's

relational model.

● SQL consists of a data definition language (DDL) and a data

manipulation language (DML).

RDBMSRelational Database Management Systems

Data Definition Language (DDL)

Data Manipulation Language (DML)

● This is a sub set of SQL.

● SQL: CREATE, DROP, ALTER statements are coming under this

definition.

● As above this also sub set of SQL.

● Popular CRUD (Create, Read, Update, Delete) operations can be

performed.

● SQL: INSERT, SELECT, UPDATE, DELETE statements are coming

under this definition.

DDL and DML

Database transactions

ACID

● Ensure some level data integrity and fault tolerance.

● Properties of database transaction are Atomicity, Consistency,

Isolation and Durability (ACID).

● Atomicity: Database operation either all occur or nothing occur.

● Consistency: Ensuring integrity constraints during database

operations.

● Isolation: One operation become visible to other concurrent

operations.

● Durability: Ensure transaction will survive permanently .

Transactions

Database normalization

● Organizing the tables and fields to minimize the redundancy and

dependency.

● This is a process of dividing large table into small tables with

relations.

● Normalization concept are 1NF, 2NF, 3NF and BCNF (Boyce-Codd

Normal Form)

● A relational database table is often described as "normalized", if it is

in the Third Normal Form (3NF).

Database Normalization

RDBMS Products

● Oracle

● Microsoft SQL Server

● MySQL

● MariaDB

● PostgreSQL

RDBMS Products

History

● In 1977 Larry Ellison saw a IBM Journal of Research andDevelopment, discovered a research paper that described a working

prototype for a relational database management system (RDBMS). Showing it to coworkers Bob Miner and Ed Oates at Ampex .

● They started Software Development Company to build the Oracle,

which was the first commercial SQL based RDBMS.

Oracle Database

MILSTONES

● 1978 Oracle version 1, written in assembly language. But Oracle

verson 1 never released.

● 1979 Oracle Version2, was the first commercial SQL Relational

Database Management System.

MILSTONES

Oracle Database Contd ...

● 1983 Oracle version 3, written in C language. Is the first RDBMS to

run on Mainframes, minicomputer and PCs.

● 1985 Oracle Version5, was the first RDBMS to support client-server

environment.

● 1988 Oracle Version6, allow multiple users to work on single table,

hot-backup reduce system maintenance, PL/SQL allow uses to

process data while it remains in the database.

● 1992 Oracle Version7, with ground breaking functionality and

several architectural changes.

● 1996 Oracle Version 7.3, to manage any type of data text, video,

maps, sound, or images.

MILSTONES


● 2001 Oracle version 9i, adds Oracle Real Application Clusters which

allow customers to run their IT on connected low-cost servers,

expanding performance, scalability and availability.

● 2003 Oracle Version 10g, the first grid computing product available

for the enterprise. This allow to process the load based on demand.

● 2007 Oracle Version 11g, It's proven to be fast, reliable, secure and

easy to manage for all types of database workloads including

enterprise applications, data warehouses and big data analysis

Oracle 11g features

● Lower IT Costs with Oracle Database 11g

● Storage management, memory management

, statistics collection, backup and recovery,

and SQL tuning have all been automated.

● Develop applications quickly as well ( All programming

languages database drivers are fully integrate with Oracle 11g).

● Oracle Application Express (with limited programming

knowledge user can build applications) .

● Maximum security and availability.

● Reduce the Hardware costs by a factor of 5X.

● Improve performance by a factor of 10x.

● Reduce storage costs by a factor of 10x.


Oracle 11g features

● Database Consolidation Onto Private Clouds

● A private cloud is an efficient way to deliver database services

because it enables IT departments to consolidate servers,

storage, and database workloads onto a shared hardware and

software.

● Business Drivers for consolidating databases onto a Private

cloud

● A Consolidating databases onto a private cloud is typically done

in one of two ways:Infrastructure Cloud or Database Cloud.

● Supports, building Private Cloud Infrastructures.

● There are many Oracle Technologies for consolidating

databases on private clouds.


Oracle 11g features

● Oracle: Big Data for the Enterprise

● Oracle Big Data Appliance : comes with full rack configuration

with 18 Sun servers for a total storage capacity of 648TB.

● CDH and Cloudera Manager : CDH consists of 100% open

source Apache Hadoop, Cloudera Manager is an end to end

management application for CDH.

● Oracle Big Data Connectors : enables an integrated data set

for analyzing all data.

● Oracle NoSQL Database : is a distributed, highly scalable, key

value database based on Oracle Berkeley DB.

● In-Database Analytics: easy to use tools for in database,

advanced analytics.


History

Milestones

● In May 1989, Ashton-Tate/Microsoft SQL Server version 1.0

shipped. Press reviews were good, but sales lagged.

● 1989 - SQL Server 1.0 (16 bit) released.


● 1993 - SQL Server 4.21 released.




● 2000 - SQL Server 2000 released.

Microsoft SQL Server

Milestones

● 2003 - SQL Server 2000 (64 bit Edition) released.



● 2010 - SQL Azure DB released (Cloud database).

● 2010 - SQL Server 2008 R2 released.


Microsoft SQL Server contd ...

SQL Server 2012

● SQL Server 2012 will provide Mission Critical Confidence with

greater uptime, blazing-fast performance and enhanced security

features for mission critical workloads; Breakthrough Insight with

managed self-service data exploration and stunning interactive data

visualizations capabilities

SQL Server 2012

● Manage data of any type or size : able to manages data of any

type, whether structured or unstructured, and of any size – from

gigabytes to petabytes.

● Enrich your data with the worlds data : By connecting to external

data sources like U.S. Census Bureau, United Nations, etc .. user can

begin to answer new types of questions and deliver new value in

ways that previously were not possible.

● Gain insight from any data : deliver new insights from all types of

data- structured, unstructured, previously archived or discarded.

Microsoft SQL Server contd ...

MySQL

Milestones

● The worlds most popular open source database.

● Original development of MySQL by Michael Widenius and David

Axmark beginning in 1994.

MySQL

● 1996 – MySQL version 3.19 released.







Milestones

MySQL 5.6

MySQL

● MySQL 5.6 is the best release ever of the world's most popular open

source database and provides a new, advanced feature set designed

to enable those who are building the next generation of web-based

and embedded applications and services.


● Sun Microsystems acquired MySQL in 2008.


● Oracle acquired Sun Microsystems on 27 January 2010.



MySQL 5.6

MySQL

● Better Performance and Scalability.

● Improved better transactional throughput.

● Improved Optimizer for better query execution times and

diagnostics.

● Better Application Availability.

● Better Developer Support.

● Improved Replication for high performance and self-healing cluster

deployments.

● Improved Performance Schema for better instrumentation and

monitoring.

● Improved Security for worry-free application deployments

and other important enhancements.

MariaDB

Milestones

● Database server developed by some of the original authors of

MySQL, offers drop-in replacement functionality.

● Its lead developer is Michael Widenius, the founder of MySQL and

Monty Program AB

● 2009 – MariaDB version 5.1 released.




● 2012 – MariaDB version 10.0.0 alpha released.

MariaDB

MariaDB 5.5 vs MySQL 5.6

● More Storage Engines than MySQL.

● Speed improved compared to MySQL.

● Extensions & new features.

● Microseconds in MariaDB .

● GIS Functionality, etc ..

● Better Testing.

● Fewer warnings and bugs.

● Truly Open Source.

MariaDB contd ….

PostgreSQL

Milestones

● The world most advance open source Database Management

System.

● is ACID-compliant, is fully transactional and has extensible data

types, operators, index methods, functions, aggregates, procedural

languages, and has a large number of extensions written by third

parties.

● 1996 – PostgreSQL version 1.0 released.





PostgreSQL

PostgreSQL 9.0

● PostgreSQL is a powerful, open source object-relational database

system. It has more than 15 years of active development and a

proven architecture that has earned it a strong reputation for reliability,

data integrity, and correctness.

● Support nested transactions.

● It supports international character sets.

● It is highly scalable both in quantity of data it can manage and in the

number of concurrent users it can accommodate.

● PostgreSQL, allowing it to be used as a spatial database for

geographic information systems (GIS).

PostgreSQL

NoSQL Databases

● A NoSQL database provides a mechanism for storage and retrieval

of data that use looser consistency models than traditional relational

databases in order to achieve horizontal scaling and higher

availability.

●In short, NoSQL database management systems are useful when

working with a huge quantity of data (especially big data) when the

data's nature does not require a relational model.

●The data can be structured, but NoSQL is used when what really

matters is the ability to store and retrieve great quantities of data, not

the relationships between the elements.

●NoSQL cannot necessarily give full ACID guarantees.

NoSQL Databases

Advantages in NoSQL

● Elastic scaling : they’re usually designed with low-cost commodity

hardware in mind therefore scaling easy to handle.

● Big data: Today, the volumes of “big data” that can be handled by

NoSQL systems, such as Hadoop, outstrip what can be handled by

the biggest RDBMS.

● Goodbye DBA : NoSQL databases are generally designed from the

ground up to require less management: automatic repair, data

distribution, and simpler data models lead to lower administration.

● Economics : NoSQL databases typically use clusters of cheap

commodity servers to manage the exploding data and transaction

volumes, while RDBMS tends to rely on expensive proprietary servers

and storage systems.

● Flexible data models : NoSQL databases have far more relaxed or

even nonexistent data model restrictions.

NoSQL Databases continued.

Challenges in NoSQL

● Maturity : RDBMS systems have been around for a long time. The

maturity of the RDBMS is reassuring. In NoSQL many key features

yet to be implemented.

● Support : In contrast, most NoSQL systems are open source

projects, and although there are usually one or more firms offering

support for each NoSQL database.

● Analytics and business intelligence :NoSQL databases offer few

facilities for ad-hoc query and analysis. Even a simple query requires

significant programming expertise.

● Administration : NoSQL today requires a lot of skill to install and a

lot of effort to maintain.

● Expertise : There are literally millions of developers throughout the

world, and in every business segment, who are familiar with RDBMS

concepts and programming. almost every NoSQL developer is in a

learning mode.

NoSQL Databases continued.

NoSQL Database Products

● MongoDB

● CouchDB

● Oracle NoSQL Database

● OrientDB

● Apache Cassandra

NoSQL Database Products

MongoDB

● MongoDB (from "humongous") is an open source

document-oriented database system developed and supported by

10gen. It is part of the NoSQL family of database systems.

●Instead of storing data in tables as is done in a "classical" relational

database, MongoDB stores structured data as JSON-like documents

with dynamic schemas (MongoDB calls the format BSON), making

the integration of data in certain types of applications easier and

faster.

● 10gen began development of MongoDB in October 2007. The

database is used by MTV Networks, Foursquare, UIDA etc...

MongoDB is the most popular NoSQL database management system.

MongoDB

Example of MongoDB Document

{

"_id": ObjectId("4efa8d2b7d284dad101e4bc9"),

"Last Name": "DUMONT",

"First Name": "Jean",

"Date of Birth": "01-22-1963"

},

{

"_id": ObjectId("4efa8d2b7d284dad101e4bc7"),

"Last Name": "PELLERIN",

"First Name": "Franck",

"Date of Birth": "09-19-1983",

"Address": "1 chemin des Loges",

"City": "VERSAILLES"

}

MongoDB contd ...

MongoDB features

● Document-Oriented Storage.

● Full Index Support.

● Replication & High Availability.

● Querying.

● Advanced security

● Professional Support by 10gen.

MongoDB contd ….

MongoDB FAQs

● What kind of database is MongoDB?

● MongoDB is a document-oriented DBMS.

● Do MongoDB databases have tables?.

● Instead of tables, a MongoDB database stores its data in

collections.

● Does MongoDB support SQL?

● No

● However, MongoDB does support a rich, ad-hoc query language

of its own.

MongoDB contd ….

MongoDB FAQs

● What are typical uses for MongoDB?

● MongoDB has a general-purpose design, making it appropriate

for a large number of use cases. Examples include content

management systems, mobile applications, gaming, e-commerce,

analytics, archiving, and logging.

● Do not use MongoDB for systems that require SQL, joins, and

multi-object transactions.

● Does MongoDB support transactions?

● MongoDB does not provide ACID transactions.

● However, MongoDB does provide some basic transactional

capabilities. Some Atomic operations are possible.

● Additionally, you can make writes in MongoDB durable (the ‘D’ in

ACID).

MongoDB contd ….

CouchDB

● Apache CouchDB, commonly referred to as CouchDB, is an open

source database that focuses on ease of use and on being "a

database that completely embraces the web". It is a NoSQL database

that uses JSON to store data, JavaScript as its query language.

● CouchDB was first released in 2005 and later became an Apache

project in 2008.

● Unlike in a relational database, CouchDB does not store data and

relationships in tables. Instead, each database is a collection of

independent documents.

CouchDB

CouchDB features

● Document Storage.

● ACID Semantics.

● Security and Validation.

● Distributed Updates and Replication.

CouchDB contd ….

CouchDB features

● Document Storage.

● ACID Semantics.

● Security and Validation.

● Distributed Updates and Replication.

CouchDB contd ….

Oracle NoSQL Database

● The Oracle NoSQL Database is a distributed key-value database. It

is designed to provide highly reliable, scalable and available data

storage across a configurable set of systems that function as storage

nodes.

● Data is stored as key-value pairs, which are written to particular

storage node(s).

● Oracle’s NoSQL Database brings enterprise quality storage and

performance to the highly available, widely distributed NoSQL

environment. Its commercially proven, write optimized storage system

delivers outstanding performance as well as robustness and reliability,

and its “No Single Point of Failure” design ensures that the system

continues to run and data remain available after any failure.

Oracle NoSQL Database

OrientDB

● OrientDB is an open source NoSQL database management system

written in Java. Even if it is a document-based database, the

relationships are managed as in graph databases with direct

connections between records.

● it's amazing fast: can store up to 150,000 records per second on

common hardware.

● Supports advanced features such as ACID Transactions, Fast

Indexes, Native and SQL queries. It imports and exports documents

in JSON.

OrientDB

OrientDB features

● Transactional: supports ACID Transactions. On crash it recovers

pending documents.

● GraphDB: native management of graphs..

● SQL: supports SQL language with extensions to handle

relationships without SQL join, manage trees and graphs of

connected documents.

● Web ready: supports natively HTTP, RESTful protocol and JSON

without use 3rd party libraries and components.

● Run everywhere: the engine is 100% pure Java: runs on Linux,

Windows and any system that supports Java technology.

OrientDB contd ...

Apache Cassandra

● Apache Cassandra is an open source distributed database

management system. It is an Apache Software Foundation top-level

project designed to handle very large amounts of data spread out

across many commodity servers while providing a highly available

service with no single point of failure.

● It is a NoSQL solution that was initially developed by Facebook and

powered their Inbox Search feature until late 2010.

● It was released as an open source project on Google code in July

2008. In March 2009, it became an Apache Incubator project. On

February 17, 2010 it graduated to a top-level project.

Apache Cassandra

Apache Cassandra Features

● Fault Tolerant : Data is automatically replicated to multiple nodes

for fault-tolerance.

● Performant : Cassandra consistently outperforms popular NoSQL

alternatives.

● Decentralized : There are no single points of failure. There are no

network bottlenecks. Every node in the cluster is identical.

● Durable : Cassandra is suitable for applications that can't afford to

lose data, even when an entire data center goes down.

● Professionally Supported : Cassandra support contracts and

services are available from third parties.

Apache Cassandra contd ...

Cloud Databases

● A cloud database is a database that typically runs on a cloud

computing platform ( a large number of computers that are connected

through a real-time communication network, typically the Internet) .

● While a cloud database can be a traditional database such as a

MySQL or SQL Server database that has been adopted for cloud use.

● Cloud databases can offer significant advantages over their

traditional counterparts, including increased accessibility, automatic

fail-over and fast automated recovery from failures, automated

on-the-go scaling, minimal investment and maintenance of in-house

hardware, and potentially better performance.

Cloud Databases

Cloud Databases

● Databases in multiple departments of a large company, for example,

can be combined in the cloud into a single hosted (CDBMS) .

● The ability to pay for storage capacity and bandwidth on a per-use

model.

● The ability to move the database from one location to another.

● Ability to choose the DBMS type whether it is RDMBS or NoSQL.

● At the same time, cloud databases have their share of potential

drawbacks, including security and privacy issues as well as the

potential loss of or inability to access critical data in the event of a

disaster of the cloud database service provider.

Cloud Databases contd ...

Cloud Database Products

● Rackspace Cloud Databases

● Amazon Cloud Databases

● Google Cloud SQL

● MongoLab

● Oracle Cloud

Cloud Databases Products

Rackspace

● The high performance MySQL database on the cloud Deliver faster

applications on the first relational database service built on

OpenStack.

● Deliver faster applications : The Cloud Databases architecture is

built for high and consistent performance, high performance,

dedicated storage network.

● Add more value to your team: Easily provision your database via

an API or the Control Panel, and save time through automation of

time-consuming tasks such as deployment, configuration, and

patching. You can also easily scale your database for bigger or

smaller memory sizes.

Rackspace

Rackspace

● Protect your data : Deliver more reliable applications and minimize

data loss. Our service includes redundant storage to protect your data

against hardware failures.

● Count on us for Fanatical Support : When you choose a

managed cloud account, we'll help you move your databases and

tables, create users, backup or restore your database, and help you

with basic optimizations.

● Speedy delivery of your data : Rackspace Cloud Databases

provides fast, scalable, fully managed hosting for your MySQL

instances.

Rackspace contd ...

Amazon Cloud Databases

● Also called Amazon Relational Database Service (Amazon RDS),

which allows, easy to set up, operate, and scale a relational database

in the cloud. It provides cost-efficient and resizable capacity while

managing time-consuming database administration tasks, freeing you

up to focus on your applications and business.

● Amazon RDS gives you access to the capabilities of a familiar

MySQL, Oracle or Microsoft SQL Server database engine.

● As with all Amazon Web Services, there are no up-front

investments required, and you pay only for the resources you use.

Amazon Cloud Databases

Amazon Cloud Databases features

● Pre-configured Parameters – Amazon RDS DB Instances are

pre-configured with an appropriate set of parameters and settings

appropriate for the DB Instance class you have selected.

● Monitoring and Metrics - user can use the AWS Management

Console to view key operational metrics for your DB Instance

deployments, including compute/memory/storage capacity utilization,

I/O activity, and DB Instance connections.

● Automated Backups - Turned on by default, the automated backup

feature of Amazon RDS enables point-in-time recovery for your DB

Instance.

Amazon Cloud Databases contd ...


● DB Snapshots – DB Snapshots are user-initiated backups of your

DB Instance.

● Push-Button Scaling – Using the Amazon RDS APIs or with a few

clicks on the AWS Management Console, you can scale the compute

and memory resources powering your deployment up or down.

● On-Demand DB Instances - On-Demand DB Instances let you pay

for compute capacity by the hour your DB Instance runs with no

long-term commitments.

● Security – you can configure firewall settings and control network

access to your DB Instances..



● DB Snapshots – DB Snapshots are user-initiated backups of your

DB Instance.

● Push-Button Scaling – Using the Amazon RDS APIs or with a few

clicks on the AWS Management Console, you can scale the compute

and memory resources powering your deployment up or down.

● On-Demand DB Instances - On-Demand DB Instances let you pay

for compute capacity by the hour your DB Instance runs with no

long-term commitments.

● Security – you can configure firewall settings and control network

access to your DB Instances..


Google Cloud SQL

● Google Cloud SQL is a web service that allows you to create,

configure, and use relational databases that live in Google's cloud. It

is a fully-managed service that maintains, manages, and administers

your databases, allowing you to focus on your applications and

services.

● By offering the capabilities of a familiar MySQL database, the

service enables you to easily move your data, applications, and

services in and out of the cloud. This enables high data portability and

helps you achieve faster time-to-market because you can quickly

leverage your existing database.

● You only pay for your instance while it is being accessed.

Google Cloud SQL

Google Cloud SQL features

● Easy to use : A graphical user interface allows you to create,

configure, manage, and monitor your database instances, with just a

click.

● Fully managed : No worrying about tasks such as replication, patch

management, or backups. These are all taken care of.

● Highly Available : Features like automatic replication across

multiple geographic regions are built in, so the service is available,

and your data is preserved, even if a datacenter becomes

unavailable.

● Exceptional Security: Google provides cloud services in a manner

drawn from its experience with operating its own business, as well as

its core services like Google Search.

Google Cloud SQL contd ..

MongoLab

● The magic of the cloud : Create and scale databases on-demand

on all the major cloud providers.

● Total data protection: Sleep well knowing that we replicate,

backup, and redundantly archive every database.

● Max uptime & performance: Our experienced robots and experts

continuously monitor and manage your databases.

● Expert care and support : Thoughtful, timely support from real

developers is why customers love us.

MongoLab

Oracle Cloud

● The Oracle Database you love, now in the cloud.

● Easy : Self-service management enables almost instantaneous

provisioning of environments.

● Standards-Based : SQL support with access through RESTful Web

Services.

● Enterprise-Grade : Enterprise security, reliability, and performance

to support business-critical applications.

● Simple Pricing : Simple, predictable pricing ($175 / Month, lowest

plan).

Oracle Cloud

Graph Databases

● A graph database is a database that uses graph structures with

nodes, edges, and properties to represent and store data.

Graph Databases

Graph Databases

● Nodes represent entities such as people, businesses, accounts, or

any other item you might want to keep track of.

● Properties are pertinent information that relate to nodes or edges.

● Edges are the lines that connect nodes to nodes and they represent

the relationship between the two.

● Compared with relational databases, graph databases are often

faster for associative data sets.

● They can scale more naturally to large data sets as they do not

typically require expensive join operations.

Graph Databases contd ...

Graph Database Products

● Neo4j

● OrientDB

● HyperGraphDB

● Titan

● GraphBase

Graph Database Products

Neo4j

● Neo4j is an open-source, high-performance, enterprise-grade NOSQL

graph database.

● Neo4j is a robust (fully ACID) transactional graph database.

● Durable and fast.

● Massively scalable, up to several billio nodes/relationships/properties

● Highly-available, when distributed across multiple machines.

● Expressive, with a powerful, human readable graph query language.

Neo4j

HyperGraphDB

● HyperGraphDB is a general purpose, open-source data storage

mechanism based on a powerful knowledge management formalism

known as directed hypergraphs.

● Graph-oriented storage.

● Graph traversals and relational-style queries.

● Customizable storage management.

● Fully transactional.

HyperGraphDB

Titan

● Titan is a highly scalable graph database optimized for storing and

querying massive-scale graphs containing hundreds of billions of vertices

and edges distributed across a multi-machine cluster.

● Titan is a transactional database that can support thousands of

concurrent users executing complex graph traversals.

● Elastic and linear scalability for a growing data and user base.

● Data distribution and replication for performance and fault tolerance.

● Support for ACID.

● Support for full-text search.

Titan

GraphBase

● GraphBase is a second generation Graph Database Management

System (DBMS). Built for 21st Century data problems, GraphBase is a

game-changer when it comes to handling large, complex data structures.

● GraphBase makes massive, highly-structured data stores possible

because it was built from scratch to manage large graphs.

● Tiny memory and storage footprint.

● Built-in support for traversal

● Graph-based transactions.

GraphBase

Thank you !