Introduction to HBase

Gokuldas K Pillai@gokool

HBase - The Hadoop Database

• Based on Google’s BigTable (OSDI’06)• Runs on top of Hadoop but provides real time

read/write access• Distributed Column Oriented Database

HBase Strengths

• Can scale to billions of rows X millions of columns

• Relatively cheap & easy to scale• Random real time access read/write access to

very large data• Support for update, delete

Who is using it

• StumpleUpon/ su.pr – Uses Hbase as a realtime data storage and analytics platform

• Twitter– Distributed read/write backup of all mySQL instances. Powers

“people search”. • Powerset (Now part of MS)• Adobe• Yahoo• Ning• Meetup• More at http://wiki.apache.org/hadoop/Hbase/PoweredBy

Key features

• Column Oriented store– Table costs only for the data stored– NULLs in rows are free

• Rows stored in sorted order• Can scale to Petabytes (At Google)

Comparing to RDBMS

• No Joins• No Query engine• No transactions• No column typing• No SQL, No ODBC/JDBC (Hbql is there now)

Data Model - Tables

• Tables consisting of rows and columns• Table cells are versioned (by timestamp)• Tables are sorted by row keys• Table access is via primary key• Row updates lock the row no matter how

many columns are involved

Column Families

• Row’s columns are grouped into families• Column family members identified by a

common ‘printable’ prefix• Column family should be predefined – but column family members can be added

dynamically– member name can be bytes

• All column family members are collocated on disk

Server Architecture

• Similar to HDFS– HbaseMaster ~ NameNode– RegionServer ~ DataNode

• HBase stores state via the Hadoop FS API• Can persist to :– Local– Amazon S3– HDFS (Default)

HBaseMaster

What it does:• Bootstrapping a new instance• Assignment and handling RegionServer problems

– Each region from every table is assigned to a RegionServer• When machines fail, move regions• When regions split, move regions to balance

What it does NOT do:– Handle write requests (Not a DB Master)– Handle location finding requests (handled by RegionServer)

RegionServer

• Carry the regions• Handle client read/write requests• Manage region splits (inform the Master)

Regions

• Horizontal Partitioning • Every region has a subset of the table’s rows• Region identified as– [table, first row(+), last row(-)]

• Table starts on a single region• Splits into two equal sized regions as the

original region grows bigger and so on..

Zookeeper

• Master election and server availability• Cluster management– Assignment transaction state management

• Client contacts ZooKeeper to bootstrap connection to the Hbase cluster

• Region key ranges, region server addresses• Guarantees consistency of data across clients

Workflow (Client connecting first time)

• Client ZooKeeper (returns –ROOT- )• Client -ROOT- (returns .META.)• Client .META. (returns RegionServer)• To avoid 3-lookups everytime, client caches

this info.– Recache on fault

Write/Read Operation

• Write request from Client RegionServer Commit log (on HDFS), memstore

• Flush to filesystem when memstore fills

• Read request from Client RegionServerLookup the memstore if available

If not, lookup flush files (reverse chrono. Order)

Integration

• Java HBase Client API• High performance Thrift gateway• A REST-ful Web service gateway (Stargate)– Supports XML, binary dat encoding options

• Cascading, Hive and Pig integration• HBase shell (jruby)• TableInput/TableOutputFormat for MR

Main Classes

• HBaseAdmin – Create table, drop table, list and alter table

• HTable– Put– Get– Scan

Alternatives to HBase

• Cassandra (From Facebook)– Based on Amazon’s Dynamo– No Master-slave but P2P– Tunable: Consistency Vs Latency

• Yahoo’s PNUTS– Not Open source– Works well for multi DC/geographical disbursed servers

References

• Hadoop – The Definitive Guide • Cloudera website• http://wiki.hbase.apache.org• Lars George,

– http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html

• Comparing Hbase, Cassandra and PNUTS– http://blog.amandeepkhurana.com/2010/05/comparing-pnuts-

hbase-and-cassandra.html• ACID compliance of Hbase -

http://hbase.apache.org/docs/r0.89.20100621/acid-semantics.html