HBase  BlockCache  101  

Nick  Dimiduk,  Hortonworks              @xefyr                n10k.com  

Agenda  •  What  is  a  BlockCache?  •  Why  cache  blocks?  •  Available  BlockCaches  •  ConfiguraFons  

RegionServer (HBase)

DataNode (Hadoop DFS)

HLog(WAL)

HRegion

HStore

StoreFile

HFile

StoreFile

HFile

MemStore

... ...

HStore

BlockCache

HRegion

...

HStoreHStore

...

1 5

23

3

2

4

Read  Path  Refresher  

BlockCache  

What  is  a  block?  

HFile  v2  Format  

HFile  v2  format  figure  reproduced  from  MaMeo  Bertozzi,  "Apache  HBase  I/O  –  HFile”,  hMp://blog.cloudera.com/blog/2012/06/hbase-­‐io-­‐

hfile-­‐input-­‐output/  

Why  cache  blocks?  Storage hierarchy: a different view

A bumpy ride that has been getting bumpier over time

Dean/2009  

Memory  is  100000x  faster  than  disk!  

Disk  seek  =  10ms  

LruBlockCache!•  On-­‐heap  ConcurrentHashMap!•  3  block  prioriFes,  based  on  access  frequency  –  Single  – MulF  –  In-­‐memory  

•  Background  evicFon  thread  •  HBASE-­‐1460,  0.20  

LruBlockCache  ConfiguraFon  •  Size  as  pct  of  heap  

–  hfile.block.cache.size  –  default:  0.4  

•  Single  access  Priority  size  –  hbase.lru.blockcache.single.percentage  –  default:  0.25  

•  MulF-­‐access  priority  size  –  hbase.lru.blockcache.multi.percentage  –  default:  0.5  

•  In-­‐memory  priority  size  –  hbase.lru.blockcache.memory.percentage  –  default:  0.25  

SlabCache!•  L2  cache;  fronted  by  LruBlockCache  (L1)  •  DoubleBlockCache  caches  blocks  in  both  L1  and  L2  •  Allocates  porFons  of  memory  with  target  block  size  •  DirectByteBuffers  manage  off-­‐heap  memory  •  Blocks  copied  on  and  off  heap  •  HBASE-­‐4027,  0.92  

SlabCache  ConfiguraFon  •  SlabCache  size  

–  hbase.offheapcache.percentage!•  Target  Block  size  

–  hbase.offheapcache.minblocksize!–  default:  64k  

•  Slab  proporFons  –  hbase.offheapcache.slab.proportions!–  default:  0.8,  0.2  

•  Slab  target  block  sizes  –  hbase.offheapcache.slab.sizes!–  default:  64k,  128k  

BucketCache!•  L2  cache;  fronted  by  LruBlockCache  (L1)  •  CombinedBlockCache  caches  data  blocks  in  L2,  everything  else  in  L1  •  Blocks  evicted  from  L1  are  promoted  to  L2  •  Allocates  porFons  of  space  with  target  block  size  •  Three  operaFng  modes:  

–  onheap  –  HeapByteBuffers  manage  on-­‐heap  memory  space  –  offheap  –  DirectByteBuffers  manage  off-­‐heap  memory  space  –  file  –  RandomAccessFile  manages  file-­‐backed  space  

•  Blocks  copied  on  and  off  heap  •  HBASE-­‐7404,  0.96  

BucketCache  ConfiguraFons  •  OperaFng  mode  

–  hbase.bucketcache.ioengine!•  Total  cache  size  as  pct  of  heap  OR  size  in  MB:  

–  hbase.bucketcache.size!•  Size  of  L1  as  pct  of  total  cache  size:  

–  hbase.bucketcache.percentage.in.combinedcache!–  default:  0.9  

•  Target  Block  size  –  hbase.offheapcache.minblocksize!–  default:  64k  

•  Bucket  target  block  sizes  –  hbase.bucketcache.bucket.sizes!–  default:  14  sizes,  clustered  around  64k  

BlockCache  Showdown  •  Compare  performance:  –  Total  cache  size  –  Cache  size  vs.  Data  size  

 n10k.com/blog/

blockcache-­‐showdown/  

L2  off-­‐heap  BucketCache  makes  a  strong  showing  

Thanks!  

M A N N I N G

Nick Dimiduk Amandeep Khurana

FOREWORD BY Michael Stack

hbaseinacFon.com  

Nick  Dimiduk              github.com/ndimiduk  

           @xefyr  

           n10k.com  

n10k.com/blog/blockcache-­‐101