Upload
hortonworks
View
115
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
© Hortonworks Inc. 2011
HBase Dev Meetup 09/11/2012
Enis Soztutar
enis [@] apache [dot] org
#enissoz
Page 1
© Hortonworks Inc. 2011
Integration Testing
• Hbase Integration Tests plan– HBASE-6201
1. Create hbase-it module
2. Ability to run integration tests on a given clustermvn verifybin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver
3. Port candidate tests
4. Add more long running and fault injection tests
5. (Most of) Integration tests should be able to run over a mini cluster or a 100+ node cluster.
6. Integration tests should run in mini mode as nightlies (apache jenkins)
7. Our plan is to run the whole suite of Integration tests nightly/weekly on medium-sized clusters. Every org. should be able to do this easily.
Page 2Architecting the Future of Big Data
© Hortonworks Inc. 2011
Current Status
• hbase-it module committed, HBASE-6203• HBaseCluster interface for interacting with the cluster from system tests
(HBASE-6241)– Patch near to being committed
• Subtask jiras for porting each test or adding a new test• Hbase book documentation available in patch for HBASE-6302• Candidate tests to be converted:
Page 3Architecting the Future of Big Data
• LoadTestTool• BigTop TestLoadAndVerify• Goraci (https://github.com/keith-
turner/goraci)• TestAcidGuarantees /
TestAtomicOperation• TestRegionBalancing (HBASE-
6053)• TestFullLogReconstruction• TestMultiVersions /
TestKeepDeletes
• TestFromClientSide• TestShell and src/test/ruby• TestRollingRestart• TestMasterFailover• TestImportExport• Test**OnCluster• Balancer tests
© Hortonworks Inc. 2011
HBASE-6241 Overview
• New junit @Category(IntegrationTests.class)• Introduces HBaseCluster
– public abstract ClusterStatus getClusterStatus();– public abstract void startRegionServer(String hostname);– Start/stop/kill RS / Master– MiniHBaseCluster and DistributedHBaseCluster implements HBaseCluster
• Introduces DistributedHBaseCluster– Keeps state about a distributed cluster – Does not know about environment. Uses ClusterManager to interact with the
cluster
• ClusterManager is an Api to manage services– public abstract void start(ServiceType service, String hostname)– Pluggable, specific implementations for every environment
• HBaseClusterManager extends ClusterManager– Knows about environment (HBASE_HOME, HBASE_CONF_DIR,
HBASE_SSH_OPTS )– Uses SSH and remotely executes ps, kill -9, bin/hbase-daemon.sh start/stop
Page 4Architecting the Future of Big Data
© Hortonworks Inc. 2011
HBASE-6241 Overview (cont.)
• Introduces HBaseIntegrationTestingUtility extends HBaseTestingUtility– Integration tests uses this
• Introduces 2 ways to run integration tests – bin/hbase o.a.h.h.IntegrationTestsDriver –config <hbase_conf_dir>
– mvn verify (runs tests using mini cluster)– mvn verify –Dhbase.test.cluster.distributed=true –Dhbase.conf.dir=<hbase_conf_dir> (further patch)
• Introduces ChaosMonkey– A utility to injects faults in a running cluster.– Does not know about environment, uses HBaseIntegrationTestingUtility– Actions (RestartActiveMaster, RestartRandomRs, RestartRsHoldingMeta,
BatchRestartRs, RollingBatchRestartRs, etc )– Policies (PeriodicRandomActionPolicy), predefined named policies
(EVERY_MINUTE_RANDOM_ACTION_POLICY)
• Introduces IntegrationTestDataIngestWithChaosMonkey– Runs LoadTestTool and ChaosMonkey for configurable running time– Adjusts data load based on the cluster size (also works with mini cluster)
Page 5Architecting the Future of Big Data
© Hortonworks Inc. 2011
Open Issues
•0.94 porting?•Hadoop NN HA testing•BIGTOP-635•Community interest•Different environments (no SSH, etc)•???
Page 6Architecting the Future of Big Data
© Hortonworks Inc. 2011
HBase on Windows
• WHY? –Because we can : )–More platforms = more users + more development
• WHO–MS + HW
• WHEN?–Patches will start rolling shortly
• WHERE?–Apache HBase Trunk–0.94? 0.92?
• WHAT– Hadoop on windows (HADOOP-8079)– Test fixes, file path name related fixes, timing issues, network related changes,
build changes, bin scripts, etc
Page 7Architecting the Future of Big Data
© Hortonworks Inc. 2011
ThanksGo bananas!
Architecting the Future of Big DataPage 8
Taken from http://99designs.com/t-shirt-design/contests/design-chaos-monkey-t-shirt-70909/entries?entriespage=1#contest-header