Upload
shon-burns
View
213
Download
0
Embed Size (px)
Citation preview
GROUP 7
TOOLS FOR BIG DATA
Sandeep PrasadDipojjwal Ray
Objectives...
Apache Hadoop Apache hadoop v1.0.3 and v1.0.4 successful
installation Wordcount functionality by hadoop mapreduce Estimating value of 'Pi' by hadoop mapreduce MapReduce and HDFS
Apache Hadoop...
High-Availability Distributed object-oriented platform Open Source Pseudo-Distributed single-node cluster A part of Apache Lucene project Handles petabytes of data
Installation of Hadoop v1.0.3 & 1.0.4...
Release Date v1.0.3 : October 12, 2012 Release Date v1.0.4 : May 16, 2012 OS : Ubuntu v12.04 Prerequisites : Sun Java, hduser Configuration
Examples...
WordCount example :
$ /bin/hadoop jar hadoop-1.0.3-examples.jar wordcount file01.txt
Estimation of 'Pi'
$ /bin/hadoop jar hadoop-1.0.3-examples.jar pi (x) (y)x= Number of mapsy= Sample per mapsRuntime 2.25 seconds (x=10 ; y=100)Estimated value 3.1480000000000
MapReduce & HDFS...
Divide and conquer algorithm Map() and Reduce() function derive roots from
functional programming JobTracker and TaskTracker NameNode and DataNode Hadoop Distributed File System Java Framework
References...
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster
http://lintool.github.io/Cloud9/
Data intensive text-processing using Mapreduce Book by Jimmy Lin and Chris Dyer
http://hadoop.apache.org/releases.html
http://www.apache.org/dyn/closer.cgi/hadoop/co
THANK YOU
framework written in Javahighly fault-tolerant distributed file system
JobTracker web UI provides information about general job statistics of the Hadoop cluster, running/completed/failed jobs and a job history log file
The task tracker web UI shows you running and non-running tasks