- 1. Medical Informatics Laboratory Department of Biomedical
engineering College of Medicine , Seoul National Univ. Eunsil Yoon
Hadoop Setting Eunsil Yoon
2. Medical Informatics Laboratory Department of Biomedical
engineering College of Medicine , Seoul National Univ. Eunsil Yoon
Linux Setting (version_2012.12) 1. Linux - USB CentOS 2. Hadoop -
hadoop-1.0.4.tar - : /usr/local/hadoop-1.0.4 - Hadoop conf -
/usr/local/hadoop-1.0.4/conf/hadoop-env.sh - screenshot 1 typing 3.
Java - jdk-7u10-linux-x64.rpm - : /usr/java/jdk1.7.0_10 - Java path
- /etc/profile - export JAVA_HOME=/usr/java/jdk1.7.0_10 - export
PATH=$PATH:$JAVA_HONE/bin - export CLASS_PATH=". . Screenshot 2 3.
Medical Informatics Laboratory Department of Biomedical engineering
College of Medicine , Seoul National Univ. Eunsil Yoon Screenshot 1
4. Medical Informatics Laboratory Department of Biomedical
engineering College of Medicine , Seoul National Univ. Eunsil Yoon
Screenshot 2 5. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon /etc/hosts 6. Medical Informatics Laboratory Department
of Biomedical engineering College of Medicine , Seoul National
Univ. Eunsil Yoon Network setting pc 7. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon
/usr/local/hadoop-1.0.4/conf/core-site.xml 8. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon
/usr/local/hadoop-1.0.4/conf/hdfs-site.xml 9. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon
/usr/local/hadoop-1.0.4/conf/mapred-site.xml 10. Medical
Informatics Laboratory Department of Biomedical engineering College
of Medicine , Seoul National Univ. Eunsil Yoon (wordcount) jar
Wordcount (jar) hadoop-env.sh hadoop-env.sh HDFS (screenshot 3 )
./bin/hadoop fs put conf/hadoop-env.sh conf/hadoop-env.sh jar
(screenshot 3 ) conf/hadoop-env.sh, output .bin/hadoop jar
hadoop-examples-*.jar wordcount conf/hadoop-env.sh wordcount_output
, HDFS (fs cat ) ./bin/hadoop fs cat wordcount_output/part-r-00000
11. Medical Informatics Laboratory Department of Biomedical
engineering College of Medicine , Seoul National Univ. Eunsil Yoon
Screenshot 3 12. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon Screenshot 4 13. Medical Informatics Laboratory
Department of Biomedical engineering College of Medicine , Seoul
National Univ. Eunsil Yoon Linux Setting (version 2013.1) Ubuntu
12.04 LTS Java jdk6u38 , 1.6.0_38 Ubuntu root , . (next page)
2013.01.15 14. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon Memory setting /boot 500MB Swap 16384 /usr 15360 /
10240 Var 5120 Tmp 5120 Home 2013.01.15 15. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon Network setting pc 2013.01.15
16. Medical Informatics Laboratory Department of Biomedical
engineering College of Medicine , Seoul National Univ. Eunsil Yoon
Root 2013.01.15 17. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon ./bin/hadoop namenode -format 2013.01.15 18. Medical
Informatics Laboratory Department of Biomedical engineering College
of Medicine , Seoul National Univ. Eunsil Yoon Wordcount 2013.01.15
19. Medical Informatics Laboratory Department of Biomedical
engineering College of Medicine , Seoul National Univ. Eunsil Yoon
wordcount_output/part-r-00000 2013.01.15 20. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon fs(filesystem) lsr 2013.01.15
21. Medical Informatics Laboratory Department of Biomedical
engineering College of Medicine , Seoul National Univ. Eunsil Yoon
yum 2013.01.15 22. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon nabi 2013.01.15 23. Medical Informatics Laboratory
Department of Biomedical engineering College of Medicine , Seoul
National Univ. Eunsil Yoon openssh-server 2013.01.15 24. Medical
Informatics Laboratory Department of Biomedical engineering College
of Medicine , Seoul National Univ. Eunsil Yoon Localhost:50070
NameNode 2013.01.15 25. Medical Informatics Laboratory Department
of Biomedical engineering College of Medicine , Seoul National
Univ. Eunsil Yoon NameNode Live 2013.01.15 26. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon Eclipse 27. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon eclipse Eclipse , Java project
Project name libraries external jar . jar , lib Finish Build.xml ,
jar Build.xml ant jar 2013.01.15 28. Medical Informatics Laboratory
Department of Biomedical engineering College of Medicine , Seoul
National Univ. Eunsil Yoon New file build.xml 2013.01.15 29.
Medical Informatics Laboratory Department of Biomedical engineering
College of Medicine , Seoul National Univ. Eunsil Yoon Build.xml
run as 2013.01.15 30. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon 81p_HDFS API Blrunner.com , .java . package .java
2013.01.16 31. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon Build.xml jar jar java 2013.01.16 32. Medical
Informatics Laboratory Department of Biomedical engineering College
of Medicine , Seoul National Univ. Eunsil Yoon jar workspace ,
hadoop 2013.01.16 33. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon Jar 2013.01.16 34. Medical Informatics Laboratory
Department of Biomedical engineering College of Medicine , Seoul
National Univ. Eunsil Yoon Fs ls HDFS 2013.01.16 35. Medical
Informatics Laboratory Department of Biomedical engineering College
of Medicine , Seoul National Univ. Eunsil Yoon -cat input.txt
2013.01.16 36. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon Example1_20130119 37. Medical Informatics Laboratory
Department of Biomedical engineering College of Medicine , Seoul
National Univ. Eunsil Yoon 1. txt hadoop 38. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon 2. Hdfs 39. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon 3. Mapreduce 40. Medical
Informatics Laboratory Department of Biomedical engineering College
of Medicine , Seoul National Univ. Eunsil Yoon 3-1. mapreduce 41.
Medical Informatics Laboratory Department of Biomedical engineering
College of Medicine , Seoul National Univ. Eunsil Yoon 3-2.
mapreduce 42. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon 4. Mapreduce output 43. Medical Informatics Laboratory
Department of Biomedical engineering College of Medicine , Seoul
National Univ. Eunsil Yoon 5. Output Text count Output:
_user_medinfo_path_output_part-r- 00000 44. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon WordCountMapper.java package
wikibooks.hadoop.chapter04; import java.io.IOException; import
java.util.StringTokenizer; import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable; import
org.apache.hadoop.io.Text; import
org.apache.hadoop.mapreduce.Mapper; public class WordCountMapper
extends Mapper { private final static IntWritable one = new
IntWritable(1); private Text word = new Text(); public void
map(LongWritable key, Text value, Context context) throws
IOException, InterruptedException { StringTokenizer itr = new
StringTokenizer(value.toString()); while (itr.hasMoreTokens()) {
word.set(itr.nextToken()); context.write(word, one ); } } } 45.
Medical Informatics Laboratory Department of Biomedical engineering
College of Medicine , Seoul National Univ. Eunsil Yoon
WordCountReducer.java 46. Medical Informatics Laboratory Department
of Biomedical engineering College of Medicine , Seoul National
Univ. Eunsil Yoon WordCount.java 47. Medical Informatics Laboratory
Department of Biomedical engineering College of Medicine , Seoul
National Univ. Eunsil Yoon 48. Medical Informatics Laboratory
Department of Biomedical engineering College of Medicine , Seoul
National Univ. Eunsil Yoon Example2. 116p. 130129 49. Medical
Informatics Laboratory Department of Biomedical engineering College
of Medicine , Seoul National Univ. Eunsil Yoon ASA(American
Standards Association: ) 2009 http://stat- computing.org/dataexpo/
2009 1987-2008 50. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon Download the data 51. Medical Informatics Laboratory
Department of Biomedical engineering College of Medicine , Seoul
National Univ. Eunsil Yoon 1987.csv (29 ) * 52. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon 22 11.4GB 53. Medical
Informatics Laboratory Department of Biomedical engineering College
of Medicine , Seoul National Univ. Eunsil Yoon HDFS ( 6) 54.
Medical Informatics Laboratory Department of Biomedical engineering
College of Medicine , Seoul National Univ. Eunsil Yoon ./bin/hadoop
fs ls input 55. Medical Informatics Laboratory Department of
Biomedical engineering College of Medicine , Seoul National Univ.
Eunsil Yoon DepartureDelayCount (19) 56. Medical Informatics
Laboratory Department of Biomedical engineering College of Medicine
, Seoul National Univ. Eunsil Yoon (117 ) 30, (1 ) 18 30 . Map
input records = 117,161,410 Map output & Reduce input records =
47,765,560 Reduce output records = 245
