hadoop计算篇 #

源码(内有WordCount)

3.4.1文档 备用

启动示例 #

hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.1.jar wordcount  /1.csv  `date +%m%d-%H%M`

源码编译 #

• WordCount.java

hadoop com.sun.tools.javac.Main WordCount.java
jar cf wc.jar WordCount*.class
hadoop jar wc.jar WordCount /1.csv `date +%m%d-%H%M`

#

https://files.pilookup.com/pi/1-100000000.zip
hadoop com.sun.tools.javac.Main WordCount.java
jar cf wc.jar WordCount*.class
hadoop jar wc.jar WordCount /1-100000000.txt `date +%m%d-%H%M`

sed -i “16 i yarn.nodemanager.resource.memory-mb1024” /opt/hadoop-3.4.1/etc/hadoop/yarn-site.xml

sed -i “16 i yarn.nodemanager.resource.cpu-vcores1” /opt/hadoop-3.4.1/etc/hadoop/yarn-site.xml