hadoop使用版本：hadoop-2.6.0-cdh5.15.1
在hadoop-2.6.0-cdh5.15.1/share/hadoop/mapreduce路徑下有乙個hadoop-mapreduce-examples-2.6.0-cdh5.15.1.jar檔案
執行命令：hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.15.1.jar pi 2 3number of maps = 2
samples per map = 3
wrote input for map #0
wrote input for map #1
starting job
19/11/07 15:52:38 info client.rmproxy: connecting to resourcemanager at /0.0.0.0:8032
19/11/07 15:52:39 info input.fileinputformat: total input paths to process : 2
19/11/07 15:52:39 info mapreduce.jobsubmitter: number of splits:2
19/11/07 15:52:39 info mapreduce.jobsubmitter: submitting tokens for job: job_1573113153178_0001
19/11/07 15:52:40 info mapreduce.job: running job: job_1573113153178_0001
19/11/07 15:52:48 info mapreduce.job: job job_1573113153178_0001 running in uber mode : false
19/11/07 15:52:48 info mapreduce.job: map 0% reduce 0%
19/11/07 15:52:54 info mapreduce.job: map 50% reduce 0%
19/11/07 15:52:55 info mapreduce.job: map 100% reduce 0%
19/11/07 15:53:00 info mapreduce.job: map 100% reduce 100%
19/11/07 15:53:00 info mapreduce.job: job job_1573113153178_0001 completed successfully
19/11/07 15:53:00 info mapreduce.job: counters: 49
file system counters
file: number of bytes read=50
file: number of bytes written=430365
file: number of read operations=0
file: number of large read operations=0
file: number of write operations=0
hdfs: number of bytes read=540
hdfs: number of bytes written=215
hdfs: number of read operations=11
hdfs: number of large read operations=0
hdfs: number of write operations=3
job counters
launched map tasks=2
launched reduce tasks=1
data-local map tasks=2
total time spent by all maps in occupied slots (ms)=6933
total time spent by all reduces in occupied slots (ms)=3732
total time spent by all map tasks (ms)=6933
total time spent by all reduce tasks (ms)=3732
total vcore-milliseconds taken by all map tasks=6933
total vcore-milliseconds taken by all reduce tasks=3732
total megabyte-milliseconds taken by all map tasks=7099392
total megabyte-milliseconds taken by all reduce tasks=3821568
map-reduce framework
map input records=2
map output records=4
map output bytes=36
map output materialized bytes=56
input split bytes=304
combine input records=0
combine output records=0
reduce input groups=2
reduce shuffle bytes=56
reduce input records=4
reduce output records=0
spilled records=8
shuffled maps =2
failed shuffles=0
merged map outputs=2
gc time elapsed (ms)=308
cpu time spent (ms)=2610
physical memory (bytes) snapshot=942092288
virtual memory (bytes) snapshot=7986814976
total committed heap usage (bytes)=1810890752
shuffle errors
bad_id=0
connection=0
io_error=0
wrong_length=0
wrong_map=0
wrong_reduce=0
file input format counters
bytes read=236
file output format counters
bytes written=97
job finished in 21.985 seconds
estimated value of pi is 4.00000000000000000000
剛開始會執行connecting to resourcemanager at /0.0.0.0:8032，首先連線到resourcemanager中。這就是乙個最簡單的yarn作業
在/wc/output/下可以看到輸出結果的檔案：
hello 1
haha 2
hello 3
meme 1
welcome 2
world 1
HUE提交MapReduce作業示例
因為hue也是通過oozie的rest提交作業，故看過oozie提交作業總結後，在hue中提交作業變得相當簡單。圖中所有的 hadoop job properties 如下，mapreduce.input fileinputformat inputdir mapreduce.output fileo...
MapReduce作業提交過程
通過提交jar包，進行mapreduce處理，那麼整個執行過程分為五個環節 1 向client端提交mapreduce job.2 隨後yarn的resourcemanager進行資源的分配.3 由nodemanager進行載入與監控containers.5 通過hdfs進行job配置檔案 jar包...
MapReduce幾種提交方式
1.在windows裡的ide直接執行main方法，會將job提交給本地執行器localjobrunner執行 本地存放hadoop安裝包 輸入輸出資料可以放在本地路徑下 c wc srcdata 輸入輸出資料也可以放在hdfs中 hdfs centos7one 9000 wc output3 2....