Hadoop除錯記錄 2

自從上次調通hbase後很久沒有碰hadoop了，今日想寫乙個mapreduce的小程式。於是先執行了下自帶的wordcount示例程式，卻報錯了。

資訊如下：

kevin@ubuntu:~/usr/hadoop/hadoop$ ./bin/hadoop jar hadoop-examples-1.2.1

.jar wordcount readme.txt output

15/05/11

08:20:04 info input.fileinputformat: total input paths to process : 1

15/05/11

08:20:04 info util.nativecodeloader: loaded the native-hadoop library

15/05/11

08:20:04

15/05/11

08:20:05

info mapred.jobclient: running job: job_201505110806_0003

15/05/11

08:20:06 info mapred.jobclient: map 0% reduce 0%

15/05/11

08:20:06

info mapred.jobclient: task id : attempt_201505110806_0003_m_000002_0, status : failed

error initializing attempt_201505110806_0003_m_000002_0:

j**a.io.ioexception: exception reading

file:/~/usr/hadoop/hadoop/tmp/mapred/local/ttprivate/tasktracker/kevin/jobcache/job_201505110806_0003/jobtoken

at org.apache.hadoop.security.credentials.readtokenstoragefile(credentials.j**a:

135)

at org.apache.hadoop.mapreduce.security.tokencache.loadtokens(tokencache.j**a:

178)

at org.apache.hadoop.mapred.tasktracker.initializejob(tasktracker.j**a:

1289

) at org.apache.hadoop.mapred.tasktracker.localizejob(tasktracker.j**a:

1226

) at org.apache.hadoop.mapred.tasktracker$

5.run(tasktracker.j**a:2603

) at j**a.lang.thread.run(thread.j**a:

745)

caused by: j**a.io.filenotfoundexception: file

file:/~/usr/hadoop/hadoop/tmp/mapred/local/ttprivate/tasktracker/kevin/jobcache/job_201505110806_0003/jobtoken does not exist.

at org.apache.hadoop.fs.rawlocalfilesystem.getfilestatus(rawlocalfilesystem.j**a:

402)

at org.apache.hadoop.fs.filterfilesystem.getfilestatus(filterfilesystem.j**a:

255)

at org.apache.hadoop.fs.checksumfilesystem$checksumfsinputchecker.

(checksumfilesystem.j**a:125

) at org.apache.hadoop.fs.checksumfilesystem.open(checksumfilesystem.j**a:

283)

at org.apache.hadoop.fs.filesystem.open(filesystem.j**a:

436)

at org.apache.hadoop.security.credentials.readtokenstoragefile(credentials.j**a:

129)

...

5more

錯誤是無法讀取jobtoken的檔案，由於程式不會出錯，推測是上次在配置hbase時修改hadoop的core-site.xml檔案出錯了。在stackoverflow上有乙個同樣的問題，有人建議刪除該配置檔案中關於快取檔案目錄的配置，因為這一部分是會被系統預設建立的，不需要手動配置。即下面這一部分：

1
2<
property
>
3<
name
>hadoop.tmp.dir
name
>
4<
value
value
>
5<
description
>a base for other temporary directories.
description
>
6property
>

刪掉這個問題就解決了。關閉hadoop，重啟hadoop。出現新問題 jobtracker is in safe mode：

15/05/11

08:47:40 error security.usergroupinformation: priviledgedactionexception as:kevin cause:org.apache.hadoop.ipc.remoteexception: org.apache.hadoop.mapred.safemodeexception: jobtracker is in

safe mode

at org.apache.hadoop.mapred.jobtracker.checksafemode(jobtracker.j**a:

5188

) at org.apache.hadoop.mapred.jobtracker.getstagingareadir(jobtracker.j**a:

3677

) at sun.reflect.nativemethodaccessorimpl.invoke0(native method)

at sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.j**a:62)

at sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.j**a:43)

at j**a.lang.reflect.method.invoke(method.j**a:

483)

at org.apache.hadoop.ipc.rpc$server.call(rpc.j**a:

587)

at org.apache.hadoop.ipc.server$handler$

1.run(server.j**a:1432

) at org.apache.hadoop.ipc.server$handler$

1.run(server.j**a:1428

) at j**a.security.accesscontroller.doprivileged(native method)

at j**ax.security.auth.subject.doas(subject.j**a:

422)

at org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation.j**a:

1190

) at org.apache.hadoop.ipc.server$handler.run(server.j**a:

1426)

bin/hadoop dfsadmin -safemode le**e

問題均解決，wordcount成功執行。

Hadoop除錯記錄 2

《Hadoop權威指南》閱讀記錄2

hadoop遠端除錯

Hadoop除錯日誌

Hadoop除錯記錄 2

《Hadoop權威指南》閱讀記錄2

hadoop遠端除錯

Hadoop除錯日誌

相關推薦