mapred job -list all | grep <username> #get your job-id
mapred job -events <job-id> # print attempts
mapred job -logs <job-id> [<attempt-id>] #all/just for attempt
That's it. I'm not going to spend another word discussing this. Grab the shell terminal and try it yourself, and everything will be clear.
tl;dr+
#here's the same example and some outputs with:
- <username> = reuts
- <job-id> = job_1392731301770_0015
- <attempt-id> = attempt_1392731301770_0015_m_000001_0
[root@hadoopb home]# mapred job -list all | grep reuts #all the
14/02/27 17:20:00 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
14/02/27 17:20:00 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
job_1392731301770_0022 FAILED 1393505972362 reuts default NORMAL 0 0 0M 0M 0M hadoopb-w02:8088/proxy/application_1392731301770_0022/jobhistory/job/job_1392731301770_0022
job_1392731301770_0021 FAILED 1393504180159 reuts default NORMAL 0 0 0M 0M 0M hadoopb-w02:8088/proxy/application_1392731301770_0021/jobhistory/job/job_1392731301770_0021
job_1392731301770_0016 FAILED 1393455977860 reuts default NORMAL 0 0 0M 0M 0M hadoopb-w02:8088/proxy/application_1392731301770_0016/jobhistory/job/job_1392731301770_0016
#... many more failures after that! I picked one of the last which is not shown here - job_1392731301770_0015
[root@hadoopb home]# mapred job -events job_1392731301770_0015 #the high-level events.
Usage: CLI [-events <job-id> <from-event-#> <#-of-events>]. Event #s start from 1.
[root@hadoopb home]# mapred job -events job_1392731301770_0015 1 100
14/02/27 17:19:44 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
14/02/27 17:19:44 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
14/02/27 17:19:45 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=FAILED. Redirecting to job history server
Task completion events for job_1392731301770_0015
Number of events (from 1) are: 6
FAILED attempt_1392731301770_0015_m_000001_0 hadoopb-w07:59031/tasklog?plaintext=true&attemptid=attempt_1392731301770_0015_m_000001_0
FAILED attempt_1392731301770_0015_m_000001_1 hadoopb-w06:45315/tasklog?plaintext=true&attemptid=attempt_1392731301770_0015_m_000001_1
FAILED attempt_1392731301770_0015_m_000000_1 hadoopb-w05:48979/tasklog?plaintext=true&attemptid=attempt_1392731301770_0015_m_000000_1
FAILED attempt_1392731301770_0015_m_000001_2 hadoopb-w07:59031/tasklog?plaintext=true&attemptid=attempt_1392731301770_0015_m_000001_2
FAILED attempt_1392731301770_0015_m_000000_2 hadoopb-w06:45315/tasklog?plaintext=true&attemptid=attempt_1392731301770_0015_m_000000_2
FAILED attempt_1392731301770_0015_m_000001_3 hadoopb-w06:45315/tasklog?plaintext=true&attemptid=attempt_1392731301770_0015_m_000001_3
[root@hadoopb home]# mapred job -logs job_1392731301770_0015 #that'd spit all the logs, so lets specify an attemptID.
[root@hadoopb home]# mapred job -logs job_1392731301770_0015 attempt_1392731301770_0015_m_000001_0
14/02/27 17:21:18 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
14/02/27 17:21:18 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
14/02/27 17:21:19 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=FAILED. Redirecting to job history server
LogType:stderr
LogLength:250
Log Contents:
Traceback (most recent call last):
File "/mnt/disk2/hadoop/yarn/usercache/reuts/appcache/application_1392731301770_0015/container_1392731301770_0015_01_000003/./mapper.py", line 11, in <module>
import config
ImportError: No module named config
LogType:stdout
LogLength:0
Log Contents:
LogType:syslog
LogLength:11461
Log Contents:
2014-02-27 01:01:50,943 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-02-27 01:01:50,944 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-02-27 01:01:51,062 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
