Yarn + Sqoop after Ctrl+C = stuck ACCEPTED

Question

我是 Hadoop 的新手，我正在使用单个节点进行 POC。第二次出现这种情况，不知道怎么解决。它发生在使用 Ctrl + C 停止 map reduce 之后。 SQLServer 正在通过 Sqoop 将数据导入 HBase，在出现错误和 Ctrl + C 后，nunhum 另一项工作正常，卡在已接受状态。已经排除了所有工作申请。我重新启动了整个服务器。资源管理器已相应配置（10Gb 内存，2 核），但不知道会发生什么。

[root@hadoop01 /]# hadoop version
Hadoop 2.6.0.2.2.6.0-2800
Subversion git@github.com:hortonworks/hadoop.git -r acb70ecfae2c3c5ab46e24b0caebceaec16fdcd0
Compiled by jenkins on 2015-05-18T20:21Z
Compiled with protoc 2.5.0
From source with checksum a25c30f622eb057f47e2155f78dba5e
This command was run using /usr/hdp/2.2.6.0-2800/hadoop/hadoop-common-2.6.0.2.2.6.0-2800.jar

[root@hadoop01 fausto.branco]# cat /etc/centos-release
CentOS release 6.6 (Final)

sudo -u hdfs sqoop import –connect “jdbc:sqlserver://SQLServerIP:1433;database=db_POC;username=hdpteste;password=xxxxxx” \
–hbase-create-table \
–hbase-table vw_hdp_Arquivo \
–hbase-row-key “id_Arquivo, Valor” \
–column-family cf_name \
–driver com.microsoft.sqlserver.jdbc.SQLServerDriver \
–table “dbo.vw_hdp_Arquivo” -m 1

15/07/14 19:43:51 INFO zookeeper.ZooKeeper: Session: 0x14e8e1385000001 closed
15/07/14 19:43:51 INFO zookeeper.ClientCnxn: EventThread shut down
15/07/14 19:43:51 INFO impl.TimelineClientImpl: Timeline service address: hxxp://hadoop01.POC.local:8188/ws/v1/timeline/
15/07/14 19:43:51 INFO client.RMProxy: Connecting to ResourceManager at hadoop01.POC.local/XXX.XXX.XXX.XXX:8050
15/07/14 19:43:53 INFO db.DBInputFormat: Using read commited transaction isolation
15/07/14 19:43:53 INFO mapreduce.JobSubmitter: number of splits:1
15/07/14 19:43:54 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1436900603357_0002
15/07/14 19:43:54 INFO impl.YarnClientImpl: Submitted application application_1436900603357_0002
15/07/14 19:43:54 INFO mapreduce.Job: The url to track the job: hxxp://hadoop01.POC.local:8088/proxy/application_1436900603357_0002/
15/07/14 19:43:54 INFO mapreduce.Job: Running job: job_1436900603357_0002

[root@hadoop01 fausto.branco]# yarn application -status application_1436900603357_0002
15/07/14 19:45:05 INFO impl.TimelineClientImpl: Timeline service address: hxxp://hadoop01.POC.local:8188/ws/v1/timeline/
15/07/14 19:45:06 INFO client.RMProxy: Connecting to ResourceManager at hadoop01.POC.local/XXX.XXX.XXX.XXX:8050
Application Report :
Application-Id : application_1436900603357_0002
Application-Name : dbo.vw_hdp_Arquivo.jar
Application-Type : MAPREDUCE
User : hdfs
Queue : default
Start-Time : 1436903034500
Finish-Time : 0
Progress : 0%
State : ACCEPTED
Final-State : UNDEFINED
Tracking-URL : N/A
RPC Port : -1
AM Host : N/A
Aggregate Resource Allocation : 0 MB-seconds, 0 vcore-seconds
Diagnostics :

[root@hadoop01 fausto.branco]# hadoop job -status job_1436900603357_0002
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it.

15/07/14 19:46:19 INFO impl.TimelineClientImpl: Timeline service address: hxxp://hadoop01.POC.local:8188/ws/v1/timeline/
15/07/14 19:46:19 INFO client.RMProxy: Connecting to ResourceManager at hadoop01.POC.local/XXX.XXX.XXX.XXX:8050

Job: job_1436900603357_0002
Job File: /user/hdfs/.staging/job_1436900603357_0002/job.xml
Job Tracking URL : hxxp://hadoop01.POC.local:8088/proxy/application_1436900603357_0002/
Uber job : false
Number of maps: 0
Number of reduces: 0
map() completion: 0.0
reduce() completion: 0.0
Job state: PREP
retired: false
reason for failure:
Counters: 0

Answer 1

找到了！

正在分析此项：聚合资源分配：0 MB 秒，0 vcore 秒

我看到节点状态不正常，并在日志中发现这条消息：

运行状况报告：1/1 本地目录有问题：/hadoop/yarn/local； 1/1 local-dirs 是坏的：/hadoop/yarn/log；

所以我放了更多磁盘并开始工作

Yarn + Sqoop after Ctrl+C = stuck ACCEPTED

Yarn + Sqoop after Ctrl+C = stuck ACCEPTED

hadoop

hadoop-yarn

sqoop