我无法通过 sqoop 所有表在 sqoop 中导入数据
I am not able to import data in sqoop via sqoop all tables
[cloudera@quickstart ~]$ **sqoop import-all-tables -m=4 --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" --username=retail_dba --password=cloudera --warehouse-dir=/user/cloudera/sqoop-import**
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/05/26 10:31:30 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.4.2
16/05/26 10:31:30 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/05/26 10:31:32 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
16/05/26 10:31:33 INFO tool.CodeGenTool: Beginning code generation
16/05/26 10:31:33 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
16/05/26 10:31:33 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
16/05/26 10:31:33 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/cc4bbc7353018a0c34e03cbc6701dc23/categories.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
16/05/26 10:31:43 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/cc4bbc7353018a0c34e03cbc6701dc23/categories.jar
16/05/26 10:31:43 WARN manager.MySQLManager: It looks like you are importing from mysql.
16/05/26 10:31:43 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
16/05/26 10:31:43 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
16/05/26 10:31:43 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
16/05/26 10:31:43 INFO mapreduce.ImportJobBase: Beginning import of categories
16/05/26 10:31:44 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
16/05/26 10:31:47 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
16/05/26 10:31:47 INFO client.RMProxy: Connecting to ResourceManager at quickstart.cloudera/127.0.0.1:8032
16/05/26 10:31:56 INFO db.DBInputFormat: Using read commited transaction isolation
16/05/26 10:31:56 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`category_id`), MAX(`category_id`) FROM `categories`
16/05/26 10:31:56 INFO mapreduce.JobSubmitter: number of splits:4
16/05/26 10:31:57 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1464283628429_0001
16/05/26 10:31:59 INFO impl.YarnClientImpl: Submitted application application_1464283628429_0001
16/05/26 10:31:59 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1464283628429_0001/
16/05/26 10:31:59 INFO mapreduce.Job: **Running job: job_1464283628429_0001**
我对这种错误进行了一些研究，最后在下面找到了
如屏幕截图中的日志中所述，在发生以下日志错误后，会弹出许多重试错误。
所以怀疑资源管理器。
Connecting to ResourceManager at quickstart.cloudera
我调查了一下 this
You shouldn't normally have to start any server after booting, but it looks like the YARN ResourceManager has died. It's possible there was just a little glitch when it was starting and could be fixed by running the following and trying again:
sudo service hadoop-yarn-resourcemanager start
If you'd like to dig a little deeper or the above doesn't work, you can find the logs in /var/log/hadoop-yarn. There will be a .out file and a .log file for the ResourceManager - check both and see if there are any stack traces, error messages, etc. (or feel free to just post those files here).
[cloudera@quickstart ~]$ **sqoop import-all-tables -m=4 --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" --username=retail_dba --password=cloudera --warehouse-dir=/user/cloudera/sqoop-import**
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/05/26 10:31:30 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.4.2
16/05/26 10:31:30 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/05/26 10:31:32 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
16/05/26 10:31:33 INFO tool.CodeGenTool: Beginning code generation
16/05/26 10:31:33 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
16/05/26 10:31:33 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
16/05/26 10:31:33 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/cc4bbc7353018a0c34e03cbc6701dc23/categories.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
16/05/26 10:31:43 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/cc4bbc7353018a0c34e03cbc6701dc23/categories.jar
16/05/26 10:31:43 WARN manager.MySQLManager: It looks like you are importing from mysql.
16/05/26 10:31:43 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
16/05/26 10:31:43 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
16/05/26 10:31:43 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
16/05/26 10:31:43 INFO mapreduce.ImportJobBase: Beginning import of categories
16/05/26 10:31:44 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
16/05/26 10:31:47 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
16/05/26 10:31:47 INFO client.RMProxy: Connecting to ResourceManager at quickstart.cloudera/127.0.0.1:8032
16/05/26 10:31:56 INFO db.DBInputFormat: Using read commited transaction isolation
16/05/26 10:31:56 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`category_id`), MAX(`category_id`) FROM `categories`
16/05/26 10:31:56 INFO mapreduce.JobSubmitter: number of splits:4
16/05/26 10:31:57 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1464283628429_0001
16/05/26 10:31:59 INFO impl.YarnClientImpl: Submitted application application_1464283628429_0001
16/05/26 10:31:59 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1464283628429_0001/
16/05/26 10:31:59 INFO mapreduce.Job: **Running job: job_1464283628429_0001**
我对这种错误进行了一些研究，最后在下面找到了 如屏幕截图中的日志中所述，在发生以下日志错误后，会弹出许多重试错误。 所以怀疑资源管理器。
Connecting to ResourceManager at quickstart.cloudera
我调查了一下 this
You shouldn't normally have to start any server after booting, but it looks like the YARN ResourceManager has died. It's possible there was just a little glitch when it was starting and could be fixed by running the following and trying again:
sudo service hadoop-yarn-resourcemanager start
If you'd like to dig a little deeper or the above doesn't work, you can find the logs in /var/log/hadoop-yarn. There will be a .out file and a .log file for the ResourceManager - check both and see if there are any stack traces, error messages, etc. (or feel free to just post those files here).