Sqoop 导入为 Parquet 文件失败

Question

我的 sqoop 导入为 Parquet 文件失败。我尝试检查错误并在线搜索但无法解决。我尝试通过设置“-Dmapreduce.job.user.classpath.first=true”来查看之前的讨论。即使那样也无济于事。提前致谢。

sqoop import -Dmapreduce.job.user.classpath.first=true \
--connect jdbc:mysql://ip/dbname \
--username  username  \
--password   password  \
--table orders \
--warehouse-dir \
--driver com.mysql.jdbc.Driver
--as-parquetfile

错误：

Application application_1492001640512_0078 failed 2 times due to AM Container for appattempt_149
2001640512_0078_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://ec2-52-44-197-32.compute-1.amazo
naws.com:8088/cluster/app/application_1492001640512_0078Then, click on links to logs of each atte
mpt.
Diagnostics: Exception from container-launch.
Container id: container_e49_1492001640512_0078_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1: 
at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
at org.apache.hadoop.util.Shell.run(Shell.java:487)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(Def
aultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.ca
ll(ContainerLaunch.java:303)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.ca
ll(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.

PS：除了 Parquet.I 尝试导入为文本文件、Avro 文件外，其他一切似乎都有效...

Answer 1

不要使用 --warehouse-dir 尝试使用 --target-dir。

Sqoop 导入为 Parquet 文件失败

Sqoop import as Parquet file is failing

sqoop