从 vertica 导入 sqoop 失败

Question

我正在尝试使用 sqoop2 将数据集从 Vertica 导入到 HDFS。

我在 sqoop 机器上进行运行查询以将数据从 Vertica v6.0.1-7 导入 hdfs

sqoop import -m 1 --driver com.vertica.jdbc.Driver --connect "jdbc:vertica://10.10.10.10:5433/MYDB" --password dbpassword --username dbusername --target-dir "/user/my/hdfs/dir" --verbose --query 'SELECT * FROM ORDER_V2 LIMIT 10;'

但是我在这里遇到了一些错误，

16/02/03 10:33:17 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Query [SELECT * FROM ORDER_V2 LIMIT 10;] must contain '$CONDITIONS' in WHERE clause.
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:300)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

任何人都知道如何做到这一点，考虑到两台机器上的用户名不同。

Answer 1

将 vertica_5.1.4_jdk_5.jar 添加到 /var/lib/sqoop

之后

并添加 WHERE $CONDITIONS 解决了问题

sqoop import -m 1 --driver com.vertica.jdbc.Driver --connect "jdbc:vertica://10.10.10.10:5433/MYDB" --password dbpassword --username dbusername --target-dir "/user/my/hdfs/dir" --verbose --query 'SELECT * FROM ORDER_V2 WHERE $CONDITIONS;'

从 vertica 导入 sqoop 失败

sqoop import from vertica failed

database

bigdata

vertica

sqoop

sqoop2