Error: [Pivotal][Greenplum JDBC Driver][Greenplum]prepared statement "PS1" already exists

Question

我在 Azure 上有一个 Greenplum 集群，当使用简单的 JDBC 连接时，我可以从本地机器访问它的数据。现在我尝试在 Spark SQL 中使用相同的 JDBC 驱动程序，如下所示：

val url = s"jdbc:pivotal:greenplum://$server:$port;DatabaseName=$database"
Spark.sqlContext.read.format("jdbc")
  .options(Map(
    "url"->url, "user"-> user, "password"-> password,
    "dbschema" -> "public", "dbtable" -> table
  ))
  .load()

执行load()时，抛出如下错误

Exception in thread "main" java.sql.SQLSyntaxErrorException: [Pivotal][Greenplum JDBC Driver][Greenplum]prepared statement "PS1" already exists. 
    at com.pivotal.jdbc.greenplumbase.ddcd.b(Unknown Source)
    at com.pivotal.jdbc.greenplumbase.ddcd.a(Unknown Source)
    at com.pivotal.jdbc.greenplumbase.ddcc.b(Unknown Source)
    at com.pivotal.jdbc.greenplumbase.ddcc.a(Unknown Source)
    at com.pivotal.jdbc.greenplum.wp.ddj.m(Unknown Source)
    at com.pivotal.jdbc.greenplum.ddg.c(Unknown Source)
    at com.pivotal.jdbc.greenplum.ddg.d(Unknown Source)
    at com.pivotal.jdbc.greenplum.ddg.a(Unknown Source)
    at com.pivotal.jdbc.greenplumbase.dddr.y(Unknown Source)
    at com.pivotal.jdbc.greenplumbase.dddr.x(Unknown Source)
    at com.pivotal.jdbc.greenplumbase.dddl.executeQuery(Unknown Source)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:61)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
    at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
    at dzlab.GreenplumSparkSample$.read1(GreenplumPropertyFactory.scala:21)
    at dzlab.GreenplumSparkSample$.main(GreenplumPropertyFactory.scala:35)
    at dzlab.GreenplumSparkSample.main(GreenplumPropertyFactory.scala)

如果我使用简单的 JDBC 连接和 read/write 数据java 方式，一切正常。所以不确定这里有什么问题？

Answer 1

Greenplum 在云中的部署使用带有 pool_mode 事务的 pgBouncer。 pgBouncer 在此模式下不支持准备好的语句。

有一些解决方法。一个是 recommended way, "The proper way to do it for JDBC is adding prepareThreshold=0 parameter to connect string."

另一种解决方法是直接连接到 Greenplum 并绕过 pgBouncer。数据库监听端口 6432 而 pgBouncer 监听端口 5432.

Error: [Pivotal][Greenplum JDBC Driver][Greenplum]prepared statement "PS1" already exists

Error: [Pivotal][Greenplum JDBC Driver][Greenplum]prepared statement "PS1" already exists

jdbc

azure

greenplum

apache-spark

apache-spark-sql