火花流中的 c3p0 错误

Question

我用c3po连接jdbc(impala),但是failed.Could你帮帮我：）

ConnectPool.scala

class ConnectPool extends Serializable{
  private val cpds: ComboPooledDataSource = new ComboPooledDataSource(true)
  private val conf = Utils.getPropmap("env.properties")
  try {
    cpds.setJdbcUrl(conf("kudu.produce.url"))
    cpds.setDriverClass(conf("jdbc.driver"))
    cpds.setMaxPoolSize(400)
    cpds.setMinPoolSize(20)
    cpds.setAcquireIncrement(5)
    cpds.setMaxStatements(380)
  } catch {
    case e: Exception => e.printStackTrace()
  }
  def getConnection: Connection = {
    try {
      return cpds.getConnection()
    } catch {
      case ex: Exception =>
        ex.printStackTrace()
        null
    }
  }
}

object ConnectManager {
  var kuduManager: ConnectPool = _
  def getConnectManager: ConnectPool = {
    synchronized {
      if (kuduManager == null) {
        kuduManager = new ConnectPool
      }
    }
    kuduManager
  }
}

main.scala

 messages.foreachRDD(rdd => {

      val conn = ConnectManager.getConnectManager.getConnection
      val stmt = conn.createStatement

      if(!rdd.isEmpty() && rdd.count() >0){
        //初始化spark
        val spark = SparkSession.builder.config(rdd.sparkContext.getConf).getOrCreate()
try{
         // use stmt
        }catch {
          case e: Exception => print("\ntest\n")
        } finally {
          stmt.close()
          conn.close()}
      }

    })

输出

18/03/16 16:56:00 INFO c3p0.SQLWarnings: [Simba]ImpalaJDBCDriver Error setting default connection property values: {0} java.sql.SQLWarning: [Simba]ImpalaJDBCDriver Error setting default connection property values: {0} at com.cloudera.jdbc.common.SWarningListener.createSQLWarning(Unknown Source) at com.cloudera.jdbc.common.SWarningListener.postWarning(Unknown Source) at com.cloudera.jdbc.common.SConnection.(Unknown Source) at com.cloudera.jdbc.common4.C4SConnection.(Unknown Source) at com.cloudera.jdbc.jdbc41.S41Connection.(Unknown Source) at com.cloudera.impala.jdbc41.ImpalaJDBC41Connection.(Unknown Source) at com.cloudera.impala.jdbc41.ImpalaJDBC41ObjectFactory.createConnection(Unknown Source) at com.cloudera.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source) at com.cloudera.jdbc.common.AbstractDriver.connect(Unknown Source) at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:119) at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:143) at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:132

Answer 1

您看到的输出是警告，而不是错误。 c3p0 重置并打印 Connection 在将它们签回池之前遇到的任何警告。这有时会让开发人员感到惊讶，因为很少有开发人员费心去检查警告，因此他们经常被忽视。

我不知道这个警告到底是什么意思。但是 JDBC 连接警告经常出现在不严重的情况下。除了警告输出之外，您的应用程序是否正常运行？

如果您决定可以忍受这些警告，您可以通过配置特殊记录器 com.mchange.v2.c3p0.SQLWarnings 重定向或关闭它们。将此记录器的日志级别设置为比 INFO 更严重的级别将阻止这些消息的出现。

火花流中的 c3p0 错误

c3p0 error in spark streaming

c3p0

spark-streaming

main.scala

输出