在 SJS 0.9.0 中添加的 CONTEXT_ID 在 table 中被设置为 null

Question

我正在尝试在我的应用程序中赶上新的 SJS 0.9.0。创建上下文后，我将尝试提交作业 -> 发生这种情况

19/04/10 21:45:06 ERROR JobDAOActor: About to restart actor due to exception:
org.postgresql.util.PSQLException: ERROR: null value in column "CONTEXT_ID" violates not-null constraint
  Detail: Failing row contains (c144684e-3dad-459d-acff-2ac353709092, SJS_512_MB_shared_prioritized@1, 1, spark.jobserver.DPAASExecutor, 2019-04-10 21:45:06.602, null, null, null, null, null, null).
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2270)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1998)
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
    at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:570)
    at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:420)
    at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:366)
    at slick.driver.JdbcActionComponent$InsertActionComposerImpl$InsertOrUpdateAction$$anonfun$nativeUpsert.apply(JdbcActionComponent.scala:560)
    at slick.driver.JdbcActionComponent$InsertActionComposerImpl$InsertOrUpdateAction$$anonfun$nativeUpsert.apply(JdbcActionComponent.scala:557)
    at slick.jdbc.JdbcBackend$SessionDef$class.withPreparedStatement(JdbcBackend.scala:347)
    at slick.jdbc.JdbcBackend$BaseSession.withPreparedStatement(JdbcBackend.scala:407)
    at slick.driver.JdbcActionComponent$InsertActionComposerImpl.preparedInsert(JdbcActionComponent.scala:498)
    at slick.driver.JdbcActionComponent$InsertActionComposerImpl$InsertOrUpdateAction.nativeUpsert(JdbcActionComponent.scala:557)
    at slick.driver.JdbcActionComponent$InsertActionComposerImpl$InsertOrUpdateAction.f(JdbcActionComponent.scala:540)
    at slick.driver.JdbcActionComponent$InsertActionComposerImpl$InsertOrUpdateAction.run(JdbcActionComponent.scala:545)
    at slick.driver.JdbcActionComponent$SimpleJdbcDriverAction.run(JdbcActionComponent.scala:32)
    at slick.driver.JdbcActionComponent$SimpleJdbcDriverAction.run(JdbcActionComponent.scala:29)
    at slick.backend.DatabaseComponent$DatabaseDef$$anon.liftedTree1(DatabaseComponent.scala:237)
    at slick.backend.DatabaseComponent$DatabaseDef$$anon.run(DatabaseComponent.scala:237)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

https://github.com/spark-jobserver/spark-jobserver/pull/1058。这是SJS 0.9.0

新增的功能

你能解释一下为什么会这样吗？我是否应该在提交作业时包含任何额外的道具，如 CONTEXT_ID，因为它仅在 SJS 0.9.0 中添加并且我已经完成了此内容 https://github.com/spark-jobserver/spark-jobserver/releases/tag/v0.9.0 .. 或者是 spark 负责 CONTEXT_ID 事情 ?

Answer 1

实际上是因为，我们在代码中使用了较早版本的SJS（0.8.0）jar，而仅在job_server目录中使用了SJS 0.9.0 jar。它使用了仅在 0.8.0 而不是 0.9.0 中需要的 manager_start.sh。替换为 0.9.0 它工作正常!!

在 SJS 0.9.0 中添加的 CONTEXT_ID 在 table 中被设置为 null

CONTEXT_ID added in SJS 0.9.0 is being set as null in the table

apache-spark

spark-jobserver