Azure 函数与数据块笔记本通信时出错
Azure functions error communicating with a databricks notebook
我在 azure functions 和 databricks notebook 之间建立了连接,以便从 notebook 中提取数据。
直到 2 月 7 日,它都运行良好,但随后我开始收到一个没有合理错误代码的错误。我有
附上堆栈跟踪和错误消息。还提供了代码中实际错误的屏幕截图。感谢任何帮助。
WorkflowException: com.databricks.NotebookExecutionException: FAILED Caused by: NotebookExecutionException: FAILED
at com.databricks.workflow.WorkflowDriver.run(WorkflowDriver.scala:71)
at com.databricks.dbutils_v1.impl.NotebookUtilsImpl.run(NotebookUtilsImpl.scala:122)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(command--1:1)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw$$iw.<init>(command--1:43)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw.<init>(command--1:45)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw.<init>(command--1:47)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw.<init>(command--1:49)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw.<init>(command--1:51)
at $line1f0f2874076241fca3dba850716c513d25.$read.<init>(command--1:53)
at $line1f0f2874076241fca3dba850716c513d25.$read$.<init>(command--1:57)
at $line1f0f2874076241fca3dba850716c513d25.$read$.<clinit>(command--1)
at $line1f0f2874076241fca3dba850716c513d25.$eval$.$print$lzycompute(<notebook>:7)
at $line1f0f2874076241fca3dba850716c513d25.$eval$.$print(<notebook>:6)
at $line1f0f2874076241fca3dba850716c513d25.$eval.$print(<notebook>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:747)
at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1020)
at scala.tools.nsc.interpreter.IMain.$anonfun$interpret(IMain.scala:568)
at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)
at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)
at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:41)
at scala.tools.nsc.interpreter.IMain.loadAndRunReq(IMain.scala:567)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:594)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:564)
at com.databricks.backend.daemon.driver.DriverILoop.execute(DriverILoop.scala:219)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.$anonfun$repl(ScalaDriverLocal.scala:235)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExitInternal$.trapExit(DriverLocal.scala:902)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExit$.apply(DriverLocal.scala:855)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.repl(ScalaDriverLocal.scala:235)
at com.databricks.backend.daemon.driver.DriverLocal.$anonfun$execute(DriverLocal.scala:541)
at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext(UsageLogging.scala:266)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:261)
at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:258)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionContext(DriverLocal.scala:50)
at com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:305)
at com.databricks.logging.UsageLogging.withAttributionTags$(UsageLogging.scala:297)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionTags(DriverLocal.scala:50)
at com.databricks.backend.daemon.driver.DriverLocal.execute(DriverLocal.scala:518)
at com.databricks.backend.daemon.driver.DriverWrapper.$anonfun$tryExecutingCommand(DriverWrapper.scala:689)
at scala.util.Try$.apply(Try.scala:213)
at com.databricks.backend.daemon.driver.DriverWrapper.tryExecutingCommand(DriverWrapper.scala:681)
at com.databricks.backend.daemon.driver.DriverWrapper.getCommandOutputAndError(DriverWrapper.scala:522)
at com.databricks.backend.daemon.driver.DriverWrapper.executeCommand(DriverWrapper.scala:634)
at com.databricks.backend.daemon.driver.DriverWrapper.runInnerLoop(DriverWrapper.scala:427)
at com.databricks.backend.daemon.driver.DriverWrapper.runInner(DriverWrapper.scala:370)
at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:221)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.databricks.NotebookExecutionException: FAILED
at com.databricks.workflow.WorkflowDriver.run0(WorkflowDriver.scala:117)
at com.databricks.workflow.WorkflowDriver.run(WorkflowDriver.scala:66)
at com.databricks.dbutils_v1.impl.NotebookUtilsImpl.run(NotebookUtilsImpl.scala:122)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(command--1:1)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw$$iw.<init>(command--1:43)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw.<init>(command--1:45)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw.<init>(command--1:47)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw.<init>(command--1:49)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw.<init>(command--1:51)
at $line1f0f2874076241fca3dba850716c513d25.$read.<init>(command--1:53)
at $line1f0f2874076241fca3dba850716c513d25.$read$.<init>(command--1:57)
at $line1f0f2874076241fca3dba850716c513d25.$read$.<clinit>(command--1)
at $line1f0f2874076241fca3dba850716c513d25.$eval$.$print$lzycompute(<notebook>:7)
at $line1f0f2874076241fca3dba850716c513d25.$eval$.$print(<notebook>:6)
at $line1f0f2874076241fca3dba850716c513d25.$eval.$print(<notebook>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:747)
at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1020)
at scala.tools.nsc.interpreter.IMain.$anonfun$interpret(IMain.scala:568)
at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)
at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)
at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:41)
at scala.tools.nsc.interpreter.IMain.loadAndRunReq(IMain.scala:567)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:594)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:564)
at com.databricks.backend.daemon.driver.DriverILoop.execute(DriverILoop.scala:219)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.$anonfun$repl(ScalaDriverLocal.scala:235)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExitInternal$.trapExit(DriverLocal.scala:902)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExit$.apply(DriverLocal.scala:855)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.repl(ScalaDriverLocal.scala:235)
at com.databricks.backend.daemon.driver.DriverLocal.$anonfun$execute(DriverLocal.scala:541)
at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext(UsageLogging.scala:266)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:261)
at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:258)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionContext(DriverLocal.scala:50)
at com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:305)
at com.databricks.logging.UsageLogging.withAttributionTags$(UsageLogging.scala:297)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionTags(DriverLocal.scala:50)
at com.databricks.backend.daemon.driver.DriverLocal.execute(DriverLocal.scala:518)
at com.databricks.backend.daemon.driver.DriverWrapper.$anonfun$tryExecutingCommand(DriverWrapper.scala:689)
at scala.util.Try$.apply(Try.scala:213)
at com.databricks.backend.daemon.driver.DriverWrapper.tryExecutingCommand(DriverWrapper.scala:681)
at com.databricks.backend.daemon.driver.DriverWrapper.getCommandOutputAndError(DriverWrapper.scala:522)
at com.databricks.backend.daemon.driver.DriverWrapper.executeCommand(DriverWrapper.scala:634)
at com.databricks.backend.daemon.driver.DriverWrapper.runInnerLoop(DriverWrapper.scala:427)
at com.databricks.backend.daemon.driver.DriverWrapper.runInner(DriverWrapper.scala:370)
at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:221)
at java.lang.Thread.run(Thread.java:748)
databricks 集群中的笔记本在集群 运行 时保留缓存。如果您添加导入语句然后将其删除,笔记本仍具有该导入的缓存实例并将继续工作。 运行 Visual Studio 中针对笔记本的代码创建了一个新的上下文,因此上述导入不再可用,并因无用错误而爆炸。开始逐行在一个新的笔记本上构建笔记本,一旦我添加了我的罪魁祸首行,我就知道问题出在哪里了。
之前:
之后:
我在 azure functions 和 databricks notebook 之间建立了连接,以便从 notebook 中提取数据。 直到 2 月 7 日,它都运行良好,但随后我开始收到一个没有合理错误代码的错误。我有 附上堆栈跟踪和错误消息。还提供了代码中实际错误的屏幕截图。感谢任何帮助。
WorkflowException: com.databricks.NotebookExecutionException: FAILED Caused by: NotebookExecutionException: FAILED
at com.databricks.workflow.WorkflowDriver.run(WorkflowDriver.scala:71)
at com.databricks.dbutils_v1.impl.NotebookUtilsImpl.run(NotebookUtilsImpl.scala:122)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(command--1:1)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw$$iw.<init>(command--1:43)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw.<init>(command--1:45)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw.<init>(command--1:47)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw.<init>(command--1:49)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw.<init>(command--1:51)
at $line1f0f2874076241fca3dba850716c513d25.$read.<init>(command--1:53)
at $line1f0f2874076241fca3dba850716c513d25.$read$.<init>(command--1:57)
at $line1f0f2874076241fca3dba850716c513d25.$read$.<clinit>(command--1)
at $line1f0f2874076241fca3dba850716c513d25.$eval$.$print$lzycompute(<notebook>:7)
at $line1f0f2874076241fca3dba850716c513d25.$eval$.$print(<notebook>:6)
at $line1f0f2874076241fca3dba850716c513d25.$eval.$print(<notebook>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:747)
at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1020)
at scala.tools.nsc.interpreter.IMain.$anonfun$interpret(IMain.scala:568)
at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)
at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)
at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:41)
at scala.tools.nsc.interpreter.IMain.loadAndRunReq(IMain.scala:567)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:594)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:564)
at com.databricks.backend.daemon.driver.DriverILoop.execute(DriverILoop.scala:219)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.$anonfun$repl(ScalaDriverLocal.scala:235)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExitInternal$.trapExit(DriverLocal.scala:902)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExit$.apply(DriverLocal.scala:855)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.repl(ScalaDriverLocal.scala:235)
at com.databricks.backend.daemon.driver.DriverLocal.$anonfun$execute(DriverLocal.scala:541)
at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext(UsageLogging.scala:266)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:261)
at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:258)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionContext(DriverLocal.scala:50)
at com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:305)
at com.databricks.logging.UsageLogging.withAttributionTags$(UsageLogging.scala:297)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionTags(DriverLocal.scala:50)
at com.databricks.backend.daemon.driver.DriverLocal.execute(DriverLocal.scala:518)
at com.databricks.backend.daemon.driver.DriverWrapper.$anonfun$tryExecutingCommand(DriverWrapper.scala:689)
at scala.util.Try$.apply(Try.scala:213)
at com.databricks.backend.daemon.driver.DriverWrapper.tryExecutingCommand(DriverWrapper.scala:681)
at com.databricks.backend.daemon.driver.DriverWrapper.getCommandOutputAndError(DriverWrapper.scala:522)
at com.databricks.backend.daemon.driver.DriverWrapper.executeCommand(DriverWrapper.scala:634)
at com.databricks.backend.daemon.driver.DriverWrapper.runInnerLoop(DriverWrapper.scala:427)
at com.databricks.backend.daemon.driver.DriverWrapper.runInner(DriverWrapper.scala:370)
at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:221)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.databricks.NotebookExecutionException: FAILED
at com.databricks.workflow.WorkflowDriver.run0(WorkflowDriver.scala:117)
at com.databricks.workflow.WorkflowDriver.run(WorkflowDriver.scala:66)
at com.databricks.dbutils_v1.impl.NotebookUtilsImpl.run(NotebookUtilsImpl.scala:122)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(command--1:1)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw$$iw.<init>(command--1:43)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw$$iw.<init>(command--1:45)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw$$iw.<init>(command--1:47)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw$$iw.<init>(command--1:49)
at $line1f0f2874076241fca3dba850716c513d25.$read$$iw.<init>(command--1:51)
at $line1f0f2874076241fca3dba850716c513d25.$read.<init>(command--1:53)
at $line1f0f2874076241fca3dba850716c513d25.$read$.<init>(command--1:57)
at $line1f0f2874076241fca3dba850716c513d25.$read$.<clinit>(command--1)
at $line1f0f2874076241fca3dba850716c513d25.$eval$.$print$lzycompute(<notebook>:7)
at $line1f0f2874076241fca3dba850716c513d25.$eval$.$print(<notebook>:6)
at $line1f0f2874076241fca3dba850716c513d25.$eval.$print(<notebook>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:747)
at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1020)
at scala.tools.nsc.interpreter.IMain.$anonfun$interpret(IMain.scala:568)
at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)
at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)
at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:41)
at scala.tools.nsc.interpreter.IMain.loadAndRunReq(IMain.scala:567)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:594)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:564)
at com.databricks.backend.daemon.driver.DriverILoop.execute(DriverILoop.scala:219)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.$anonfun$repl(ScalaDriverLocal.scala:235)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExitInternal$.trapExit(DriverLocal.scala:902)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExit$.apply(DriverLocal.scala:855)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.repl(ScalaDriverLocal.scala:235)
at com.databricks.backend.daemon.driver.DriverLocal.$anonfun$execute(DriverLocal.scala:541)
at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext(UsageLogging.scala:266)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:261)
at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:258)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionContext(DriverLocal.scala:50)
at com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:305)
at com.databricks.logging.UsageLogging.withAttributionTags$(UsageLogging.scala:297)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionTags(DriverLocal.scala:50)
at com.databricks.backend.daemon.driver.DriverLocal.execute(DriverLocal.scala:518)
at com.databricks.backend.daemon.driver.DriverWrapper.$anonfun$tryExecutingCommand(DriverWrapper.scala:689)
at scala.util.Try$.apply(Try.scala:213)
at com.databricks.backend.daemon.driver.DriverWrapper.tryExecutingCommand(DriverWrapper.scala:681)
at com.databricks.backend.daemon.driver.DriverWrapper.getCommandOutputAndError(DriverWrapper.scala:522)
at com.databricks.backend.daemon.driver.DriverWrapper.executeCommand(DriverWrapper.scala:634)
at com.databricks.backend.daemon.driver.DriverWrapper.runInnerLoop(DriverWrapper.scala:427)
at com.databricks.backend.daemon.driver.DriverWrapper.runInner(DriverWrapper.scala:370)
at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:221)
at java.lang.Thread.run(Thread.java:748)
databricks 集群中的笔记本在集群 运行 时保留缓存。如果您添加导入语句然后将其删除,笔记本仍具有该导入的缓存实例并将继续工作。 运行 Visual Studio 中针对笔记本的代码创建了一个新的上下文,因此上述导入不再可用,并因无用错误而爆炸。开始逐行在一个新的笔记本上构建笔记本,一旦我添加了我的罪魁祸首行,我就知道问题出在哪里了。
之前:
之后: