Kafka 流:InvalidStateStoreException

Kafka Streams: InvalidStateStoreException

如果有状态流应用单节点6线程启动,会出现上述异常吗?

如果有状态流应用程序在节点 1 上启动并使用特定主题,并在不同节点上 运行,是否需要遵循任何流程?

如果有状态流应用在2个节点上启动,如果出现上述异常,流应用会立即终止吗?


跟进:

If the stateful stream application is started with 6 threads on a single node, would the above exception occur?

It can

基本上我想知道如果我们将整个主题消费保持在单个节点上,如果发生重新平衡,它是否会避免从内部主题重新构建存储,因为其中一个线程正在运行 down/terminates?

store is not ready yet: you can wait until the store is ready -- best to register a restore callback (check the docs for details) to get informed when restore is finished and you can retry to query the store.

抱歉,我想澄清一下,是 StateRestoreCallback 还是 StateRestoreListener?我估计是后者。另外,是否需要覆盖 StateRestoreCallback 并包含恢复商店的逻辑?

InvalidStateStoreException 可能有不同的原因,因此,如果没有更多上下文,很难回答您的问题。

If the stateful stream application is started with 6 threads on a single node, would the above exception occur?

可以。

Is there any process that needs to be followed, if a stateful stream application started on node 1 consuming a particular topic, is made to run on different node?

没有

If the stateful stream application is started on 2 nodes and if the above exception occurs, would the stream application terminate immediately?

取决于抛出异常的位置:

  • 要么,相应的StreamThread会死掉,但应用程序不会自动终止。您应该在 KafkaStreams 实例上注册一个未捕获的异常处理程序,并使用自定义代码对垂死的线程做出反应(例如,终止应用程序)。
  • 如果它是使用交互式查询从 KafkaStreams 抛出的,StreamThread 不会受到影响。

Where can this exception be caught in a try-catch block?

通常是的,尤其是当您提到交互式查询功能时。

if we add sleep for 10 mins, would the store automatically gets to valid state?

如果你提到交互式查询功能,睡觉不是一个好策略。异常的原因有多种,您需要做出相应的反应:

  • 商店不在本地而是在不同的节点上:您可以通过检查商店元数据来解决这个问题。
  • store 还没有准备好:您可以等到 store 准备好——最好注册一个恢复侦听器(查看文档了解详细信息),以便在恢复完成时得到通知,您可以重试查询商店。

更新

Essentially I was wondering if we keep the entire topic consumption on a single node, would it avoid re-building the store from an internal topic if a re-balancing occurs, due to one of the thread going down/terminates?

是(对于非 EOS 情况)。其他线程将检测本地存储并重新使用它。

StateRestoreCallback OR StateRestoreListener

是的,是 StateRestoreListener。只有编写自定义状态存储才能实现 StateRestoreCallback