如何在 Kafka Streams 应用程序中处理 Offset Commit 期间的超时异常
How to handle Timeout exceptions during Offset Commit in a Kafka Streams application
如果在 Kafka Streams 应用程序中提交偏移量期间出现超时异常，ProductionExceptionHandler 是否起作用？
Timeout of 60000ms expired before successfully committing offsets
我正在使用 Spring 基于云流的 Kafka 流应用程序，并且我确实有一个配置为 LogAndContinue 的 ProductionExceptionHandler 实现。但是似乎控件没有到达此超时异常的异常处理程序，并且所有流线程都死了。
正确的处理方法是什么？这是完整的错误。
[-StreamThread-1] o.a.k.s.p.i.AssignedStreamsTasks : stream-thread [stream-app-0d77ace1-4d20-413b-a71a-94cca3e620e0-StreamThread-1] Failed to commit stream task 0_3 due to the following error:
org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before successfully committing offsets {example-topic-3=OffsetAndMetadata{offset=139191, leaderEpoch=null, metadata='AQAAAXU0ut1W'}}
2020-10-17 04:05:23.410 ERROR [ratelimit-transformer,,,] 19 --- [-StreamThread-1] o.a.k.s.p.internals.StreamThread : stream-thread [stream-app-0d77ace1-4d20-413b-a71a-94cca3e620e0-StreamThread-1] Encountered the following unexpected Kafka exception during processing, this usually indicate Streams internal errors:
org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before successfully committing offsets {example-topic-3=OffsetAndMetadata{offset=139191, leaderEpoch=null, metadata='AQAAAXU0ut1W'}}
2020-10-17 04:05:24.149 ERROR [ratelimit-transformer,,,] 19 --- [-StreamThread-1] org.apache.kafka.streams.KafkaStreams : stream-client [stream-app-0d77ace1-4d20-413b-a71a-94cca3e620e0] All stream threads have died. The instance will be in error state and should be closed.
Exception in thread "stream-app-0d77ace1-4d20-413b-a71a-94cca3e620e0-StreamThread-1" org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before successfully committing offsets {example-topic-3=OffsetAndMetadata{offset=139191, leaderEpoch=null, metadata='AQAAAXU0ut1W'}}
目前 TimeoutException Kafka Streams 管理得不是很好，最终会导致您的 StreamThread 崩溃。大多数情况下，由于网络问题或 kafka 不可用会导致超时。
Kafka Streams 记录的错误消息是明确的“所有流线程已死亡。实例将处于错误状态，应关闭。”;因此，您可以停止并重新启动您的 KafkaStreams 应用程序，但如果是网络问题，重新启动您的应用程序将无济于事。
请注意，有一些工作正在进行中：
- Gracefully handle timeout exceptions on Kafka Streams | KAFKA-9274
- KIP-572 Improve timeouts and retries in Kafka+Streams
您可以在 KafkaStreams 实例上配置 java.lang.Thread.UncaughtExceptionHandler 来处理该异常 (KafkaStreams JavaDoc)
如果在 Kafka Streams 应用程序中提交偏移量期间出现超时异常，ProductionExceptionHandler 是否起作用？
Timeout of 60000ms expired before successfully committing offsets
我正在使用 Spring 基于云流的 Kafka 流应用程序，并且我确实有一个配置为 LogAndContinue 的 ProductionExceptionHandler 实现。但是似乎控件没有到达此超时异常的异常处理程序，并且所有流线程都死了。
正确的处理方法是什么？这是完整的错误。
[-StreamThread-1] o.a.k.s.p.i.AssignedStreamsTasks : stream-thread [stream-app-0d77ace1-4d20-413b-a71a-94cca3e620e0-StreamThread-1] Failed to commit stream task 0_3 due to the following error:
org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before successfully committing offsets {example-topic-3=OffsetAndMetadata{offset=139191, leaderEpoch=null, metadata='AQAAAXU0ut1W'}}
2020-10-17 04:05:23.410 ERROR [ratelimit-transformer,,,] 19 --- [-StreamThread-1] o.a.k.s.p.internals.StreamThread : stream-thread [stream-app-0d77ace1-4d20-413b-a71a-94cca3e620e0-StreamThread-1] Encountered the following unexpected Kafka exception during processing, this usually indicate Streams internal errors:
org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before successfully committing offsets {example-topic-3=OffsetAndMetadata{offset=139191, leaderEpoch=null, metadata='AQAAAXU0ut1W'}}
2020-10-17 04:05:24.149 ERROR [ratelimit-transformer,,,] 19 --- [-StreamThread-1] org.apache.kafka.streams.KafkaStreams : stream-client [stream-app-0d77ace1-4d20-413b-a71a-94cca3e620e0] All stream threads have died. The instance will be in error state and should be closed.
Exception in thread "stream-app-0d77ace1-4d20-413b-a71a-94cca3e620e0-StreamThread-1" org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before successfully committing offsets {example-topic-3=OffsetAndMetadata{offset=139191, leaderEpoch=null, metadata='AQAAAXU0ut1W'}}
目前 TimeoutException Kafka Streams 管理得不是很好，最终会导致您的 StreamThread 崩溃。大多数情况下，由于网络问题或 kafka 不可用会导致超时。
Kafka Streams 记录的错误消息是明确的“所有流线程已死亡。实例将处于错误状态，应关闭。”;因此，您可以停止并重新启动您的 KafkaStreams 应用程序，但如果是网络问题，重新启动您的应用程序将无济于事。
请注意，有一些工作正在进行中：
- Gracefully handle timeout exceptions on Kafka Streams | KAFKA-9274
- KIP-572 Improve timeouts and retries in Kafka+Streams
您可以在 KafkaStreams 实例上配置 java.lang.Thread.UncaughtExceptionHandler 来处理该异常 (KafkaStreams JavaDoc)