RSocket:本地和远程状态不一致
RSocket: Local and remote state disagreement
足够稳定的系统每天处理约 5000 个 WS 请求,并且 1%(约 50 个连接)连接因错误而关闭,下一个异常:在恢复过程中,我们在帧删除过程中存在分歧。
有人遇到过这个问题吗?知道发生了什么事吗?
java 11
kotlin 1.4.10
org.springframework.boot:spring-boot-starter-rsocket:2.4.0
-> io.rsocket:rsocket-core:1.1.0
-> io.rsocket:rsocket-transport-netty:1.1.0
15:34:05.998 [reactor-http-epoll-3] ERROR reactor.core.publisher.Operators - Operator called default onErrorDropped
reactor.core.Exceptions$ErrorCallbackNotImplemented: java.lang.IllegalStateException: Local and remote state disagreement: need to remove additional 194 bytes, but cache is empty
Caused by: java.lang.IllegalStateException: Local and remote state disagreement: need to remove additional 194 bytes, but cache is empty
at io.rsocket.resume.InMemoryResumableFramesStore.releaseFrames(InMemoryResumableFramesStore.java:120)
at io.rsocket.resume.ServerRSocketSession.doResume(ServerRSocketSession.java:153)
at io.rsocket.resume.ServerRSocketSession.lambda$resumeWith(ServerRSocketSession.java:109)
at io.rsocket.resume.ServerRSocketSession.resumeWith(ServerRSocketSession.java:117)
at io.rsocket.core.ServerSetup$ResumableServerSetup.acceptRSocketResume(ServerSetup.java:137)
at io.rsocket.core.RSocketServer.acceptResume(RSocketServer.java:356)
at io.rsocket.core.RSocketServer.accept(RSocketServer.java:368)
at io.rsocket.core.RSocketServer.lambda$acceptor[=10=](RSocketServer.java:350)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:125)
at com.company.mdc.config.MdcContextLifter.onNext(MdcConfig.kt:40)
at reactor.core.publisher.FluxFirstWithSignal$FirstEmittingSubscriber.onNext(FluxFirstWithSignal.java:329)
at com.company.mdc.config.MdcContextLifter.onNext(MdcConfig.kt:40)
at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:160)
at io.rsocket.core.SetupHandlingDuplexConnection.onNext(SetupHandlingDuplexConnection.java:114)
at io.rsocket.core.SetupHandlingDuplexConnection.onNext(SetupHandlingDuplexConnection.java:19)
at com.company.mdc.config.MdcContextLifter.onNext(MdcConfig.kt:40)
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:120)
at reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:265)
at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:371)
at reactor.netty.channel.ChannelOperations.onInboundNext(ChannelOperations.java:381)
at reactor.netty.http.server.HttpServerOperations.onInboundNext(HttpServerOperations.java:544)
at reactor.netty.http.server.WebsocketServerOperations.onInboundNext(WebsocketServerOperations.java:161)
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:94)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
at io.netty.util.concurrent.SingleThreadEventExecutor.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
那是 rsoket 库中的错误。问题已经解决
https://github.com/rsocket/rsocket-java/issues/973
足够稳定的系统每天处理约 5000 个 WS 请求,并且 1%(约 50 个连接)连接因错误而关闭,下一个异常:在恢复过程中,我们在帧删除过程中存在分歧。
有人遇到过这个问题吗?知道发生了什么事吗?
java 11
kotlin 1.4.10
org.springframework.boot:spring-boot-starter-rsocket:2.4.0
-> io.rsocket:rsocket-core:1.1.0
-> io.rsocket:rsocket-transport-netty:1.1.0
15:34:05.998 [reactor-http-epoll-3] ERROR reactor.core.publisher.Operators - Operator called default onErrorDropped
reactor.core.Exceptions$ErrorCallbackNotImplemented: java.lang.IllegalStateException: Local and remote state disagreement: need to remove additional 194 bytes, but cache is empty
Caused by: java.lang.IllegalStateException: Local and remote state disagreement: need to remove additional 194 bytes, but cache is empty
at io.rsocket.resume.InMemoryResumableFramesStore.releaseFrames(InMemoryResumableFramesStore.java:120)
at io.rsocket.resume.ServerRSocketSession.doResume(ServerRSocketSession.java:153)
at io.rsocket.resume.ServerRSocketSession.lambda$resumeWith(ServerRSocketSession.java:109)
at io.rsocket.resume.ServerRSocketSession.resumeWith(ServerRSocketSession.java:117)
at io.rsocket.core.ServerSetup$ResumableServerSetup.acceptRSocketResume(ServerSetup.java:137)
at io.rsocket.core.RSocketServer.acceptResume(RSocketServer.java:356)
at io.rsocket.core.RSocketServer.accept(RSocketServer.java:368)
at io.rsocket.core.RSocketServer.lambda$acceptor[=10=](RSocketServer.java:350)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:125)
at com.company.mdc.config.MdcContextLifter.onNext(MdcConfig.kt:40)
at reactor.core.publisher.FluxFirstWithSignal$FirstEmittingSubscriber.onNext(FluxFirstWithSignal.java:329)
at com.company.mdc.config.MdcContextLifter.onNext(MdcConfig.kt:40)
at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:160)
at io.rsocket.core.SetupHandlingDuplexConnection.onNext(SetupHandlingDuplexConnection.java:114)
at io.rsocket.core.SetupHandlingDuplexConnection.onNext(SetupHandlingDuplexConnection.java:19)
at com.company.mdc.config.MdcContextLifter.onNext(MdcConfig.kt:40)
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:120)
at reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:265)
at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:371)
at reactor.netty.channel.ChannelOperations.onInboundNext(ChannelOperations.java:381)
at reactor.netty.http.server.HttpServerOperations.onInboundNext(HttpServerOperations.java:544)
at reactor.netty.http.server.WebsocketServerOperations.onInboundNext(WebsocketServerOperations.java:161)
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:94)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
at io.netty.util.concurrent.SingleThreadEventExecutor.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
那是 rsoket 库中的错误。问题已经解决 https://github.com/rsocket/rsocket-java/issues/973