IMap InternalKey 的 HazelcastSerializationException - 如何调试或记录

Question

我们正在使用实现 Cache<K, V> 的 Class。缓存存储在 Hazelcast IMap<InternalKey<K, C>, V> 中。有一个 "remove" 逻辑删除了 <String, String> 的 InternalKey。

K、C 和 V 都是字符串。

public class HazelcastMapJCacheAdapter<K, C, V> implements Cache<K, V> {
    private IMap<InternalKey<K, C>, V> cachedData;

    public void removeEntriesByKeyAndContext(BiPredicate<K, C> removeCondition) {
        Predicate<InternalKey<K, C>, V> predicate = (Predicate<InternalKey<K, C>, V> & Serializable)
            mapEntry -> (removeCondition.test(mapEntry.getKey().getOriginalKey(), mapEntry.getKey().getContext()));
        cachedData.removeAll(predicate);
    }
}

我一直收到无法理解的 HazelcastSerializationException。字符串序列化会出什么问题？我怎样才能正确记录这个场景以获得更多数据（我手头的参数是一个谓词，它是一个函数接口......）？我无法在我的开发环境中重现这种情况，所以调试它是一个问题。

谢谢

2020-03-23 02:07:34 WARN | [10.212.179.245]:5701 [MyApp] [3.12.5] Error while logging processing event
com.hazelcast.nio.serialization.HazelcastSerializationException: Failed to serialize 'com.hazelcast.spi.impl.operationservice.impl.operations.PartitionIteratingOperation'
    at com.hazelcast.internal.serialization.impl.SerializationUtil.handleSerializeException(SerializationUtil.java:82) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:157) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:133) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:124) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.spi.impl.operationservice.impl.OutboundOperationHandler.send(OutboundOperationHandler.java:56) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeRemote(Invocation.java:656) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:631) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:592) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:256) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:61) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.spi.impl.operationservice.impl.InvokeOnPartitions.invokeOnAllPartitions(InvokeOnPartitions.java:121) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.spi.impl.operationservice.impl.InvokeOnPartitions.invokeAsync(InvokeOnPartitions.java:99) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.spi.impl.operationservice.impl.InvokeOnPartitions.invoke(InvokeOnPartitions.java:88) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.spi.impl.operationservice.impl.OperationServiceImpl.invokeOnAllPartitions(OperationServiceImpl.java:385) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.map.impl.proxy.MapProxySupport.removeAllInternal(MapProxySupport.java:618) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.map.impl.proxy.NearCachedMapProxyImpl.removeAllInternal(NearCachedMapProxyImpl.java:330) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.hazelcast.map.impl.proxy.MapProxyImpl.removeAll(MapProxyImpl.java:285) ~[hazelcast-3.12.5.jar:3.12.5]
    at com.myapp.cache.impl.HazelcastMapJCacheAdapter.removeEntriesByKeyAndContext(HazelcastMapJCacheAdapter.java:578) ~[myapp-distributed-cache-impl.jar:?]

Answer 1

您需要在服务器的 class 路径上设置 class InternalKey 的定义，因为条目是为谓词反序列化的。

稍微断章取义，您也可以尝试将 in-memory-format 设置为 OBJECT 以提高谓词执行的性能。

Answer 2

以下是 removeAll（以及任何其他类型的服务器端处理，如 executeOnKey 等）的要求摘要：

键和值必须是可序列化的，并且它们的 classes 必须在服务器 classpath
Predicate（或 EntryProcessor 等）必须是可序列化的并且在服务器 classpath

我的猜测是，在您的情况下，BiPredicate<K, C> removeCondition 不可序列化；但是它必须作为传递给 removeAll 的 lambda 的一部分进行序列化。例如考虑这个代码：

class Scratch {
    public static void main(String[] args) {
        remove((x, y) -> true);
    }

    private static void remove(BiPredicate<String, String> predicate) {
        HazelcastInstance hz = Hazelcast.newHazelcastInstance();
        // put 100 items in the map
        IMap<String, String> map = hz.getMap("map");
        for (int i = 0; i < 100; i++) {
            map.put("" + i, "" + i);
        }

        map.removeAll((Predicate<String, String> & Serializable)
                entry -> predicate.test(entry.getKey(), entry.getValue()));

        // now size is 0
        System.out.println("Map size after removeAll " + map.size());
    }
}

以上代码将毫无问题地执行。在集群中添加另一个 HazelcastInstance 的那一刻（只需在 remove 方法主体中添加 Hazelcast.newHazelcastInstance()），Predicate 实例必须被序列化并通过网络在 2 个集群之间发送成员。这失败了，因为 predicate 参数不是 Serializable，而是作为参数传递给 removeAll 的 lambda Predicate 的一部分。解决方案是确保 lambda 中引用的所有内容都是可序列化的。所以在上面的例子中，解决方法是像这样更新 main 方法：

public static void main(String[] args) {
    remove((BiPredicate<String, String> & Serializable) (x, y) -> true);
}

一般来说，使用 lambda 时要小心，因为很容易意外捕获外部 class 字段，在这种情况下，您将需要解决包含 class 和 lambda 的序列化的可序列化问题表格大小会意外膨胀。引用 https://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html#serialization "You can serialize a lambda expression if its target type and its captured arguments are serializable. However, like inner classes, the serialization of lambda expressions is strongly discouraged."

IMap InternalKey 的 HazelcastSerializationException - 如何调试或记录

HazelcastSerializationException for an IMap InternalKey - how to debug or log

exception

jcache

hazelcast

hazelcast-imap