找不到 hazelcast 映射器 class

Question

我正在尝试运行 map/reduce 在 hazelcast 3.4 上执行任务，但我一直收到 class未找到异常

我创建了几个播放器并将它们存储到 IMap

    this.conf = new ClientConfig();
    HazelcastInstance cluster = HazelcastClient.newHazelcastClient(this.conf);
    Map<String, Player> mapPlayers = cluster.getMap("players");

    for (int playerID = 0; playerID < 10000; playerID++) {
        Player p = new Player();
        mapPlayers.put(Integer.toString(playerID), p);
        System.out.println("inserted player nuber " + Integer.toString(playerID));
    }

之后我创建了 map reduce class MRCount

public class MRCount {
    private ClientConfig conf;

    public MRCount() {
        this.conf = new ClientConfig();
        this.conf.getNetworkConfig();
    }

    public void getCount() throws ExecutionException, InterruptedException {
        HazelcastInstance cluster = HazelcastClient.newHazelcastClient(this.conf);
        IMap<String, Player> mapPlayers = cluster.getMap("players");
        KeyValueSource<String, Player> source = KeyValueSource.fromMap(mapPlayers);
        JobTracker jobTracker = cluster.getJobTracker("default");
        Job<String, Player> job = jobTracker.newJob(source);

        ICompletableFuture<Map<String, Integer>> future = job.mapper(new MyMapper())
                                                                .reducer(new MyReducerFactory()).submit();

//        future.andThen(buildCallback());
        Map<String, Integer> result = future.get();

        for (Map.Entry<String, Integer> e: result.entrySet()) {
            System.out.println(e.getKey() + ":  " + Integer.toString(e.getValue()) );
        }

        cluster.shutdown();
    }
}

映射器class:

public class MyMapper implements Mapper<String, Player, String, Integer> {

    @Override
    public void map(String key, Player value, Context<String, Integer> context) {

        context.emit("total", 1);
    }
}

public class MyReducerFactory implements ReducerFactory<String, Integer, Integer>{

    @Override
    public Reducer<Integer, Integer> newReducer(String key) {
        return new MyReducer();
    }

    private class MyReducer extends Reducer<Integer, Integer> {
        private volatile Integer result = 0;
        @Override
        public void reduce(Integer value) {
            result += value;
        }

        @Override
        public Integer finalizeReduce() {
            return result;
        }
    }
}

我用 maven 构建了我的项目。当我启动导入功能时，一切顺利，我已经将播放器存储在 HZ 集群中，但是当我启动 MRCount.getCount() 函数时，我不断收到 HazelcastSerializationException

    Exception in thread "main" java.util.concurrent.ExecutionException: com.hazelcast.nio.serialization.HazelcastSerializationException: java.lang.ClassNotFoundException: com.test.queries.models.MyMapper
    at com.hazelcast.client.spi.impl.ClientCallFuture.resolveResponse(ClientCallFuture.java:214)
    at com.hazelcast.client.spi.impl.ClientCallFuture.access[=14=]0(ClientCallFuture.java:53)
    at com.hazelcast.client.spi.impl.ClientCallFuture.run(ClientCallFuture.java:286)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
    at com.hazelcast.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76)
    at com.hazelcast.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:92)
Caused by: com.hazelcast.nio.serialization.HazelcastSerializationException: java.lang.ClassNotFoundException: com.test.queries.models.MyMapper
    at com.hazelcast.nio.serialization.DefaultSerializers$ObjectSerializer.read(DefaultSerializers.java:201)
    at com.hazelcast.nio.serialization.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
    at com.hazelcast.nio.serialization.SerializationServiceImpl.readObject(SerializationServiceImpl.java:309)
    at com.hazelcast.nio.serialization.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:439)
    at com.hazelcast.mapreduce.impl.client.ClientMapReduceRequest.readData(ClientMapReduceRequest.java:226)
    at com.hazelcast.mapreduce.impl.client.ClientMapReduceRequest.read(ClientMapReduceRequest.java:181)
    at com.hazelcast.client.impl.client.ClientRequest.readPortable(ClientRequest.java:116)
    at com.hazelcast.nio.serialization.PortableSerializer.read(PortableSerializer.java:88)
    at com.hazelcast.nio.serialization.PortableSerializer.read(PortableSerializer.java:30)
    at com.hazelcast.nio.serialization.StreamSerializerAdapter.toObject(StreamSerializerAdapter.java:65)
    at com.hazelcast.nio.serialization.SerializationServiceImpl.toObject(SerializationServiceImpl.java:260)
    at com.hazelcast.client.impl.ClientEngineImpl$ClientPacketProcessor.loadRequest(ClientEngineImpl.java:364)
    at com.hazelcast.client.impl.ClientEngineImpl$ClientPacketProcessor.run(ClientEngineImpl.java:340)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
    at com.hazelcast.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76)
    at com.hazelcast.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:92)
    at ------ End remote and begin local stack-trace ------.(Unknown Source)
    at com.hazelcast.client.spi.impl.ClientCallFuture.resolveResponse(ClientCallFuture.java:201)
    ... 7 more
Caused by: java.lang.ClassNotFoundException: com.test.queries.models.MyMapper
    at java.net.URLClassLoader.run(URLClassLoader.java:366)
    at java.net.URLClassLoader.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at com.hazelcast.nio.ClassLoaderUtil.tryLoadClass(ClassLoaderUtil.java:124)
    at com.hazelcast.nio.ClassLoaderUtil.loadClass(ClassLoaderUtil.java:113)
    at com.hazelcast.nio.IOUtil.resolveClass(IOUtil.java:113)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1612)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
    at com.hazelcast.nio.serialization.DefaultSerializers$ObjectSerializer.read(DefaultSerializers.java:196)
    at com.hazelcast.nio.serialization.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
    at com.hazelcast.nio.serialization.SerializationServiceImpl.readObject(SerializationServiceImpl.java:309)
    at com.hazelcast.nio.serialization.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:439)
    at com.hazelcast.mapreduce.impl.client.ClientMapReduceRequest.readData(ClientMapReduceRequest.java:226)
    at com.hazelcast.mapreduce.impl.client.ClientMapReduceRequest.read(ClientMapReduceRequest.java:181)
    at com.hazelcast.client.impl.client.ClientRequest.readPortable(ClientRequest.java:116)
    at com.hazelcast.nio.serialization.PortableSerializer.read(PortableSerializer.java:88)
    at com.hazelcast.nio.serialization.PortableSerializer.read(PortableSerializer.java:30)
    at com.hazelcast.nio.serialization.StreamSerializerAdapter.toObject(StreamSerializerAdapter.java:65)
    at com.hazelcast.nio.serialization.SerializationServiceImpl.toObject(SerializationServiceImpl.java:260)
    at com.hazelcast.client.impl.ClientEngineImpl$ClientPacketProcessor.loadRequest(ClientEngineImpl.java:364)
    at com.hazelcast.client.impl.ClientEngineImpl$ClientPacketProcessor.run(ClientEngineImpl.java:340)
    ... 5 more

Answer 1

您还必须在服务器端部署所有类。

Answer 2

首先，使用 com.test.queries.models.MyMapper class 创建自定义 Jar 文件并将其放置在 Hazelcast lib 文件夹中。然后，在服务器 .bat 文件中添加以下 classpath 条目并重新启动。

set CLASSPATH=%~dp0..\lib\hazelcast-all-3.5.2.jar
set CLASSPATH=%CLASSPATH%;..\lib\Custom-0.0.1-SNAPSHOT.jar

找不到 hazelcast 映射器 class

hazelcast mapper class not found

java

mapreduce

mapper

hazelcast