调用通过 GAR 文件部署的远程任务

Calling remotely tasks deployed via a GAR file

我使用 UriDeploymentSpi bean 从我的一个节点中的目录加载 GAR 文件

我有以下 GAR ignite.xml 文件(顺便说一句,我花了一些时间才弄明白这个文件,但没有记录在案?)

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
    http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.5.xsd">


    <util:list id="myList" value-type="java.lang.String">
        <value>myproject.HelloWorldTask</value>
        <value>myproject.SimpleTask</value>
    </util:list>

</beans>

HelloWorldTask:

package myproject;

public class HelloWorldTask extends ComputeTaskAdapter<String, Integer> {

    static {
        System.out.println("TheGlue: Loading HelloWorldTask ");
    }

    public HelloWorldTask() {
    }

    @Nullable
    @Override
    public Map<? extends ComputeJob, ClusterNode> map(List<ClusterNode> nodes, @Nullable String arg) throws IgniteException {
        System.out.println("Hello from GAR file");
        return null;  //To change body of implemented methods use File | Settings | File Templates.
    }

    @Nullable
    @Override
    public Integer reduce(List<ComputeJobResult> results) throws IgniteException {
        return null;  //To change body of implemented methods use File | Settings | File Templates.
    }
}

简单任务:

package myproject;

@ComputeTaskName("SimpleTaskName")
public class SimpleTask implements ComputeTask<String, Integer> {


    static {
        System.out.println("Loading SimpleTask");
    }

    public SimpleTask() {
    }

    @Override
    public Map<? extends ComputeJob, ClusterNode> map(List<ClusterNode> subgrid, String arg) throws IgniteException {
        System.out.println("Computing Job in SimpleTask ");
        return null;  //To change body of implemented methods use File | Settings | File Templates.
    }

    @Override
    public ComputeJobResultPolicy result(ComputeJobResult res, List<ComputeJobResult> rcvd) throws IgniteException {
        return null;  //To change body of implemented methods use File | Settings | File Templates.
    }

    @Override
    public Integer reduce(List<ComputeJobResult> results) throws IgniteException {
        return null;  //To change body of implemented methods use File | Settings | File Templates.
    }
}

Ignite 可以找到这 2 个 classes(通过 GridUriDeploymentSpringDocumentGridUriDeploymentFileProcessor 调试并找到并加载它们)。 Ignite 说它找到了 GAR,但据我所知,classes 没有实例化。日志文件中没有错误,也没有任务已部署的迹象。

我正在尝试在未部署 GAR 文件的节点(即集群的客户端节点)上执行以下代码,但任务未在集群上执行:

public class _03GarTest {

    public static void main(String[] args) {

        System.out.println("Start urideployment test");

        IgniteConfiguration cfg = new IgniteConfiguration();
        cfg.setPeerClassLoadingEnabled(true); //needs to be the same as in the XML for the server
        cfg.setClientMode(true);

        try(Ignite ignite = Ignition.start(cfg)) {
            ignite.compute(ignite.cluster().forRemotes()).execute("SimpleTaskName", null);
        }

    }
}

我执行 _03GarTest class 的日志文件(如果我 运行 与“SimpleTaskName”或“myproject.SimpleTaskName”相同),转储客户端节点上的以下堆栈跟踪:

Exception in thread "main" class org.apache.ignite.IgniteDeploymentException: Unknown task name or failed to auto-deploy task (was task (re|un)deployed?): SimpleTaskName
    at org.apache.ignite.internal.util.IgniteUtils.apply(IgniteUtils.java:761)
    at org.apache.ignite.internal.util.IgniteUtils.apply(IgniteUtils.java:759)
    at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:877)
    at org.apache.ignite.internal.IgniteComputeImpl.execute(IgniteComputeImpl.java:154)
    at _03GarTest.main(_03GarTest.java:55)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: class org.apache.ignite.internal.IgniteDeploymentCheckedException: Unknown task name or failed to auto-deploy task (was task (re|un)deployed?): SimpleTaskName
    at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:515)
    at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:447)
    at org.apache.ignite.internal.IgniteComputeImpl.execute(IgniteComputeImpl.java:151)
    ... 6 more

并且在服务器上,产生了以下日志:

[13:13:33,057][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da, addrs=[0:0:0:0:0:0:0:1, 10.1.26.59, 127.0.0.1, 192.168.8.103, 192.168.99.1], sockAddrs=[/192.168.8.103:0, /0:0:0:0:0:0:0:1:0, /192.168.99.1:0, /10.1.26.59:0, /10.1.26.59:0, /127.0.0.1:0, /192.168.8.103:0, /192.168.99.1:0], discPort=0, order=12, intOrder=7, lastExchangeTime=1452600812926, loc=false, ver=1.5.0#20151229-sha1:f1f8cda2, isClient=true]
    [13:13:33,063][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Topology snapshot [ver=12, servers=1, clients=1, CPUs=8, heap=1.5GB]
    [13:13:33,085][WARNING][disco-event-worker-#48%null%][CourtesyConfigNotice] 

    >>> +-------------------------------------------------------------------+
    >>> + Courtesy notice that joining node has inconsistent configuration. +
    >>> + Ignore this message if you are sure that this is done on purpose. +
    >>> +-------------------------------------------------------------------+
    >>> Remote Node ID: B70DCE5E-C0FD-4FFE-8DC2-B72B18DB76DA
    >>> Remote SPI with the same name is not configured: UriDeploymentSpi
    >>> => Local node:  o.a.i.spi.deployment.uri.UriDeploymentSpi

    [13:13:33,103][INFO][exchange-worker-#51%null%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=12, minorTopVer=0], evt=NODE_JOINED, node=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da]
    [13:13:33,907][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Node left topology: TcpDiscoveryNode [id=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da, addrs=[0:0:0:0:0:0:0:1, 10.1.26.59, 127.0.0.1, 192.168.8.103, 192.168.99.1], sockAddrs=[/192.168.8.103:0, /0:0:0:0:0:0:0:1:0, /192.168.99.1:0, /10.1.26.59:0, /10.1.26.59:0, /127.0.0.1:0, /192.168.8.103:0, /192.168.99.1:0], discPort=0, order=12, intOrder=7, lastExchangeTime=1452600812926, loc=false, ver=1.5.0#20151229-sha1:f1f8cda2, isClient=true]
    [13:13:33,908][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Topology snapshot [ver=13, servers=1, clients=0, CPUs=8, heap=1.0GB]
    [13:13:33,918][INFO][exchange-worker-#51%null%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=13, minorTopVer=0], evt=NODE_LEFT, node=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da]
    [13:14:03,193][INFO][grid-timeout-worker-#33%null%][IgniteKernal] 

关于如何调用通过另一个节点上的 GAR 文件部署的任务有什么想法吗?

----更新----

根据其中一个答案的建议,我在客户端添加了以下代码

        System.out.println("Start urideployment test");

        IgniteConfiguration cfg = new IgniteConfiguration();
        cfg.setPeerClassLoadingEnabled(true); //needs to be the same as in the XML for the server
        cfg.setClientMode(true);

        UriDeploymentSpi deploymentSpi = new UriDeploymentSpi();

        deploymentSpi.setUriList(Arrays.asList("file:///Users/sbeaupre/Dropbox/prorabel/Projects/IgniteTests/ignite/gar"));

        cfg.setDeploymentSpi(deploymentSpi);

        try(Ignite ignite = Ignition.start(cfg)) {
...

但这也不起作用,我在客户端节点上得到了以下堆栈跟踪,而在服务器节点上什么也没有:

Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger info
INFO: Topology snapshot [ver=4, servers=1, clients=1, CPUs=8, heap=1.5GB]
Jan 14, 2016 5:42:23 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from resource loaded from byte array
Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger info
INFO: User version is not explicitly defined (will use default version) [file=META-INF/ignite.xml, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]]]
Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger info
INFO: Task locally deployed: class myproject.SimpleTask
Loading SimpleTask
Computing Job in SimpleTask 
Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Failed to map task jobs to nodes: GridTaskSessionImpl [taskName=SimpleTaskName, dep=GridDeployment [ts=1452789743727, depMode=SHARED, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], clsLdrId=cc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, userVer=0, loc=true, sampleClsName=myproject.SimpleTask, pendingUndeploy=false, undeployed=false, usage=1], taskClsName=myproject.SimpleTask, sesId=bc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, startTime=1452789743638, endTime=9223372036854775807, taskNodeId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, subjId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, mapFut=IgniteFuture [orig=GridFutureAdapter [resFlag=0, res=null, startTime=1452789743739, endTime=0, ignoreInterrupts=false, lsnr=null, state=INIT]]]
class org.apache.ignite.IgniteCheckedException: Task map operation produced no mapped jobs: GridTaskSessionImpl [taskName=SimpleTaskName, dep=GridDeployment [ts=1452789743727, depMode=SHARED, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], clsLdrId=cc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, userVer=0, loc=true, sampleClsName=myproject.SimpleTask, pendingUndeploy=false, undeployed=false, usage=1], taskClsName=myproject.SimpleTask, sesId=bc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, startTime=1452789743638, endTime=9223372036854775807, taskNodeId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, subjId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, mapFut=IgniteFuture [orig=GridFutureAdapter [resFlag=0, res=null, startTime=1452789743739, endTime=0, ignoreInterrupts=false, lsnr=null, state=INIT]]]
    at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:497)
    at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
    at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:678)
    at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:447)
    at org.apache.ignite.internal.IgniteComputeImpl.execute(IgniteComputeImpl.java:151)
    at _03GarTest.main(_03GarTest.java:55)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

斯文,

您还应该在客户端节点上配置 URI 部署 SPI,以使 GAR 部署正常工作。

当您调用 compute.execute("taskName"); 时,在将第一个请求发送到拓扑中的任何节点之前以及在结果开始返回之后,必须在客户端本地完成很多事情。至少,Ignite 应该能够获取映射作业并能够处理所有远程作业的结果并减少所有结果 - 请参阅 ComputeTask.map()ComputeTask.result()ComputeTask.reduce()。因此,您应该能够在客户端节点上实例化任务,这就是为什么您应该有任务 类 可用。

我认为在客户端节点上配置 URI 部署后,您的代码应该可以正常工作。

如果您需要任何其他信息,请post在此处发表评论。

谢谢!

2016 年 1 月 18 日更新

这是针对问题更新的更新。

请注意,有问题的任务 returns 来自 map() 方法的 null 是非法的。您可以在二进制版本中参考 org.apache.ignite.examples.computegrid.ComputeTaskMapExample 或直接通过 https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob;f=examples/src/main/java/org/apache/ignite/examples/computegrid/ComputeTaskMapExample.java;h=3de5293a814e527b57e3984f6d3ab96bb1b62daf;hb=HEAD