如何使用 Openshift Wildfly 8 Cartridge 打开 JChannel (JGroups)

How to open a JChannel (JGroups) using Openshift Wildfly 8 Cartridge

我们正在 Wildfly-Openshift-Cartridge 中的 Wildfly 8 上开发 Java-EE 应用程序 运行。 Openshift 应用程序设置为缩放,因此,多个 JVM 节点将 运行 作为一个集群。

不同节点之间的通信,我喜欢用JGroups。但是,即使 JGroups 本身似乎可以工作,我也无法打开新的 JChannel。如果我将应用程序本地部署到例如3 个 Wildfly 独立实例。

wildfly-cartrige 的默认 standalone.xml 已包含 JGroups 的配置:

....
<subsystem xmlns="urn:jboss:domain:jgroups:2.0" default-stack="tcp">
    <stack name="tcp">
        <transport type="TCP" socket-binding="jgroups-tcp">
            <property name="external_addr">${env.OPENSHIFT_GEAR_DNS}</property>
            <property name="external_port">${env.OPENSHIFT_WILDFLY_CLUSTER_PROXY_PORT}</property>
            <property name="bind_port">${env.OPENSHIFT_WILDFLY_CLUSTER_PORT}</property>
            <property name="bind_addr">${env.OPENSHIFT_WILDFLY_IP}</property>
            <property name="defer_client_bind_addr">true</property>
        </transport>
        <protocol type="TCPPING">
            <property name="timeout">30000</property>
            <property name="initial_hosts">${env.OPENSHIFT_WILDFLY_CLUSTER}</property>
            <property name="port_range">0</property>
            <property name="num_initial_members">1</property>
        </protocol>
        <protocol type="MERGE2"/>
        <protocol type="FD"/>
        <protocol type="VERIFY_SUSPECT"/>
        <protocol type="BARRIER"/>
        <protocol type="pbcast.NAKACK"/>
        <protocol type="UNICAST2"/>
        <protocol type="pbcast.STABLE"/>
        <protocol type="AUTH">
            <property name="auth_class">org.jgroups.auth.MD5Token</property>
            <property name="token_hash">SHA</property>
            <property name="auth_value">${env.OPENSHIFT_SECRET_TOKEN}</property>
        </protocol>
        <protocol type="pbcast.GMS"/>
        <protocol type="UFC"/>
        <protocol type="MFC"/>
        <protocol type="FRAG2"/>
        <!--protocol type="pbcast.STATE_TRANSFER"/>
        <protocol type="pbcast.FLUSH"/-->
    </stack>
</subsystem>
....

据此,默认堆栈应为 tcp - 这是有道理的,因为 OpenShift 环境中不允许 UDP 多播。

如果我将应用程序部署到 OpenShift(缩放,有两个 运行 齿轮),JGroups 本身似乎在 JVM 之间进行通信:来自 wildfly/log/server.log 的输出:

....
INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 59) ISPN000078: Starting JGroups Channel
INFO  [stdout] (ServerService Thread Pool -- 59)
INFO  [stdout] (ServerService Thread Pool -- 59) -------------------------------------------------------------------
INFO  [stdout] (ServerService Thread Pool -- 59) GMS: address=xxx.rhcloud.com/ejb, cluster=ejb, physical address=172.30.0.163:62417
....
INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-5,shared=tcp) ISPN000094: Received new cluster view: [xxx.rhcloud.com/ejb|4] (1) xxx.rhcloud.com/ejb]
....
INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-6,shared=tcp) ISPN000094: Received new cluster view: [yyyy.rhcloud.com/ejb|5] (2) [xxx.rhcloud.com/ejb, 55b89cd750044609f0000043-xx.rhcloud.com/ejb]
...

开箱即用,无需任何自己的编程。

但是现在,我想打开一个 JChannel 将消息从一个 JVM 发送到所有其他 JVM。

...
import org.jgroups.*;
...
public class MyNotifier extends ReceiverAdapter
{
    ...

    JChannel channel;

    public void start()
    {
        try
        {
            channel = new JChannel();
            channel.setReceiver(this);
            channel.connect("MyNotifier");
        }
        catch (ChannelException e)
        {
            log.error("JGroups: Error starting channel.", e);
        }
    }

    public void receive(Message msg)
    {
        // process message
    }

    public void send(DataObject data)
    {
        Message msg = new Message(null, null, data);

        try
        {
            channel.send(msg);
        }
        catch (Exception e)
        {
            log.error("JGroups: Error sending message.", e);
        }
    }

    public void viewAccepted(View new_view)
    {
        log.info("JGroups: View accepted: " + new_view );
    }
}

channel = new JChannel()期间出现异常:

INFO  [stdout] (ServerService Thread Pool -- 82) 72190 [ERROR] MyNotifier: JGroups: Error starting channel.
INFO  [stdout] (ServerService Thread Pool -- 82) org.jgroups.ChannelException: failed to start protocol stack
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.JChannel.startStack(JChannel.java:1787) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.JChannel.connect(JChannel.java:413) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.JChannel.connect(JChannel.java:388) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    at xxxxx.MyNotifier.start(MyNotifier.java:41) [classes:?]
INFO  [stdout] (ServerService Thread Pool -- 82)    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_45]
INFO  [stdout] (ServerService Thread Pool -- 82)    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_45]
INFO  [stdout] (ServerService Thread Pool -- 82)    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_45]
INFO  [stdout] (ServerService Thread Pool -- 82)    at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_45]
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [wildfly-ee-8.2.0.Final.jar!/:8.2.0.Final]
...
INFO  [stdout] (ServerService Thread Pool -- 82) Caused by: java.lang.Exception: problem creating sockets (bind_addr=localhost/127.0.0.1, mcast_addr=null)
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.protocols.UDP.start(UDP.java:238) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.stack.ProtocolStack.startStack(ProtocolStack.java:990) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.JChannel.startStack(JChannel.java:1784) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    ... 136 more
INFO  [stdout] (ServerService Thread Pool -- 82) Caused by: java.lang.IllegalArgumentException: port out of range:65536
INFO  [stdout] (ServerService Thread Pool -- 82)    at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143) ~[?:1.8.0_45]
INFO  [stdout] (ServerService Thread Pool -- 82)    at java.net.InetSocketAddress.<init>(InetSocketAddress.java:188) ~[?:1.8.0_45]
INFO  [stdout] (ServerService Thread Pool -- 82)    at java.net.DatagramSocket.<init>(DatagramSocket.java:299) ~[?:1.8.0_45]
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.util.DefaultSocketFactory.createDatagramSocket(DefaultSocketFactory.java:65) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.protocols.UDP.createEphemeralDatagramSocket(UDP.java:458) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.protocols.UDP.createSockets(UDP.java:342) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.protocols.UDP.start(UDP.java:234) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.stack.ProtocolStack.startStack(ProtocolStack.java:990) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    at org.jgroups.JChannel.startStack(JChannel.java:1784) ~[jgroups-2.12.1.Final.jar:2.12.1.Final]
INFO  [stdout] (ServerService Thread Pool -- 82)    ... 136 more

JChannel 似乎甚至尝试使用 UDP。但是我怎样才能让它使用standalone.xml中设置为子系统的东西?

更多信息: 正如我在顶部所写,如果我使用 UDP 多播将应用程序部署到某些本地 运行 wildfly 实例,则成功创建 JChannel。然后我尝试在本地应用与 OpenShift 相同的堆栈配置,结果它仍然有效,但即使是通过 UDP。所以我的 channel = new JChannel(); 似乎没有使用子系统配置中的设置。但是为什么?

编辑:

我了解子系统配置,即子系统中设置的默认堆栈与 new JChannel() 将使用的内容无关;因此,我现在尝试创建自己的 tcp.xml。 然而,第一个问题是如何在此配置中使用环境变量(扩展到它们的值),例如我需要像 <TCP bind_addr=${env.OPENSHIFT_WILDFLY_IP}/> 这样的东西。我怎样才能做到这一点?

对于第一步,我尝试插入修复正确的值:

<config xmlns="urn:org:jgroups"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/JGroups-3.3.xsd">
    <TCP
            external_addr="backend-evendingtest.rhcloud.com"
            external_port="62417"
            bind_port="7600"
            bind_addr="127.11.122.129"
            defer_client_bind_addr="true"/>

    <TCPPING timeout="3000"
             initial_hosts="backend-evendingtest.rhcloud.com[62417],55b89cd750044609f0000043-evendingtest.rhcloud.com[56432]"
             port_range="0"
             num_initial_members="1"/>
    <MERGE2/>
    <FD/>
    <VERIFY_SUSPECT/>
    <BARRIER/>
    <pbcast.NAKACK2
            use_mcast_xmit="false"/>
    <UNICAST3/>
    <pbcast.STABLE/>
    <AUTH auth_class="org.jgroups.auth.MD5Token"
          token_hash="SHA"
          auth_value="2QVXd4rUYmxOtZNlxsJ7fBbWb4J8eFJup39PSC1IZck1JHDr_y4qUzFZVqUzLVZ7rDHnZWPtOpyY4pTrPAZ__D06GHHz5Uq9D5ZPMtyHPkZuvRh3cWchUX3JxA5Qsq4B"/>
    <pbcast.GMS/>
    <MFC/>
    <FRAG2/>

...但我失败并出现异常:

java.net.BindException: Permission denied
      at java.net.PlainDatagramSocketImpl.bind0(Native Method)
      at java.net.AbstractPlainDatagramSocketImpl.bind(AbstractPlainDatagramSocketImpl.java:94)
      at java.net.DatagramSocket.bind(DatagramSocket.java:392)
      at java.net.MulticastSocket.<init>(MulticastSocket.java:172)
      at java.net.MulticastSocket.<init>(MulticastSocket.java:136)
      at org.jgroups.util.DefaultSocketFactory.createMulticastSocket(DefaultSocketFactory.java:70)
      at org.jgroups.stack.DiagnosticsHandler.start(DiagnosticsHandler.java:83)
      at org.jgroups.protocols.TP.start(TP.java:1222)
      at org.jgroups.protocols.TCP.start(TCP.java:82)
      at org.jgroups.stack.ProtocolStack.startStack(ProtocolStack.java:952)
      at org.jgroups.JChannel.startStack(JChannel.java:864)

为什么这条轨迹中有类似 'createMulticastSocket' 的东西? - 我不想使用 UDP / 多播...

在此先感谢您的帮助! - 巴德拉

new JChannel() 使用默认配置,即 udp.xml(在 JGroups JAR 中)。您收到的错误消息是因为 JGroups 无法绑定到 127.0.0.1:0(临时端口)。

我建议重新使用Wildfly已经提供的频道,详情见[1]。 IIRC,Wildfly 9 实际上公开了 JGroups 频道,但这可能还没有在 8...

[1] https://developer.jboss.org/wiki/ClusteringChangesInWildFly8