Wildfly 10.1 中的会话复制

Session Replication in Wildfly 10.1

我正在尝试在我的 Wildfly 10.1 应用程序中使用可分发的 WAR 启用会话复制。

我正在 运行 在托管主机提供商上使用 2 个 RedHat 7.2 实例,可以完全访问 OS 和防火墙。我无权访问为我们的流量提供服务的路由器,但主机已确认已启用多播 UDP。

我将 SeLinux 设置为最小值,端口在 iptables 中打开,多播 IP 已被订阅,我的 wildfly 域模式配置使用带有 full-ha-sockets 的克隆 full-ha 配置文件:

这是域配置文件,除数据源外是普通配置文件:

<profile name="ha-dev2">
    <subsystem xmlns="urn:jboss:domain:logging:3.0">
        <add-logging-api-dependencies value="false"/>
        <console-handler name="CONSOLE">
            <level name="INFO"/>
            <formatter>
                <named-formatter name="COLOR-PATTERN"/>
            </formatter>
        </console-handler>
        <periodic-rotating-file-handler name="FILE" autoflush="true">
            <formatter>
                <named-formatter name="PATTERN"/>
            </formatter>
            <file relative-to="jboss.server.log.dir" path="server.log"/>
            <suffix value=".yyyy-MM-dd"/>
            <append value="true"/>
        </periodic-rotating-file-handler>
        <logger category="com.arjuna">
            <level name="WARN"/>
        </logger>
        <logger category="org.jboss.as.config">
            <level name="DEBUG"/>
        </logger>
        <logger category="sun.rmi">
            <level name="WARN"/>
        </logger>
        <root-logger>
            <level name="INFO"/>
            <handlers>
                <handler name="CONSOLE"/>
                <handler name="FILE"/>
            </handlers>
        </root-logger>
        <formatter name="PATTERN">
            <pattern-formatter pattern="%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n"/>
        </formatter>
        <formatter name="COLOR-PATTERN">
            <pattern-formatter pattern="%K{level}%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n"/>
        </formatter>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:batch-jberet:1.0">
        <default-job-repository name="in-memory"/>
        <default-thread-pool name="batch"/>
        <job-repository name="in-memory">
            <in-memory/>
        </job-repository>
        <thread-pool name="batch">
            <max-threads count="10"/>
            <keepalive-time time="30" unit="seconds"/>
        </thread-pool>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:bean-validation:1.0"/>
    <subsystem xmlns="urn:jboss:domain:datasources:4.0">
        <datasources>
            <datasource jndi-name="java:jboss/datasources/ExampleDS" pool-name="ExampleDS" enabled="true" use-java-context="true">
                <connection-url>jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE</connection-url>
                <driver>h2</driver>
                <security>
                    <user-name>sa</user-name>
                    <password>sa</password>
                </security>
            </datasource>
            <!-- DATASOURCES REDACTED -->
            <drivers>
                <driver name="h2" module="com.h2database.h2">
                    <xa-datasource-class>org.h2.jdbcx.JdbcDataSource</xa-datasource-class>
                </driver>
            </drivers>
        </datasources>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:ee:4.0">
        <spec-descriptor-property-replacement>false</spec-descriptor-property-replacement>
        <concurrent>
            <context-services>
                <context-service name="default" jndi-name="java:jboss/ee/concurrency/context/default" use-transaction-setup-provider="true"/>
            </context-services>
            <managed-thread-factories>
                <managed-thread-factory name="default" jndi-name="java:jboss/ee/concurrency/factory/default" context-service="default"/>
            </managed-thread-factories>
            <managed-executor-services>
                <managed-executor-service name="default" jndi-name="java:jboss/ee/concurrency/executor/default" context-service="default" hung-task-threshold="60000" keepalive-time="5000"/>
            </managed-executor-services>
            <managed-scheduled-executor-services>
                <managed-scheduled-executor-service name="default" jndi-name="java:jboss/ee/concurrency/scheduler/default" context-service="default" hung-task-threshold="60000" keepalive-time="3000"/>
            </managed-scheduled-executor-services>
        </concurrent>
        <default-bindings context-service="java:jboss/ee/concurrency/context/default" datasource="java:jboss/datasources/ExampleDS" jms-connection-factory="java:jboss/DefaultJMSConnectionFactory" managed-executor-service="java:jboss/ee/concurrency/executor/default" managed-scheduled-executor-service="java:jboss/ee/concurrency/scheduler/default" managed-thread-factory="java:jboss/ee/concurrency/factory/default"/>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:ejb3:4.0">
        <session-bean>
            <stateless>
                <bean-instance-pool-ref pool-name="slsb-strict-max-pool"/>
            </stateless>
            <stateful default-access-timeout="5000" cache-ref="distributable" passivation-disabled-cache-ref="simple"/>
            <singleton default-access-timeout="5000"/>
        </session-bean>
        <mdb>
            <resource-adapter-ref resource-adapter-name="${ejb.resource-adapter-name:activemq-ra.rar}"/>
            <bean-instance-pool-ref pool-name="mdb-strict-max-pool"/>
        </mdb>
        <pools>
            <bean-instance-pools>
                <strict-max-pool name="slsb-strict-max-pool" derive-size="from-worker-pools" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/>
                <strict-max-pool name="mdb-strict-max-pool" derive-size="from-cpu-count" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/>
            </bean-instance-pools>
        </pools>
        <caches>
            <cache name="simple"/>
            <cache name="distributable" passivation-store-ref="infinispan" aliases="passivating clustered"/>
        </caches>
        <passivation-stores>
            <passivation-store name="infinispan" cache-container="ejb" max-size="10000"/>
        </passivation-stores>
        <async thread-pool-name="default"/>
        <timer-service thread-pool-name="default" default-data-store="default-file-store">
            <data-stores>
                <file-data-store name="default-file-store" path="timer-service-data" relative-to="jboss.server.data.dir"/>
            </data-stores>
        </timer-service>
        <remote connector-ref="http-remoting-connector" thread-pool-name="default"/>
        <thread-pools>
            <thread-pool name="default">
                <max-threads count="10"/>
                <keepalive-time time="100" unit="milliseconds"/>
            </thread-pool>
        </thread-pools>
        <iiop enable-by-default="false" use-qualified-name="false"/>
        <default-security-domain value="other"/>
        <default-missing-method-permissions-deny-access value="true"/>
        <log-system-exceptions value="true"/>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:io:1.1">
        <worker name="default"/>
        <buffer-pool name="default"/>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:infinispan:4.0">
        <cache-container name="server" aliases="singleton cluster" default-cache="default" module="org.wildfly.clustering.server">
            <transport lock-timeout="60000"/>
            <replicated-cache name="default" mode="SYNC">
                <transaction mode="BATCH"/>
            </replicated-cache>
        </cache-container>
        <cache-container name="web" default-cache="dist" module="org.wildfly.clustering.web.infinispan">
            <transport lock-timeout="60000"/>
            <distributed-cache name="dist" mode="ASYNC" l1-lifespan="0" owners="2">
                <locking isolation="REPEATABLE_READ"/>
                <transaction mode="BATCH"/>
                <file-store/>
            </distributed-cache>
            <distributed-cache name="concurrent" mode="SYNC" l1-lifespan="0" owners="2">
                <file-store/>
            </distributed-cache>
        </cache-container>
        <cache-container name="ejb" aliases="sfsb" default-cache="dist" module="org.wildfly.clustering.ejb.infinispan">
            <transport lock-timeout="60000"/>
            <distributed-cache name="dist" mode="ASYNC" l1-lifespan="0" owners="2">
                <locking isolation="REPEATABLE_READ"/>
                <transaction mode="BATCH"/>
                <file-store/>
            </distributed-cache>
        </cache-container>
        <cache-container name="hibernate" default-cache="local-query" module="org.hibernate.infinispan">
            <transport lock-timeout="60000"/>
            <local-cache name="local-query">
                <eviction strategy="LRU" max-entries="10000"/>
                <expiration max-idle="100000"/>
            </local-cache>
            <invalidation-cache name="entity" mode="SYNC">
                <transaction mode="NON_XA"/>
                <eviction strategy="LRU" max-entries="10000"/>
                <expiration max-idle="100000"/>
            </invalidation-cache>
            <replicated-cache name="timestamps" mode="ASYNC"/>
        </cache-container>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:iiop-openjdk:1.0">
        <orb socket-binding="iiop" ssl-socket-binding="iiop-ssl"/>
        <initializers security="identity" transactions="spec"/>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:jaxrs:1.0"/>
    <subsystem xmlns="urn:jboss:domain:jca:4.0">
        <archive-validation enabled="true" fail-on-error="true" fail-on-warn="false"/>
        <bean-validation enabled="true"/>
        <default-workmanager>
            <short-running-threads>
                <core-threads count="50"/>
                <queue-length count="50"/>
                <max-threads count="50"/>
                <keepalive-time time="10" unit="seconds"/>
            </short-running-threads>
            <long-running-threads>
                <core-threads count="50"/>
                <queue-length count="50"/>
                <max-threads count="50"/>
                <keepalive-time time="10" unit="seconds"/>
            </long-running-threads>
        </default-workmanager>
        <cached-connection-manager/>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:jdr:1.0"/>
    <subsystem xmlns="urn:jboss:domain:jgroups:4.0">
        <channels default="ee">
            <channel name="ee" stack="udp"/>
        </channels>
        <stacks>
            <stack name="udp">
                <transport type="UDP" socket-binding="jgroups-udp"/>
                <protocol type="PING"/>
                <protocol type="MERGE3"/>
                <protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
                <protocol type="FD_ALL"/>
                <protocol type="VERIFY_SUSPECT"/>
                <protocol type="pbcast.NAKACK2"/>
                <protocol type="UNICAST3"/>
                <protocol type="pbcast.STABLE"/>
                <protocol type="pbcast.GMS"/>
                <protocol type="UFC"/>
                <protocol type="MFC"/>
                <protocol type="FRAG2"/>
            </stack>
            <stack name="tcp">
                <transport type="TCP" socket-binding="jgroups-tcp"/>
                <protocol type="MPING" socket-binding="jgroups-mping"/>
                <protocol type="MERGE3"/>
                <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
                <protocol type="FD"/>
                <protocol type="VERIFY_SUSPECT"/>
                <protocol type="pbcast.NAKACK2"/>
                <protocol type="UNICAST3"/>
                <protocol type="pbcast.STABLE"/>
                <protocol type="pbcast.GMS"/>
                <protocol type="MFC"/>
                <protocol type="FRAG2"/>
            </stack>
        </stacks>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:jmx:1.3">
        <expose-resolved-model/>
        <expose-expression-model/>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:jpa:1.1">
        <jpa default-datasource="" default-extended-persistence-inheritance="DEEP"/>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:jsf:1.0"/>
    <subsystem xmlns="urn:jboss:domain:jsr77:1.0"/>
    <subsystem xmlns="urn:jboss:domain:mail:2.0">
        <mail-session name="default" jndi-name="java:jboss/mail/Default">
            <smtp-server outbound-socket-binding-ref="mail-smtp"/>
        </mail-session>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:messaging-activemq:1.0">
        <server name="default">
            <cluster password="${jboss.messaging.cluster.password:@password@}"/>
            <bindings-directory/>
            <journal-directory/>
            <large-messages-directory/>
            <paging-directory/>
            <security-setting name="#">
                <role name="guest" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/>
            </security-setting>
            <address-setting name="#" dead-letter-address="jms.queue.DLQ" expiry-address="jms.queue.ExpiryQueue" max-size-bytes="10485760" page-size-bytes="2097152" message-counter-history-day-limit="10" redistribution-delay="1000"/>
            <http-connector name="http-connector" socket-binding="http" endpoint="http-acceptor"/>
            <http-connector name="http-connector-throughput" socket-binding="http" endpoint="http-acceptor-throughput">
                <param name="batch-delay" value="50"/>
            </http-connector>
            <in-vm-connector name="in-vm" server-id="0"/>
            <http-acceptor name="http-acceptor" http-listener="default"/>
            <http-acceptor name="http-acceptor-throughput" http-listener="default">
                <param name="batch-delay" value="50"/>
                <param name="direct-deliver" value="false"/>
            </http-acceptor>
            <in-vm-acceptor name="in-vm" server-id="0"/>
            <broadcast-group name="bg-group1" jgroups-channel="activemq-cluster" connectors="http-connector"/>
            <discovery-group name="dg-group1" jgroups-channel="activemq-cluster"/>
            <cluster-connection name="my-cluster" address="jms" connector-name="http-connector" discovery-group="dg-group1"/>
            <jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/>
            <jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/>
            <connection-factory name="InVmConnectionFactory" entries="java:/ConnectionFactory" connectors="in-vm"/>
            <connection-factory name="RemoteConnectionFactory" entries="java:jboss/exported/jms/RemoteConnectionFactory" connectors="http-connector" ha="true" block-on-acknowledge="true" reconnect-attempts="-1"/>
            <pooled-connection-factory name="activemq-ra" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory" connectors="in-vm" transaction="xa"/>
        </server>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:modcluster:2.0">
        <mod-cluster-config advertise-socket="modcluster" balancer="dev-ha-server-group" connector="ajp">
            <dynamic-load-provider>
                <load-metric type="busyness"/>
            </dynamic-load-provider>
        </mod-cluster-config>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:naming:2.0">
        <remote-naming/>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:pojo:1.0"/>
    <subsystem xmlns="urn:jboss:domain:remoting:3.0">
        <http-connector name="http-remoting-connector" connector-ref="default" security-realm="ApplicationRealm"/>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:resource-adapters:4.0"/>
    <subsystem xmlns="urn:jboss:domain:request-controller:1.0"/>
    <subsystem xmlns="urn:jboss:domain:sar:1.0"/>
    <subsystem xmlns="urn:jboss:domain:security:1.2">
        <security-domains>
            <security-domain name="other" cache-type="default">
                <authentication>
                    <login-module code="Remoting" flag="optional">
                        <module-option name="password-stacking" value="useFirstPass"/>
                    </login-module>
                    <login-module code="RealmDirect" flag="required">
                        <module-option name="password-stacking" value="useFirstPass"/>
                    </login-module>
                </authentication>
            </security-domain>
            <security-domain name="jboss-web-policy" cache-type="default">
                <authorization>
                    <policy-module code="Delegating" flag="required"/>
                </authorization>
            </security-domain>
            <security-domain name="jboss-ejb-policy" cache-type="default">
                <authorization>
                    <policy-module code="Delegating" flag="required"/>
                </authorization>
            </security-domain>
            <security-domain name="jaspitest" cache-type="default">
                <authentication-jaspi>
                    <login-module-stack name="dummy">
                        <login-module code="Dummy" flag="optional"/>
                    </login-module-stack>
                    <auth-module code="Dummy"/>
                </authentication-jaspi>
            </security-domain>
        </security-domains>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:security-manager:1.0">
        <deployment-permissions>

            <maximum-set>
                <permission class="java.security.AllPermission"/>
            </maximum-set>
        </deployment-permissions>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:singleton:1.0">
        <singleton-policies default="default">
            <singleton-policy name="default" cache-container="server">
                <simple-election-policy/>
            </singleton-policy>
        </singleton-policies>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:transactions:3.0">
        <core-environment>
            <process-id>
                <uuid/>
            </process-id>
        </core-environment>
        <recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager"/>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:undertow:3.1">
        <buffer-cache name="default"/>
        <server name="default-server">
            <ajp-listener name="ajp" socket-binding="ajp" max-post-size="26214400"/>
            <http-listener name="default" socket-binding="http" max-post-size="26214400" redirect-socket="https" enable-http2="true"/>
            <https-listener name="https" socket-binding="https" max-post-size="26214400" security-realm="ApplicationRealm" enable-http2="true"/>
            <host name="default-host" alias="localhost">
                <location name="/" handler="welcome-content"/>
                <filter-ref name="server-header"/>
                <filter-ref name="x-powered-by-header"/>
                <filter-ref name="gzipFilter" predicate="exists['%{o,Content-Type}'] and regex[pattern='(?:application/javascript|text/css|text/html|text/xml|application/json)(;.*)?', value=%{o,Content-Type}, full-match=true]"/>
            </host>
        </server>
        <servlet-container name="default">
            <jsp-config trim-spaces="true"/>
            <websockets/>
        </servlet-container>
        <handlers>
            <file name="welcome-content" path="${jboss.home.dir}/welcome-content"/>
        </handlers>
        <filters>
            <response-header name="server-header" header-name="Server" header-value="WildFly/10"/>
            <response-header name="x-powered-by-header" header-name="X-Powered-By" header-value="Undertow/1"/>
            <gzip name="gzipFilter"/>
        </filters>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:webservices:2.0">
        <wsdl-host>${jboss.bind.address:127.0.0.1}</wsdl-host>
        <endpoint-config name="Standard-Endpoint-Config"/>
        <endpoint-config name="Recording-Endpoint-Config">
            <pre-handler-chain name="recording-handlers" protocol-bindings="##SOAP11_HTTP ##SOAP11_HTTP_MTOM ##SOAP12_HTTP ##SOAP12_HTTP_MTOM">
                <handler name="RecordingHandler" class="org.jboss.ws.common.invocation.RecordingServerHandler"/>
            </pre-handler-chain>
        </endpoint-config>
        <client-config name="Standard-Client-Config"/>
    </subsystem>
    <subsystem xmlns="urn:jboss:domain:weld:3.0"/>
</profile>

这是套接字绑定组:

    <socket-binding-group name="dev-full-ha-sockets" default-interface="public">
        <socket-binding name="ajp" port="${jboss.ajp.port:8009}"/>
        <socket-binding name="http" port="${jboss.http.port:8080}"/>
        <socket-binding name="https" port="${jboss.https.port:18443}"/>
        <socket-binding name="iiop" interface="unsecure" port="3528"/>
        <socket-binding name="iiop-ssl" interface="unsecure" port="3529"/>
        <socket-binding name="jgroups-mping" port="0" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/>
        <socket-binding name="jgroups-tcp" port="7600"/>
        <socket-binding name="jgroups-tcp-fd" port="57600"/>
        <socket-binding name="jgroups-udp" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/>
        <socket-binding name="jgroups-udp-fd" port="54200"/>
        <socket-binding name="modcluster" port="0" multicast-address="224.0.1.105" multicast-port="23364"/>
        <socket-binding name="txn-recovery-environment" port="4712"/>
        <socket-binding name="txn-status-manager" port="4713"/>
        <outbound-socket-binding name="mail-smtp">
            <remote-destination host="localhost" port="25"/>
        </outbound-socket-binding>
    </socket-binding-group>

最后,服务器组:

<server-group name="dev-ha-server-group" profile="ha-dev2">
            <jvm name="default">
                <heap size="256m" max-size="1024m"/>
            </jvm>
            <socket-binding-group ref="dev-full-ha-sockets"/>
            <deployments>
             <!-- Deployments redacted -->
            </deployments>
        </server-group>

使用 TCPDump,我可以从我的两台服务器上看到 modcluster 的流量。

我还有 运行 McastReceiverTest(来源:https://github.com/belaban/JGroups/blob/master/tests/other/org/jgroups/tests/McastReceiverTest.java)通过:

java org.jgroups.tests.McastReceiverTest -mcast_addr 230.0.0.4 -port 45700

并使用了以下命令:

printf "GET / HTTP/1.0\r\n\r\n" | nc -vu 230.0.0.4 45700

并查看通过的 UDP 流量。

虽然我没有看到来自 wildfly 的 jgroups 的任何东西。在我的日志中,当我查找订阅的频道时,我看到

"Received new cluster view for channel server: [app-one:server-one|0] (1) [app-one:server-one]"

但我从未在任何一个实例上看到我的第二台服务器。当我测试时,我在两个 RedHat 实例前面都有一个 F5 负载平衡器。

我有一些假设希望澄清:

  1. 我需要为 Modcluster 启用 Apache 吗?如果您没有单独的平衡机制,我会假设它不会用于负载平衡。

  2. 会话复制是否与 运行 宁 "hot-hot" 方案相同,以便 2 个应用程序可以 运行 并保持会话活动?我的理解是,我应该能够登录到服务器 A 上的应用程序,而服务器 B 也能够保持该会话有效。我们正在努力避免对会话粘性的需求。

  3. 这 运行 可以在多服务器环境中使用还是仅适用于同一台本地计算机上的 2 个实例?我认为无论是在本地计算机上还是在同一防火墙后面的 2 个单独的 physicals/vms 上都没有关系。

谢谢。


我找到了一直困扰我的解决方案。我的配置问题出在我的绑定 IP 中。我为我的 jboss.bind.address 使用 127.0.0.1,因为我的 Wildfly 配置前面有 Apache。我将绑定 IP 更改为 NIC 的 IP(在我的例子中是 10.* 地址)。

如果您 post 编辑了整个 standalone.xml 文件,将会很有帮助。

回答您的问题:

  1. 不,您可以使用任何负载平衡器。 Mod 集群非常好,因为 Wildfly 直接与其对话,因此在关闭时可以更快地删除服务器。

  2. 是的。通过复制,所有会话数据都存储在 Infinispan 中,然后将其复制到集群中的其他服务器。

  3. 集群的主要用例是在不同的服务器上拥有实例。

因为我没有您的完整配置并且对您的托管一无所知这是我对问题所在的最佳猜测以便我检查:

  • 您在 AWS 中 运行 - UDP 多播在 AWS 中不起作用。您必须使用其他 ping 方法。
  • 您的服务器设置为不接受 UDP 多播广播。检查所有防火墙和网络规则。
  • 您的配置存在缺陷。如果你post整个配置我可以看一下。
  • 您的 web.xml 文件缺少 标签

我的配置问题出在我的绑定 IP 上。我为我的 jboss.bind.address 使用 127.0.0.1,因为我的 Wildfly 配置前面有 Apache。我将绑定 IP 更改为 NIC 的 IP(在我的例子中它是一个 10.* 地址)。