Apache Ignite 连接查询 Returns 由于数据未位于同一位置而导致的结果不正确

Question

我是 Ignite 的新手，关注各种博客以了解所需的配置。

作为初学者，我使用了 "movie lens" 数据（电影和评级）并使用 Ignite Web 控制台生成了所需的配置。

请在下面找到有关我的代码的所需详细信息：

1) MySQL Table 结构

CREATE TABLE movies( movie_id INTEGER, movie_name VARCHAR(250), genre VARCHAR(250), CONSTRAINT pk_movie PRIMARY KEY (movie_id) );

CREATE TABLE ratings( user_id INTEGER, movie_id INTEGER, rating FLOAT, timestamp TIMESTAMP, CONSTRAINT pk_rating PRIMARY KEY (user_id, movie_id) );

2) Spring 引导项目中的 Ignite 配置文件（由 Ignite Web 控制台自动生成）

<?xml version="1.0" encoding="UTF-8"?>

<!-- This file was generated by Ignite Web Console (05/15/2019, 18:37) -->

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/util
                           http://www.springframework.org/schema/util/spring-util.xsd">
    <!-- Load external properties file. -->
    <bean id="placeholderConfig" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="location" value="classpath:secret.properties"/>
    </bean>

    <!-- Data source beans will be initialized from external properties file. -->
    <bean id="dsGeneric_Movielens" class="com.mchange.v2.c3p0.ComboPooledDataSource">
        <property name="jdbcUrl" value="${dsGeneric_Movielens.jdbc.url}"/>
        <property name="user" value="${dsGeneric_Movielens.jdbc.username}"/>
        <property name="password" value="${dsGeneric_Movielens.jdbc.password}"/>
    </bean>

    <bean class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="igniteInstanceName" value="ImportedCluster"/>

        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>
                                <value>127.0.0.1:47500..47510</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>

        <property name="cacheConfiguration">
            <list>
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="MoviesCache"/>
                    <property name="cacheMode" value="REPLICATED"/>
                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="cacheStoreFactory">
                        <bean class="org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStoreFactory">
                            <property name="dataSourceBean" value="dsGeneric_Movielens"/>
                            <property name="dialect">
                                <bean class="org.apache.ignite.cache.store.jdbc.dialect.BasicJdbcDialect">
                                </bean>
                            </property>

                            <property name="types">
                                <list>
                                    <bean class="org.apache.ignite.cache.store.jdbc.JdbcType">
                                        <property name="cacheName" value="MoviesCache"/>
                                        <property name="keyType" value="java.lang.Integer"/>
                                        <property name="valueType" value="com.learnwithmanoj.ignite.model.Movies"/>
                                        <property name="databaseSchema" value="movielens"/>
                                        <property name="databaseTable" value="movies"/>

                                        <property name="keyFields">
                                            <list>
                                                <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                    <constructor-arg>
                                                        <util:constant static-field="java.sql.Types.INTEGER"/>
                                                    </constructor-arg>
                                                    <constructor-arg value="movie_id"/>
                                                    <constructor-arg value="int"/>
                                                    <constructor-arg value="movieId"/>
                                                </bean>
                                            </list>
                                        </property>

                                        <property name="valueFields">
                                            <list>
                                                <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                    <constructor-arg>
                                                        <util:constant static-field="java.sql.Types.VARCHAR"/>
                                                    </constructor-arg>
                                                    <constructor-arg value="movie_name"/>
                                                    <constructor-arg value="java.lang.String"/>
                                                    <constructor-arg value="movieName"/>
                                                </bean>

                                                <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                    <constructor-arg>
                                                        <util:constant static-field="java.sql.Types.VARCHAR"/>
                                                    </constructor-arg>
                                                    <constructor-arg value="genre"/>
                                                    <constructor-arg value="java.lang.String"/>
                                                    <constructor-arg value="genre"/>
                                                </bean>
                                            </list>
                                        </property>
                                    </bean>
                                </list>
                            </property>
                        </bean>
                    </property>

                    <property name="readThrough" value="true"/>
                    <property name="writeThrough" value="true"/>

                    <property name="queryEntities">
                        <list>
                            <bean class="org.apache.ignite.cache.QueryEntity">
                                <property name="keyType" value="java.lang.Integer"/>
                                <property name="valueType" value="com.learnwithmanoj.ignite.model.Movies"/>
                                <property name="keyFieldName" value="movieId"/>

                                <property name="keyFields">
                                    <list>
                                        <value>movieId</value>
                                    </list>
                                </property>

                                <property name="fields">
                                    <map>
                                        <entry key="movieName" value="java.lang.String"/>
                                        <entry key="genre" value="java.lang.String"/>
                                        <entry key="movieId" value="java.lang.Integer"/>
                                    </map>
                                </property>

                                <property name="aliases">
                                    <map>
                                        <entry key="movieId" value="movie_id"/>
                                        <entry key="movieName" value="movie_name"/>
                                    </map>
                                </property>
                            </bean>
                        </list>
                    </property>
                </bean>

                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="RatingsCache"/>
                    <property name="cacheMode" value="REPLICATED"/>
                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="cacheStoreFactory">
                        <bean class="org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStoreFactory">
                            <property name="dataSourceBean" value="dsGeneric_Movielens"/>
                            <property name="dialect">
                                <bean class="org.apache.ignite.cache.store.jdbc.dialect.BasicJdbcDialect">
                                </bean>
                            </property>

                            <property name="types">
                                <list>
                                    <bean class="org.apache.ignite.cache.store.jdbc.JdbcType">
                                        <property name="cacheName" value="RatingsCache"/>
                                        <property name="keyType" value="com.learnwithmanoj.ignite.model.RatingsKey"/>
                                        <property name="valueType" value="com.learnwithmanoj.ignite.model.Ratings"/>
                                        <property name="databaseSchema" value="movielens"/>
                                        <property name="databaseTable" value="ratings"/>

                                        <property name="keyFields">
                                            <list>
                                                <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                    <constructor-arg>
                                                        <util:constant static-field="java.sql.Types.INTEGER"/>
                                                    </constructor-arg>
                                                    <constructor-arg value="user_id"/>
                                                    <constructor-arg value="int"/>
                                                    <constructor-arg value="userId"/>
                                                </bean>

                                                <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                    <constructor-arg>
                                                        <util:constant static-field="java.sql.Types.INTEGER"/>
                                                    </constructor-arg>
                                                    <constructor-arg value="movie_id"/>
                                                    <constructor-arg value="int"/>
                                                    <constructor-arg value="movieId"/>
                                                </bean>
                                            </list>
                                        </property>

                                        <property name="valueFields">
                                            <list>
                                                <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                    <constructor-arg>
                                                        <util:constant static-field="java.sql.Types.REAL"/>
                                                    </constructor-arg>
                                                    <constructor-arg value="rating"/>
                                                    <constructor-arg value="java.lang.Double"/>
                                                    <constructor-arg value="rating"/>
                                                </bean>

                                                <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                    <constructor-arg>
                                                        <util:constant static-field="java.sql.Types.TIMESTAMP"/>
                                                    </constructor-arg>
                                                    <constructor-arg value="timestamp"/>
                                                    <constructor-arg value="java.sql.Timestamp"/>
                                                    <constructor-arg value="timestamp"/>
                                                </bean>
                                            </list>
                                        </property>
                                    </bean>
                                </list>
                            </property>
                        </bean>
                    </property>

                    <property name="readThrough" value="true"/>
                    <property name="writeThrough" value="true"/>

                    <property name="queryEntities">
                        <list>
                            <bean class="org.apache.ignite.cache.QueryEntity">
                                <property name="keyType" value="com.learnwithmanoj.ignite.model.RatingsKey"/>
                                <property name="valueType" value="com.learnwithmanoj.ignite.model.Ratings"/>

                                <property name="keyFields">
                                    <list>
                                        <value>userId</value>
                                        <value>movieId</value>
                                    </list>
                                </property>

                                <property name="fields">
                                    <map>
                                        <entry key="userId" value="java.lang.Integer"/>
                                        <entry key="movieId" value="java.lang.Integer"/>
                                        <entry key="rating" value="java.lang.Double"/>
                                        <entry key="timestamp" value="java.sql.Timestamp"/>
                                    </map>
                                </property>

                                <property name="aliases">
                                    <map>
                                        <entry key="userId" value="user_id"/>
                                        <entry key="movieId" value="movie_id"/>
                                    </map>
                                </property>

                                <property name="indexes">
                                    <list>
                                        <bean class="org.apache.ignite.cache.QueryIndex">
                                            <property name="name" value="fk_movie_id"/>
                                            <property name="indexType" value="SORTED"/>

                                            <property name="fields">
                                                <map>
                                                    <entry key="movieId" value="false"/>
                                                </map>
                                            </property>
                                        </bean>
                                    </list>
                                </property>
                            </bean>
                        </list>
                    </property>
                </bean>
            </list>
        </property>
    </bean>
</beans>

3) Ignite 安装文件夹中的 Ignite 配置文件

<?xml version="1.0" encoding="UTF-8"?>

<!-- This file was generated by Ignite Web Console (05/15/2019, 08:52) -->

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/util
                           http://www.springframework.org/schema/util/spring-util.xsd">
    <!-- Load external properties file. -->
    <bean id="placeholderConfig" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="location" value="classpath:secret.properties"/>
    </bean>

    <!-- Data source beans will be initialized from external properties file. -->
    <bean id="datasource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
        <property name="jdbcUrl" value="${datasource.jdbc.url}"/>
        <property name="user" value="${datasource.jdbc.username}"/>
        <property name="password" value="${datasource.jdbc.password}"/>
    </bean>

    <bean class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="igniteInstanceName" value="MovieRatingsCluster"/>

        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>
                                <value>127.0.0.1:47500..47510</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>

    </bean>
</beans>

问题陈述：

当我将 Spring 引导应用程序作为独立的 Ignite 节点启动时，MySQL 数据库中的所有数据都会按预期同步到 Ignite 节点中。此外，如果我尝试在评级和电影之间加入 table，它会正确加入并给出正确的结果。

使用的加入查询：

select r.movie_id, m.movie_name, count(r.rating) as count from "RatingsCache".ratings r inner join "MoviesCache".movies m where r.movie_id = m.movie_id group by r.movie_id order by count desc limit 5;

现在进入问题。如果我使用多个节点进行相同的测试，则由于数据未并置，Join 开始给出不正确的结果。

任何人都可以指导我正确的配置集，需要添加这些配置来解决分区模式下的加入问题吗？

我什至尝试添加以下配置。

<property name="cacheKeyConfiguration">
    <list>
        <bean class="org.apache.ignite.cache.CacheKeyConfiguration">
            <property name="typeName" value="com.learnwithmanoj.ignite.model.RatingsKey" />
            <property name="affinityKeyFieldName" value="movie_id" />
        </bean>
    </list>
</property>

但是出现以下错误。

Caused by: org.apache.ignite.binary.BinaryObjectException: Binary type has different affinity key fields [typeName=com.learnwithmanoj.ignite.model.RatingsKey, affKeyFieldName1=null, affKeyFieldName2=movie_id]

Answer 1

您的解决方案应该有效。首先，您需要清理所有节点上 Ignite 主目录下的 marshaller/ 目录，然后重新启动这些节点。

您可能还需要指定大写的关键字段名称：MOVIE_ID.

Apache Ignite 连接查询 Returns 由于数据未位于同一位置而导致的结果不正确

Apache Ignite Join Query Returns Incorrect Results Due to Data not Being Co-located

gridgain

ignite