驱动程序升级后，Cassandra 列元数据显示 "DateType" 而不是 "timestamp"

Question

我有一些 Java 代码可以对 Cassandra table 的模式进行内省。升级 Cassandra 驱动程序依赖项后，此代码不再按预期运行。对于旧驱动程序版本，timestamp 列的类型从 ColumnMetadata#getType() as DataType.Name#TIMESTAMP. With the new driver, the same call returns DataType.Name#CUSTOM and CustomType#getCustomTypeClassName 返回 org.apache.cassandra.db.marshal.DateType。

旧驱动版本为com.datastax.cassandra:cassandra-driver-core:2.1.9:

<dependency>
    <groupId>com.datastax.cassandra</groupId>
    <artifactId>cassandra-driver-core</artifactId>
    <version>2.1.9</version>
</dependency>

新驱动版本为com.datastax.cassandra:dse-driver:1.1.2:

<dependency>
    <groupId>com.datastax.cassandra</groupId>
    <artifactId>dse-driver</artifactId>
    <version>1.1.2</version>
</dependency>

集群版本为DataStax Enterprise 2.1.11.969:

cqlsh> SELECT release_version FROM system.local;

 release_version
-----------------
      2.1.11.969

为了说明问题，我创建了一个简单的控制台应用程序，用于打印指定 table 的列元数据。（见下文。）使用旧驱动程序构建时，输出如下所示：

# old driver
mvn -Pcassandra-driver clean package
java -jar target/cassandra-print-column-metadata-cassandra-driver.jar <address> <user> <password> <keyspace> <table>
...
ts timestamp
...

使用新驱动程序构建时，输出如下所示：

# new driver
mvn -Pdse-driver clean package
java -jar target/cassandra-print-column-metadata-dse-driver.jar <address> <user> <password> <keyspace> <table>
...
ts 'org.apache.cassandra.db.marshal.DateType'
...

到目前为止，我只在 timestamp 列中遇到过这个问题。我还没有看到它适用于任何其他数据类型，尽管我的模式没有详尽使用所有支持的数据类型。

DESCRIBE TABLE表示该列为timestamp。 system.schema_columns 表示 validator 是 org.apache.cassandra.db.marshal.DateType.

[cqlsh 3.1.7 | Cassandra 2.1.11.969 | CQL spec 3.0.0 | Thrift protocol 19.39.0]

cqlsh:my_keyspace> DESCRIBE TABLE my_table;

CREATE TABLE my_table (
  prim_addr text,
  ch text,
  received_on timestamp,
  ...
  PRIMARY KEY (prim_addr, ch, received_on)
) WITH
  bloom_filter_fp_chance=0.100000 AND
  caching='{"keys":"ALL", "rows_per_partition":"NONE"}' AND
  comment='emm_ks' AND
  dclocal_read_repair_chance=0.000000 AND
  gc_grace_seconds=864000 AND
  read_repair_chance=0.100000 AND
  compaction={'sstable_size_in_mb': '160', 'class': 'LeveledCompactionStrategy'} AND
  compression={'sstable_compression': 'SnappyCompressor'};

cqlsh:system> SELECT * FROM system.schema_columns WHERE keyspace_name = 'my_keyspace' AND columnfamily_name = 'my_table' AND column_name IN ('prim_addr', 'ch', 'received_on');

 keyspace_name | columnfamily_name | column_name | component_index | index_name | index_options | index_type | type           | validator
---------------+-------------------+-------------+-----------------+------------+---------------+------------+----------------+------------------------------------------
     my_keyspace |  my_table |          ch |               0 |       null |          null |       null | clustering_key | org.apache.cassandra.db.marshal.UTF8Type
     my_keyspace |  my_table |   prim_addr |            null |       null |          null |       null |  partition_key | org.apache.cassandra.db.marshal.UTF8Type
     my_keyspace |  my_table | received_on |               1 |       null |          null |       null | clustering_key | org.apache.cassandra.db.marshal.DateType

这是驱动程序中的错误、行为的故意更改还是我的某种错误配置？

pom.xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>cnauroth</groupId>
    <artifactId>cassandra-print-column-metadata</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <description>Console application that prints Cassandra table column metadata</description>
    <name>cassandra-print-column-metadata</name>
    <packaging>jar</packaging>

    <properties>
        <maven.compiler.source>1.7</maven.compiler.source>
        <maven.compiler.target>1.7</maven.compiler.target>
        <slf4j.version>1.7.25</slf4j.version>
    </properties> 

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <archive>
                        <manifest>
                            <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
                            <mainClass>cnauroth.Main</mainClass>
                        </manifest>
                    </archive>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                    <finalName>${project.artifactId}</finalName>
                    <appendAssemblyId>false</appendAssemblyId>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <profiles>
        <profile>
            <id>dse-driver</id>
            <activation>
                <activeByDefault>true</activeByDefault>
            </activation>
            <dependencies>
                <dependency>
                    <groupId>com.datastax.cassandra</groupId>
                    <artifactId>dse-driver</artifactId>
                    <version>1.1.2</version>
                </dependency>
            </dependencies>
            <build>
                <plugins>
                    <plugin>
                        <artifactId>maven-assembly-plugin</artifactId>
                        <configuration>
                            <finalName>${project.artifactId}-dse-driver</finalName>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
        <profile>
            <id>cassandra-driver</id>
            <activation>
                <activeByDefault>false</activeByDefault>
            </activation>
            <dependencies>
                <dependency>
                    <groupId>com.datastax.cassandra</groupId>
                    <artifactId>cassandra-driver-core</artifactId>
                    <version>2.1.9</version>
                </dependency>
            </dependencies>
            <build>
                <plugins>
                    <plugin>
                        <artifactId>maven-assembly-plugin</artifactId>
                        <configuration>
                            <finalName>${project.artifactId}-cassandra-driver</finalName>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

    <dependencies>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>${slf4j.version}</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>${slf4j.version}</version>
        </dependency>
    </dependencies>
</project>

Main.java

package cnauroth;

import java.util.List;

import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.ColumnMetadata;
import com.datastax.driver.core.Session;

class Main {

    public static void main(String[] args) throws Exception {
        // Skipping validation for brevity
        String address = args[0];
        String user = args[1];
        String password = args[2];
        String keyspace = args[3];
        String table = args[4];

        try (Cluster cluster = new Cluster.Builder()
                .addContactPoints(address)
                .withCredentials(user, password)
                .build()) {
            List<ColumnMetadata> columns =
                    cluster.getMetadata().getKeyspace(keyspace).getTable(table).getColumns();
            for (ColumnMetadata column : columns) {
                System.out.println(column);
            }
        }
    }
}

Answer 1

看起来用于时间戳的内部 Cassandra 类型在 Cassandra 1.2 和 2.0 (CASSANDRA-5723) 之间从 org.apache.cassandra.db.marshal.DateType 和 org.apache.cassandra.db.marshal.TimestampType 改变了。如果您使用 Cassandra 1.2（或 DSE 兼容版本）创建 table，将使用 DateType（即使您稍后升级了集群）。

看来 java 驱动程序的 2.1 版本能够解决这个问题 (source) but starting with 3.0 it does not (source)。相反，它将其解析为自定义类型。

幸运的是，驱动程序仍然能够序列化和反序列化此列，因为 cql 时间戳类型在响应中通过协议进行通信，但驱动程序将其解析为错误类型是一个错误。我继续创建 JAVA-1561 来跟踪这个。

如果您要将集群迁移到 C* 3.0+ 或 DSE 5.0+，我怀疑问题会消失，因为架构 tables 引用了 cql 名称而不是代表 Java class名称（除非确实是自定义类型）。

驱动程序升级后，Cassandra 列元数据显示 "DateType" 而不是 "timestamp"

Cassandra column metadata shows "DateType" instead of "timestamp" after driver upgrade

cassandra

datastax-java-driver

datastax

pom.xml

Main.java