具有记录数组类型的嵌套 Avro 记录反序列化

Question

我正在编写代码来反序列化其中包含数组类型记录的 avro 嵌套记录，方法是将 POJO 用于数组类型记录，并在主 POJO class 中作为列表调用以进行反序列化。但是我不明白如何使用多个模式来反序列化记录。

架构结构：

{
"type": "record",
"name": "MainSchemaName",
"version": "2",
"namespace": "com.cmain",
"doc": "AExample",
"fields": [
 {
  "name": "MainABC",
  "type": {
    "type": "array",
    "items": {
      "name": "ABCarr",
      "type": "record",
      "fields": [
        {
          "name": "prod1",
          "type": "double"
        },
        {
          "name": "prod2",
          "type": "string"
        }
      ]
    }
  }
 },
 {
  "name": "comnsu1",
  "type": "int"
 }
 ]
}

Answer 1

您需要为每条记录指定一个文件，如下所示：

ABCarr.avsc:

{
      "name": "ABCarr",
      "namespace": "com.cmain",
      "type": "record",
      "fields": [
        {
          "name": "prod1",
          "type": "double"
        },
        {
          "name": "prod2",
          "type": "string"
        }
      ]
}

MainSchemaName.avsc:

{
"type": "record",
"name": "MainSchemaName",
"version": "2",
"namespace": "com.cmain",
"doc": "AExample",
"fields": [
 {
  "name": "MainABC",
  "type": {
       "type": "array",
       "items": "com.cmain.ABCarr",
       "java-class": "java.util.List"
     }
 },
 {
  "name": "comnsu1",
  "type": "int"
 }
 ]
}

然后您应该配置 avro-maven-plugin 以构建其他人（在您的情况下为 ABCarr）所需的模式。假设您的 avro 模式文件位于 src/main/resources/schema/avro 路径上，您应该指定以下内容来构建 ABCarr，然后在 MainSchemaName 上使用它作为类型：

<plugin>
    <groupId>org.apache.avro</groupId>
    <artifactId>avro-maven-plugin</artifactId>
    <version>1.9.2</version>
    <executions>
        <execution>
            <phase>generate-sources</phase>
            <goals>
                <goal>schema</goal>
            </goals>
            <configuration>
                <sourceDirectory>${project.basedir}/src/main/resources/schema/avro</sourceDirectory>
                <stringType>String</stringType>
                <createSetters>true</createSetters>
                <fieldVisibility>private</fieldVisibility>
                <imports>
                    <import>${project.basedir}/src/main/resources/schema/avro/ABCarr.avsc</import>
                </imports>
            </configuration>
        </execution>
    </executions>
</plugin>

因此只需在导入选项中导入所有常用模式。

具有记录数组类型的嵌套 Avro 记录反序列化

Nested Avro record deserialization with array type of records

java

avro

deserialization

apache-kafka