在 Apache Beam PCollection 中使用 AutoValueSchema 会出现 `RuntimeException: Creator parameter arg0 Doesn't correspond to a schema field`

Question

我正在尝试创建一个由 AutoValue 定义的对象组成的 PCollection，并且我添加了适当的注释以通过 DefaultSchema(AutoValueSchema.class) 推断架构。像这样：

@DefaultSchema(AutoValueSchema.class)
@AutoValue
public abstract class MyAutoClas {
  public abstract String getMyStr();
  public abstract Integer getMyInt();

  @CreateSchema
  public static MyAutoClass create(String myStr, Integer myInt) {
    return new AutoValue_MyAutoClass(myStr, myInt);
  }
}

我有一个看起来像这样的小测试用例：

PCollection<KV<String, MyAutoClass>> result = pipeline
    .apply(Create.of(MyAutoClass.create("abc", 1)))
    .apply(WithKeys.of(in -> in.getMyStr()));

PAssert.that(result).containsInAnyOrder(KV.of("abc", MyAutoClass.create("abc", 1)));
pipeline.run().waitUntilFinish();

当我尝试运行时，我看到以下错误：

[ERROR] testMyAutoValueClass(.....)  Time elapsed: 1.891 s  <<< ERROR!
java.lang.RuntimeException: Creator parameter arg0 Doesn't correspond to a schema field
    at org.apache.beam.sdk.schemas.utils.ByteBuddyUtils$InvokeUserCreateInstruction.<init>(ByteBuddyUtils.java:717)
    at org.apache.beam.sdk.schemas.utils.ByteBuddyUtils$StaticFactoryMethodInstruction.<init>(ByteBuddyUtils.java:660)
    at org.apache.beam.sdk.schemas.utils.JavaBeanUtils.createStaticCreator(JavaBeanUtils.java:284)
    at org.apache.beam.sdk.schemas.utils.JavaBeanUtils.lambda$getStaticCreator(JavaBeanUtils.java:273)
    at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1664)
    at org.apache.beam.sdk.schemas.utils.JavaBeanUtils.getStaticCreator(JavaBeanUtils.java:269)
    at org.apache.beam.sdk.schemas.AutoValueSchema.lambda$schemaTypeCreatorFactory3bce5b(AutoValueSchema.java:80)
    at org.apache.beam.sdk.schemas.UserTypeCreatorFactory.create(UserTypeCreatorFactory.java:21)
  ....... [ETCETERA] ......

Answer 1

发生此错误是因为 ByteBuddy 和 Java 反射实用程序无法推断您的 @SchemaCreate 方法的参数名称（因此抱怨一些未知参数 arg0 - 这是默认名称）。

为了确保Java反射能够找到参数名称，您需要指示编译器包含此信息。如果你正在使用 Maven 构建，你可以这样做：

<properties>
    <!-- PLUGIN VERSIONS -->
    <maven-compiler-plugin.version>3.1</maven-compiler-plugin.version>

    <!-- OTHER PROPERTIES -->
    <java.version>1.8</java.version>
</properties>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>${maven-compiler-plugin.version}</version>
            <configuration>
                <!-- Original answer -->
                <compilerArgument>-parameters</compilerArgument>
                <!-- Or, if you use the plugin version >= 3.6.2 -->
                <parameters>true</parameters>
                <testCompilerArgument>-parameters</testCompilerArgument>
                <source>${java.version}</source>
                <target>${java.version}</target>
            </configuration>
        </plugin>
    </plugins>
</build>

注意： 要用 Gradle 完成同样的事情，请使用 .

在 Apache Beam PCollection 中使用 AutoValueSchema 会出现 `RuntimeException: Creator parameter arg0 Doesn't correspond to a schema field`

Using AutoValueSchema in Apache Beam PCollection gives `RuntimeException: Creator parameter arg0 Doesn't correspond to a schema field`

java

auto-value

apache-beam