运行个 Apache Flink 作业时链接失败

Question

我有一个用 Flink 0.9 开发的工作，它使用了图形模块 (Gelly)。该作业在 IDE (Eclipse) 中成功运行但在使用 maven (mvn clean install) 将其导出到 JAR 后，它无法在本地 flink 实例上执行并出现以下错误

"The program's entry point class 'myclass' could not be loaded due to a linkage failure"

java.lang.NoClassDefFoundError: org/apache/flink/graph/GraphAlgorithm

知道为什么会发生这种情况以及如何解决吗？

Answer 1

flink-gelly 的代码似乎没有出现在您的 jar 文件中。此问题最明显的原因是项目的 pom 文件中缺少 maven 依赖项。但我假设存在依赖关系，否则无法在 IDE 中开发作业。

很可能，jar 文件是由 maven-jar-plugin 创建的，不包括依赖项。尝试将以下片段添加到您的 pom.xml:

    <build>
    <plugins>
        <!-- We use the maven-shade plugin to create a fat jar that contains all dependencies
        except flink and it's transitive dependencies. The resulting fat-jar can be executed
        on a cluster. Change the value of Program-Class if your program entry point changes. -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>2.3</version>
            <executions>
                <!-- Run shade goal on package phase -->
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <filters>
                            <filter>
                                <artifact>org.apache.flink:*</artifact>
                                <excludes>
                                    <exclude>org/apache/flink/shaded/**</exclude>
                                    <exclude>web-docs/**</exclude>
                                </excludes>
                            </filter>
                        </filters>
                        <transformers>
                            <!-- add Main-Class to manifest file -->
                            <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                <mainClass>YOURMAINCLASS</mainClass>
                            </transformer>
                        </transformers>
                        <createDependencyReducedPom>false</createDependencyReducedPom>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>

</build>
<profiles>
    <profile>
        <!-- A profile that does everyting correctly:
        We set the Flink dependencies to provided -->
        <id>build-jar</id>
        <activation>
            <activeByDefault>false</activeByDefault>
        </activation>
        <dependencies>
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-java</artifactId>
                <version>0.9-SNAPSHOT</version>
                <scope>provided</scope>
            </dependency>
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-streaming-core</artifactId>
                <version>0.9-SNAPSHOT</version>
                <scope>provided</scope>
            </dependency>
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-clients</artifactId>
                <version>0.9-SNAPSHOT</version>
                <scope>provided</scope>
            </dependency>
        </dependencies>
    </profile>
</profiles>

现在，您可以使用 mvn clean package -Pbuild-jar 构建 jar。 jar 文件现在位于 target/ 目录中。

您可以手动检查jar（zip）文件是否包含/org/apache/flink/graph/

中的class个文件

运行个 Apache Flink 作业时链接失败

Linkage failure when running Apache Flink jobs

maven

apache-flink

gelly

运行 个 Apache Flink 作业时链接失败

Linkage failure when running Apache Flink jobs

maven

apache-flink

gelly

运行个 Apache Flink 作业时链接失败