为什么 flink-quickstart-scala 建议在默认范围内添加连接器依赖项,而 Flink Hive 集成文档建议相反
Why does flink-quickstart-scala suggests adding connector dependencies in the default scope, while Flink Hive integration docs suggest the opposite
连接器依赖项应在默认范围内
这就是 flink-quickstart-scala 建议的内容:
<!-- Add connector dependencies here. They must be in the default scope (compile). -->
<!-- Example:
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
-->
它也符合 Flink project configuration:
We recommend packaging the application code and all its required dependencies into one jar-with-dependencies which we refer to as the application jar. The application jar can be submitted to an already running Flink cluster, or added to a Flink application container image.
Important: For Maven (and other build tools) to correctly package the dependencies into the application jar, these application dependencies must be specified in scope compile (unlike the core dependencies, which must be specified in scope provided).
Hive 连接器依赖项应在提供的范围内
然而,Flink Hive Integration docs建议相反:
If you are building your own program, you need the following dependencies in your mvn file. It’s recommended not to include these dependencies in the resulting jar file. You’re supposed to add dependencies as stated above at runtime.
为什么?
造成这种差异的原因是,对于 Hive,建议使用各自的 Hive 依赖项启动集群。 documentation 声明最好在启动集群之前将依赖项放入 lib
目录。这样,集群就可以 运行 使用 Hive 的作业。同时,您不必将此依赖项捆绑在用户 jar 中,从而减小其大小。但是,如果您愿意,应该没有任何东西可以阻止您将 Hive 依赖项与您的用户代码捆绑在一起。
连接器依赖项应在默认范围内
这就是 flink-quickstart-scala 建议的内容:
<!-- Add connector dependencies here. They must be in the default scope (compile). -->
<!-- Example:
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
-->
它也符合 Flink project configuration:
We recommend packaging the application code and all its required dependencies into one jar-with-dependencies which we refer to as the application jar. The application jar can be submitted to an already running Flink cluster, or added to a Flink application container image.
Important: For Maven (and other build tools) to correctly package the dependencies into the application jar, these application dependencies must be specified in scope compile (unlike the core dependencies, which must be specified in scope provided).
Hive 连接器依赖项应在提供的范围内
然而,Flink Hive Integration docs建议相反:
If you are building your own program, you need the following dependencies in your mvn file. It’s recommended not to include these dependencies in the resulting jar file. You’re supposed to add dependencies as stated above at runtime.
为什么?
造成这种差异的原因是,对于 Hive,建议使用各自的 Hive 依赖项启动集群。 documentation 声明最好在启动集群之前将依赖项放入 lib
目录。这样,集群就可以 运行 使用 Hive 的作业。同时,您不必将此依赖项捆绑在用户 jar 中,从而减小其大小。但是,如果您愿意,应该没有任何东西可以阻止您将 Hive 依赖项与您的用户代码捆绑在一起。