Spring 启动 + Kafka + Kerberos 配置

Question

我正在使用 Spring Boot 1.5.6.RELEASE 使用 Kerberos 身份验证连接到 Kafka 0.11。这些是我用于 Kafka 的依赖项：

        <dependency>
            <groupId>org.springframework.integration</groupId>
            <artifactId>spring-integration-kafka</artifactId>
            <version>3.0.0.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
            <version>2.0.8.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-kafka</artifactId>
            <version>2.1.1.RELEASE</version>
        </dependency>

我必须向不在我们管理下的 Kafka 服务器发送消息，我获得了 Kafka 用户名、keytab 文件和 krb5.conf 文件。

这些是在没有 Kerberos 的情况下用于测试的属性：

spring:
  kafka:
    bootstrap-servers: "10.10.20.185:9092"
    producer:
      value-serializer: org.springframework.kafka.support.serializer.JsonSerializer

工作正常。

如何在我的应用程序配置中实施 Kerberos？由于我是 Kafka 和 Kerberos 的新手，我们将不胜感激。

Answer 1

参见 the kafka documentation "Authentication using SASL/Kerberos"。

To configure SASL authentication on the clients: Clients (producers, consumers, connect workers, etc) will authenticate to the cluster with their own principal (usually with the same name as the user running the client), so obtain or create these principals as needed. Then configure the JAAS configuration property for each client. Different clients within a JVM may run as different users by specifiying different principals. The property sasl.jaas.config in producer.properties or consumer.properties describes how clients like producer and consumer can connect to the Kafka Broker. The following is an example configuration for a client using a keytab (recommended for long-running processes):

sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
    useKeyTab=true \
    storeKey=true  \
    keyTab="/etc/security/keytabs/kafka_client.keytab" \
    principal="kafka-client-1@EXAMPLE.COM";

For command-line utilities like kafka-console-consumer or kafka-console-producer, kinit can be used along with "useTicketCache=true" as in:

sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
    useTicketCache=true;

JAAS configuration for clients may alternatively be specified as a JVM parameter similar to brokers as described here. Clients use the login section named KafkaClient. This option allows only one user for all client connections from a JVM.

Make sure the keytabs configured in the JAAS configuration are readable by the operating system user who is starting kafka client. Optionally pass the krb5 file locations as JVM parameters to each client JVM (see here for more details):

-Djava.security.krb5.conf=/etc/kafka/krb5.conf

在producer.properties或consumer.properties中配置以下属性：

security.protocol=SASL_PLAINTEXT (or SASL_SSL)
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka

Answer 2

从Confluent

看到这个

您需要在 spring 引导 application.yml 文件中输入以下内容。 Spring 启动 Kafka 将自动选择这些配置并使用 Kerberos 身份验证创建您的消费者客户端。

spring:
  kafka:
    producer:
      bootstrap-servers: address:port
    properties:
      security:
        protocol: SASL_PLAINTEXT
      sasl:
        mechanism: GSSAPI
        kerberos:
          service:
            name: kafka
        jaas:
          config: com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="/path/to/keytab" principal="servicename/hostname@REALM";

您还需要将 -Djava.security.krb5.conf=/etc/krb5.conf 传递给 JVM 或在您的 main class 中使用静态初始化程序来设置 Kerberos configuration information（这包含有关您的 KDC 的详细信息）。阅读更多关于 Kerberos Requirements here

static {
        System.setProperty("java.security.krb5.conf", "/etc/krb5.conf");
        System.out.println(System.getProperty("java.security.krb5.conf"));
    }

确保您的代理也支持 spring 启动配置中提供的安全协议。您可以通过检查 Kafka 代理中的 server.properties 文件来确认这些，即

With GSSAPI (Kerberos) enabled mechanism (Note: More than mechanism can be enabled in Kafka brokers hence supporting multiple authentication mechanisms)

sasl.enabled.mechanisms=GSSAPI

With SCRAM enabled mechanism

sasl.enabled.mechanisms=SCRAM-SHA-256

With PLAIN enabled mechanism

sasl.enabled.mechanisms=PLAIN

Spring 启动 + Kafka + Kerberos 配置

Spring Boot + Kafka + Kerberos configuration

spring-boot

apache-kafka

kerberos

spring-kafka