Kafka 不会从 PEM 证书开始
Kafka won't start with PEM certificate
我发现 Kafka 2.7.0 支持 PEM 证书，因此我决定尝试使用 DigiCert SSL 证书设置代理。我使用了新的选项，并且像 KIP-651 中的示例一样做了所有事情。但我收到错误：
[2021-01-20 17:54:55,787] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.config.ConfigException: Invalid value javax.net.ssl.SSLHandshakeException: no cipher suites in common for configuration A client SSLEngine created with the provided settings can't connect to a server SSLEngine created with those settings.
at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:98)
at org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:72)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:157)
at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:97)
at kafka.network.Processor.<init>(SocketServer.scala:790)
at kafka.network.SocketServer.newProcessor(SocketServer.scala:415)
at kafka.network.SocketServer.$anonfun$addDataPlaneProcessors(SocketServer.scala:288)
at kafka.network.SocketServer.addDataPlaneProcessors(SocketServer.scala:287)
at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors(SocketServer.scala:254)
at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$adapted(SocketServer.scala:251)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551)
at scala.collection.AbstractIterable.foreach(Iterable.scala:920)
at kafka.network.SocketServer.createDataPlaneAcceptorsAndProcessors(SocketServer.scala:251)
at kafka.network.SocketServer.startup(SocketServer.scala:125)
at kafka.server.KafkaServer.startup(KafkaServer.scala:303)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
at kafka.Kafka$.main(Kafka.scala:82)
at kafka.Kafka.main(Kafka.scala)
openssl x509 -in certificate.pem -text:
Certificate:
...
Signature Algorithm: ecdsa-with-SHA384
...
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
我试过使用不同版本的 Oracle JDK 和 OpenJDK，但没有效果。
我还尝试使用以下选项设置证书：
ssl.keystore.type=PEM
ssl.keystore.location=/opt/kafka/certs/certificate.pem
ssl.key.password=null
我遇到了一个新错误：
[2021-01-23 20:33:21,552] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.errors.InvalidConfigurationException: Failed to load PEM SSL keystore /opt/kafka/certs/certificate.pem
Caused by: org.apache.kafka.common.errors.InvalidConfigurationException: Invalid PEM keystore configs
Caused by: java.io.IOException: overrun, bytes = 111
at javax.crypto.EncryptedPrivateKeyInfo.<init>(EncryptedPrivateKeyInfo.java:92)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.privateKey(DefaultSslEngineFactory.java:512)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.createKeyStoreFromPem(DefaultSslEngineFactory.java:462)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.<init>(DefaultSslEngineFactory.java:435)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedPemStore.load(DefaultSslEngineFactory.java:412)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.<init>(DefaultSslEngineFactory.java:349)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedPemStore.<init>(DefaultSslEngineFactory.java:405)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.createKeystore(DefaultSslEngineFactory.java:293)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.configure(DefaultSslEngineFactory.java:161)
at org.apache.kafka.common.security.ssl.SslFactory.instantiateSslEngineFactory(SslFactory.java:136)
at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:93)
at org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:72)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:157)
at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:97)
at kafka.network.Processor.<init>(SocketServer.scala:790)
at kafka.network.SocketServer.newProcessor(SocketServer.scala:415)
at kafka.network.SocketServer.$anonfun$addDataPlaneProcessors(SocketServer.scala:288)
at kafka.network.SocketServer.addDataPlaneProcessors(SocketServer.scala:287)
at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors(SocketServer.scala:254)
at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$adapted(SocketServer.scala:251)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551)
at scala.collection.AbstractIterable.foreach(Iterable.scala:920)
at kafka.network.SocketServer.createDataPlaneAcceptorsAndProcessors(SocketServer.scala:251)
at kafka.network.SocketServer.startup(SocketServer.scala:125)
at kafka.server.KafkaServer.startup(KafkaServer.scala:303)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
at kafka.Kafka$.main(Kafka.scala:82)
at kafka.Kafka.main(Kafka.scala)
但如果将此证书转换为 jks:
openssl pkcs12 -export -in certificate.pem -out certificate.p12
keytool -importkeystore -srckeystore certificate.p12 -srcstoretype pkcs12 -destkeystore certificate.jks
echo 'ssl.keystore.location=/opt/kafka/certs/certificate.jks' >>server.properties
echo 'ssl.keystore.password=password' >>server.properties
经纪人工作正常。
这可能是一个错误还是我做错了什么？完整日志 here
我认为这可能是因为您使用的私钥是使用 PBES2 方案加密的。您可以使用 OpenSSL 转换原始密钥并使用 PBES1 代替：
openssl pkcs8 -in old_kafka.key -passout "pass:password" -topk8 -v1 PBE-SHA1-3DES -out kafka.key
您可以找到 PKCS#5 encryption algorithms are available with PBES1 and PBES2 in OpenSSL's PKCS#8 man page 的更多信息。
此外，this issue 中提到了 OpenJDK 对 PBES2 的支持。
我发现 Kafka 2.7.0 支持 PEM 证书，因此我决定尝试使用 DigiCert SSL 证书设置代理。我使用了新的选项，并且像 KIP-651 中的示例一样做了所有事情。但我收到错误：
[2021-01-20 17:54:55,787] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.config.ConfigException: Invalid value javax.net.ssl.SSLHandshakeException: no cipher suites in common for configuration A client SSLEngine created with the provided settings can't connect to a server SSLEngine created with those settings.
at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:98)
at org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:72)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:157)
at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:97)
at kafka.network.Processor.<init>(SocketServer.scala:790)
at kafka.network.SocketServer.newProcessor(SocketServer.scala:415)
at kafka.network.SocketServer.$anonfun$addDataPlaneProcessors(SocketServer.scala:288)
at kafka.network.SocketServer.addDataPlaneProcessors(SocketServer.scala:287)
at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors(SocketServer.scala:254)
at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$adapted(SocketServer.scala:251)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551)
at scala.collection.AbstractIterable.foreach(Iterable.scala:920)
at kafka.network.SocketServer.createDataPlaneAcceptorsAndProcessors(SocketServer.scala:251)
at kafka.network.SocketServer.startup(SocketServer.scala:125)
at kafka.server.KafkaServer.startup(KafkaServer.scala:303)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
at kafka.Kafka$.main(Kafka.scala:82)
at kafka.Kafka.main(Kafka.scala)
openssl x509 -in certificate.pem -text:
Certificate:
...
Signature Algorithm: ecdsa-with-SHA384
...
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
我试过使用不同版本的 Oracle JDK 和 OpenJDK，但没有效果。
我还尝试使用以下选项设置证书：
ssl.keystore.type=PEM
ssl.keystore.location=/opt/kafka/certs/certificate.pem
ssl.key.password=null
我遇到了一个新错误：
[2021-01-23 20:33:21,552] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.errors.InvalidConfigurationException: Failed to load PEM SSL keystore /opt/kafka/certs/certificate.pem
Caused by: org.apache.kafka.common.errors.InvalidConfigurationException: Invalid PEM keystore configs
Caused by: java.io.IOException: overrun, bytes = 111
at javax.crypto.EncryptedPrivateKeyInfo.<init>(EncryptedPrivateKeyInfo.java:92)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.privateKey(DefaultSslEngineFactory.java:512)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.createKeyStoreFromPem(DefaultSslEngineFactory.java:462)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.<init>(DefaultSslEngineFactory.java:435)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedPemStore.load(DefaultSslEngineFactory.java:412)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.<init>(DefaultSslEngineFactory.java:349)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedPemStore.<init>(DefaultSslEngineFactory.java:405)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.createKeystore(DefaultSslEngineFactory.java:293)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.configure(DefaultSslEngineFactory.java:161)
at org.apache.kafka.common.security.ssl.SslFactory.instantiateSslEngineFactory(SslFactory.java:136)
at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:93)
at org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:72)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:157)
at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:97)
at kafka.network.Processor.<init>(SocketServer.scala:790)
at kafka.network.SocketServer.newProcessor(SocketServer.scala:415)
at kafka.network.SocketServer.$anonfun$addDataPlaneProcessors(SocketServer.scala:288)
at kafka.network.SocketServer.addDataPlaneProcessors(SocketServer.scala:287)
at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors(SocketServer.scala:254)
at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$adapted(SocketServer.scala:251)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551)
at scala.collection.AbstractIterable.foreach(Iterable.scala:920)
at kafka.network.SocketServer.createDataPlaneAcceptorsAndProcessors(SocketServer.scala:251)
at kafka.network.SocketServer.startup(SocketServer.scala:125)
at kafka.server.KafkaServer.startup(KafkaServer.scala:303)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
at kafka.Kafka$.main(Kafka.scala:82)
at kafka.Kafka.main(Kafka.scala)
但如果将此证书转换为 jks:
openssl pkcs12 -export -in certificate.pem -out certificate.p12
keytool -importkeystore -srckeystore certificate.p12 -srcstoretype pkcs12 -destkeystore certificate.jks
echo 'ssl.keystore.location=/opt/kafka/certs/certificate.jks' >>server.properties
echo 'ssl.keystore.password=password' >>server.properties
经纪人工作正常。
这可能是一个错误还是我做错了什么？完整日志 here
我认为这可能是因为您使用的私钥是使用 PBES2 方案加密的。您可以使用 OpenSSL 转换原始密钥并使用 PBES1 代替：
openssl pkcs8 -in old_kafka.key -passout "pass:password" -topk8 -v1 PBE-SHA1-3DES -out kafka.key
您可以找到 PKCS#5 encryption algorithms are available with PBES1 and PBES2 in OpenSSL's PKCS#8 man page 的更多信息。
此外，this issue 中提到了 OpenJDK 对 PBES2 的支持。