Apache Kafka 客户端/生产者:无效的 PEM 密钥库配置

问题描述 投票:0回答:1

我正在尝试实现一个 Kafka Producer,包括 SSL 配置。由于我们在容器中运行服务并且不可能在本地文件系统中传送密钥库(使用 ssl.keystore.locationssl.truststore.location 选项),我必须使用 PEM 格式传递密钥和证书相应/备用选项 - 自版本 2.7.0 起可用(我在 JDK8 上运行 Kafka 3.5.0 和 kafka-clients:3.5.0)

这是我的完整客户端配置(密钥仅用于测试)

security.protocol=SSL
ssl.enabled.protocols=TLSv1.3

ssl.keystore.type=PEM
ssl.key.password=test1234
ssl.keystore.key=-----BEGIN RSA PRIVATE KEY----- \
MIIEpQIBAAKCAQEAq2UXYa2I/ebR89f9Y7IA7u9PIKOAz2jyK4Y+Zm3lNzIj+7Sf \
1zVFt8KwIG+KSOpvS7i3FoUZhqSr6KnkOZnd5QBuNkAo+20J1FTGSMlHHxxvYSb/ \
03M1Ehwu5FO87jGX4KhZdiFO4glaC+ww7X3Rk9NFcONI+/r7YRS6lmux/yZ45bJb \
NYg7dIkVdl2evZOM3qu+uLbeAU/R7gu2G94XpKPe7OqdIwN5oNch41eoj1Mucm4j \
BT0b4BrJXsB82pHsKtDfpRWw+HjSvysgGhietcj8AfiO9U3SU7frRaFnQg6qrYzd \
6QxQlvBa2T62hoJvfjctXoHTuywpbAQTEzpLlwIDAQABAoIBACMJ+au4ZGczxvxI \
zb6R52rxdYXAp1a/p7KKMTuTf+a203BjzsGaXHRi0sruwqCBfNtIGfX6o+tIwvQ/ \
ty6nbBui6OUiXL553iaQZjD/JRERKRv9cxNxXAolxNhc3iT78oa1JATobk37h3Mk \
iCQlMrE74dhgPs100+tW54ZU4gC9L9OY8M7tQuXFOOK7cydTMmuiC1SMQEHR9WUD \
CJQMyn8zcPxHPGcfyc5Sr7W54FQ2quvWNmxTDGAp80bYl/wuAEbuig3QRyLv055G \
GRMZN0nAM6J3LWbFx/NtWEErN6RoFvc4C0h+H3V+DYiQ2SQNrmBKb3N9gQqs5mXM \
3c/DnYECgYEA+MY3WRDeD1DjHK/q2/NLHLGmSqoQ2hN7ea0j+8VLvu7X4r1TrQJ4 \
qv5j9v4vRnXRW5dCo7tyWmwbY9g93foPGmtYFD6UzaZRu0jO6oyWL2rsEm015NuT \
9Faf5oDMkAlWrVusUTHsnMlwTKCDWRqqa6Jum1wfqsRm0hGGqs7ZiGECgYEAsF+D \
e/ldvxfWdCIK24AQOrKcYgIy0iwTuDrvfkhA0v9I578U15/odBoXLyrUuvJuxWUa \
aQl8oaxvOS5jw/f5nraquOXUECllwMoh1GNkMxX7zTaLtt16JkSHzXZjBZ8QDFAt \
KRRXOqtsDLgrpSvRjHAxltNmPSNKxCzp8cBbdvcCgYEA8cdDV46rItuMPv9CFM6Z \
u6N0aLBsfGXe5Vy+0hzoBwL+UmGLB2SVfiqLYSudKq+q74dMvvCYrQTQgfNR/YDg \
WN7m1ZYU7OGuIYUbhJ1qAV+7XBVEAc7eNuRRt6aKKZrJ4OZtzOPmgGLOf8qonVrr \
MnzFXrzkEgWboKJj7XHnvUECgYEAmZ3HTp6GOV9QyPuxizcCa62PqYmSrB352vdy \
CwJxe5Uf8cErLRc5Bo6G7O0YNe/b7Q+/Lgc2RzaB9ULjYBdGkic1kS+UQbNczcsA \
JhWfeyNFmppMnCxRLK15vsnDO20sNB1NCD+QLiiKRpXOZeFBdVlux5g+rzdi8nTQ \
GmDZCUcCgYEA2X6cJ4uCngOv7tQdILZpzYeZAb2crNa9vlngcfLbi8+7zTyS4Lop \
8Nmn979OrmCtHKgrMOcfFiObIBgnFPSufr+svp/y9Jd5TXE6dVmNLX13rki3HHgS \
uvxYM9UV5XdtSSb4BPYgBYmuXP8QjiK+V7TqU6Cc0ZK/Ma8DoBi3IEY= \
-----END RSA PRIVATE KEY-----
ssl.keystore.certificate.chain=-----BEGIN CERTIFICATE----- \
MIIDODCCAiACFBB23JdRC3qHpoGeo+qFdT2pGgOYMA0GCSqGSIb3DQEBCwUAMEUx \
CzAJBgNVBAYTAmF0MRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRl \
cm5ldCBXaWRnaXRzIFB0eSBMdGQwHhcNMjMwODE3MDgyMDQzWhcNMjQwODE2MDgy \
MDQzWjBsMRAwDgYDVQQGEwdVbmtub3duMRAwDgYDVQQIEwdVbmtub3duMRAwDgYD \
VQQHEwdVbmtub3duMRAwDgYDVQQKEwdVbmtub3duMRAwDgYDVQQLEwdVbmtub3du \
MRAwDgYDVQQDEwdVbmtub3duMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC \
AQEAq2UXYa2I/ebR89f9Y7IA7u9PIKOAz2jyK4Y+Zm3lNzIj+7Sf1zVFt8KwIG+K \
SOpvS7i3FoUZhqSr6KnkOZnd5QBuNkAo+20J1FTGSMlHHxxvYSb/03M1Ehwu5FO8 \
7jGX4KhZdiFO4glaC+ww7X3Rk9NFcONI+/r7YRS6lmux/yZ45bJbNYg7dIkVdl2e \
vZOM3qu+uLbeAU/R7gu2G94XpKPe7OqdIwN5oNch41eoj1Mucm4jBT0b4BrJXsB8 \
2pHsKtDfpRWw+HjSvysgGhietcj8AfiO9U3SU7frRaFnQg6qrYzd6QxQlvBa2T62 \
hoJvfjctXoHTuywpbAQTEzpLlwIDAQABMA0GCSqGSIb3DQEBCwUAA4IBAQBciKIN \
KCPTnSTPJEMHHunhC12WvbfF//BGqh3gn1eviGHRB1pnylQLPQXBIOJlTtgdlt0h \
0xSYgvWouD4UwctWTZoVRV/yHhMuZ99LXl8OKKCpZKGXfYTB2QjSkdWN+BIBhPz5 \
L7PIsNQJVmY/FpV8Es93lsjS2nH8fqaCOpVLsc604GDVD4cbUJEbCmB6faZ1Fy7x \
GX7yBkqTSEwE9TAJAgsPIWZZNsGD44ZGj0VAy/kRCEHigcToOvxPN+3nVWE3qacu \
QFyxNNX6waCc7rwxiIYarbxiGWxZo9pMEQC5ilxOzu1dShxbZNha8pyk3ClcGpCo \
naBeqW3X6uSM7CMr \
-----END CERTIFICATE----- \
-----BEGIN CERTIFICATE----- \
MIIDazCCAlOgAwIBAgIUEGrJ27wUi2LZ+2dE2DVJyDsPkwowDQYJKoZIhvcNAQEL \
BQAwRTELMAkGA1UEBhMCYXQxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM \
GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMzA4MTcwODE0MDRaFw0yNDA4 \
MTYwODE0MDRaMEUxCzAJBgNVBAYTAmF0MRMwEQYDVQQIDApTb21lLVN0YXRlMSEw \
HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB \
AQUAA4IBDwAwggEKAoIBAQDbCz4asRwDChWKL6VMX6STMugfn2X+QISSrad21VFH \
G9GHgRY36CpD/dd1AmoJKHChe5idR+Wq/522rg0AWDcSRF0ueI092JIYhEicnLIa \
cu5CehQwrFp+eaWh+ycgun9o1BXoUZVwBz/8I+02PqsgdpPfWdzIEvMd3xGYyI0w \
darsrWrvOxfzmtZcNtj5OWyRX6Gx0ql3eq5i423BHAkKKzMwLw8TLtoIC+7i0fkR \
7BckDexgdEMAPBgLxMwc9/ZI5LPad4BwizNiFxIkA38vwVHpgeNLD58aZNh0wlOB \
c/Vk/Lq8zK3YHHLcXASudUvjR8cDMRNYwcHmrWjl5S1PAgMBAAGjUzBRMB0GA1Ud \
DgQWBBQtJjgJdIYZyJqTACDjVBoD5hPtoTAfBgNVHSMEGDAWgBQtJjgJdIYZyJqT \
ACDjVBoD5hPtoTAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAQ \
/CS0lo8QiK/6oCyo6rcsh3XjTvbBXTW/h25RNXkQbEDmMErCI9UVHNFA+JtxfTW4 \
YDyp7ccfXdqEVzYGMlqfVeWxVSw3PvC+PCuJeAc2l8PtW9gMfmoM/N6Lg6ckw4Dy \
vEhLy34s0x+mtPQM9owxiPlKMYIzUnGhbdGF4cb+IYDONgrsBIQVa7UHrw8r43DE \
41pVFUqVvgrbHs3Oao8lEPJLyolPLuHdgVafp2urXRCqkWEwM9pTiffWDR2USX5B \
mL3mVNTnVXSpOErpjkQvp73zOjSG/h4CEgGdrLc8kjv1jkSyi3R8c3MRDhKlZnWK \
omAr3CbvLmjrFWnzDN7d \
-----END CERTIFICATE-----

ssl.truststore.type=PEM
ssl.truststore.certificates=-----BEGIN CERTIFICATE----- \
MIIDazCCAlOgAwIBAgIUEGrJ27wUi2LZ+2dE2DVJyDsPkwowDQYJKoZIhvcNAQEL \
BQAwRTELMAkGA1UEBhMCYXQxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM \
GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMzA4MTcwODE0MDRaFw0yNDA4 \
MTYwODE0MDRaMEUxCzAJBgNVBAYTAmF0MRMwEQYDVQQIDApTb21lLVN0YXRlMSEw \
HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB \
AQUAA4IBDwAwggEKAoIBAQDbCz4asRwDChWKL6VMX6STMugfn2X+QISSrad21VFH \
G9GHgRY36CpD/dd1AmoJKHChe5idR+Wq/522rg0AWDcSRF0ueI092JIYhEicnLIa \
cu5CehQwrFp+eaWh+ycgun9o1BXoUZVwBz/8I+02PqsgdpPfWdzIEvMd3xGYyI0w \
darsrWrvOxfzmtZcNtj5OWyRX6Gx0ql3eq5i423BHAkKKzMwLw8TLtoIC+7i0fkR \
7BckDexgdEMAPBgLxMwc9/ZI5LPad4BwizNiFxIkA38vwVHpgeNLD58aZNh0wlOB \
c/Vk/Lq8zK3YHHLcXASudUvjR8cDMRNYwcHmrWjl5S1PAgMBAAGjUzBRMB0GA1Ud \
DgQWBBQtJjgJdIYZyJqTACDjVBoD5hPtoTAfBgNVHSMEGDAWgBQtJjgJdIYZyJqT \
ACDjVBoD5hPtoTAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAQ \
/CS0lo8QiK/6oCyo6rcsh3XjTvbBXTW/h25RNXkQbEDmMErCI9UVHNFA+JtxfTW4 \
YDyp7ccfXdqEVzYGMlqfVeWxVSw3PvC+PCuJeAc2l8PtW9gMfmoM/N6Lg6ckw4Dy \
vEhLy34s0x+mtPQM9owxiPlKMYIzUnGhbdGF4cb+IYDONgrsBIQVa7UHrw8r43DE \
41pVFUqVvgrbHs3Oao8lEPJLyolPLuHdgVafp2urXRCqkWEwM9pTiffWDR2USX5B \
mL3mVNTnVXSpOErpjkQvp73zOjSG/h4CEgGdrLc8kjv1jkSyi3R8c3MRDhKlZnWK \
omAr3CbvLmjrFWnzDN7d \
-----END CERTIFICATE-----

但是,当我尝试运行生产者时,我总是收到以下错误,表明提供的私钥有问题,但我不确定是什么......

(当我跳过客户端身份验证时,信任链起作用)。

org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:837)
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:671)
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:652)
        at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:67)
        at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:54)
        at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)
Caused by: org.apache.kafka.common.errors.InvalidConfigurationException: Invalid PEM keystore configs
Caused by: java.io.IOException: overrun, bytes = 925
        at java.base/javax.crypto.EncryptedPrivateKeyInfo.<init>(EncryptedPrivateKeyInfo.java:98)
        at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.privateKey(DefaultSslEngineFactory.java:510)
        at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.createKeyStoreFromPem(DefaultSslEngineFactory.java:460)
        at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.<init>(DefaultSslEngineFactory.java:433)
        at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.createKeystore(DefaultSslEngineFactory.java:284)
        at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.configure(DefaultSslEngineFactory.java:161)
        at org.apache.kafka.common.security.ssl.SslFactory.instantiateSslEngineFactory(SslFactory.java:140)

我的现实应用程序在运行时设置属性,并从通过远程 API 调用加载的密钥库转换密钥(使用 Bouncy castle

Key clientPrivateKey = keyStore.getKey(clientCertAlias, keyStorePassword.toCharArray());
try (Writer writer = new StringWriter(); JcaPEMWriter pemWriter = new JcaPEMWriter(writer)) {
   pemWriter.writeObject(clientPrivateKey);
   pemWriter.flush();
   properties.put("ssl.keystore.key", writer.toString());
}

当我将 JKS 密钥库选项与相同的密钥库(包括可信密钥和我的私钥)一起使用时,初始化可以正常工作,因此私钥导出/转换为 PEM 格式的方式一定存在缺陷

security.protocol=SSL
ssl.truststore.type=JKS
ssl.truststore.location=/tmp/client.keystore.jks
ssl.enabled.protocols=TLSv1.3
ssl.truststore.password=test1234
ssl.keystore.type=JKS
ssl.keystore.location=/tmp/client.keystore.jks
ssl.keystore.password=test1234
ssl.endpoint.identification.algorithm=
java ssl apache-kafka ssl-certificate pem
1个回答
0
投票

我发现 OpenJDK 和内部库中缺乏对 PBES2 格式密钥的支持是问题所在。

另请参阅:Kafka 不会以 PEM 证书启动

© www.soinside.com 2019 - 2024. All rights reserved.