当我使用Kerberos身份验证通过SpringBoot连接到Kafka时遇到问题。我正在使用具有以下详细信息的自定义Kafka连接管理器-
bootstrap-servers-sasl: node1:9094, node2:9094, node3:9094
protocol: SASL_SSL
mechanism: GSSAPI
kerberos:
service:
name: kfkusr
jaas:
config: "com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab=\"#keytab-name#\" principal=\"abc/[email protected]\";"
其中#keytab-name#将在运行时替换为-
我的本地PC-C:/Users/MyPC/AppData/Local/Temp/abc.node2_d2254866264751402128.keytab
PCF-/home/vcap/tmp/abc.node2_d2215947326380395062.keytab
[该应用程序在本地运行正常，消息将发送到Kafka。但是在PCF上运行时失败，并带有以下异常-
2019-08-09T14:40:46.481-05:00 [APP/PROC/WEB/0] [OUT] WARN [9f-3868cbe47d81] org.apache.kafka.clients.NetworkClient o.a.k.c.NetworkClient.processDisconnection(NetworkClient.java:585) - ||||||||||||||Connection to node -1 terminated during authentication. This may indicate that authentication failed due to invalid credentials.
...
...
Failed to send; nested exception is org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.: org.springframework.kafka.core.KafkaProducerException: Failed to send; nested exception is org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms
...
...
Exception thrown when sending a message with key='null' and payload='<my payload>' to topic <test_topic> :: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
UPDATE 1-
添加krb5.conf文件后，它具有默认领域
[libdefaults]
default_realm = mydomain.NET
认证错误消失，但仍然存在以下错误
Failed to send; nested exception is org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.: org.springframework.kafka.core.KafkaProducerException: Failed to send; nested exception is org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms
org.apache.kafka: DEBUG启用调试后，新错误显示为-
2019-08-14T09:49:51.947-05:00 [APP/PROC/WEB/0] [OUT] DEBUG [d3-5b28248c661c] org.apache.kafka.clients.NetworkClient o.a.k.c.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:907) - ||||||||||||||Initialize connection to node node1:9094 (id: -1 rack: null) for sending metadata request
2019-08-14T09:49:51.947-05:00 [APP/PROC/WEB/0] [OUT] DEBUG [d3-5b28248c661c] org.apache.kafka.clients.NetworkClient o.a.k.c.NetworkClient.initiateConnect(NetworkClient.java:762) - ||||||||||||||Initiating connection to node node1:9094 (id: -1 rack: null)
2019-08-14T09:49:51.948-05:00 [APP/PROC/WEB/0] [OUT] DEBUG [d3-5b28248c661c] o.a.k.c.s.a.SaslClientAuthenticator o.a.k.c.s.a.SaslClientAuthenticator.setSaslState(SaslClientAuthenticator.java:209) - ||||||||||||||Set SASL client state to SEND_HANDSHAKE_REQUEST
2019-08-14T09:49:51.948-05:00 [APP/PROC/WEB/0] [OUT] DEBUG [d3-5b28248c661c] o.a.k.c.s.a.SaslClientAuthenticator o.a.k.c.s.a.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:134) - ||||||||||||||Creating SaslClient: client=abc/[email protected];service=kfkusr;serviceHostname=node1;mechs=[GSSAPI]
2019-08-14T09:49:51.949-05:00 [APP/PROC/WEB/0] [OUT] INFO [d3-5b28248c661c] o.a.k.common.network.SaslChannelBuilder o.a.k.c.n.SaslChannelBuilder.buildChannel(SaslChannelBuilder.java:119) - ||||||||||||||Failed to create channel due to : org.apache.kafka.common.KafkaException: Failed to configure SaslClientAuthenticator at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.configure(SaslClientAuthenticator.java:125) at org.apache.kafka.common.network.SaslChannelBuilder.buildChannel(SaslChannelBuilder.java:116) at org.apache.kafka.common.network.Selector.connect(Selector.java:203) at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:764) at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:60) at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:908) at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:819) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:431) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:224) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:162) at java.lang.Thread.run(Thread.java:748)Caused by: org.apache.kafka.common.KafkaException: Failed to create SaslClient with mechanism GSSAPI at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslClient(SaslClientAuthenticator.java:140) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.configure(SaslClientAuthenticator.java:123) ... 10 common frames omittedCaused by: javax.security.sasl.SaslException: Failure to initialize security context at com.sun.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:149) at com.sun.security.sasl.gsskerb.FactoryImpl.createSaslClient(FactoryImpl.java:63) at javax.security.sasl.Sasl.createSaslClient(Sasl.java:384) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:136) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:131) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslClient(SaslClientAuthenticator.java:131) ... 11 common frames omittedCaused by: org.ietf.jgss.GSSException: Invalid name provided (Mechanism level: KrbException: Cannot locate default realm) at sun.security.jgss.krb5.Krb5NameElement.getInstance(Krb5NameElement.java:129) at sun.security.jgss.krb5.Krb5MechFactory.getNameElement(Krb5MechFactory.java:95) at sun.security.jgss.GSSManagerImpl.getNameElement(GSSManagerImpl.java:203) at sun.security.jgss.GSSNameImpl.getElement(GSSNameImpl.java:477) at sun.security.jgss.GSSNameImpl.init(GSSNameImpl.java:201) at sun.security.jgss.GSSNameImpl.<init>(GSSNameImpl.java:170) at sun.security.jgss.GSSManagerImpl.createName(GSSManagerImpl.java:138) at com.sun.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:107) ... 18 common frames omitted
2019-08-14T09:49:51.949-05:00 [APP/PROC/WEB/0] [OUT] DEBUG [d3-5b28248c661c] org.apache.kafka.clients.NetworkClient o.a.k.c.NetworkClient.initiateConnect(NetworkClient.java:773) - ||||||||||||||Error connecting to node gtcrd-ckbla01d.nam.nsroot.net:9094 (id: -1 rack: null): java.io.IOException: Channel could not be created for socket java.nio.channels.SocketChannel[closed] at org.apache.kafka.common.network.Selector.connect(Selector.java:210) at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:764) at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:60) at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:908) at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:819) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:431) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:224) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:162) at java.lang.Thread.run(Thread.java:748)Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.common.KafkaException: Failed to configure SaslClientAuthenticator at org.apache.kafka.common.network.SaslChannelBuilder.buildChannel(SaslChannelBuilder.java:120) at org.apache.kafka.common.network.Selector.connect(Selector.java:203) ... 8 common frames omittedCaused by: org.apache.kafka.common.KafkaException: Failed to configure SaslClientAuthenticator at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.configure(SaslClientAuthenticator.java:125) at org.apache.kafka.common.network.SaslChannelBuilder.buildChannel(SaslChannelBuilder.java:116) ... 9 common frames omittedCaused by: org.apache.kafka.common.KafkaException: Failed to create SaslClient with mechanism GSSAPI at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslClient(SaslClientAuthenticator.java:140) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.configure(SaslClientAuthenticator.java:123) ... 10 common frames omittedCaused by: javax.security.sasl.SaslException: Failure to initialize security context at com.sun.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:149) at com.sun.security.sasl.gsskerb.FactoryImpl.createSaslClient(FactoryImpl.java:63) at javax.security.sasl.Sasl.createSaslClient(Sasl.java:384) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:136) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:131) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslClient(SaslClientAuthenticator.java:131) ... 11 common frames omittedCaused by: org.ietf.jgss.GSSException: Invalid name provided (Mechanism level: KrbException: Cannot locate default realm) at sun.security.jgss.krb5.Krb5NameElement.getInstance(Krb5NameElement.java:129) at sun.security.jgss.krb5.Krb5MechFactory.getNameElement(Krb5MechFactory.java:95) at sun.security.jgss.GSSManagerImpl.getNameElement(GSSManagerImpl.java:203) at sun.security.jgss.GSSNameImpl.getElement(GSSNameImpl.java:477) at sun.security.jgss.GSSNameImpl.init(GSSNameImpl.java:201) at sun.security.jgss.GSSNameImpl.<init>(GSSNameImpl.java:170) at sun.security.jgss.GSSManagerImpl.createName(GSSManagerImpl.java:138) at com.sun.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:107) ... 18 common frames omitted
好吧..我去解决这个问题..可能对其他陷入该问题的人有所帮助。
首先，我在Producer Config属性中设置krb5.conf文件。该文件包含主机的详细信息，领域以及使用kerberos发现服务所需的其他详细信息
System.setProperty("java.security.krb5.conf", <path to conf file>);
而不是直接在属性中设置jaas配置，我创建了jaas conf文件并在系统属性中设置它-
在代码之前是这样-
props.put("sasl.jaas.config", jaasConfig);
更改后-
System.setProperty("java.security.auth.login.config", jaasFile.getPath());
样本Jaas配置文件jaas_client.conf结构-
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
keyTab="#keytab-name#"
principal="#principal#";
};
P.S。在jaas_client.conf文件中，在将#keytab-name#更新为密钥表文件的实际路径和具有实际值的#principal#值之后，在设置为SystemProperties
当使用以下属性启用调试时，可以设置调试模式以查看JaaS身份验证和票证日志-
System.setProperty("sun.security.krb5.debug", ""+<true/false>);