Kafka Broker 身份验证失败 - 凭据无效

Question

在最后一天左右的时间里，我尝试使用融合的 docker 镜像在本地设置一个节点的 Kafka 集群。不幸的是，一直未能做到这一点。以下是我的所有配置文件：

/etc/kafka/secrets/zookeeper_server_jaas.conf

Server {
    org.apache.zookeeper.server.auth.DigestLoginModule required
    user_admin="admin_secret";
};

/etc/kafka/secrets/kafka_server_jaas.conf

KafkaServer {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="admin"
    password="admin_secret";
};

Client {
   org.apache.zookeeper.server.auth.DigestLoginModule required
   username="admin"
   password="admin_secret";
};

docker-compose.yml

version: '3.5'

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
      KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/secrets/zookeeper_server_jaas.conf
          -Dzookeeper.authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
          -Dzookeeper.allowSaslFailedClients=false
          -Dzookeeper.requireClientAuthScheme=sasl
    volumes:
      - ./secrets:/etc/kafka/secrets
  
  broker:
    image: confluentinc/cp-kafka:latest
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENERS: SASL_SSL://:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: SASL_SSL:SASL_SSL
      KAFKA_ADVERTISED_LISTENERS: SASL_SSL://broker:9092
      KAFKA_SASL_ENABLED_MECHANISMS: SCRAM-SHA-512
      KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: SCRAM-SHA-512
      KAFKA_INTER_BROKER_LISTENER_NAME: SASL_SSL
      KAFKA_SSL_KEYSTORE_FILENAME: kafka.broker.keystore.jks
      KAFKA_SSL_KEYSTORE_CREDENTIALS: broker_keystore_creds
      KAFKA_SSL_KEY_CREDENTIALS: broker_sslkey_creds
      KAFKA_SSL_TRUSTSTORE_FILENAME: kafka.broker.truststore.jks
      KAFKA_SSL_TRUSTSTORE_CREDENTIALS: broker_truststore_creds
      KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/secrets/kafka_server_jaas.conf
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: false
      KAFKA_SSL_CLIENT_AUTH: "required"
      KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: "HTTPS"
    volumes:
      - ./secrets:/etc/kafka/secrets

我在 docker compose 中引用的证书位于本地计算机上 docker-compose 文件本身的

/secrets

目录下。

我在运行时遇到的错误

docker-compose up

是：

broker     | [2023-06-23 09:22:15,816] INFO [Controller id=1, targetBrokerId=1] Failed authentication with broker/192.168.16.3 (channelId=1) (Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512) (org.apache.kafka.common.network.Selector)
broker     | [2023-06-23 09:22:15,818] INFO [Controller id=1, targetBrokerId=1] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient)
broker     | [2023-06-23 09:22:15,818] ERROR [Controller id=1, targetBrokerId=1] Connection to node 1 (broker/192.168.16.3:9092) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512 (org.apache.kafka.clients.NetworkClient)

Answer 1

虽然我个人没有尝试过，但您能否将

user_admin

字段更新为

admin

中的

/etc/kafka/secrets/zookeeper_server_jaas.conf

值。据我所知，kafka 和 Zookeeper 应该具有相同的用户详细信息。我可能错了。

Answer 2

在您的zookeeper配置文件中，您正在使用DigestLoginModule：

org.apache.zookeeper.server.auth.DigestLoginModule required

但是在 docker compoes 环境规范中：

KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/secrets/zookeeper_server_jaas.conf
          -Dzookeeper.authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider

您正在使用不同的提供程序：SASLAuthenticationProvider。 而不是这么长的 OPTS，这足以传递环境变量：

EXTRA_ARGS: "-Djava.security.auth.login.config=/etc/kafka/zookeeper_jaas.conf"