在 Windows 机器上使用 KRaft 启动 Kafka(3.4.0) 时出现错误

Question

我正在尝试按照此处的文档使用 KRaft 启动 Kafka - https://kafka.apache.org/quickstart 在 Windows 计算机上。

注意-相同版本的kafka与zookeeper完美配合

第 1 步 - 生成 uuid

(.\bin\windows\kafka-storage.bat random-uuid)

第 2 步 - 格式化日志目录

(.\bin\windows\kafka-storage.bat format -t d1hZRwFeRuqyNTTwR9grGw -c .\config\kraft\server.properties)

第 3 步 - 启动服务器

(.\bin\windows\kafka-server-start.bat .\config\kraft\server.properties)

但是它没有启动并给出错误 -

    [2023-03-26 23:34:36,921] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2023-03-26 23:34:37,252] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2023-03-26 23:34:37,466] INFO Starting controller (kafka.server.ControllerServer)
[2023-03-26 23:34:37,866] INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas)
[2023-03-26 23:34:37,926] INFO Awaiting socket connections on 0.0.0.0:9093. (kafka.network.DataPlaneAcceptor)
[2023-03-26 23:34:37,983] INFO [SocketServer listenerType=CONTROLLER, nodeId=1] Created data-plane acceptor and processors for endpoint : ListenerName(CONTROLLER) (kafka.network.SocketServer)
[2023-03-26 23:34:37,994] INFO [SharedServer id=1] Starting SharedServer (kafka.server.SharedServer)
[2023-03-26 23:34:38,111] INFO [LogLoader partition=__cluster_metadata-0, dir=E:\tmp\kraft-combined-logs] Loading producer state till offset 0 with message format version 2 (kafka.log.UnifiedLog$)
[2023-03-26 23:34:38,113] INFO [LogLoader partition=__cluster_metadata-0, dir=E:\tmp\kraft-combined-logs] Reloading from producer snapshot and rebuilding producer state from offset 0 (kafka.log.UnifiedLog$)
[2023-03-26 23:34:38,116] INFO [LogLoader partition=__cluster_metadata-0, dir=E:\tmp\kraft-combined-logs] Producer state recovery took 2ms for snapshot load and 0ms for segment recovery from offset 0 (kafka.log.UnifiedLog$)
[2023-03-26 23:34:38,175] INFO Initialized snapshots with IDs SortedSet() from E:\tmp\kraft-combined-logs\__cluster_metadata-0 (kafka.raft.KafkaMetadataLog$)
[2023-03-26 23:34:38,204] INFO [raft-expiration-reaper]: Starting (kafka.raft.TimingWheelExpirationService$ExpiredOperationReaper)
[2023-03-26 23:34:38,423] ERROR [SharedServer id=1] Got exception while starting SharedServer (kafka.server.SharedServer)
java.io.UncheckedIOException: Error while writing the Quorum status from the file E:\tmp\kraft-combined-logs\__cluster_metadata-0\quorum-state
        at org.apache.kafka.raft.FileBasedStateStore.writeElectionStateToFile(FileBasedStateStore.java:155)
        at org.apache.kafka.raft.FileBasedStateStore.writeElectionState(FileBasedStateStore.java:128)
        at org.apache.kafka.raft.QuorumState.transitionTo(QuorumState.java:477)
        at org.apache.kafka.raft.QuorumState.initialize(QuorumState.java:212)
        at org.apache.kafka.raft.KafkaRaftClient.initialize(KafkaRaftClient.java:369)
        at kafka.raft.KafkaRaftManager.buildRaftClient(RaftManager.scala:240)
        at kafka.raft.KafkaRaftManager.<init>(RaftManager.scala:166)
        at kafka.server.SharedServer.start(SharedServer.scala:228)
        at kafka.server.SharedServer.startForController(SharedServer.scala:128)
        at kafka.server.ControllerServer.startup(ControllerServer.scala:188)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:98)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:98)
        at scala.Option.foreach(Option.scala:437)
        at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:98)
        at kafka.Kafka$.main(Kafka.scala:115)
        at kafka.Kafka.main(Kafka.scala)
Caused by: java.nio.file.FileSystemException: E:\tmp\kraft-combined-logs\__cluster_metadata-0\quorum-state.tmp -> E:\tmp\kraft-combined-logs\__cluster_metadata-0\quorum-state: The process cannot access the file because it is being used by another process
        at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
        at java.base/sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:403)
        at java.base/sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:293)
        at java.base/java.nio.file.Files.move(Files.java:1430)
        at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:949)
        at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:932)
        at org.apache.kafka.raft.FileBasedStateStore.writeElectionStateToFile(FileBasedStateStore.java:152)
        ... 15 more
        Suppressed: java.nio.file.FileSystemException: E:\tmp\kraft-combined-logs\__cluster_metadata-0\quorum-state.tmp -> E:\tmp\kraft-combined-logs\__cluster_metadata-0\quorum-state: The process cannot access the file because it is being used by another process
                at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
                at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
                at java.base/sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:317)
                at java.base/sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:293)
                at java.base/java.nio.file.Files.move(Files.java:1430)
                at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:946)
                ... 17 more
[2023-03-26 23:34:38,430] ERROR [ControllerServer id=1] Fatal error during controller startup. Prepare to shutdown (kafka.server.ControllerServer)
java.lang.NullPointerException: Cannot invoke "kafka.raft.KafkaRaftManager.apiVersions()" because the return value of "kafka.server.SharedServer.raftManager()" is null
        at kafka.server.ControllerServer.startup(ControllerServer.scala:197)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:98)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:98)
        at scala.Option.foreach(Option.scala:437)
        at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:98)
        at kafka.Kafka$.main(Kafka.scala:115)
        at kafka.Kafka.main(Kafka.scala)
[2023-03-26 23:34:38,434] INFO [ControllerServer id=1] shutting down (kafka.server.ControllerServer)
[2023-03-26 23:34:38,435] INFO [SocketServer listenerType=CONTROLLER, nodeId=1] Stopping socket server request processors (kafka.network.SocketServer)
[2023-03-26 23:34:38,442] INFO [SocketServer listenerType=CONTROLLER, nodeId=1] Stopped socket server request processors (kafka.network.SocketServer)
[2023-03-26 23:34:38,444] INFO [SocketServer listenerType=CONTROLLER, nodeId=1] Shutting down socket server (kafka.network.SocketServer)
[2023-03-26 23:34:38,478] INFO [SocketServer listenerType=CONTROLLER, nodeId=1] Shutdown completed (kafka.network.SocketServer)
[2023-03-26 23:34:38,480] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$)
java.lang.NullPointerException: Cannot invoke "kafka.raft.KafkaRaftManager.apiVersions()" because the return value of "kafka.server.SharedServer.raftManager()" is null
        at kafka.server.ControllerServer.startup(ControllerServer.scala:197)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:98)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:98)
        at scala.Option.foreach(Option.scala:437)
        at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:98)
        at kafka.Kafka$.main(Kafka.scala:115)
        at kafka.Kafka.main(Kafka.scala)
[2023-03-26 23:34:38,488] INFO App info kafka.server for 1 unregistered (org.apache.kafka.common.utils.AppInfoParser)

Answer 1

使用管理权限运行命令，就可以了。 注意：如果是 WSL2，请使用 sudo 运行命令。

Answer 2

作为在 KRaft 模式下启动 Kafka 的一种解决方法，我建议尝试这样的配置：https://medium.com/gitconnected/kraft-kafka-cluster-with-docker-e79a97d19f2c

一旦运行，您就可以玩它了。