console-producer连接到节点kafka1：9092（org.apache.kafka.clients.NetworkClient）java.net.UnknownHostException

Question

我正在通过docker-compose用kafka启动一个容器。我的意思是，我可以成功登录这样的kafka容器，然后直接从那里产生和使用消息。我也可以看到另一个容器（filebeat），具体取决于成功连接的kafka容器。

docker-compose.yml

version: '3.2'
services:

  zoo1:
    image: elevy/zookeeper:latest
    environment:
      MYID: 1
      SERVERS: zoo1
    ports:
      - "2181:2181"

  kafka1:
    image: wurstmeister/kafka
    command: [start-kafka.sh]
    depends_on:
      - zoo1
    links:
      - zoo1
    ports:
      - "9092:9092"
    environment:
      KAFKA_LISTENERS: PLAINTEXT://:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092
      KAFKA_BROKER_ID: 1
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_LOG_RETENTION_HOURS: "168"
      KAFKA_LOG_RETENTION_BYTES: "100000000"
      KAFKA_ZOOKEEPER_CONNECT:  zoo1:2181
      KAFKA_CREATE_TOPICS: "log:1:1"
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'

  filebeat:
    image: docker.elastic.co/beats/filebeat:7.5.2
    command: filebeat -e -strict.perms=false
    volumes:
      - "//c/Users/Cast/megalog-try-1/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro"
      - "//c/Users/Cast/megalog-try-1/sample-logs:/sample-logs"
    links:
      - kafka1
    depends_on:
      - kafka1

但是，当我尝试从本地kafka生成一个非常简单的消息到该容器kafka时，我拒绝连接。

我发现了一个例外情况相同但情景相似的人。我尝试his solution失败。基本上，它指向“将advertised.listeners调整为外部IP”。我是在docker-compose中完成的，但没有任何改变。即使这个方法行得通，我想这也不是最终的解决方案，因为对我而言，这是因为他正在KAFKA_ADVERTISED_LISTENERS]中对他的计算机IP地址进行硬编码。

所以我的问题是如何从运行Windows 10的本地kafka连接到kafka容器以发送消息？

这里是我到目前为止尝试过的一切：

1-尝试使用本地ip

ipconfig
...
Endereço IPv4. . . . . . . .  . . . . . . . : 192.168.129.97

kafka-console-producer.bat --broker-list 192.168.129.97:9092 --topic log

C:\tools\kafka\bin\windows>kafka-console-producer.bat --broker-list 192.168.129.97:9092 --topic log
>[2020-03-02 16:54:44,155] WARN [Producer clientId=console-producer] Connection to node -1 (/192.168.129.97:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-03-02 16:54:46,212] WARN [Producer clientId=console-producer] Connection to node -1 (/192.168.129.97:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
2-尝试使用docker machine ip

C:\tools\kafka\bin\windows>docker-machine ip
192.168.99.100

kafka-console-producer.bat --broker-list 192.168.99.100:9092 --topic log


[2020-03-02 17:00:22,500] WARN [Producer clientId=console-producer] Error connecting to node kafka1:9092 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: kafka1
        at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797)
        at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
        at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1364)
        at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
        at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:104)
        at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:403)
        at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:363)
        at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:151)
        at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:955)
        at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:69)
        at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1126)
        at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1017)
        at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:538)
        at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:335)
        at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)
        at java.base/java.lang.Thread.run(Thread.java:834)
3-尝试使用容器ip

docker ps
...
3eb114e24b53        wurstmeister/kafka                                    "start-kafka.sh"      0.0.0.0:9092->9092/tcp                                     megalog-try-1_kafka1_1

docker inspect 3eb114e24b53
...
"IPAddress": "172.18.0.4",

C:\tools\kafka\bin\windows>kafka-console-producer.bat --broker-list 172.18.0.4:9092 --topic log
>testing
[2020-03-02 16:57:54,471] WARN [Producer clientId=console-producer] Connection to node -1 (/172.18.0.4:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
org.apache.kafka.common.KafkaException: Producer closed while send in progress
        at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:888)
***在CRU的建议之后添加

C:\tools\kafka\bin\windows>docker exec -it megalog-try-1_zoo1_1 bash
bash-4.3# ls
bin            entrypoint.sh  home           lib64          mnt            root           sbin           sys            usr            zookeeper
dev            etc            lib            media          proc           run            srv            tmp            var
bash-4.3# cd zookeeper/
bash-4.3# ls
bin                   conf                  data                  lib                   log                   wal                   zookeeper-3.4.10.jar
bash-4.3# cd conf
bash-4.3# ls
log4j.properties  zoo.cfg
bash-4.3# cat zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/zookeeper/data
dataLogDir=/zookeeper/wal
#snapCount=100000
autopurge.purgeInterval=1
clientPort=2181
quorumListenOnAllIPs=true
server.1=zoo1:2888:3888bash-4.3#
和来自kafka容器

C:\tools\kafka\bin\windows>docker exec -it megalog-try-1_kafka1_1 bash
bash-4.4# pwd
/
bash-4.4# ls
bin    dev    etc    home   kafka  lib    lib64  media  mnt    opt    proc   root   run    sbin   srv    sys    tmp    usr    var
bash-4.4# cd kafka
bash-4.4# ls
kafka-logs-0f385d1d435e
bash-4.4# cd kafka-logs-0f385d1d435e/
bash-4.4# ls
app_logs-0                        log-0                             meta.properties                   replication-offset-checkpoint
cleaner-offset-checkpoint         log-start-offset-checkpoint       recovery-point-offset-checkpoint  request_logs-0
bash-4.4# cat meta.properties
#
#Mon Mar 02 21:09:33 GMT 2020
cluster.id=-qitBVmjSUGo7Zd7P5cetw
version=0
broker.id=1
bash-4.4#
添加了***

在偶然发现this article之后，我可以编写一个docker-compose来运行我的本地SpringBoot可以发送消息的kafka容器。不过，我必须在docker-compose中手动对docker-machine ip进行硬编码。好吧，我认为这不是唯一的方法，否则每次启动Docker ToolBox时，我都必须运行docker-machine ip并替换IP地址。我想知道是否有更友好的解决方案。

下面的192.168.99.100是我的docker-machine ip。

version: "3.2"
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENERS:
        "INTERNAL://kafka:9090,\
         EXTERNAL://:9092"
      KAFKA_ADVERTISED_LISTENERS:
        "INTERNAL://kafka:9090,\
         EXTERNAL://192.168.99.100:9092"
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP:
        "INTERNAL:PLAINTEXT,\
         EXTERNAL:PLAINTEXT"
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
基于Cricket_007的建议，我仔细阅读了another article，但得出的结论是我必须使用DNS解析，但据我所知，我无法在Windows计算机中设置专有的dns解析。

我正在通过docker-compose用kafka启动一个容器。我的意思是，我可以成功登录这样的kafka容器，然后直接从那里产生和使用消息。我也可以看到...

Answer 1

1. KAFKA_ADVERTISED_PORT已过时。只需使用公告的侦听器即可。