Filebeat可以发送日志到Fluentd吗？我需要配置文件示例

Question

我尝试使用 fluent-plugin-beats 将日志消息从 Filebeat 发送到 Fluentd，两者都在我的 Docker 桌面上运行，但 Filebeat 不断发送错误。

2023-11-08 17:02:31 2023-11-08T10:02:31.482Z    ERROR   [publisher_pipeline_output]     pipeline/output.go:154  Failed to connect to backoff(async(tcp://127.0.0.1:5044)): dial tcp 127.0.0.1:5044: connect: connection refused
2023-11-08 17:02:31 2023-11-08T10:02:31.482Z    INFO    [publisher_pipeline_output]     pipeline/output.go:145  Attempting to reconnect to backoff(async(tcp://127.0.0.1:5044)) with 11 reconnect attempt(s)

这是我的文件bat.yml

filebeat.inputs:
- type: filestream
  id: my-filestream-id
  paths:
    - /log/test/*.log
    
output.logstash:
  # The Logstash hosts
  hosts: ["127.0.0.1:5044"]

我的 Fluent.conf

<source>
  @type beats
  metadata_as_tag  
  # port 5044
  # bind localhost
</source>


<match *beat>
  @type copy
  <store>
    @type file
    path /output/test
    add_path_suffix true
    path_suffix ".txt"
  </store>
  <store>
    @type stdout
  </store>
</match> 

我尝试在 docker-compose.yml 中将端口从 Filebeat 更改为 Fluentd，这是我在 docker-compose 中的配置。

version: "3"
services:
  fluentd:
    container_name: fluentd
    build: 
      context: ./fluentd
      dockerfile: Dockerfile
    volumes:
      - ./fluentd/conf:/fluentd/etc
      - ./fluentd/buffer:/fluentd/buffer
      - ./fluentd/data:/output:rw
    ports:
      - "24224:24224"
      - "24224:24224/udp"
      - "1514:1514/udp"
      - "13543:13543/udp"
      - "5044:5044"
    networks:
      - fluent
  filebeat:
    container_name: filebeat
    build:
      context: ./filebeat
      dockerfile: Dockerfile
    volumes: 
      - ./filebeat/log:/log/test/:rw
    networks:
      - fluent
networks:
  fluent:
    driver: bridge

我尝试使用 Filebeat 而不是使用 Docker Desktop，但却收到此错误

2023-11-09T08:15:24+00:00   fluent.error    {"error":"undefined method `[]' for nil:NilClass","message":"unexpected error error=\"undefined method `[]' for nil:NilClass\""}

Answer 1

我首先尝试将日志发送到输出文件以查看它是否有效。

filebeat.inputs:
- type: filestream
  id: my-filestream-id
  paths:
    - /log/test/*.log

output.file:
  path: /output/test.log