ELK FileBeat - 管道未摄取

Question

我在同一个中有多个索引

filebeat.yml

即使管道测试没有抛出任何错误并且能够成功解析文档 ID，管道也没有被摄取

#=========================== Filebeat inputs =============================
filebeat.inputs:
- type: aws-cloudwatch
  enabled: true
  access_key_id: "xxxxxxxxxxxxx"
  secret_access_key: "xxxxxxxxxxxx"
  region_name: "xxxxxxxxx"
  log_group_arn: "arn:aws:logs:xxxxxxxx:apache_logs"
  log_group_name: "xxxxxxxxx/apache"
  fields:
    name: "prod-cloudwatch_apache_logs"
  ignore_older: 720h
  start_position: end
- type: aws-cloudwatch
  enabled: true
  access_key_id: "xxxxxxxxxxxxx"
  secret_access_key: "xxxxxxxxxxxx"
  region_name: "xxxxxxxxx"
  log_group_arn: "arn:aws:logs:xxxxxxxx:apache_logs"
  log_group_name: "xxxxxxxxx/apache"
  fields:
    name: "uat-cloudwatch_apache_logs"
  ignore_older: 720h
  start_position: end

output.elasticsearch:
  hosts: ["https://xxxxxxx.aws.elastic-cloud.com:xxxxxx"]
  protocol: "https"
  username: "${ESUSER}"
  password: "${ESPASS}"
  output.elasticsearch.allow_older_versions: true
  indices:
   - index: "cloudwatch_uat_apache_logs-%{+yyyy.MM}"
     when.contains:
      fields.name: "uat-cloudwatch_apache_logs"
   - index: "cloudwatch_prod_apache_logs%{+yyyy.MM}"
     when.contains:
      fields.name: "prod-cloudwatch_apache_logs"
  pipelines:
    - pipeline: "apache_pipeline"
      when:
       has_fields: ['name.prod-cloudwatch_apache_logs']

我的管道未运行（字段在 kibana 上不可见）

修复配置问题

Answer 1

您的配置中有 2 个小问题。

output.elasticsearch.allow_older_versions: true

应该只是

allow_older_versions: true

因为它已经在output.elasticsearch块下。

第二个问题（可能不是主要问题）是一个小问题。缺少一个破折号。

  - index: "cloudwatch_prod_apache_logs%{+yyyy.MM}"

  - index: "cloudwatch_prod_apache_logs-%{+yyyy.MM}"