Filebeat 未从使用 co.elastic.logs/enabled: "true" 注释的 Pod 收集日志

Question

我正在尝试在 kubernetes 上设置 ELK 堆栈。为了从我们的 pod 收集日志，我为 filebeat 设置基于提示的自动发现，然后根据文档用

co.elastic.logs/enabled: "true"

注释 pod。我可以看到 Pod 已被注释并且正在生成日志。但这些日志不是由 filebeat 收集的。这是到目前为止我的配置。

# Setup Logstash.
cat <<EOF | kubectl apply -f -
apiVersion: logstash.k8s.elastic.co/v1alpha1
kind: Logstash
metadata:
  name: jaegerpoc-elastic
  namespace: elastic-system
spec:
  count: 1
  elasticsearchRefs:
    - name: jaegerpoc-elastic
      clusterName: jes
  version: 8.9.1
  pipelines:
    - pipeline.id: main
      config.string: |
        input {
          beats {
            port => 5044
          }
        }
        output {
          elasticsearch {
            hosts => "http://jaegerpoc-elastic-es-http.elastic-system.svc:9200"
            user => "${JES_ES_USER}"
            password => "${JES_ES_PASSWORD}"
            index => "logs"
          }
        }
  services:
    - name: beats
      service:
        spec:
          type: ClusterIP
          ports:
            - port: 5044
              name: "filebeat"
              protocol: TCP
              targetPort: 5044
EOF

# Setup FileBeat
cat <<EOF | kubectl apply -f -
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: jaegerpoc-elastic
  namespace: elastic-system
spec:
  type: filebeat
  version: 8.9.1
  elasticsearchRef:
    name: jaegerpoc-elastic
  config:
    filebeat.autodiscover:
      providers:
        - type: kubernetes
          node: "minikube"
          hints:
            enabled: true
            default_config:
              enabled: false
              type: container
              paths:
                - "/var/log/containers/*\${data.kubernetes.container.id}.log"
    output.elasticsearch:
      enabled: false
    output.logstash:
      hosts: ["jaegerpoc-elastic-ls-beats.elastic-system.svc.cluster.local:5044"]
  daemonSet:
    podTemplate:
      spec:
        serviceAccountName: jaegerpoc-elastic-beat-sa
        automountServiceAccountToken : true
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        securityContext:
          runAsUser: 0
        containers:
        - name: filebeat
          volumeMounts:
          - name: varlogcontainers
            mountPath: /var/log/containers
          - name: varlogpods
            mountPath: /var/log/pods
          - name: varlibdockercontainers
            mountPath: /var/lib/docker/containers
        volumes:
        - name: varlogcontainers
          hostPath:
            path: /var/log/containers
        - name: varlogpods
          hostPath:
            path: /var/log/pods
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
EOF

现在，如果我使用容器类型的输入配置设置 filebeat，我可以看到来自所有 pod 的日志，而不是自动发现。

filebeat.inputs:
    - type: container
      paths:
      - /var/log/containers/*.log

如果我遗漏了什么，请告诉我。

Answer 1

尝试使用这种格式。

   filebeat.autodiscover:
      providers:
        - type: kubernetes
          in_cluster: true
          node: "${NODE_NAME}"
          hints.enabled: true
          hints.default_config:
            type: container
            paths:
              - /var/log/pods/*/*.log
    
    output.logstash:
      hosts: ["logstash-host:5044"]

如果不起作用，请尝试以下方法。

filebeat.autodiscover:
  providers:
    - type: kubernetes
      in_cluster: true
      node: "${NODE_NAME}"
      templates:
        - condition:
            equals:
              kubernetes.labels.app: "nginx"
          config:
            - type: container
              paths:
                - /var/log/pods/*/*.log

output.logstash:
  hosts: ["logstash-host:5044"]

让我们看看会发生什么。

参考： https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover-hints.html