在 promtail-loki-grafana 中使用 logfmt 时间的问题

Question

我在 loki 中跟踪注册日期有问题，我想使用日志日期 (logfmt)，在 grafana 中查看它时，我看到了上传日期。

docker.compose

version: "3"

networks:
  tracing:

services:
  grafana:
    container_name: grafana-2
    image: grafana/grafana:10.4.1
    volumes:
      - './run/resources/datasource.yml:/etc/grafana/provisioning/datasources/datasource.yaml'
      - './dockerVolumes/grafana_data:/var/lib/grafana'
    environment:
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
      - TZ=UTC
    ports:
      - "3000:3000"
    networks:
      - tracing

  loki:
    container_name: loki
    image: grafana/loki:2.9.6
    volumes:
      - './dockerVolumes/loki_data:/loki'
    environment:
      - TZ=UTC
    ports:
      - "3100:3100"
    networks:
      - tracing

  promtail:
    container_name: promtail
    image: grafana/promtail:2.9.6
    volumes:
      - './run/resources/promtail-config.yml:/etc/promtail/config.yml'
      - './run/logs:/var/log'
    command: -config.file=/etc/promtail/config.yml
    environment:
      - TZ=UTC
    networks:
      - tracing

配置promtail

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
  - job_name: od
    static_configs:
      - targets:
          - localhost
        labels:
          job: od_logs
          __path__: /var/log/od/*.log
    relabel_configs:
      - source_labels: ['__path__']
        regex: '/var/log/od/(.*)\.log'
        action: replace
        target_label: logfilename
        replacement: '${1}'
    pipeline_stages:
      - logfmt:
          mapping:
            timestamp: time
      - timestamp:
          source: time
          format: RFC3339

配置洛基

auth_enabled: false

server:
  http_listen_port: 3100

ingester:
  max_transfer_retries: 0
  rate_limit_bytes: 8388608
  chunk_idle_period: 1h
  chunk_retain_period: 30s
  lifecycler:
    address: localhost
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
  client:
    backoff_config:
      max_period: 5s


schema_config:
  configs:
    - from: 2020-05-15
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h

storage_config:
  boltdb_shipper:
    active_index_directory: /loki/index
    cache_location: /loki/index_cache
    cache_ttl: 24h
    shared_store: filesystem
  filesystem:
    directory: /loki/chunks
    index_queries_cache_config:
      cache:
        max_size: 50000
        item_size_bytes: 5000
        max_age: 1h
    index_queries_cache:
      enabled: true

compactor:
  working_directory: /loki/compactor
  shared_store: filesystem

limits_config:
  enforce_metric_name: false
  reject_old_samples: false
  reject_old_samples_max_age: 999999h 
  unordered_writes: true

chunk_store_config:
  max_look_back_period: 0s

table_manager:
  retention_deletes_enabled: false
  retention_period: 0s

示例日志

time=2024-04-04T18:15:55.434 level=trace thread=3214 origin=libOD msg="9000"
time=2012-11-01T22:08:41+00:00 level=trace thread=3214 origin=libOD msg="9000"
time=2024-04-04T18:15:55.434 level=trace thread=3214 origin=libOD msg="event processed successfully"

问题时间grafana

https://grafana.com/docs/loki/latest/send-data/promtail/stages/logfmt/

https://grafana.com/docs/loki/latest/send-data/promtail/installation/

https://grafana.com/docs/loki/latest/send-data/promtail/troubleshooting/

Answer 1

时间戳阶段在提取的映射中查找字段。但是，在提取的映射中只能找到时间戳字段。所以，你可以试试这个方法。

    pipeline_stages:
      - logfmt:
          mapping:
            time:
      - timestamp:
          source: time
          format: RFC3339