即使导出指标，Grafana Prometheus 数据源也不显示指标

Question

Spring Boot 3.2.0，JDK 17

使用在 Docker 中运行的 Grafana Stack

放弃 JavaAgent 解决方案并使用 Spring Boot 实现手动检测来推送指标和跟踪后，我现在在收集器中看到 Prometheus 导出日志。但是，我在 Prometheus 指标查询下拉列表中没有显示任何内容。此外，普罗米修斯没有显示任何目标。我认为这很好，因为执行器被 JWT 锁定，所以我没有抓取指标。

14:13:04.035 [main] INFO  i.m.c.i.push.PushMeterRegistry - publishing metrics for OtlpMeterRegistry every 10s

收集器日志：

debug [email protected]/accumulator.go:79accumulating metric: jvm.threads.states{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: executor.active{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: executor.active{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: executor.queue.remaining{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.gc.live.data.size{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: executor.completed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.buffer.memory.used{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: spring.integration.channels{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.threads.states{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: rabbitmq.acknowledged{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.memory.used{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.buffer.count{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: logback.events{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: rabbitmq.rejected{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: tasks.scheduled.execution{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.memory.max{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.threads.live{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.classes.unloaded{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.classes.loaded{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.threads.peak{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: application.started.time{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: rabbitmq.not_acknowledged_published{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: rabbitmq.failed_to_publish{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: tomcat.sessions.rejected{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: jvm.gc.memory.promoted{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug [email protected]/accumulator.go:79accumulating metric: disk.total{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
info MetricsExporter{"kind": "exporter", "data_type": "metrics", "name": "logging", "resource metrics": 1, "metrics": 109, "data points": 109}
info ResourceMetrics #0
 Resource SchemaURL: 
 Resource attributes:
      -> telemetry.sdk.name: Str(io.micrometer)
      -> telemetry.sdk.language: Str(java)
      -> telemetry.sdk.version: Str(1.12.0)
      -> service.name: Str(host-validation)

收藏家：

exporters:
  logging:
    verbosity: detailed

  #Metrics to Prometheus
  prometheus:
    endpoint: "0.0.0.0:8889"
    enable_open_metrics: true
    const_labels:
      label1: osint
  otlp/tempo:
    endpoint: tempo:4317
    tls:
      insecure: true
  loki:
    endpoint: "http://loki:3100/loki/api/v1/push"

# The Collector pipeline.
service:

  extensions: [health_check, pprof, zpages]
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [prometheus,logging]
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/tempo,logging]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [loki,logging]

顺便说一句，我可以看到 Tempo 中的跨度和 Loki 中的日志。 我在启动时看到异常，但它似乎没有任何影响：

java.lang.Throwable: null
    at io.opentelemetry.context.ContextStorageWrappers.addWrapper(ContextStorageWrappers.java:32)
    at io.opentelemetry.context.ContextStorage.addWrapper(ContextStorage.java:89)
    at org.springframework.boot.actuate.autoconfigure.tracing.OpenTelemetryAutoConfiguration.otelCurrentTraceContext(OpenTelemetryAutoConfiguration.java:159)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)

Answer 1

普罗米修斯导出器无意用于推送指标。

已使用 https://grafana.com/docs/mimir/latest/get-started/

导出到 Mimir