Consul 连接注入器无法连接到我的 consul 服务器

Question

如果我使用 consul helm 图表将 consul 客户端的实例部署到我的 k3s 集群中。连接注射器 pod 将无法正常启动。

k3s版本：k3s版本v1.25.6+k3s1（9176e03c）
Helm版本：version.BuildInfo{版本：“v3.12.1”，GitCommit：“f32a527a060157990e2aa86bf45010dfb3cc8b8d”，GitTreeState：“clean”，GoVersion：“go1.20.4”}
Terraform 版本：Terraform v1.5.2
领事版本：v1.16
Docker compose 版本：Docker Compose 版本 v2.19.1
Docker 版本：

Client: Docker Engine - Community
 Version:           24.0.4
 API version:       1.43
 Go version:        go1.20.5
 Git commit:        3713ee1
 Built:             Fri Jul  7 14:50:55 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.4
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.5
  Git commit:       4ffc614
  Built:            Fri Jul  7 14:50:55 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

这是我的 consul server.hcl 文件

server           = true
bootstrap        = false
bootstrap_expect = 1

node_name  = "dev-consul"
datacenter = "dev-dc"

encrypt                 = "<encrypt-key>"
encrypt_verify_incoming = true
encrypt_verify_outgoing = true

tls {
  defaults {
    ca_file         = "/consul/config/certs/consul-agent-ca.pem"
    cert_file       = "/consul/config/certs/dev-dc-server-consul-0.pem"
    key_file        = "/consul/config/certs/dev-dc-server-consul-0-key.pem"
    verify_incoming = true
    verify_outgoing = true
  }
}

data_dir  = "/consul-data"
log_level = "INFO"

advertise_addr = "<advertise-addr>"
bind_addr      = "0.0.0.0"

addresses = {
  "http" = "0.0.0.0"
}

auto_encrypt = {
  "allow_tls" = true
}

connect = {
  "enabled" = true
}

ui_config = {
  "enabled" = true
}

我正在使用这个 docker compose 文件来部署我的服务器节点：

version: "3.9"

services:
  consul:
    image: hashicorp/consul:1.16
    volumes:
      - ./config/server.hcl:/consul/config/server.hcl:ro
      - consul_data:/titanium/consul-data
      - ./certs:/consul/config/certs/
    ports:
      - "8600:8600/tcp"
      - "8600:8600/udp"
      - "8500:8500/tcp"
      - "8500:8500/udp"
      - "8301:8301/tcp"
      - "8301:8301/udp"
      - "8302:8302/tcp"
      - "8302:8302/udp"
      - "8502:8502"
      - "21000-21255:21000-21255"
      - "8300:8300"
      - "8300:8300/udp"
    command: "agent"

volumes:
  consul_data:

目前consul服务器运行良好。当我使用带有以下值的领事舵图表时.yml

global:
  name: consul
  image: hashicorp/consul:1.16
  domain: dev.local
  datacenter: dev-dc
  exposeGossipPorts: true
  gossipEncryption:
    secretName: "gossip-encryption-key-secret"
    secretKey: "key"
  tls:
    enabled: true
    enableAutoEncrypt: true
    verify: true
    caCert:
      secretName: "consul-certs"
      secretKey: "ca.pem"
connectInject:
  enabled: true
  default: true
  cni:
    enabled: true
    logLevel: info
    cniBinDir: "/opt/cni/bin"
    cniNetDir: "/etc/cni/net.d"
  namespaceSelector: |
    matchLabels:
      connect-inject : enabled
  failurePolicy: "Ignore"
server:
  enabled: false
client:
  enabled: true
  join: [ "<consul-service-addr>" ]
  grpc: true

使用以下值，consul 客户端将连接到集群，但包含 consul-connect-injector 的 pod 将给出以下错误消息

2023-07-23T19:23:44.851Z [INFO]  consul-server-connection-manager: trying to connect to a Consul server 2023-07-23T19:23:44.854Z [ERROR] consul-server-connection-manager: connection error: error="failed to discover Consul server addresses: failed to resolve DNS name: consul-server.consul.svc: lookup consul-server.consul.svc on 10.43.0.10:53: no such host

如果我检查 pod 的 kubernetes 事件，我会收到如下错误：

MountVolume.SetUp failed for volume "consul-ca-cert

MountVolume.SetUp failed for volume "certs" : secret "consul-connect-inject-webhook-cert" not found

。

我通过 kubernetes 资源文件创建 gossip 加密密钥和 tls 证书的秘密。

我认为没有必要知道，但这是我使用的 terraform 脚本：

provider "helm" {
  kubernetes {
    config_path = "~/.kube/config"
  }
}

resource "helm_release" "consul-connect" {
  name       = "consul-connect"
  chart      = "consul"
  repository = "https://helm.releases.hashicorp.com"
  namespace  = "consul"

  values = [file("config/values.yml")]
}

我尝试多次修改我的values.yml，但连接注入器pod一遍又一遍地打印相同的错误消息。我还尝试使用简化的values.yml

global:
  name: consul
  domain: dev.local
  datacenter: dev-dc
  gossipEncryption:
    secretName: "gossip-encryption-key-secret"
    secretKey: "key"
  tls:
    enabled: true
    enableAutoEncrypt: true
    verify: true
    caCert:
      secretName: "consul-certs"
      secretKey: "ca.pem"
connectInject:
  enabled: true
  failurePolicy: "Ignore"
controller:
  enabled: true
server:
  enabled: false
client:
  enabled: true
  image: hashicorp/consul:1.15.3
  join: [ "<consul-server-addr>" ]

但即便如此也没有改变任何事情。我还尝试通过 helm 图表安装 consul 服务器，但它没有解决我的 connect-injector-pod 问题。

Answer 1

看起来它试图访问您的配置中禁用的 consul-server.consul.svc。

我会尝试启用外部服务器并将

externalServers.hosts

设置为与

client.join

相同的值