Azure 应用服务证书 ssl 到 AKS 入口

问题描述 投票：0回答：2

我已从 Azure 应用服务证书购买了通配符 ssl 证书。我还有一个 AKS 集群。我想把它放在秘密中并在入口中使用。购买后，它将秘密文件存储在 Azure Key Vault 中。我下载了它，然后导入以创建 Azure Key Vault 证书。然后，我使用 akv2k8s 在 AKS 中创建了一个秘密文件，并在 ingress 中使用它。在我的应用程序抛出“err_cert_authority_invalid”错误之后。 我做错了什么吗？ 关于 ssl 和 ingress 的文档并不多。在许多文章中，他们使用“让加密”或“证书管理器”。

https://akv2k8s.io/

kubernetes kubernetes-ingress azure-aks azure-keyvault
2个回答
2
投票

• 这可能是由于误解证书是由临时环境颁发的，反之亦然。因此，为此目的，我建议您检查 ‘stable/wordpress’ helm 图表，其中包含入口注释 'certmanager.k8s.io/cluster-issuer': 'letsencrypt-staging'。这将导致伪造的颁发者颁发证书。因此，即使您的证书作为机密进入 AKS，它也会显示为由虚假颁发者颁发，因为证书哈希验证链在中间被破坏。请在下面找到用于此目的的卷曲：-

   ‘ # curl -vkI https://blog.my-domain.com/
     ...
     * Server certificate:
     *  subject: CN=blog.my-domain.com
     *  start date: May 13 08:51:13 2019 GMT
     *  expire date: Aug 11 08:51:13 2019 GMT
     *  issuer: CN=Fake LE Intermediate X1
     ... ‘

然后，列出入口如下：-

  ‘ # kubectl get ing
    NAME             HOSTS                              ADDRESS          PORTS     AGE
    blog-wordpress   blog.my-domain.com   35.200.214.186   80, 443   8m48s ’

还有证书：-

  ‘ # kubectl get certificates
    NAME                  READY   SECRET                AGE
    wordpress.local-tls   True    wordpress.local-tls   9m ’

然后，将证书的颁发者切换为最初颁发证书的颁发者，如下所示：-

   ‘ # kubectl edit ing blog-wordpress ’

并更新注释如下：-

  ‘ certmanager.k8s.io/cluster-issuer: letsencrypt-prod ’

一旦更新入口清单，证书清单将自动更新。要验证它，请打开 ‘wordpress.local-tls’ 证书资源的清单，如下所示：-

 ‘ kubectl edit certificate wordpress.local-tls ’

发行人将被视为更新如下：-

‘ kubectl edit certificate wordpress.local-tls ’

因此，通过这种方式，您将能够在 AKS 中导入证书机密。如需了解更多详情，我建议您参考以下链接了解更多详情：-

https://github.com/vmware-archive/kube-prod-runtime/issues/532

0
投票

我在通配符证书方面遇到了同样的问题，我考虑了不同的方法：

  1. 拥有证书的部署管道来在 kubernetes 中创建密钥
  2. 拥有带有 
    null_resource
    的 terraform，它将等待配置证书
  3. 将证书配置嵌入到 nginx 入口的部署中

我对解决方案 1 和 2 不满意，因为就我而言，我不知道证书何时会在 Azure 端准备就绪。

因此，为了使其与 Nginx 同时部署，我结合使用了 SecretProviderClass 和 Helm 预同步挂钩。

  1. 我创建一个 SecretProviderClass 来从 Azure Key Vault 挂载证书
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: az-kv-domain-certificate
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "false"
    clientID: {{ .Values.azureKVCSI.clientId }}
    keyvaultName: {{ .Values.azureKVCSI.keyVaultName }}       # Set to the name of your key vault
    objects:  |
      array:
        - |
          objectName: {{ .Values.azureKVCSI.secretName }}
          objectType: secret              # object types: secret, key, or cert
          objectVersion: ""               # [OPTIONAL] object versions, default to latest if empty
    tenantId: {{ .Values.azureKVCSI.tenantId }}
  secretObjects:
    - secretName: {{ .Release.Name }}-ingress-nginx-domain-certificate
      type: Opaque
      data:
        - objectName: wildcard-cert
          key: cert.pkcs12
  1. 我创建了一个与此类似的作业，以从 CSI 获取证书，将其转换并创建 kubernetes 密钥
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  annotations:
    "helm.sh/hook": pre-install,pre-upgrade
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
    "helm.sh/hook-weight": "-5"
  name: ingress-nginx-wildcard-reload-role
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - list
  - watch
  - create
  - update
  - patch
  - delete
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ingress-nginx-wildcard-reload-rolebinding
  annotations:
    "helm.sh/hook": pre-install,pre-upgrade
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
    "helm.sh/hook-weight": "-5"
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-wildcard-reload-role
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: {{ .Release.Namespace }}
---
apiVersion: v1
kind: Secret
metadata:
  name: ingress-nginx-wildcard-reload-token
  annotations:
    kubernetes.io/service-account.name: ingress-nginx
    "helm.sh/hook": pre-install,pre-upgrade
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
    "helm.sh/hook-weight": "-5"
type: kubernetes.io/service-account-token
---
apiVersion: batch/v1
kind: Job
metadata:
  name: {{ include "ingress-nginx.fullname" . }}-wildcard-reload
  namespace: {{ .Release.Namespace }}
  annotations:
    "helm.sh/hook": pre-install,pre-upgrade
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
  labels:
    {{- include "ingress-nginx.labels" . | nindent 4 }}
    app.kubernetes.io/component: admission-webhook
    azure.workload.identity/use: "true"
spec:
{{- if .Capabilities.APIVersions.Has "batch/v1alpha1" }}
  # Alpha feature since k8s 1.12
  ttlSecondsAfterFinished: 0
{{- end }}
  template:
    metadata:
      name: {{ include "ingress-nginx.fullname" . }}-wildcard-reload
      labels:
        {{- include "ingress-nginx.labels" . | nindent 8 }}
        app.kubernetes.io/component: admission-webhook
    spec:
      containers:
        - name: create
          image: todo # put here an image containing kubectl and openssl
          command:
          - "/bin/bash"
          - "-c"
          args:
          - |
            #!/bin/bash
            set -e

            IN_FILE="/mnt/azure-kv-cert-csi/wildcard-pprd"
            mkdir /tmp/certs 
            cp "${IN_FILE}" /tmp/certs
            cd /tmp/certs
            base64 -d "${IN_FILE}" > "/tmp/certs/wildcard-pprd_base64_decoded"
            IN_FILE="/tmp/certs/wildcard-pprd_decoded"

            echo "Converting the certificate from PFX to KEY and CRT..."
            echo "" > pfx_pass
            echo "0000" > pfx_pass_in_out
            IMPORT_PASSWORD="file:pfx_pass"
            PASSPH="file:pfx_pass_in_out"
            openssl pkcs12 -in "${IN_FILE}" -nocerts -out "${IN_FILE}_encrypted.key" -password $IMPORT_PASSWORD -passin $PASSPH -passout $PASSPH && \
            openssl rsa -in "${IN_FILE}_encrypted.key" -passin $PASSPH | sed -ne '/-BEGIN PRIVATE KEY-/,/-END PRIVATE KEY-/p' > "${IN_FILE}.key" && echo "OK: Converted to KEY"
            openssl pkcs12 -in "${IN_FILE}" -clcerts -nokeys -password $IMPORT_PASSWORD -passin $PASSPH -passout $PASSPH | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > "${IN_FILE}.crt" && echo "OK: Converted to CRT"


            # configure kubectl 
            kubectl config set-cluster local --server=https://kubernetes.default --certificate-authority=/kube-api-ca/ca.crt
            kubectl config set-credentials ingress-nginx --token="${KUBE_SA_TOKEN}" --certificate-authority=/kube-api-ca/ca.crt
            kubectl config set-context local --cluster=local --user=ingress-nginx --namespace=nginx-ingress-controller

            echo "Creating the kubernetes secret..."
            kubectl create secret tls default-tls-test --key="${IN_FILE}.key" --cert="${IN_FILE}.crt"
            echo "Done, exiting..."

          env:
          - name: KUBE_SA_TOKEN
            valueFrom:
              secretKeyRef:
                name: ingress-nginx-wildcard-reload-token
                key: token
          volumeMounts:
          - mountPath: /mnt/azure-kv-cert-csi
            name: azure-kv-cert-csi
            readOnly: true
          - mountPath: "/kube-api-ca"
            name: kube-ca
            readOnly: true
      restartPolicy: OnFailure
      serviceAccountName: ingress-nginx
      volumes:
      - name: kube-ca
        secret: 
          secretName: ingress-nginx-wildcard-reload-token
          items:
          - key: ca.crt
            path: ca.crt
      - csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: az-kv-domain-certificate
        name: azure-kv-cert-csi
  1. 然后您可以通过在 nginx 的 helm 图表中修改此证书来在 Nginx 中使用该证书作为默认证书：
    extraArgs:
      default-ssl-certificate: 'nginx-ingress-controller/default-tls'

作为补充，我建议也将其添加为 cronjob，这样您就可以确保该作业每隔一段时间运行一次，并且在证书到期时不会感到意外。

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.