如何在GKE中使用CronJob中的kubectl？

Question

[我在GKE群集中部署了一个CronJob以定期复制名称空间中的机密（对于cert-manager，但是我总是收到以下错误：

The connection to the server localhost:8080 was refused - did you specify the right host or port?

这是我的部署：

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: certificate-replicator-cron-job
  namespace: default
spec:
  jobTemplate:
    spec:
      template:
        metadata:
          labels:
            app: default
            release: default
        spec:
          automountServiceAccountToken: false
          containers:
          - command:
            - /bin/bash
            - -c
            - for i in $(kubectl get ns -o json |jq -r ".items[].metadata.name" |grep
              "^bf-"); do kubectl get secret -o json --namespace default dev.botfront.cloud-staging-tls
              --export |jq 'del(.metadata.namespace)' |kubectl apply -n ${i}-f -;  done
            image: bitnami/kubectl:latest
            name: certificate-replicator-container
          restartPolicy: OnFailure
          serviceAccountName: sa-certificate-replicator
  schedule: '* * * * *'

我还为服务帐户设置了角色：

$ kubectl describe role certificate-replicator-role                                                                                                                                 
Name:         certificate-replicator-role
Labels:       <none>
Annotations:  <none>
PolicyRule:
  Resources   Non-Resource URLs  Resource Names  Verbs
  ---------   -----------------  --------------  -----
  secrets     []                 []              [list create get]
  namespaces  []                 []              [list get]

$ kubectl describe rolebinding certificate-replicator-role-binding                                                                                                                  git:(master|✚4…
Name:         certificate-replicator-role-binding
Labels:       <none>
Annotations:  <none>
Role:
  Kind:  Role
  Name:  certificate-replicator-role
Subjects:
  Kind            Name                       Namespace
  ----            ----                       ---------
  ServiceAccount  sa-certificate-replicator  default

$ kubectl describe serviceaccount sa-certificate-replicator                                                                                                                         git:(master|✚4…
Name:                sa-certificate-replicator
Namespace:           default
Labels:              <none>
Annotations:         <none>
Image pull secrets:  <none>
Mountable secrets:   sa-certificate-replicator-token-ljsfb
Tokens:              sa-certificate-replicator-token-ljsfb
Events:              <none>

[我知道我可能会创建另一个预安装gcloud的Docker映像，并使用服务帐户密钥进行身份验证，但是我想与云提供商无关，并且也避免了由于kubectl正在向集群进行身份验证从内部调用。

有可能吗？

Answer 1

Gcloud要求您以某种方式进行身份验证。每当我想远程运行kubectl时，我都会使用.json文件来验证google-cloud的服务帐户。但是，这是一个非常肮脏的解决方案。

相反，我建议使用kubernetes api实现您的目标。创建一个角色，使您可以在名称空间和configmaps资源上进行操作。将其与服务帐户关联，然后卷曲以从cronjob内部进行复制。

这里是默认名称空间的示例。

首先创建一个角色，并将其与您的服务帐户关联（此示例中为默认值。）>

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nssc-clusterrole
  namespace: default
rules:
- apiGroups: [""]
  resources: ["namespaces", "configmaps", "secrets"]
  verbs: ["*"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: nssc-clusterrolebinding
  namespace: default
roleRef:
  name: nssc-clusterrole
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
subjects:
  - name: default
    namespace: default
    kind: ServiceAccount
第二，创建一个秘密进行测试。

---
apiVersion: v1
kind: Secret
metadata:
  name: secrets-test
  namespace: default
type: Opaque
stringData:
  mysecret1: abc123
  mysecret2: def456
第三，请求卷曲以获取秘密。

curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt -H "Authorization: Bearer $(cat /var/run/secrets/kubernet
es.io/serviceaccount/token)" -H "Accept: application/json"  https://kubernetes.default.svc/api/v1/namespaces/default/secrets/sec
rets-test
您将获得包含您的机密内容的json。 

{
  "kind": "Secret",
  "apiVersion": "v1",
  "metadata": {
    "name": "secrets-test",
    "namespace": "default",
    "selfLink": "/api/v1/namespaces/default/secrets/secrets-test",
    "uid": "...",
    "resourceVersion": "...",
    "creationTimestamp": "2019-10-26T01:52:29Z",
    "annotations": {
      "kubectl.kubernetes.io/last-applied-configuration": "{...}\n"
    }
  },
  "data": {
    "mysecret1": "base64value",
    "mysecret2": "base64value"
  },
  "type": "Opaque"
}
[第四，通过更改json并提出新的curl请求，在新的名称空间中创建机密。还要将服务帐户与角色相关联。

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: nssc-clusterrolebinding
  namespace: new-namespace
roleRef:
  name: nssc-handler-clusterrole
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
subjects:
  - name: default
    namespace: default
    kind: ServiceAccount
{
    "apiVersion": "v1",
    "data": {
        "mysecret1": "Y29udHJvbDEyMyE=",
        "mysecret2": "Y29udHJvbDQ1NiE="
    },
    "kind": "Secret",
    "metadata": {
        "name": "secrets-test",
        "namespace": "new-namespace"
    },
    "type": "Opaque"
}
curl -X POST -d @test.json --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt -H "Authorization: Bearer $(cat /va
r/run/secrets/kubernetes.io/serviceaccount/token)" -H "Accept: application/json" -H "Content-Type: application/json" https://kub
ernetes.default.svc/api/v1/namespaces/new-namespace/secrets