kubernetes 上的 Spring Cloud Data Flow jar 任务

Question

如何在 Kubernetes 中运行的 SCDF 中将任务作为 Java jar 文件运行？

为了确定启动平台，我使用以下配置图：

apiVersion: v1
kind: ConfigMap
metadata:
  name: scdf-server
  labels:
    app: scdf-server
data:
  application.yaml: |-
    spring:
      cloud:
        dataflow:
          task:
            platform:
              kubernetes:
                accounts:
                  default:
                    limits:
                      memory: 2048Mi

在这种情况下，Java 文件作为 Docker 容器在命名空间中启动，但它们不是。

我尝试使用此配置图确定本地平台：

apiVersion: v1
kind: ConfigMap
metadata:
  name: scdf-server
  labels:
    app: scdf-server
data:
  application.yaml: |-
    spring:
      cloud:
        dataflow:
          task:
            platform:
              local:
                accounts:
                  default:
                    javaOpts: "-Xdebug -Xmx2048m"

没有帮助。

我收到错误：

kubelet  Failed to apply default image tag "/opt/dataflow/predefined_tasks/scdf_jdbc_exec_query-0.0.4.jar": couldn't parse image name "/opt/dataflow/predefined_tasks/scdf_jdbc_exec_query-0.0.4.jar": invalid reference format

文件 scdf-deployment.yaml：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: scdf
spec:
  selector:
    matchLabels:
      run: scdf
  replicas: 1
  template:
    metadata:
      labels:
        run: scdf
    spec:
      securityContext:
        runAsNonRoot: false
        runAsUser: 0
      containers:
        - name: scdf
          image: springcloud/spring-cloud-dataflow-server:2.11.1-jdk17
          resources:
            limits:
              memory: 10240Mi
            requests:
              memory: 10240Mi
          ports:
            - containerPort: 9393
          env:
            - name: JDK_JAVA_OPTIONS
              value: "-Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -Xmx8192m"
            - name: SPRING_CLOUD_DATAFLOW_APPLICATIONPROPERTIES_TASK_SPRING_CLOUD_TASK_CLOSECONTEXTENABLED
              value: "true"
            - name: SPRING_CLOUD_SKIPPER_CLIENT_SERVER_URI
              value: "http://skipper-server:7577/api"
            - name: SPRING_CLOUD_DATAFLOW_APPLICATIONPROPERTIES_STREAM_SPRING_CLOUD_STREAM_KAFKA_BINDER_BROKERS
              value: "PLAINTEXT://kafka-broker:9092"
            - name: SPRING_CLOUD_DATAFLOW_APPLICATIONPROPERTIES_STREAM_SPRING_CLOUD_STREAM_KAFKA_STREAMS_BINDER_BROKERS
              value: "PLAINTEXT://kafka-broker:9092"
            - name: SPRING_CLOUD_DATAFLOW_APPLICATIONPROPERTIES_STREAM_SPRING_CLOUD_STREAM_KAFKA_BINDER_ZKNODES
              value: "zookeeper:2181"
            - name: SPRING_CLOUD_DATAFLOW_APPLICATIONPROPERTIES_STREAM_SPRING_CLOUD_STREAM_KAFKA_STREAMS_BINDER_ZKNODES
              value: "zookeeper:2181"
            - name: SPRING_CLOUD_DATAFLOW_APPLICATIONPROPERTIES_STREAM_SPRING_KAFKA_STREAMS_PROPERTIES_METRICS_RECORDING_LEVEL
              value: "ERROR"
            - name: SPRING_DATASOURCE_URL
              value: "jdbc:postgresql://scdf-postgres:5432/dataflow"
            - name: SPRING_DATASOURCE_USERNAME
              value: "postgres"
            - name: SPRING_DATASOURCE_PASSWORD
              value: "postgres"
            - name: SPRING_DATASOURCE_DRIVER_CLASS_NAME
              value: "org.postgresql.Driver"
            - name: SPRING_DATASOURCE_HIKARI_MAXIMUMPOOLSIZE
              value: "100"
          volumeMounts:
            - name: config
              mountPath: /workspace/config
              readOnly: true
            - name: scdf-tasks
              mountPath: /opt/dataflow/predefined_tasks
      serviceAccountName: scdf-sa
      volumes:
        - name: config
          configMap:
            name: scdf-server
            items:
            - key: application.yaml
              path: application.yaml
        - name: scdf-tasks
          persistentVolumeClaim:
            claimName: scdf-tasks-pvc
      hostname: scdf

如何在 Kubernetes 上配置 SCDF 中 Java 文件的启动？

Answer 1

您需要执行以下操作：

• 创建一个容器来运行Java应用程序。使用 Spring Boot 构建镜像插件或 paketo

  pack

  。
• 将容器推送到容器注册表。
• 通过配置容器拉取映像凭据来提供对容器的访问
• 将 docker 容器的 uri 注册到 SCDF 作为任务
• 为任务创建任务定义
• 启动任务。