在使用 MinIO 作为工件存储库的本地 Minikube K8s 集群上运行 Argo 工作流程时出现错误

问题描述 投票:0回答:2

我正在本地 MinIO K8s 集群上运行 Argo 工作流程。我正在 MinIO 上设置一个工件存储库,可以存储工作流程中的输出工件。我按照此处的说明进行操作 https://argoproj.github.io/argo-workflows/configure-artifact-repository/#configuring-minio .

我遇到的错误是:无法创建新的 S3 客户端:端点 url 不能具有完全限定的路径。

我的 MinIO 端点位于 http://127.0.0.1:52139

这是我的工作流程 YAML 文件:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: artifactory-repository-ref-
spec:
  archiveLogs: true
  entrypoint: main
  templates:
    - name: main
      container:
        image: docker/whalesay:latest
        command: [ sh, -c ]
        args: [ "cowsay hello world | tee /tmp/hello_world.txt" ]
      archiveLocation:
        archiveLogs: true
      outputs:
        artifacts:
          - name: hello_world
            path: /tmp/hello_world.txt

这是我的工作流控制器配置映射 YAML,它部署在与工作流相同的命名空间中:

# This file describes the config settings available in the workflow controller configmap
apiVersion: v1
kind: ConfigMap
metadata:
  name: workflow-controller-configmap
data:                      # "config: |" key is optional in 2.7+!
  artifactRepository: |    # However, all nested maps must be strings
   archiveLogs: true
   s3:
     endpoint: argo-artifacts:9000
     bucket: my-bucket
     insecure: true
     accessKeySecret:                #omit if accessing via AWS IAM
       name: my-minio-cred
       key: accessKey
     secretKeySecret:                #omit if accessing via AWS IAM
       name: my-minio-cred
       key: secretKey
     useSDKCreds: true

我在工作流程运行的同一命名空间中创建了一个名为

my-minio-cred
的秘密。

以下是运行工作流的 Pod 中的日志:

time="2023-02-16T21:39:05.044Z" level=info msg="Starting Workflow Executor" version=v3.4.5
time="2023-02-16T21:39:05.047Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2023-02-16T21:39:05.047Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=argo podName=artifactory-repository-ref-5tcmt template="{\"name\":\"main\",\"inputs\":{},\"outputs\":{\"artifacts\":[{\"name\":\"hello_world\",\"path\":\"/tmp/hello_world.txt\"}]},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"docker/whalesay:latest\",\"command\":[\"sh\",\"-c\"],\"args\":[\"cowsay hello world | tee /tmp/hello_world.txt\"],\"resources\":{}},\"archiveLocation\":{\"archiveLogs\":true,\"s3\":{\"endpoint\":\"http://127.0.0.1:52897\",\"bucket\":\"my-bucket\",\"insecure\":true,\"accessKeySecret\":{\"name\":\"my-minio-cred\",\"key\":\"accessKey\"},\"secretKeySecret\":{\"name\":\"my-minio-cred\",\"key\":\"secretKey\"},\"useSDKCreds\":true,\"key\":\"artifactory-repository-ref-5tcmt/artifactory-repository-ref-5tcmt\"}}}" version="&Version{Version:v3.4.5,BuildDate:2023-02-07T12:36:25Z,GitCommit:1253f443baa8ad1610d2e62ec26ecdc85fe1b837,GitTag:v3.4.5,GitTreeState:clean,GoVersion:go1.18.10,Compiler:gc,Platform:linux/arm64,}"
time="2023-02-16T21:39:05.047Z" level=info msg="Starting deadline monitor"
time="2023-02-16T21:39:08.048Z" level=info msg="Main container completed" error="<nil>"
time="2023-02-16T21:39:08.048Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2023-02-16T21:39:08.048Z" level=info msg="No output parameters"
time="2023-02-16T21:39:08.048Z" level=info msg="Saving output artifacts"
time="2023-02-16T21:39:08.048Z" level=info msg="stopping progress monitor (context done)" error="context canceled"
time="2023-02-16T21:39:08.048Z" level=info msg="Deadline monitor stopped"
time="2023-02-16T21:39:08.048Z" level=info msg="Staging artifact: hello_world"
time="2023-02-16T21:39:08.049Z" level=info msg="Copying /tmp/hello_world.txt from container base image layer to /tmp/argo/outputs/artifacts/hello_world.tgz"
time="2023-02-16T21:39:08.049Z" level=info msg="/var/run/argo/outputs/artifacts/tmp/hello_world.txt.tgz -> /tmp/argo/outputs/artifacts/hello_world.tgz"
time="2023-02-16T21:39:08.049Z" level=info msg="S3 Save path: /tmp/argo/outputs/artifacts/hello_world.tgz, key: artifactory-repository-ref-5tcmt/artifactory-repository-ref-5tcmt/hello_world.tgz"
time="2023-02-16T21:39:08.049Z" level=info msg="Creating minio client using static credentials" endpoint="http://127.0.0.1:52897"
time="2023-02-16T21:39:08.049Z" level=warning msg="Non-transient error: Endpoint url cannot have fully qualified paths."
time="2023-02-16T21:39:08.049Z" level=info msg="Save artifact" artifactName=hello_world duration="282.917µs" error="failed to create new S3 client: Endpoint url cannot have fully qualified paths." key=artifactory-repository-ref-5tcmt/artifactory-repository-ref-5tcmt/hello_world.tgz
time="2023-02-16T21:39:08.049Z" level=error msg="executor error: failed to create new S3 client: Endpoint url cannot have fully qualified paths."
time="2023-02-16T21:39:08.049Z" level=info msg="S3 Save path: /tmp/argo/outputs/logs/main.log, key: artifactory-repository-ref-5tcmt/artifactory-repository-ref-5tcmt/main.log"
time="2023-02-16T21:39:08.049Z" level=info msg="Creating minio client using static credentials" endpoint="http://127.0.0.1:52897"
time="2023-02-16T21:39:08.049Z" level=warning msg="Non-transient error: Endpoint url cannot have fully qualified paths."
time="2023-02-16T21:39:08.049Z" level=info msg="Save artifact" artifactName=main-logs duration="28.5µs" error="failed to create new S3 client: Endpoint url cannot have fully qualified paths." key=artifactory-repository-ref-5tcmt/artifactory-repository-ref-5tcmt/main.log
time="2023-02-16T21:39:08.049Z" level=error msg="executor error: failed to create new S3 client: Endpoint url cannot have fully qualified paths."
time="2023-02-16T21:39:08.056Z" level=info msg="Create workflowtaskresults 403"
time="2023-02-16T21:39:08.056Z" level=warning msg="failed to patch task set, falling back to legacy/insecure pod patch, see https://argoproj.github.io/argo-workflows/workflow-rbac/" error="workflowtaskresults.argoproj.io is forbidden: User \"system:serviceaccount:argo:default\" cannot create resource \"workflowtaskresults\" in API group \"argoproj.io\" in the namespace \"argo\""
time="2023-02-16T21:39:08.057Z" level=info msg="Patch pods 403"
time="2023-02-16T21:39:08.057Z" level=warning msg="Non-transient error: pods \"artifactory-repository-ref-5tcmt\" is forbidden: User \"system:serviceaccount:argo:default\" cannot patch resource \"pods\" in API group \"\" in the namespace \"argo\""
time="2023-02-16T21:39:08.057Z" level=error msg="executor error: pods \"artifactory-repository-ref-5tcmt\" is forbidden: User \"system:serviceaccount:argo:default\" cannot patch resource \"pods\" in API group \"\" in the namespace \"argo\""
time="2023-02-16T21:39:08.057Z" level=info msg="Alloc=6350 TotalAlloc=12366 Sys=18642 NumGC=4 Goroutines=5"
time="2023-02-16T21:39:08.057Z" level=fatal msg="failed to create new S3 client: Endpoint url cannot have fully qualified paths."

我尝试将工作流控制器-config.yaml 中的端点键从

127.0.0.1:52139
更改为
127.0.0.1:9000
以及
argo-artifacts:9000
但它仍然不起作用。 argo-artifacts 是由
helm install argo-artifacts minio/minio
命令创建的 LoadBalancer 服务的名称。

我从以下位置获取了 MinIO 存储桶的端点

minikube service --url argo-artifacts
如“配置 MinIO”部分所示 https://argoproj.github.io/argo-workflows/configure-artifact-repository/#configuring-minio

一切都在同一个命名空间中。

这里可能出了什么问题?

我尝试更改 MinIO 存储桶的端点 URL、更改不同组件的命名空间以及更改部署

argo-artifacts
服务的命名空间。

kubernetes workflow minikube minio argo-workflows
2个回答
0
投票

您的日志中有多个问题。其中之一是:

time="2023-02-16T21:39:08.049Z" level=info msg="Creating minio client using static credentials" endpoint="http://127.0.0.1:52897"
time="2023-02-16T21:39:08.049Z" level=warning msg="Non-transient error: Endpoint url cannot have fully qualified paths."

首先 - 我不认为你的 minio 端点会暴露在 localhost (127.0.0.1) IP 地址上,无论如何也不会暴露在 minikube 集群内部。我假设这个值是从“my-minio-cred”秘密中获取的,所以请将其更改为“argo-artifacts:9000”。其次 - 请注意,您不需要在那里有“http”,这就是上面警告的内容。

我看到的另一个问题 -

time="2023-02-16T21:39:08.056Z" level=warning msg="failed to patch task set, falling back to legacy/insecure pod patch, see https://argoproj.github.io/argo-workflows/workflow-rbac/" error="workflowtaskresults.argoproj.io is forbidden: User \"system:serviceaccount:argo:default\" cannot create resource \"workflowtaskresults\" in API group \"argoproj.io\" in the namespace \"argo\""
time="2023-02-16T21:39:08.057Z" level=info msg="Patch pods 403"

根据 argo worklfows 文档此处,不鼓励使用“默认”服务帐户。如果您坚持使用它 - 您需要授予它一些权限。您可以通过创建角色,然后为“服务帐户”创建角色绑定来实现此目的。如果您使用 helm 安装 argo - 它也会创建一个具有适当角色的服务帐户。只需确保在提交工作流程时使用它即可。希望这有帮助


0
投票

您可以检查一下这个问题,恕我直言,它与您的问题有关: 由于 RBAC 错误,快速启动失败

您可以尝试通过

解决此问题
  • 使用选项
    --serviceaccount argo
  • 使用最新入门指南的安装程序部署 Argo 工作流程:快速入门,应用清单
    quick-start-minimal.yaml
© www.soinside.com 2019 - 2024. All rights reserved.