使用 GCP 工作流程将 CSV 文件从 Google Cloud Storage 导入到 Google Cloud SQL

Question

我正在编写一个工作流程，将 csv 从 GCS 存储桶导入到云 sql（my_sql 实例）中。我正在使用 sql admin api v1 来执行导入。以下是我的工作流程代码：

main:
  params: [event]
  steps:
  - log_event:
      call: sys.log
      args:
          text: ${event}
          severity: INFO
  - extract_bucket_and_file:
      assign:
        - bucket: ${event.data.bucket}
        - file: ${event.data.name}
  - init:
      assign:
        - project_id: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
        - instance: "xxx-instance"
        - database_name: "xxx-db"    
  - start_sql_instance:
      call: googleapis.sqladmin.v1.instances.patch
      args:
          instance: ${instance}
          project: ${project_id}
          body:
              settings:
                activationPolicy: 'ALWAYS'
      result: startSqlResponse
  - log_start_sql_instance_response:
      call: sys.log
      args:
          text: ${startSqlResponse}
          severity: INFO
  - import_data:
      call: http.post
      args: 
        url: ${"https://sqladmin.googleapis.com/v1/projects/"+project_id+"/instances/"+instance+"/import"}
        auth:
          type: OAuth2
        body:
          importContext:
            fileType: CSV
            uri: gs://${bucket}/${file}
            database: ${database_name}
            kind: sql#importContext
            csvImportOptions:
              table: 'xxx'
      result: importResponse
  - log_import_data_response:
      call: sys.log
      args:
          text: ${importResponse}
          severity: INFO                          
  - the_end:
      return: "SUCCESS"

执行此工作流程时遇到的错误如下：

HTTP server responded with error code 403
in step "import_data", routine "main", line: 33
{
  "body": {
    "error": {
      "code": 403,
      "errors": [
        {
          "domain": "global",
          "message": "The service account does not have the required permissions for the bucket.",
          "reason": "notAuthorized"
        }
      ],
      "message": "The service account does not have the required permissions for the bucket."
    }
  },
  "code": 403,
  "headers": {
    "Alt-Svc": "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000",
    "Cache-Control": "private",
    "Content-Length": "328",
    "Content-Type": "application/json; charset=UTF-8",
    "Date": "Wed, 13 Sep 2023 05:47:04 GMT",
    "Server": "ESF",
    "Vary": "Origin, X-Origin, Referer",
    "X-Content-Type-Options": "nosniff",
    "X-Frame-Options": "SAMEORIGIN",
    "X-Xss-Protection": "0"
  },
  "message": "HTTP server responded with error code 403",
  "tags": [
    "HttpError"
  ]
}

根据文档here，建议授予

Storage Object Admin

以方便

Cloud SQL Instance Service Account

，您可以从实例控制台或 gcloud 命令

gcloud sql instances describe xxx-instance

获取。我完全按照文档所述进行操作，但仍然收到 403 错误。

如果有人能指导我哪里出错了，我将非常感激。

谢谢

Answer 1

这不是 Cloud SQL 错误，而是工作流程错误。 Workflow 服务帐号没有在 Cloud SQL 中导入数据的权限。

授予工作流服务帐户的 Cloud SQL 管理员角色。