在 Azure Pipeline 中运行 Google Cloud Spanner 模拟器以进行集成测试

Question

我有一个 go 应用程序，正在编写使用 Azure Pipelines 的集成测试。在我的 Azure 构建 .yaml 中：

...
    - job: Run_Go_Integration_Tests
        displayName: 'Run Go Integration Tests'
        steps:
          - script: |
              sudo add-apt-repository ppa:longsleep/golang-backports
              sudo apt update
              sudo apt install golang-go
            displayName: 'Install Go'
          - script: |
              go version
            displayName: 'Check Go Version'
          - task: GoogleCloudSdkTool@1
            inputs:
              checkLatest: true
          - script: |
              make ci-itest
            displayName: 'Running Integration Tests'

make ci-test

在收到管道错误之前运行以下命令：

##@ Initialize Spanner
gcloud-config: ## Initialize Cloud Spanner Emulator and create emulator config
    @echo "Creating emulator configuration..."
    gcloud config configurations create emulator
    gcloud config set auth/disable_credentials true
    gcloud config set project ${DEV_SPANNER_PROJECT}
    gcloud config set api_endpoint_overrides/spanner http://localhost:9020/
    gcloud config configurations activate emulator

ci-start-spanner: ## Start Cloud Spanner Emulator for CI
    gcloud components update --quiet
    @echo "Starting cloud spanner..."
    gcloud emulators spanner start --quiet &

当管道执行命令时

gcloud emulators spanner start

我收到此错误：

$ gcloud emulators spanner start --quiet

Executing: /opt/hostedtoolcache/gcloud/448.0.0/x64/bin/cloud_spanner_emulator/gateway_main --hostname localhost --grpc_port 9010 --http_port 9020 
[cloud-spanner-emulator] WARNING: proto: file "google/rpc/status.proto" is already registered
[cloud-spanner-emulator]    previously from: "google.golang.org/genproto/googleapis/rpc/status"
[cloud-spanner-emulator]    currently from:  "unknown"
[cloud-spanner-emulator] See https://protobuf.dev/reference/go/faq#namespace-conflict
[cloud-spanner-emulator] 
[cloud-spanner-emulator] WARNING: proto: file "google/rpc/status.proto" has a name conflict over google.rpc.Status
[cloud-spanner-emulator]    previously from: "google.golang.org/genproto/googleapis/rpc/status"
[cloud-spanner-emulator]    currently from:  "unknown"
[cloud-spanner-emulator] See https://protobuf.dev/reference/go/faq#namespace-conflict
[cloud-spanner-emulator] 
[cloud-spanner-emulator] WARNING: proto: message google.rpc.Status is already registered
[cloud-spanner-emulator]    previously from: "google.golang.org/genproto/googleapis/rpc/status"
[cloud-spanner-emulator]    currently from:  "unknown"
[cloud-spanner-emulator] See https://protobuf.dev/reference/go/faq#namespace-conflict
[cloud-spanner-emulator] 
[cloud-spanner-emulator] WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
[cloud-spanner-emulator] I0000 00:00:1695827664.387924    4075 emulator_main.cc:39] Cloud Spanner Emulator running.
[cloud-spanner-emulator] I0000 00:00:1695827664.387942    4075 emulator_main.cc:40] Server address: localhost:9010
ERROR: gcloud crashed (ConnectionError): HTTPConnectionPool(host='localhost', port=9020): Max retries exceeded with url: /v1/projects/local/instances?alt=json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fecb45060d0>: Failed to establish a new connection: [Errno 111] Connection refused'))

关于如何解决

NewConnectionError

错误有什么建议吗？

Answer 1

这最终对我有用。我需要创建一个包含所需云扳手模拟器图像的

docker-compose.yml

：

这个文件位于我的go项目下

./pipelines/docker-compose.yml

version: "3.9"
services:
  gcp-cloud-spanner:
    restart: always
    image: gcr.io/cloud-spanner-emulator/emulator
    ports:
      - 9010:9010
      - 9020:9020

然后我修改了 Azure 管道代码以使用 Cloud Spanner 模拟器引用新创建的 Docker compose 文件：

- job: Run_Go_Integration_Tests
        displayName: 'Run Go Integration Tests'
        steps:
          - script: |
              sudo add-apt-repository ppa:longsleep/golang-backports
              sudo apt update
              sudo apt install golang-go
            displayName: 'Install Go'
          - script: |
              go version
            displayName: 'Check Go Version'
          - task: GoogleCloudSdkTool@1
            inputs:
              checkLatest: true
            displayName: 'Install Google Cloud SDK'
          - task: DockerCompose@0
            inputs:
              action: Run services
              dockerComposeFile: $(Build.SourcesDirectory)/pipelines/docker-compose.yml
              buildImages: false
            displayName: 'Run Integration Test Services'
          - script: |
              make ci-itest
            displayName: 'Running Integration Tests'

在 ci-test 中，我们现在可以针对 Docker Compose 模拟器运行任何所需的

gcloud

命令。在运行任何测试之前，您需要首先使用云模拟器配置、创建实例和数据库。

生成文件：

ci-itest: gcloud-config create-spanner-instance create-spanner-db ci-cloud-spanner-migrate-up run-integration-tests-here

gcloud-config: # Create emulator configuration
    @echo "Creating emulator configuration..."
    gcloud config configurations create emulator
    gcloud config set auth/disable_credentials true
    gcloud config set project ${DEV_SPANNER_PROJECT}
    gcloud config set api_endpoint_overrides/spanner http://localhost:9020/
    gcloud config configurations activate emulator

create-spanner-instance: ## Create cloud spanner instance
    export SPANNER_EMULATOR_HOST=${SPANNER_EMULATOR_HOST} && \
    gcloud spanner instances create ${DEV_SPANNER_INSTANCE} --config=emulator \
    --description="Cloud Spanner emulator" --nodes=3

create-spanner-db: ## Create cloud spanner database
    export SPANNER_EMULATOR_HOST=${SPANNER_EMULATOR_HOST} && \
    gcloud spanner databases create ${DEV_SPANNER_DB} --instance ${DEV_SPANNER_INSTANCE}

ci-cloud-spanner-migrate-up: ## Setup DB schema
    gcloud spanner databases ddl update ${DEV_SPANNER_DB} --instance='${DEV_SPANNER_INSTANCE}' --ddl-file=./migrations/{YOUR_DDL_FILE}.ddl