在 Bitbucket 管道中运行 Elastic 的问题

问题描述 投票：0回答：1

嗨，谁能帮我调试 CI/CD 环境中 Elastic 的问题？

Elastic 在本地运行（用于开发）并在其托管的 SaaS 平台上运行以用于暂存和生产。

我在 bitbucket 管道中有以下步骤

    - step: &unit_tests
    image: python:3.9
    name: Testing
    caches:
      - pip
    script:
      - pip install -r requirements.txt
      - export GC_API_DB_PORT=5432
      - export CS_DEPLOYMENT=development
      - docker run -d -p 9200:9200 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:8.6.2
      - python manage.py search_index --rebuild -f
      - python -m pytest --cov-report term --cov=gc_api --cov=gc_api_auth --cov-fail-under=85
    services:
      - postgres
      - docker

但是我似乎无法连接到弹性实例（我也尝试过弹性单独运行的配置）

    elasticsearch:
  image: docker.elastic.co/elasticsearch/elasticsearch:8.6.2
  container_name: elasticsearch
  cap_add:
    - IPC_LOCK
  volumes:
    - elasticsearch-data:/usr/share/elasticsearch/data
  ports:
    - 9200:9200
    - 9300:9300
....
    - step: &unit_tests
        image: python:3.9
        name: Testing
        caches:
          - pip
        script:
          - pip install -r requirements.txt
          - export GC_API_DB_PORT=5432
          - export ELASTIC_SEARCH_HOST=$BITBUCKET_DOCKER_HOST_INTERNAL
          - python -m pytest --cov-report term --cov=gc_api --cov=gc_api_auth --cov-fail-under=85
        services:
          - postgres
          - elasticsearch

这是我的错误

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.9/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.9/http/client.py", line 1377, in getresponse
    response.begin()
  File "/usr/local/lib/python3.9/http/client.py", line 320, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.9/http/client.py", line 289, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/connection/http_requests.py", line 166, in perform_request
    response = self.session.send(prepared_request, **send_kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/opt/atlassian/pipelines/agent/build/manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.9/site-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 301, in handle
    for index in self.es_conn.indices.get_alias().values():
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/client/utils.py", line 347, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/client/indices.py", line 642, in get_alias
    return self.transport.perform_request(
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/transport.py", line 417, in perform_request
    self._do_verify_elasticsearch(headers=headers, timeout=timeout)
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/transport.py", line 606, in _do_verify_elasticsearch
    raise error
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/transport.py", line 569, in _do_verify_elasticsearch
    _, info_headers, info_response = conn.perform_request(
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/connection/http_requests.py", line 194, in perform_request
    raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) caused by: ConnectionError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')))

您的任何意见将不胜感激

python elasticsearch cicd bitbucket-pipelines
1个回答
0
投票

如果您手动启动和分离 docker 容器，请考虑添加睡眠以等待容器中的进程实际接受连接

docker run -d -p 9200:9200 docker.elastic.co/elasticsearch/elasticsearch
sleep 10 ?
python -m pytest

如果您选择该服务，请注意您正在将 docker-compose 服务片段粘贴到 bitbucket-pipelines.yml 中。虽然服务部分有一些相似之处，但它是一个完全不同的应用程序，该定义是无效的。检查 https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/ 但所有这些 

ports
、
valumes
和 
cap_add
属性最多将被忽略。如果 Bitbcucket Pipelines 由于 yml 无效而失败可能会更好，因为这是反复出现的混乱。

现在，Bitbucket Pipelines 服务将与运行脚本的代理共享网络。我不知道您期望“BITBUCKET_DOCKER_HOST_INTERNAL”变量的评估值是多少，但它不存在，请查看https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/。如果它是一个用户定义的变量，请注意使用该前缀是非常规的并且会造成混淆，通常为具有记录变量的现有应用程序保留。

无论如何：你应该简单地使用

localhost
.

definitions:
  services:
    elasticsearch:
      image: docker.elastic.co/elasticsearch/elasticsearch
      variables:
        discovery.type: single-node
pipelines:
  default:
    - step:
      services: [elasticsearch]
      script:
        - pip install -r requirements.txt
        - export ELASTIC_SEARCH_HOST=localhost
        - python -m pytest


pip install
指令应该足够慢，以便在测试套件开始运行时弹性搜索服务准备就绪，因此这里不需要 
sleep
指令。

还有另一种使用测试容器的方法https://testcontainers-python.readthedocs.io/en/latest/elasticsearch/。 AFAIU 图书馆已经非常小心地阻止执行，直到服务真正准备好。

缺点是

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.