将PyPI软件包添加到Composer环境时出现错误

Question

我刚刚创建了一个新的Cloud Composer环境，尝试向该环境中添加一些PyPI软件包时遇到错误（例如google-cloud-pubsub。

这是有关环境的数据：

我尝试添加google-cloud-pubsub时遇到的错误仅仅是这样：无法安装PyPI软件包。

有人知道吗？

谢谢：）

云构建：

+ python3 -m pip install -r requirements.txt
Collecting google-cloud-pubsub==1.2.0 (from -r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/c5/9e/e95b75789ba589e022f856754da8c6a7e7d607861254c7fd8ccdf5d7bdac/google_cloud_pubsub-1.2.0-py2.py3-none-any.whl (121kB)
Requirement already satisfied: apache-airflow==1.10.2-composer in /usr/local/lib/airflow (from -r requirements.txt (line 2))
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0 (from google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/63/7e/a523169b0cc9ce62d56e07571db927286a94b1a5f51ac220bd97db825c77/google_api_core-1.16.0-py2.py3-none-any.whl (70kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3 (from google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/65/19/2060c8faa325fddc09aa67af98ffcb6813f39a0ad805679fa64815362b3a/grpc-google-iam-v1-0.12.3.tar.gz
Requirement already satisfied: alembic<0.9,>=0.8.3 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: configparser<3.6.0,>=3.5.0 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: croniter<0.4,>=0.3.17 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: dill<0.3,>=0.2.2 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: enum34~=1.1.6 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: flask<0.13,>=0.12.4 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: flask-appbuilder==1.12.1 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: flask-admin==1.5.2 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: flask-caching<1.4.0,>=1.3.3 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: flask-login<0.5,>=0.3 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: flask-swagger==0.2.13 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: flask-wtf<0.15,>=0.14.2 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: funcsigs==1.0.0 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: future<0.17,>=0.16.0 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: gitpython>=2.0.2 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: gunicorn<20.0,>=19.4.0 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: iso8601>=0.1.12 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: jsonschema~=3.0 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: json-merge-patch==0.2 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: jinja2<=2.10.0,>=2.7.3 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: markdown<3.0,>=2.5.2 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: pandas<1.0.0,>=0.17.1 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: pendulum==1.4.4 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: psutil<6.0.0,>=4.2.0 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: pygments<3.0,>=2.0.1 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: python-daemon<2.2,>=2.1.1 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: python-dateutil<3,>=2.3 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: python-nvd3==0.15.0 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: requests<3,>=2.20.0 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: setproctitle<2,>=1.1.8 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: sqlalchemy<1.3.0,>=1.1.15 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: tabulate<=0.8.2,>=0.7.5 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: tenacity==4.8.0 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: thrift>=0.9.2 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: tzlocal>=1.4 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: unicodecsv>=0.14.1 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: werkzeug<0.15.0,>=0.14.1 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: zope.deprecation<5.0,>=4.0 in /opt/python3.6/lib/python3.6/site-packages (from apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Collecting googleapis-common-protos<2.0dev,>=1.6.0 (from google-api-core[grpc]<2.0.0dev,>=1.14.0->google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/05/46/168fd780f594a4d61122f7f3dc0561686084319ad73b4febbf02ae8b32cf/googleapis-common-protos-1.51.0.tar.gz
Requirement already satisfied: six>=1.10.0 in /opt/python3.6/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
Requirement already satisfied: pytz in /opt/python3.6/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
Requirement already satisfied: google-auth<2.0dev,>=0.4.0 in /opt/python3.6/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
Requirement already satisfied: setuptools>=34.0.0 in /opt/python3.6/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
Requirement already satisfied: protobuf>=3.4.0 in /opt/python3.6/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
Requirement already satisfied: grpcio<2.0dev,>=1.8.2; extra == "grpc" in /opt/python3.6/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
Requirement already satisfied: python-editor>=0.3 in /opt/python3.6/lib/python3.6/site-packages (from alembic<0.9,>=0.8.3->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: Mako in /opt/python3.6/lib/python3.6/site-packages (from alembic<0.9,>=0.8.3->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: itsdangerous>=0.21 in /opt/python3.6/lib/python3.6/site-packages (from flask<0.13,>=0.12.4->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: click>=2.0 in /opt/python3.6/lib/python3.6/site-packages (from flask<0.13,>=0.12.4->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: colorama<1,>=0.3.9 in /opt/python3.6/lib/python3.6/site-packages (from flask-appbuilder==1.12.1->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: Flask-OpenID<2,>=1.2.5 in /opt/python3.6/lib/python3.6/site-packages (from flask-appbuilder==1.12.1->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: Flask-Babel<1,>=0.11.1 in /opt/python3.6/lib/python3.6/site-packages (from flask-appbuilder==1.12.1->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: Flask-SQLAlchemy<3,>=2.3 in /opt/python3.6/lib/python3.6/site-packages (from flask-appbuilder==1.12.1->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: wtforms in /opt/python3.6/lib/python3.6/site-packages (from flask-admin==1.5.2->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: PyYAML>=3.0 in /opt/python3.6/lib/python3.6/site-packages (from flask-swagger==0.2.13->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: ordereddict in /opt/python3.6/lib/python3.6/site-packages (from funcsigs==1.0.0->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: gitdb2>=2.0.0 in /opt/python3.6/lib/python3.6/site-packages (from gitpython>=2.0.2->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: attrs>=17.4.0 in /opt/python3.6/lib/python3.6/site-packages (from jsonschema~=3.0->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: pyrsistent>=0.14.0 in /opt/python3.6/lib/python3.6/site-packages (from jsonschema~=3.0->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: MarkupSafe>=0.23 in /opt/python3.6/lib/python3.6/site-packages (from jinja2<=2.10.0,>=2.7.3->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: numpy>=1.12.0 in /opt/python3.6/lib/python3.6/site-packages (from pandas<1.0.0,>=0.17.1->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: pytzdata>=2018.3.0.0 in /opt/python3.6/lib/python3.6/site-packages (from pendulum==1.4.4->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: lockfile>=0.10 in /opt/python3.6/lib/python3.6/site-packages (from python-daemon<2.2,>=2.1.1->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: docutils in /opt/python3.6/lib/python3.6/site-packages (from python-daemon<2.2,>=2.1.1->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: python-slugify>=1.2.5 in /opt/python3.6/lib/python3.6/site-packages (from python-nvd3==0.15.0->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/python3.6/lib/python3.6/site-packages (from requests<3,>=2.20.0->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: idna<2.9,>=2.5 in /opt/python3.6/lib/python3.6/site-packages (from requests<3,>=2.20.0->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: certifi>=2017.4.17 in /opt/python3.6/lib/python3.6/site-packages (from requests<3,>=2.20.0->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/python3.6/lib/python3.6/site-packages (from requests<3,>=2.20.0->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: monotonic>=0.6 in /opt/python3.6/lib/python3.6/site-packages (from tenacity==4.8.0->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: rsa>=3.1.4 in /opt/python3.6/lib/python3.6/site-packages (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
Requirement already satisfied: cachetools>=2.0.0 in /opt/python3.6/lib/python3.6/site-packages (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/python3.6/lib/python3.6/site-packages (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
Requirement already satisfied: python3-openid>=2.0 in /opt/python3.6/lib/python3.6/site-packages (from Flask-OpenID<2,>=1.2.5->flask-appbuilder==1.12.1->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: Babel>=2.3 in /opt/python3.6/lib/python3.6/site-packages (from Flask-Babel<1,>=0.11.1->flask-appbuilder==1.12.1->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: smmap2>=2.0.0 in /opt/python3.6/lib/python3.6/site-packages (from gitdb2>=2.0.0->gitpython>=2.0.2->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: text-unidecode==1.2 in /opt/python3.6/lib/python3.6/site-packages (from python-slugify>=1.2.5->python-nvd3==0.15.0->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Requirement already satisfied: pyasn1>=0.1.3 in /opt/python3.6/lib/python3.6/site-packages (from rsa>=3.1.4->google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->google-cloud-pubsub==1.2.0->-r requirements.txt (line 1))
Requirement already satisfied: defusedxml in /opt/python3.6/lib/python3.6/site-packages (from python3-openid>=2.0->Flask-OpenID<2,>=1.2.5->flask-appbuilder==1.12.1->apache-airflow==1.10.2-composer->-r requirements.txt (line 2))
Building wheels for collected packages: grpc-google-iam-v1, googleapis-common-protos
  Running setup.py bdist_wheel for grpc-google-iam-v1: started
  Running setup.py bdist_wheel for grpc-google-iam-v1: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/de/3a/83/77a1e18e1a8757186df834b86ce6800120ac9c79cd8ca4091b
  Running setup.py bdist_wheel for googleapis-common-protos: started
  Running setup.py bdist_wheel for googleapis-common-protos: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/2c/f9/7f/6eb87e636072bf467e25348bbeb96849333e6a080dca78f706
Successfully built grpc-google-iam-v1 googleapis-common-protos
Installing collected packages: googleapis-common-protos, google-api-core, grpc-google-iam-v1, google-cloud-pubsub
  Found existing installation: googleapis-common-protos 1.5.8
    Uninstalling googleapis-common-protos-1.5.8:
      Successfully uninstalled googleapis-common-protos-1.5.8
  Found existing installation: google-api-core 1.8.1
    Uninstalling google-api-core-1.8.1:
      Successfully uninstalled google-api-core-1.8.1
  Found existing installation: grpc-google-iam-v1 0.11.4
    Uninstalling grpc-google-iam-v1-0.11.4:
      Successfully uninstalled grpc-google-iam-v1-0.11.4
Successfully installed google-api-core-1.16.0 google-cloud-pubsub-1.2.0 googleapis-common-protos-1.51.0 grpc-google-iam-v1-0.12.3
You are using pip version 9.0.3, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
+ python3 -m pip check
google-cloud-spanner 1.8.0 has requirement grpc-google-iam-v1<0.12dev,>=0.11.4, but you have grpc-google-iam-v1 0.12.3.
google-cloud-bigtable 0.32.0 has requirement grpc-google-iam-v1<0.12dev,>=0.11.4, but you have grpc-google-iam-v1 0.12.3.
You are using pip version 9.0.3, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
The command '/bin/sh -c bash installer.sh $COMPOSER_PYTHON_VERSION' returned a non-zero code: 1
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1

Answer 1

错误“无法安装PyPI软件包”表示Cloud Build无法为您的Cloud Composer环境构建Docker映像。通常是因为存在PyPI软件包安装失败，或者存在不兼容的依赖项。您的情况就是后者：

+ python3 -m pip check
google-cloud-spanner 1.8.0 has requirement grpc-google-iam-v1<0.12dev,>=0.11.4, but you have grpc-google-iam-v1 0.12.3.
google-cloud-bigtable 0.32.0 has requirement grpc-google-iam-v1<0.12dev,>=0.11.4, but you have grpc-google-iam-v1 0.12.3.

为了检查不兼容的依赖项（并防止它们破坏环境），Composer在所有软件包安装结束时运行pip check。您需要解决这些冲突才能使环境更新成功。