Scrapy 和远大期望 (great_expectations) - 不能一起工作

Question

我正在尝试在同一虚拟环境中使用包 scrapy 和 great_expectations 。这两个包之间的兼容性似乎存在问题，具体取决于我导入它们的顺序。

示例：

• 我创建了一个虚拟环境并 pip 安装了每个包的最新版本。

有效：

import great_expectations
import scrapy

print("done")

不起作用：

import scrapy
import great_expectations

print("done")

错误：

Traceback (most recent call last):
  File 

"/Users/grant/vs_code_projects/grants_projects/test_environment.py", line 2, in <module>
    import great_expectations
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/__init__.py", line 32, in <module>
    register_core_expectations()
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/registry.py", line 187, in register_core_expectations
    from great_expectations.expectations import core  # noqa: F401
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/core/__init__.py", line 1, in <module>
    from .expect_column_distinct_values_to_be_in_set import (
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/core/expect_column_distinct_values_to_be_in_set.py", line 12, in <module>
    from great_expectations.expectations.expectation import (
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/expectation.py", line 2350, in <module>
    class BatchExpectation(Expectation, ABC):
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/expectation.py", line 287, in __new__
    newclass._register_renderer_functions()
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/expectation.py", line 369, in _register_renderer_functions
    attr_obj: Callable = getattr(cls, candidate_renderer_fn_name)
AttributeError: __provides__

编辑：

• 我已经尝试了Scrapy的所有主要版本（当前版本，2.0.0，1.0.0）和远大前程的所有主要版本，结果都是一样的。

Answer 1

如果我将“

attr_obj: Callable = getattr(cls, candidate_renderer_fn_name)

”行更改为“

attr_obj: Callable = getattr(cls, candidate_renderer_fn_name, None)

”，它确实允许导入包。

True：修改该行以包含默认值

None

和

getattr

将有效绕过

AttributeError

，如果

 上不存在名为 

attr_obj

 的属性，则允许将 

None

 设置为 

candidate_renderer_fn_name

 cls

。

虽然这允许您的导入顺利进行，但它也可能掩盖其他问题或导致意外行为，特别是如果预计

attr_obj

始终成功检索属性。

您可以尝试将Great Expectations的导入延迟到Scrapy初始化之后：使用函数来封装Great Expectations的导入和使用，确保仅在需要时才导入。

import scrapy

def use_great_expectations():
    import great_expectations
    # Your Great Expectations code here
    print("Great Expectations used")

print("Scrapy and other initializations")
# Other Scrapy or general code here

# Now, call the function to use Great Expectations
use_great_expectations()

或者，在阅读了 An Truong 的“当懒惰是好的：懒惰导入”之后：

import scrapy
import lazy_import

great_expectations = lazy_import.lazy_module("great_expectations")

def use_great_expectations():
    # Assuming `great_expectations` functionalities are required here
    # Now, the Great Expectations module will be actually loaded
    print("Using Great Expectations")
    # Example Great Expectations code here