我尝试运行模型“AdapterHub/bert-base-uncased-pf-conll2003”（此处的模型描述）进行 NLP 中的标记分类。我使用 Google Colab 来完成此任务。
首先我尝试安装变压器
pip install -U adapter-transformers
上述命令的输出是
Collecting adapter-transformers
Downloading adapter_transformers-3.2.1.post0-py3-none-any.whl (6.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.4/6.4 MB 9.9 MB/s eta 0:00:00
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from adapter-transformers) (3.13.1)
Collecting huggingface-hub<0.14.0,>=0.11.0 (from adapter-transformers)
Downloading huggingface_hub-0.13.4-py3-none-any.whl (200 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 200.1/200.1 kB 10.6 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from adapter-transformers) (1.23.5)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from adapter-transformers) (23.2)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from adapter-transformers) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from adapter-transformers) (2023.6.3)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from adapter-transformers) (2.31.0)
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from adapter-transformers)
Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 31.9 MB/s eta 0:00:00
Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from adapter-transformers) (4.66.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<0.14.0,>=0.11.0->adapter-transformers) (4.5.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->adapter-transformers) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->adapter-transformers) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->adapter-transformers) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->adapter-transformers) (2023.11.17)
Installing collected packages: tokenizers, huggingface-hub, adapter-transformers
Attempting uninstall: tokenizers
Found existing installation: tokenizers 0.15.0
Uninstalling tokenizers-0.15.0:
Successfully uninstalled tokenizers-0.15.0
Attempting uninstall: huggingface-hub
Found existing installation: huggingface-hub 0.19.4
Uninstalling huggingface-hub-0.19.4:
Successfully uninstalled huggingface-hub-0.19.4
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
transformers 4.35.2 requires huggingface-hub<1.0,>=0.16.4, but you have huggingface-hub 0.13.4 which is incompatible.
transformers 4.35.2 requires tokenizers<0.19,>=0.14, but you have tokenizers 0.13.3 which is incompatible.
Successfully installed adapter-transformers-3.2.1.post0 huggingface-hub-0.13.4 tokenizers-0.13.3
我使用了以下代码块
from transformers import AutoModelWithHeads
from transformers import pipeline
token_classification = pipeline("token-classification", model = "AdapterHub/bert-base-uncased-pf-conll2003")
res = token_classification("Take out the trash bag from the bin and replace it.")
print(res)
我收到错误
EntryNotFoundError: 404 Client Error. (Request ID: Root=1-657e793c-0ce0c1936aff5e5741676650)
Entry Not Found for url: https://huggingface.co/AdapterHub/bert-base-uncased-pf-conll2003/resolve/main/config.json.
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
<ipython-input-3-030dfe0e128d> in <cell line: 3>()
1 from transformers import AutoModelWithHeads
2 from transformers import pipeline
----> 3 token_classification = pipeline("token-classification", model = "AdapterHub/bert-base-uncased-pf-conll2003")
4 res = token_classification("Take out the trash bag from the bin and replace it.")
5 print(res)
/usr/local/lib/python3.10/dist-packages/transformers/pipelines/__init__.py in pipeline(task, model, config, tokenizer, feature_extractor, framework, revision, use_fast, use_auth_token, device, device_map, torch_dtype, trust_remote_code, model_kwargs, pipeline_class, **kwargs)
673 hub_kwargs["_commit_hash"] = config._commit_hash
674 elif config is None and isinstance(model, str):
--> 675 config = AutoConfig.from_pretrained(model, _from_pipeline=task, **hub_kwargs, **model_kwargs)
676 hub_kwargs["_commit_hash"] = config._commit_hash
677
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
850 kwargs["name_or_path"] = pretrained_model_name_or_path
851 trust_remote_code = kwargs.pop("trust_remote_code", False)
--> 852 config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
853 if "auto_map" in config_dict and "AutoConfig" in config_dict["auto_map"]:
854 if not trust_remote_code:
/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py in get_config_dict(cls, pretrained_model_name_or_path, **kwargs)
563 original_kwargs = copy.deepcopy(kwargs)
564 # Get config dict associated with the base config file
--> 565 config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
566 if "_commit_hash" in config_dict:
567 original_kwargs["_commit_hash"] = config_dict["_commit_hash"]
/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py in _get_config_dict(cls, pretrained_model_name_or_path, **kwargs)
624 try:
625 # Load from local folder or from cache or download from model Hub and cache
--> 626 resolved_config_file = cached_file(
627 pretrained_model_name_or_path,
628 configuration_file,
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py in cached_file(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, use_auth_token, revision, local_files_only, subfolder, user_agent, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash)
452 if revision is None:
453 revision = "main"
--> 454 raise EnvironmentError(
455 f"{path_or_repo_id} does not appear to have a file named {full_filename}. Checkout "
456 f"'https://huggingface.co/{path_or_repo_id}/{revision}' for available files."
OSError: AdapterHub/bert-base-uncased-pf-conll2003 does not appear to have a file named config.json. Checkout 'https://huggingface.co/AdapterHub/bert-base-uncased-pf-conll2003/main' for available files.
我该如何解决这个问题？
提前致谢
# be sure you have the dependencies
pip install adapters
pip install -U adapter-transformers
在管道之外创建模型
from transformers import AutoModelWithHeads
from transformers import pipeline
model = AutoModelWithHeads.from_pretrained("bert-base-uncased")
adapter_name = model.load_adapter("AdapterHub/bert-base-uncased-pf-conll2003", source="hf")
model.active_adapters = adapter_name
token_classification = pipeline("token-classification", model=model)
res = token_classification("Take out the trash bag from the bin and replace it.")
print(res)