我（或多或少）正在使用我自己的数据遵循 Training_quora_duplicate_questions.py 示例。
正如我从交叉编码器页面中了解到的，我引用：
对于二元任务和具有连续分数的任务（如STS），我们设置num_labels=1。对于分类任务，我们将其设置为我们拥有的标签数量。
因此我这样编写了模型：
model = CrossEncoder("distilroberta-base", num_labels= 2)
评估者如此：
evaluator = CEBinaryClassificationEvaluator.from_input_examples(dev_examples, name= "Rooms-dev")
但是，当我第一次尝试评估时运行代码时，它会给出以下错误：
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
我之前尝试过相同的代码，但使用 num_labels=1 并且它有效，“尽管（我认为）不是我想要的，因为我想建模来预测 0 或 1，而不是从 0 到连续的1.
有什么想法可能导致这种情况吗？
代码和错误消息如下：
# extracting rooms & labels into lists
rooms_1 = rooms_df["room_ner1"].tolist()
rooms_2 = rooms_df["room_ner2"].tolist()
labels = rooms_df["label"].tolist()
# creating array of room-pairs
dataset = [
[original, candidate, label]
for original, candidate, label
in zip(rooms_1, rooms_2, labels)
]
random.shuffle(dataset)
dev_sample_size = int(len(dataset) * 0.10)
dev_data = dataset[: dev_sample_size]
train_data = dataset[dev_sample_size: ]
# preparing training dataset
train_examples = list()
n_examples = len(train_data)
# creating training dataset
for i in trange(n_examples):
example = train_data[i]
train_examples.append(InputExample(texts= [example[0], example[1]], label= example[2]))
train_examples.append(InputExample(texts= [example[1], example[0]], label= example[2]))
# preparing test dataset
dev_examples = list()
d_examples = len(dev_data)
# creating test dataset
for i in trange(d_examples):
example = dev_data[i]
dev_examples.append(InputExample(texts= [example[0], example[1]], label= example[2]))
train_batch_size = 16
num_epochs = 4
model_save_path = "output/training_rooms-" + datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
model = CrossEncoder("distilroberta-base", num_labels= 2)#, device= "mps")
train_dataloader = DataLoader(
train_examples,
shuffle= True,
batch_size= train_batch_size
)
evaluator = CEBinaryClassificationEvaluator.from_input_examples(dev_examples, name= "Rooms-dev")
warmup_steps = math.ceil(len(train_dataloader) * num_epochs * 0.1) # 10% train data for warm-up
logger.info(f"Warmup-steps: {warmup_steps:_}")
model.fit(
train_dataloader= train_dataloader,
evaluator= evaluator,
epochs= num_epochs,
evaluation_steps= 5_000,
warmup_steps= warmup_steps,
output_path= model_save_path
)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[67], [line 1](vscode-notebook-cell:?execution_count=67&line=1)
----> [1](vscode-notebook-cell:?execution_count=67&line=1) model.fit(
[2](vscode-notebook-cell:?execution_count=67&line=2) train_dataloader= train_dataloader,
[3](vscode-notebook-cell:?execution_count=67&line=3) evaluator= evaluator,
[4](vscode-notebook-cell:?execution_count=67&line=4) epochs= num_epochs,
[5](vscode-notebook-cell:?execution_count=67&line=5) evaluation_steps= 5_000,
[6](vscode-notebook-cell:?execution_count=67&line=6) warmup_steps= warmup_steps,
[7](vscode-notebook-cell:?execution_count=67&line=7) output_path= model_save_path
[8](vscode-notebook-cell:?execution_count=67&line=8) )
File [~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:275](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:275), in CrossEncoder.fit(self, train_dataloader, evaluator, epochs, loss_fct, activation_fct, scheduler, warmup_steps, optimizer_class, optimizer_params, weight_decay, evaluation_steps, output_path, save_best_model, max_grad_norm, use_amp, callback, show_progress_bar)
[272](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:272) training_steps += 1
[274](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:274) if evaluator is not None and evaluation_steps > 0 and training_steps % evaluation_steps == 0:
--> [275](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:275) self._eval_during_training(
[276](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:276) evaluator, output_path, save_best_model, epoch, training_steps, callback
[277](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:277) )
[279](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:279) self.model.zero_grad()
[280](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:280) self.model.train()
File [~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:450](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:450), in CrossEncoder._eval_during_training(self, evaluator, output_path, save_best_model, epoch, steps, callback)
[448](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:448) """Runs evaluation during the training"""
[449](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:449) if evaluator is not None:
--> [450](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:450) score = evaluator(self, output_path=output_path, epoch=epoch, steps=steps)
[451](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:451) if callback is not None:
[452](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py:452) callback(score, epoch, steps)
File [~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/evaluation/CEBinaryClassificationEvaluator.py:81](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/evaluation/CEBinaryClassificationEvaluator.py:81), in CEBinaryClassificationEvaluator.__call__(self, model, output_path, epoch, steps)
[76](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/evaluation/CEBinaryClassificationEvaluator.py:76) logger.info("CEBinaryClassificationEvaluator: Evaluating the model on " + self.name + " dataset" + out_txt)
[77](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/evaluation/CEBinaryClassificationEvaluator.py:77) pred_scores = model.predict(
[78](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/evaluation/CEBinaryClassificationEvaluator.py:78) self.sentence_pairs, convert_to_numpy=True, show_progress_bar=self.show_progress_bar
[79](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/evaluation/CEBinaryClassificationEvaluator.py:79) )
---> [81](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/evaluation/CEBinaryClassificationEvaluator.py:81) acc, acc_threshold = BinaryClassificationEvaluator.find_best_acc_and_threshold(pred_scores, self.labels, True)
[82](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/evaluation/CEBinaryClassificationEvaluator.py:82) f1, precision, recall, f1_threshold = BinaryClassificationEvaluator.find_best_f1_and_threshold(
[83](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/evaluation/CEBinaryClassificationEvaluator.py:83) pred_scores, self.labels, True
[84](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/evaluation/CEBinaryClassificationEvaluator.py:84) )
[85](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/cross_encoder/evaluation/CEBinaryClassificationEvaluator.py:85) ap = average_precision_score(self.labels, pred_scores)
File [~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/evaluation/BinaryClassificationEvaluator.py:226](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/evaluation/BinaryClassificationEvaluator.py:226), in BinaryClassificationEvaluator.find_best_acc_and_threshold(scores, labels, high_score_more_similar)
[223](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/evaluation/BinaryClassificationEvaluator.py:223) assert len(scores) == len(labels)
[224](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/evaluation/BinaryClassificationEvaluator.py:224) rows = list(zip(scores, labels))
--> [226](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/evaluation/BinaryClassificationEvaluator.py:226) rows = sorted(rows, key=lambda x: x[0], reverse=high_score_more_similar)
[228](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/evaluation/BinaryClassificationEvaluator.py:228) max_acc = 0
[229](https://file+.vscode-resource.vscode-cdn.net/Users/duarteharris/IronHack/GitHub/room-matching/~/IronHack/GitHub/room-matching/.venv/lib/python3.11/site-packages/sentence_transformers/evaluation/BinaryClassificationEvaluator.py:229) best_threshold = -1
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
提前感谢您的帮助。
在这种情况下（num_labels = 2）评估必须不同，例如：
from sentence_transformers.cross_encoder.evaluation import CEF1Evaluator, CESoftmaxAccuracyEvaluator
from sentence_transformers.evaluation import SequentialEvaluator
accuracy_evaluator = CESoftmaxAccuracyEvaluator.from_input_examples(dev_examples, name= "Rooms-dev")
f1_evaluator = CEF1Evaluator.from_input_examples(dev_examples, name= "Rooms-dev")
evaluator = SequentialEvaluator([accuracy_evaluator, f1_evaluator])
然后你就可以通过了：
model.fit(
train_dataloader= train_dataloader,
evaluator= evaluator,
epochs= num_epochs,
evaluation_steps= 5_000,
warmup_steps= warmup_steps,
output_path= model_save_path
)