尝试使用pytorch。收到错误“尚未在 MPS 设备上分配占位符存储！”

Question

我正在尝试训练 BERT 模型，并且正在使用 torch。当我尝试在我的 M1 mac 上运行它时，我不断收到此错误。然后我切换到我的 intel i9 Mac，我遇到了同样的错误。这是我的代码：

# Import the necessary packages
import torch
import numpy as np
import pandas as pd
from transformers import BertTokenizer, BertForSequenceClassification, TrainingArguments, Trainer
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

# Set the device (CPU or GPU)
# device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
device = torch.device("cpu")

# Load the pre-trained BERT model and tokenizer
model_name = "bert-base-uncased"
tokenizer = BertTokenizer.from_pretrained(model_name)
model = BertForSequenceClassification.from_pretrained(model_name, num_labels=2).to(device)

# Define the training arguments
training_args = TrainingArguments(
    output_dir="bert_model",
    evaluation_strategy="epoch",
    per_device_train_batch_size=32,
    per_device_eval_batch_size=64,
    learning_rate=2e-5,
    num_train_epochs=4,
    weight_decay=0.01,
    metric_for_best_model="accuracy",
)

# Define the compute metrics function
def compute_metrics(pred):
    labels = pred.label_ids
    logits = pred.predictions
    preds = np.argmax(logits, axis=1)
    
    accuracy = accuracy_score(labels, preds)
    precision = precision_score(labels, preds, average='weighted')
    recall = recall_score(labels, preds, average='weighted')
    f1 = f1_score(labels, preds, average='weighted')

    return {
        'accuracy': accuracy,
        'precision': precision,
        'recall': recall,
        'f1': f1
    }

# Prepare the train dataset
train_df = preprocessed_df 
X_train = train_df["text"].values.tolist()
y_train = train_df["label"].values.tolist()

# Tokenize the input text for training
tokenized_inputs_train = tokenizer(X_train, truncation=True, padding=True, max_length=512, return_tensors="pt")

# Create a list of dictionaries for training
train_dataset = []
for i in range(len(X_train)):
    example = {
        "input_ids": tokenized_inputs_train["input_ids"][i],
        "attention_mask": tokenized_inputs_train["attention_mask"][i],
        "labels": y_train[i]
    }
    train_dataset.append(example)


# Create a Trainer instance with the train dataset
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    compute_metrics=compute_metrics,
)

# Train the model
trainer.train()

这是错误：

Output exceeds the size limit. Open the full output data in a text editor
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[15], line 75
     67 trainer = Trainer(
     68     model=model,
     69     args=training_args,
     70     train_dataset=train_dataset,
     71     compute_metrics=compute_metrics,
     72 )
     74 # Train the model
---> 75 trainer.train()

File ~/Library/CloudStorage/OneDrive-Personal/Uni/L6/Final Project/FakeNewsDetection/myenv/lib/python3.11/site-packages/transformers/trainer.py:1645, in Trainer.train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1640     self.model_wrapped = self.model
   1642 inner_training_loop = find_executable_batch_size(
   1643     self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size
   1644 )
-> 1645 return inner_training_loop(
   1646     args=args,
   1647     resume_from_checkpoint=resume_from_checkpoint,
   1648     trial=trial,
   1649     ignore_keys_for_eval=ignore_keys_for_eval,
   1650 )

File ~/Library/CloudStorage/OneDrive-Personal/Uni/L6/Final Project/FakeNewsDetection/myenv/lib/python3.11/site-packages/transformers/trainer.py:1938, in Trainer._inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
...
   2208     # remove once script supports set_grad_enabled
   2209     _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 2210 return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)

RuntimeError: Placeholder storage has not been allocated on MPS device!

首先，我将设备设置为 CPU，因为我读到这是与 GPU 相关的错误。那没用。

然后我在网上看到了一个安装 pytorch nightly 版本的解决方案，但无论我怎么做，我总是最终安装稳定版本。我具体是这样做的：

pip install --pre torch torchvision torchaudio -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html 

不幸的是，当我检查torch版本时，它显示的是稳定版本，并且CUDA_HOME的值为None。我不知道还能做什么。

编辑：我想指定我将继续使用我的英特尔 Macbook，因此面向此的解决方案将非常有帮助

Answer 1

我遇到了同样的问题，我通过更新变压器和火炬修复了它（使用它并选择夜间 - mac：https://pytorch.org/）