我们可以从 Python 脚本而不是 CLI 启动加速吗？

Question

我正在学习 HuggingFace 课程，其中提供了以下代码。

from accelerate import Accelerator
from transformers import AdamW, AutoModelForSequenceClassification, get_scheduler

accelerator = Accelerator()

model = AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2)
optimizer = AdamW(model.parameters(), lr=3e-5)

train_dl, eval_dl, model, optimizer = accelerator.prepare(
    train_dataloader, eval_dataloader, model, optimizer
)

num_epochs = 3
num_training_steps = num_epochs * len(train_dl)
lr_scheduler = get_scheduler(
    "linear",
    optimizer=optimizer,
    num_warmup_steps=0,
    num_training_steps=num_training_steps,
)

progress_bar = tqdm(range(num_training_steps))

model.train()
for epoch in range(num_epochs):
    for batch in train_dl:
        outputs = model(**batch)
        loss = outputs.loss
        accelerator.backward(loss)

        optimizer.step()
        lr_scheduler.step()
        optimizer.zero_grad()
        progress_bar.update(1)

要利用加速库，我可以简单地将该脚本作为典型的 Python 程序执行 (

python train.py

)，还是必须使用加速 CLI 命令 (

accelerate launch train.py

) 来启用分布式训练？

Answer 1

也许你可以尝试“notebook_launcher”模式

提示：加速 >= 0.23.0