pyTorch LSTM模型未学习，准确度为0％

Question

我正在尝试在https://pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html之后使用LSTM构建POS标记器。

[当我对一个句子进行培训和测试时，确实得到了相当准确的标记。但是，当我运行以下训练和验证功能时，我得到0％的准确性（所有时期）。我相信我在某处做着非常错误的事情，但是对此我是陌生的，我似乎找不到。

似乎每个时期也只输出几个标签，：

Epoch 1: ['CD', 'EX', 'NNS', 'CD', 'CD', 'NNS', 'NNS', 'RBR', 'CD', 'CD', 'CD', 'NNS', 'CD', 'NNS', 'NNS'] 
Avg Train Loss: 4.0158  Avg Val Loss: 4.0151     Val Accuracy: 0

Epoch 2: ['IN', 'VBD', 'VBD', 'VBD', 'VBD', 'VBD', 'IN', 'VBD', 'IN', 'VBD', 'VBD', 'VBD', 'IN', 'IN', 'IN']
Avg Train Loss: 3.8330  Avg Val Loss: 3.8328     Val Accuracy: 0

Epoch 3: ['$', 'PDT', 'PDT', 'PDT', '$', '$', '$', '$', '.', 'PDT', 'PDT', '$', '$', '.', 'PDT'] 

依此类推。我的功能是：

def train(epoch, model, loss_function, optimizer):
    train_loss = 0
    train_examples = 0
    model = BasicPOSTagger(EMBEDDING_DIM, HIDDEN_DIM, len(word_to_idx), len(tag_to_idx))
    corr = 0
    for sentence, tags in training_data:
        model.zero_grad()
        sent_in = prepare_sequence(sentence, word_to_idx)
        targets = prepare_sequence(tags, tag_to_idx)

        # run forward pass
        tag_scores = model(sent_in)
        # compute loss, gradients, update params
        loss = loss_function(tag_scores, targets)
        train_loss += loss
        loss = Variable(train_loss, requires_grad = True)
        loss.backward()
        optimizer.step()        
        train_examples += 1

    avg_train_loss = train_loss / train_examples
    avg_val_loss, val_accuracy = evaluate(model, loss_function, optimizer)

    print("Epoch: {}/{}\tAvg Train Loss: {:.4f}\tAvg Val Loss: {:.4f}\t Val Accuracy: {:.0f}".format(epoch, 
                                                                      EPOCHS, 
                                                                      avg_train_loss, 
                                                                      avg_val_loss,
                                                                      val_accuracy))

def evaluate(model, loss_function, optimizer):
  # returns:: avg_val_loss (float)
  # returns:: val_accuracy (float)
    val_loss = 0
    correct = 0
    val_examples = 0
    with torch.no_grad():
        for sentence, tags in val_data:
            # model.zero_grad()
            # get inputs ready
            sent_in = prepare_sequence(sentence, word_to_idx)
            targets = prepare_sequence(tags, tag_to_idx)

            # run forward pass
            tag_scores = model(sent_in)
            # compute loss, gradients, update params
            newloss = loss_function(tag_scores, targets)
            val_loss += newloss
            indices = torch.argmax(tag_scores,dim=1)
            pred = [tag for a in indices for (tag, index) in tag_to_idx.items() if index == a.item()]
            correct += sum([1 for i,j in zip(pred,targets.tolist()) if i==j])
            # print(pred, training_data[val_examples][0])
            val_examples += 1            
    val_accuracy = 100. * correct / val_examples
    avg_val_loss = val_loss / val_examples
    return avg_val_loss, val_accuracy

调用函数：

for epoch in range(1, 31): 
    train(epoch, model, loss_function, optimizer)

下面是这段代码，并在验证集中的单个句子上进行了测试，得出的准确度约为0.5。

for epoch in range(1):
    for sentence, tags in training_data:
        model.zero_grad()
        sent_in = prepare_sequence(sentence, word_to_idx)
        targets = prepare_sequence(tags, tag_to_idx)
        tag_scores = model(sent_in)
        loss = loss_func(tag_scores, targets) # using NLLLoss
        loss.backward()
        optimizer.step()

with torch.no_grad():
    inputs = prepare_sequence(val_data[0][0], word_to_idx)
    tag_scores = model(inputs)

非常感谢您的帮助，谢谢！

Answer 1

将训练循环更改为

for sentence, tags in training_data:    
    model.zero_grad()

    sent_in = prepare_sequence(sentence, word_to_idx)
    targets = prepare_sequence(tags, tag_to_idx)

    tag_scores = model(sent_in)
    loss = loss_function(tag_scores, targets)
    total_loss += loss.item()

    loss.backward()
    optimizer.step() 

loss是一个包含梯度的张量。 train_loss += loss将更新计算图。正确的方法是从张量中获取与损失相对应的标量并将其累加在train_loss

中