仅供参考:我正在尝试将重新学习机制应用于分类任务。我知道做cus深度学习在任务中可以超越rl是没用的。无论如何,在研究目的我正在做。
我奖励代理人,如果他是正确的肯定1或不负-1并使用predict_action(predict_class)和奖励计算损失FUNC
这是我的错误,我仍然无法解决...看了S.O的一些答案,但仍然遇到麻烦
错误是;张量的元素0不需要grad,也没有grad_fn
如果我的英语技能让你觉得不舒服,我很抱歉谢谢先进
# creating model
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.pipe = nn.Sequential(nn.Linear(9, 120),
nn.ReLU(),
nn.Linear(120, 64),
nn.ReLU(),
nn.Linear(64,2),
nn.Softmax()
)
def forward(self, x):
return self.pipe(x)
def env_step(action, label, size):
total_reward = []
for i in range(size):
reward = 0
if action[i] == label[i]:
total_reward.append(reward+1)
continue
else:
total_reward.append(reward-1)
continue
return total_reward
if __name__=='__main__':
epoch_size = 100
net = Net()
criterion = nn.MSELoss()
optimizer = optim.Adam(params=net.parameters(), lr=0.01)
total_loss = deque(maxlen = 50)
for epoch in range(epoch_size):
batch_index = 0
for i in range(13):
# batch sample
batch_xs = torch.FloatTensor(train_state[batch_index: batch_index+50]) # make tensor
batch_ys = torch.from_numpy(train_label[batch_index: batch_index+50]).type('torch.LongTensor') # make tensor
# action_prob; e.g classification prob
actions_prob = net(batch_xs)
#print(actions_prob)
action = torch.argmax(actions_prob, dim=1).unsqueeze(1)
#print(action)
reward = np.array(env_step(action, batch_ys, 50))
#print(reward)
reward = torch.from_numpy(reward).unsqueeze(1).type('torch.FloatTensor')
#print(reward)
action = action.type('torch.FloatTensor')
optimizer.zero_grad()
loss = criterion(action, reward)
loss.backward()
optimizer.step()
batch_index += 50
qazxsw poi由argmax函数生成,这是不可区分的。相反,您希望在所采取的行动中获得奖励和负责概率之间的损失。
通常,在强化学习中为政策选择的“损失”就是所谓的qazxsw poi:
这是qazxsw poi采取行动的负责概率的日志的产物,获得的奖励。