在Keras中使用return_sequences = True进行简单的LSTM训练

Question

我正在尝试使用return_sequences在Keras中训练一个简单的基于LSTM的RNN模型。因为我想要所有的时间步输出。这是我正在使用的代码：

model = Sequential()
model.add(LSTM(10, return_sequences = True, input_shape = (8, 8)))
model.add(Activation('softmax'))

model.compile(loss = 'categorical_crossentropy', optimizer = adam, metrics = ['accuracy'])
model.fit(X_train, y_train, batch_size = 1, epochs = 1000, verbose = 1)

但是我收到以下错误：

model.fit(X_train, y_train, batch_size = 1, epochs = 1000, verbose = 1)
File "C:\Program Files\Anaconda3\lib\site-packages\keras\models.py", line 1002, in fit
validation_steps=validation_steps)
File "C:\Program Files\Anaconda3\lib\site-packages\keras\engine\training.py", line 1630, in fit
batch_size=batch_size)
File "C:\Program Files\Anaconda3\lib\site-packages\keras\engine\training.py", line 1480, in _standardize_user_data
exception_prefix='target')
File "C:\Program Files\Anaconda3\lib\site-packages\keras\engine\training.py", line 113, in _standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking target: expected activation_1 to have 3 dimensions, but got array with shape (1437, 10)

但是当我更改以下行时，相同的代码工作正常：

model.add(LSTM(10, return_sequences = True, input_shape = (8, 8)))

至：

model.add(LSTM(10, return_sequences = False, input_shape = (8, 8)))

我假设我将不得不更改激活（'softmax'）图层定义，但不知道如何。有人可以帮我解决这个问题吗？

TIA

Answer 1

我在代码中看到了两个主要问题。

首先，您是否尝试使用softmax激活来获取LSTM单元的输出？作为return_sequences=True，输出形状将是[batch_size, time_steps, 10]。您可能需要使用TimeDistributed图层来确保激活层可以将时间分布式输出作为TimeDistributed(Activation('softmax'))。

接下来，你的y_train的形状[1437,10]显然不是时间分布的。要使用标签优化输出形状输出的模型[batch_size，time_steps，10]，您的标签需要具有相同的形状。因此，如果所有时间步骤重复相同的标签，我将更改您的代码，如下所示，

# Making y_train in to time distributed format, i.e. [1437,8,10]
y_train = y_train.reshape(-1,1,10)
y_train = np.repeat(y_train,8,axis=1)

model = Sequential()
model.add(LSTM(10, return_sequences = True, input_shape = (8, 8)))

# Making sure Activation layer is TimeDistributed
model.add(TimeDistributed(Activation('softmax')))

model.compile(loss = 'categorical_crossentropy', optimizer = Adam(), metrics = ['accuracy'])
model.fit(X_train, y_train, batch_size = 1, epochs = 10, verbose = 1)