无法将额外数据连接到 CNN + RNN 架构

Question

我正在尝试创建一个 CNN + LSTM 网络，但除了 CNN 输出之外，LSTM 还会添加其他数据作为输入。我正在尝试连接 CNN 输出和附加输入，但我无法让它工作。

`cnn_input = 输入(形状=(256, 256, 3), name="cnn_input") cnn_layer1 = Conv2D(32, (3, 3), strides=1, padding="相同", 激活="relu")( cnn_输入 ） cnn_layer1_pooling = MaxPooling2D((2, 2), strides=2, padding="相同")(cnn_layer1)

    cnn_layer2 = Conv2D(64, (3, 3), strides=1, padding="same", activation="relu")(
        cnn_layer1_pooling
    )
    cnn_layer2_pooling = MaxPooling2D((2, 2), strides=2, padding="same")(cnn_layer2)

    cnn_layer3 = Conv2D(128, (3, 3), strides=1, padding="same", activation="relu")(
        cnn_layer2_pooling
    )
    cnn_layer3_pooling = MaxPooling2D((2, 2), strides=2, padding="same")(cnn_layer3)

    cnn_layer4 = Conv2D(256, (3, 3), strides=1, padding="same", activation="relu")(
        cnn_layer3_pooling
    )
    cnn_output = TimeDistributed(Flatten())(cnn_layer4)
    cnn_output_dense = Dense(700, activation="relu")(cnn_output)
    # Define the input for weather data
    weather_input = Input(shape=(4,), name="weather_input")

    # Concatenate the CNN output and weather data
    merged = Concatenate(cnn_output_dense, weather_input)

    # Apply the ReshapeLayer and LSTM
    lstm_layer1 = LSTM(64, return_sequences=True)(cnn_output_dense)
    lstm_layer2 = LSTM(32, return_sequences=True)(lstm_layer1)
    lstm_output = LSTM(32)(lstm_layer2)

    # Dense layer for final output
    dense_output = Dense(2, activation="softmax")(lstm_output)

    # Build the model
    model = Model(inputs=cnn_input, outputs=dense_output)
    model.compile(
        loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"]
    )
    model.summary()`

Answer 1

TensorFlow 提供的连接层采用张量列表作为输入。您可以在 Concatenate 层的文档页面上查看更多详细信息，并查看那里的代码示例。

只需在 Concatenate() 中添加方括号，结果如下：

merged = Concatenate([cnn_output_dense, weather_input])

试试这个，你的代码就会运行得很好。