在推理时去除Keras模型中的辅助分支。

问题描述 投票:0回答:1

我正在尝试实现一个结合了LSTM和CNN模型的预测算法,从 本文. 本质上,该论文提出了一个具有三个分支的模型:一个CNN分支,一个LSTM分支,以及一个结合两者的合并分支。前两个分支只在训练过程中出现,以防止过拟合,并确保最终模型同时针对CNN和LSTM特征进行训练。这是论文中的图(总损失函数中的α、β和gamma只是这些特定损失的权重)。lstm-cnn model structure据我了解,这些类似于ResNet和Inception模型中的辅助分支,以确保每一层都对模型输出有贡献。我据此实现了这一点。

def construct_lstm_cnn(look_forward, look_back=30):
    cnn = construct_cnn(look_forward, fc=False)
    cnn_flatten = Flatten()(cnn.output)
    lstm = construct_lstm(look_forward, look_back, 2, fc=False)

    #Merged layer (the main branch that will be making prediction after training)
    cnn_lstm = concatenate([cnn_flatten, lstm.output])
    fc_merged    = Dense(500, activation='relu')(cnn_lstm)
    drop_merged  = Dropout(0.5)(fc_merged)
    fc2_merged   = Dense(100, activation='relu')(drop_merged)
    drop2_merged = Dropout(0.5)(fc2_merged)
    fc3_merged   = Dense(25 , activation='relu')(drop2_merged)
    drop3_merged = Dropout(0.5)(fc3_merged)
    pred_merged  = Dense(look_forward, activation='linear')(drop3_merged)

    #Auxiliary branch for cnn (want to remove at inference time)
    fc_cnn    = Dense(500, activation='relu')(cnn_flatten)
    drop_cnn  = Dropout(0.5)(fc_cnn)
    fc2_cnn   = Dense(100, activation='relu')(drop_cnn)
    drop2_cnn = Dropout(0.5)(fc2_cnn)
    fc3_cnn   = Dense(25 , activation='relu')(drop2_cnn)
    drop3_cnn = Dropout(0.5)(fc3_cnn)
    pred_cnn_aux  = Dense(look_forward, activation='linear')(drop3_cnn)

    #Auxiliary branch for lstm (want to remove at inference time)
    fc_lstm    = Dense(500, activation='relu')(lstm.output)
    drop_lstm  = Dropout(0.5)(fc_lstm)
    fc2_lstm   = Dense(100, activation='relu')(drop_lstm)
    drop2_lstm = Dropout(0.5)(fc2_lstm)
    fc3_lstm   = Dense(25 , activation='relu')(drop2_lstm)
    drop3_lstm = Dropout(0.5)(fc3_lstm)
    pred_lstm_aux  = Dense(look_forward, activation='linear')(drop3_lstm)

    #Final model with three branches
    model = Model(inputs=[cnn.input, lstm.input], outputs=[pred_merged, pred_cnn_aux, pred_lstm_aux],    name="lstm-cnn")
    return model

然而,我似乎在Keras中找不到一种方法来移除列出的辅助分支。有什么方法可以让我删除那些在推理时没用的层吗?

python tensorflow keras deep-learning keras-layer
1个回答
1
投票

我为你提供一个简化的例子

这里有完整的模型和所有的分支... 这是要适合的模型。

def construct_lstm_cnn():

    inp_lstm = Input((20,30))
    lstm = LSTM(32, activation='relu')(inp_lstm)
    inp_cnn = Input((32,32,3))
    cnn = Conv2D(32, 3, activation='relu')(inp_cnn)
    cnn = Flatten()(cnn)

    cnn_lstm = Concatenate()([cnn, lstm])
    cnn_lstm = Dense(1)(cnn_lstm)

    fc_cnn = Dense(32, activation='relu')(cnn)
    fc_cnn = Dropout(0.5)(fc_cnn)
    fc_cnn = Dense(1)(fc_cnn)

    fc_lstm = Dense(32, activation='relu')(lstm)
    fc_lstm = Dropout(0.5)(fc_lstm)
    fc_lstm = Dense(1)(fc_lstm)

    model = Model(inputs=[inp_cnn, inp_lstm], outputs=[cnn_lstm, fc_cnn, fc_lstm])
    return model

lstm_cnn = construct_lstm_cnn()
lstm_cnn.compile(...)
lstm_cnn.summary()

lstm_cnn.fit(...)

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_10 (InputLayer)           [(None, 32, 32, 3)]  0                                            
__________________________________________________________________________________________________
conv2d_18 (Conv2D)              (None, 30, 30, 32)   896         input_10[0][0]                   
__________________________________________________________________________________________________
input_9 (InputLayer)            [(None, 20, 30)]     0                                            
__________________________________________________________________________________________________
flatten_3 (Flatten)             (None, 28800)        0           conv2d_18[0][0]                  
__________________________________________________________________________________________________
lstm_5 (LSTM)                   (None, 32)           8064        input_9[0][0]                    
__________________________________________________________________________________________________
dense_13 (Dense)                (None, 32)           921632      flatten_3[0][0]                  
__________________________________________________________________________________________________
dense_15 (Dense)                (None, 32)           1056        lstm_5[0][0]                     
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 28832)        0           flatten_3[0][0]                  
                                                                 lstm_5[0][0]                     
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 32)           0           dense_13[0][0]                   
__________________________________________________________________________________________________
dropout_4 (Dropout)             (None, 32)           0           dense_15[0][0]                   
__________________________________________________________________________________________________
dense_12 (Dense)                (None, 1)            28833       concatenate_1[0][0]              
__________________________________________________________________________________________________
dense_14 (Dense)                (None, 1)            33          dropout_3[0][0]                  
__________________________________________________________________________________________________
dense_16 (Dense)                (None, 1)            33          dropout_4[0][0]                  
==================================================================================================

为推理时间,在训练后,我们可以简单地用这种方式去除无用的分支。

lstm_cnn_inference = Model(lstm_cnn.input, lstm_cnn.output[0])
lstm_cnn_inference.summary()

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_10 (InputLayer)           [(None, 32, 32, 3)]  0                                            
__________________________________________________________________________________________________
conv2d_18 (Conv2D)              (None, 30, 30, 32)   896         input_10[0][0]                   
__________________________________________________________________________________________________
input_9 (InputLayer)            [(None, 20, 30)]     0                                            
__________________________________________________________________________________________________
flatten_3 (Flatten)             (None, 28800)        0           conv2d_18[0][0]                  
__________________________________________________________________________________________________
lstm_5 (LSTM)                   (None, 32)           8064        input_9[0][0]                    
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 28832)        0           flatten_3[0][0]                  
                                                                 lstm_5[0][0]                     
__________________________________________________________________________________________________
dense_12 (Dense)                (None, 1)            28833       concatenate_1[0][0]              
==================================================================================================

这样一来,我们就只保留了中心分支

© www.soinside.com 2019 - 2024. All rights reserved.