如何连接两个CNN

Question

我有两个基于 MobileNet 的 CNN 代码（第一和第二个 CNN 相同）：

img_height, img_width = 224, 224
num_classes = 30
input_shape = (img_height, img_width, 3)

epochs = 1


base_model_fingerprint = MobileNet(weights='imagenet', include_top=False, input_shape=input_shape)

x = base_model_fingerprint.output
x = GlobalAveragePooling2D()(x)
x = Dropout(0.5)(x)
output_fingerprint = Dense(num_classes, activation='softmax')(x)

model_fingerprint = Model(inputs=base_model_fingerprint.input, outputs=output_fingerprint)


for layer in model_fingerprint.layers:
    layer._name = 'fingerprint_' + layer.name


for layer in base_model_fingerprint.layers:
    layer.trainable = False

model_fingerprint.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])


train_datagen_fingerprint = ImageDataGenerator(
        rescale=1./255,
        rotation_range=20,
        width_shift_range=0.2,
        height_shift_range=0.2,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True)

test_datagen_fingerprint = ImageDataGenerator(rescale=1. / 255)

train_generator_fingerprint = train_datagen_fingerprint.flow_from_directory(
        'C:/Users/giova/Desktop/CNN_FINGER_RESIZE/TRAIN',
        target_size=(img_height, img_width),
        batch_size=32,
        class_mode='categorical')

test_generator_fingerprint = test_datagen_fingerprint.flow_from_directory(
        'C:/Users/giova/Desktop/CNN_FINGER_RESIZE/TEST',
        target_size=(img_height, img_width),
        batch_size=32,
        class_mode='categorical')


reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=0.0001)
early_stop = EarlyStopping(monitor='val_loss', patience=10)


history = model_fingerprint.fit(train_generator_fingerprint,
                                validation_data=test_generator_fingerprint,
                                epochs=epochs,
                                callbacks=[reduce_lr, early_stop])


test_loss, test_acc = model_fingerprint.evaluate(test_generator_fingerprint, verbose=2)
print('Test accuracy:', test_acc)

如何创建第三个 CNN 以获得更高的准确性？

我试过使用 model.save 所以我可以使用 .h5 文件但是它不起作用 第二个 CNN 仅更改从中获取输入数据的目录。

更新：我写了这段代码但是无法解决错误：

import tensorflow as tf
from tensorflow.keras.models import Model, load_model

# Carica i modelli delle due CNN
model1 = load_model('pesi_fingerprint.h5')
model2 = load_model('pesi_palmprint.h5')

# Rimuove l'ultimo livello di ciascuna CNN
model1.layers.pop()
model2.layers.pop()

# Imposta i layer delle due CNN come non trainabili
for layer in model1.layers:
    layer.trainable = False
    layer._name = 'model1_' + layer.name
for layer in model2.layers:
    layer.trainable = False
    layer._name = 'model2_' + layer.name

# Crea il nuovo modello concatenando le feature maps
concatenated = tf.keras.layers.Concatenate()([model1.layers[-1].output, model2.layers[-1].output])

x = tf.keras.layers.Reshape((6, 5, 3))(concatenated) # aumenta le dimensioni
x = tf.keras.layers.Conv2D(16, kernel_size=(3, 3), activation='relu', input_shape=(6, 5, 3))(x) # utilizza un kernel di convoluzione 3x3
x = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(x)
x = tf.keras.layers.Conv2D(8, kernel_size=(3,3), activation='relu', padding='same')(x)
x = tf.keras.layers.Conv2D(8, kernel_size=(3,3), activation='relu', padding='same')(x)
x = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(x)
x = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(x)
x = tf.keras.layers.Flatten()(x)
x = tf.keras.layers.Dense(128, activation='relu')(x)

output = tf.keras.layers.Dense(num_classes, activation='softmax')(x)

# Crea il modello finale
model = Model(inputs=[model1.input, model2.input], outputs=output)

# Compila il modello
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# Stampa la struttura del modello
model.summary()

from tensorflow.keras.preprocessing.image import ImageDataGenerator

train_datagen = ImageDataGenerator(
    rescale=1./255, # normalizza i valori dei pixel tra 0 e 1
    rotation_range=20, # ruota le immagini in modo casuale
    width_shift_range=0.2, # sposta le immagini in orizzontale in modo casuale
    height_shift_range=0.2, # sposta le immagini in verticale in modo casuale
    shear_range=0.2, # applica la deformazione di taglio alle immagini
    zoom_range=0.2, # applica lo zoom alle immagini
    horizontal_flip=True, # inverte le immagini in orizzontale in modo casuale
    fill_mode='nearest',
    target_size = (224,224) # riempie i pixel mancanti con il valore più vicino
)

test_datagen = ImageDataGenerator(rescale=1./255,
                                  target_size=(224, 224))

# specifica il percorso della cartella che contiene i dati di training e di test
train_dir = 'C:\\Users\\giova\\Desktop\\MERGE CNN\\TRAIN'
test_dir = 'C:\\Users\\giova\\Desktop\\MERGE CNN\\TEST'

# imposta il numero di classi
num_classes = 30

# leggi i dati di training dalla cartella specificata
train_generator = train_datagen.flow_from_directory(
    train_dir,
    target_size=(224, 224),
    batch_size=32,
    class_mode='categorical'
)

# leggi i dati di test dalla cartella specificata
test_generator = test_datagen.flow_from_directory(
    test_dir,
    target_size=(224, 224),
    batch_size=32,
    class_mode='categorical'
)

# Addestramento del modello
# Addestramento del modello
model.fit(train_generator, epochs=10, validation_data=test_generator)

错误： ValueError：调用层“reshape”（类型 Reshape）时遇到异常。

新数组的总大小必须不变，input_shape = [60], output_shape = [6, 5, 3]

“reshape”层接收的调用参数（类型 Reshape）： • inputs=tf.Tensor(shape=(None, 60), dtype=float32)

Answer 1

串联模型的输出形状是

[60]

，但您试图将其重塑为

[6,5,3]

，这是不可能的，因为它改变了数组的总大小，例如：6x5x3！= 60，请尝试将其重塑为

[6,5,2] 

相反。

Answer 2

错误是Reshape层的输出形状不正确造成的。 Reshape 层的输入有 60 个元素，但指定的输出形状 (6, 5, 3) 包含 90 个元素。要解决此问题，您应该将输出形状更改为 (6, 10, 1) 或总共具有 60 个元素的任何其他形状：

替换这一行：

x = tf.keras.layers.Reshape((6, 5, 3))(concatenated) # aumenta le dimensioni

搭配：

x = tf.keras.layers.Reshape((6, 10, 1))(concatenated) # aumenta le dimensioni

现在新数组的总大小将保持不变（即 60 个元素）。