自定义损失函数：如何使用Tensorflow在keras中的损失函数中添加隐藏层的输出

Question

在我的模型中，隐藏层的输出（即“编码”）有两个通道（例如形状：[none, 128, 128, 2]）。我希望在损失函数中这两个通道之间添加SSIM：

损失= ssim（输入，输出）+ theta * ssim（编码（通道1），编码（通道2））。

我该如何实现这个？以下是我的模型的架构。

def structural_similarity_index(y_true, y_pred):
    loss = 1 - tf.image.ssim(y_true, y_pred, max_val=1.0) 
    return loss

def mymodel():
    input_img = Input(shape=(256, 256, 1))

    # encoder
    x = Conv2D(4, (3, 3), activation='relu', padding='same')(input_img)
    x = MaxPooling2D((2, 2), padding='same')(x)
    encoded = Conv2D(2, (3, 3), activation='relu', padding='same', name='encoder')(x)

    # decoder    
    x = Conv2D(4, (3, 3), activation='relu', padding='same')(encoded)
    x = UpSampling2D((2, 2))(x)
    decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)

    autoencoder = Model(input_img, decoded)    
    autoencoder.compile(optimizer = 'adadelta', loss = structural_similarity_index)
    autoencoder.summary()        
    return autoencoder

我尝试定义一个“loss_warper”函数，如下所示，但它不起作用。这就是我添加损失函数的方式：

autoencoder.add_loss(loss_wrapper(encoded[:,:,:,0],encoded[:,:,:,1])(input_img, decoded))

“loss_warper”函数：

def loss_wrapper(CH1, CH2):
    def structural_similarity_index(y_true, y_pred):
        regweight = 0.01
        loss = 1 - tf.image.ssim(y_true, y_pred, max_val=1.0)
        loss = loss + regweight*(1-tf.image.ssim(CH1, CH2, max_val=1.0))
        return loss
    return structural_similarity_index

错误信息：

File "E:/Autoencoder.py", line 160, in trainprocess
    validation_data= (x_validate, x_validate))
...
ValueError: ('Error when checking model target: expected no data, but got:', array([...]...[...]))

有人知道如何实现吗？非常感谢任何帮助！

Answer 1

您遇到的错误可能是由于您向模型添加附加损失函数的方式造成的。您可以修改自定义损失函数以直接包含两个通道之间的 SSIM 计算，而不是使用 add_loss() 方法。您可以尝试下面更新的代码，其中 ssim_loss_encoded 计算为编码张量的通道 1 和通道 2 之间的 SSIM 损失。然后，combined_loss 函数使用指定的权重 theta 将 y_true 和 y_pred 之间的 SSIM 损失与 ssim_loss_encoded 组合起来：

import tensorflow.keras.backend as K

def ssim_loss(y_true, y_pred):
    ssim_loss = 1 - K.mean(tf.image.ssim(y_true, y_pred, max_val=1.0))
    return ssim_loss

def mymodel():
    input_img = Input(shape=(256, 256, 1))


# encoder
x = Conv2D(4, (3, 3), activation='relu', padding='same')(input_img)
x = MaxPooling2D((2, 2), padding='same')(x)
encoded = Conv2D(2, (3, 3), activation='relu', padding='same', name='encoder')(x)

# decoder    
x = Conv2D(4, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)

# Calculate SSIM loss between the two channels of 'encoded'
channel1 = encoded[:,:,:,0]
channel2 = encoded[:,:,:,1]
ssim_loss_encoded = 1 - K.mean(tf.image.ssim(channel1, channel2, max_val=1.0))

# Define the combined loss function
theta = 0.01
combined_loss = lambda y_true, y_pred: ssim_loss(y_true, y_pred) + theta * ssim_loss_encoded

autoencoder = Model(input_img, decoded)    
autoencoder.compile(optimizer='adadelta', loss=combined_loss)
autoencoder.summary()        
return autoencoder