如何将 KeraCV MixUp 和 CuMix Augmentation 集成到图像数据生成器中？

Question

我有以下几行代码，用于读取 csv 文件并准备一个数据集，以便使用图像数据生成器类将其输入到张量流模型中。这使用子目录名称作为类。这里如何集成KeraCV的CutMix和MixUp？

    import pandas as pd
    from collections import defaultdict
    import os
    
    
    def find_duplicate_filenames(root_dir):
        files_dict = defaultdict(list)
        for subdir, dirs, files in os.walk(root_dir):
            for file in files:
                file_path = os.path.join(subdir, file)
                files_dict[file].append(file_path)
        duplicates = {file: paths for file, paths in files_dict.items() if len(paths) > 1}
        return duplicates
    
    # Path to the directory with training images
    root_directory = 'train_images/'
    
    # Identifying duplicates
    duplicates_by_name = find_duplicate_filenames(root_directory)
    
    # Create a set of paths to exclude
    exclude_paths = {path for paths in duplicates_by_name.values() for path in paths}
    
    # Collect all valid image paths and their labels
    valid_files = []
    labels = []
    for subdir, dirs, files in os.walk(root_directory):
        for file in files:
            file_path = os.path.join(subdir, file)
            if file_path not in exclude_paths:
                valid_files.append(file_path)
                labels.append(subdir.split('/')[-1])  # assuming folder names are class labels
    
    # Create DataFrame for the image paths and labels
    data = pd.DataFrame({'filename': valid_files, 'class': labels})
    
    
    data = data[data['class']!='']
    
    print(len(data['class'].unique()))

from tensorflow.keras.preprocessing.image import ImageDataGenerator
import numpy as np

def custom_preprocessing(image):
    # Example of custom preprocessing logic
    # Adjust the image as required, this is just a placeholder
    image = np.clip(image, a_min=0, a_max=1)  # Ensuring the image is within the range [0, 1]
    return image

# ImageDataGenerator with custom preprocessing
train_datagen = ImageDataGenerator(
    preprocessing_function=custom_preprocessing,  # Using the custom preprocessing function
    width_shift_range=0.5,                         # Randomly shifts images horizontally
    horizontal_flip=False,                         # Randomly flips images horizontally
    vertical_flip=False                            # Randomly flips images vertically
)

# Continue with your model training setup


# Setup train and validation generators
train_generator = train_datagen.flow_from_dataframe(
    dataframe=train_data,
    x_col='filename',
    y_col='class',
    target_size=(224, 224),
    batch_size=64,
    class_mode='categorical',    
    color_mode='rgb'
)

validation_generator = train_datagen.flow_from_dataframe(
    dataframe=valid_data,
    x_col='filename',
    y_col='class',
    target_size=(224, 224),
    batch_size=64,
    class_mode='categorical',
    color_mode='rgb'
)

如何在此处添加 KerasCV CutMix 和 MixUp 层以添加到增强中？我尝试将

keras_cv.layers.CutMix()

和

keras_cv.layers.MixUp()

添加到模型中，但没有成功。预先感谢！

Answer 1

ImageDataGenerator 已弃用，您应该切换到他们在here展示的工作流程以进行增强和一般的 tf.data.Dataset。您可以阅读这里为什么

Dataset

真的很方便。
对于增强来说，这变得非常简单。您可以定义一个

Sequential

模型，您可以在其中添加不同的增强层：

import tensorflow as tf
import keras
import keras_cv

augmentations = tf.keras.Sequential([
    keras.layers.RandomTranslation(0, 0.5),
    keras.layers.RandomFlip('horizontal'),
    keras_cv.layers.CutMix()
])

# example how to use in a net
real_model = tf.keras.Sequential([
    tf.keras.Input([input_size]),
    augmentations,
    tf.keras.Dense(3)

虽然我通常不会混合

tf.keras

和

keras

调用，但在这里我什至将它们与

keras_cv

混合使用，并且它适用于模型。您可以将这些层用作网络中的第一层，就像最后的代码行一样。它们仅用于训练数据。