深度学习 CNN ValueError：as_list() 未在未知 TensorShape 上定义

Question

我已经使用以下代码加载了对图像数据进行预处理的数据集：

data = tf.keras.utils.image_dataset_from_directory('/content/drive/MyDrive/PengantarSainsData/Capstone2/dataset_revisi')
data_iterator = data.as_numpy_iterator()
batch = data_iterator.next()
def preprocess(x, y):
    x_normalized = x / 255
    y_one_hot = tf.keras.utils.to_categorical(y, num_classes=5)
    return x_normalized, y_one_hot
data = data.map(lambda x, y: tf.py_function(func=preprocess, inp=[x, y], Tout=[tf.float32, tf.float32]))
scaled_iterator = data.as_numpy_iterator()
batch = scaled_iterator.next()

然后使用以下代码将其划分为训练、验证和测试数据：

train_size = int(len(data) * .7)
val_size = int(len(data) * .2)
test_size = int(len(data) * .1)
train = data.take(train_size)
val = data.skip(train_size).take(val_size)
test = data.skip(train_size + val_size).take(test_size)

还创建了模型架构，这样：

model.add(Conv2D(16, (3, 3), 1, activation = 'relu', input_shape = (256, 256, 3)))
model.add(MaxPooling2D())

model.add(Conv2D(32, (3, 3), 1, activation = 'relu'))
model.add(MaxPooling2D())

model.add(Conv2D(16, (3, 3), 1, activation = 'relu'))
model.add(MaxPooling2D())

model.add(Flatten())
model.add(Dense(128, activation = 'relu')) # 256 number of units used in dense layer
model.add(Dense(5, activation = 'softmax')) # sigmoid represents 0 and 1 output

model.compile('adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

model.summary()

然后当我想要进行训练过程时：

logdir = 'logs'
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir = logdir)
hist = model.fit(train, epochs = 16, validation_data = val, callbacks = [tensorboard_callback])

它不断抛出图片所附的错误。我应该怎么办？已经为此苦苦挣扎了2天

我尝试了不同的模型，并使用下面的代码更具体地定义其架构

model.build(input_shape=(None, 256, 256, 3))

但还是没成功

Answer 1

我认为您使用

as_numpy_iterator

做了一些不必要的步骤。您的

image_dataset_from_directory

是一个

tf.data.Dataset

，您可以直接操作它，而无需将其转换为 NumPy 迭代器。您只需要使用

tf.one_hot

而不是

tf.keras.utils.to_categorical

。

这是一个完整的示例，使用 MNIST 的本地版本（您必须更改路径和类别数量：

import tensorflow as tf
from tensorflow.keras.layers import *

data = tf.keras.utils.image_dataset_from_directory(r'path\to\mnist\test')


def preprocess(x, y):
    x_normalized = x / 255
    y_one_hot = tf.one_hot(tf.cast(y, tf.int32), depth=10)
    return x_normalized, y_one_hot


data = data.map(preprocess)

train_size = int(len(data) * .7)
val_size = int(len(data) * .2)
test_size = int(len(data) * .1)
train = data.take(train_size)
val = data.skip(train_size).take(val_size)
test = data.skip(train_size + val_size).take(test_size)

model = tf.keras.models.Sequential()
model.add(Conv2D(16, (3, 3), 1, activation='relu', input_shape=(256, 256, 3)))
model.add(MaxPooling2D())

model.add(Conv2D(32, (3, 3), 1, activation='relu'))
model.add(MaxPooling2D())

model.add(Conv2D(16, (3, 3), 1, activation='relu'))
model.add(MaxPooling2D())

model.add(Flatten())
model.add(Dense(128, activation='relu'))   
model.add(Dense(10, activation='softmax')) 

model.compile('adam', loss='categorical_crossentropy', metrics=['accuracy'])

model.summary()

logdir = 'logs'
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=logdir)
hist = model.fit(train, epochs=1, validation_data=val, callbacks=[tensorboard_callback])