刚接触tensorflow,已经开始使用tensorflow 2.0
我已经为多类分类问题构建了一个张量流数据集。让我们称之为
labeled_ds
。我通过从各自的类明智目录加载所有图像文件来准备这个数据集。我在这里跟随教程:tensorflow guide to load image dataset
现在,我需要将
labeld_ds
分成三个不相交的部分:训练、验证和测试。我正在浏览 tensorflow API,但没有允许指定拆分百分比的示例。我在load方法中找到了一些东西,但我不确定如何使用它。此外,我怎样才能让分裂分层?
# labeled_ds contains multi class data, which is unbalanced.
train_ds, val_ds, test_ds = tf.data.Dataset.tfds.load(labeled_ds, split=["train", "validation", "test"])
我被困在这里,如果有任何关于如何从这里取得进展的建议,我将不胜感激。提前致谢。
请参考以下代码使用张量流数据集“oxford_flowers102”创建训练、测试和验证拆分
!pip install tensorflow==2.0.0
import tensorflow as tf
print(tf.__version__)
import tensorflow_datasets as tfds
labeled_ds, summary = tfds.load('oxford_flowers102', split='train+test+validation', with_info=True)
labeled_all_length = [i for i,_ in enumerate(labeled_ds)][-1] + 1
train_size = int(0.8 * labeled_all_length)
val_test_size = int(0.1 * labeled_all_length)
df_train = labeled_ds.take(train_size)
df_test = labeled_ds.skip(train_size)
df_val = df_test.skip(val_test_size)
df_test = df_test.take(val_test_size)
df_train_length = [i for i,_ in enumerate(df_train)][-1] + 1
df_val_length = [i for i,_ in enumerate(df_val)][-1] + 1
df_test_length = [i for i,_ in enumerate(df_test)][-1] + 1
print('Original: ', labeled_all_length)
print('Train: ', df_train_length)
print('Validation :', df_val_length)
print('Test :', df_test_length)
我有同样的问题
取决于数据集,大部分都有训练集和测试集。在这种情况下,您可以执行以下操作(假设 80-10-10 拆分):
splits, info = tfds.load('fashion_mnist', with_info=True, as_supervised=True,
split=['train+test[:80]','train+test[80:90]', 'train+test[90:]'],
data_dir=filePath)
Francesco Boi Soultion 对我很有用。
splits, info = tfds.load('fashion_mnist', with_info=True, as_supervised=True, split=['train+test[:80]','train+test[80:90]', 'train+test[90:]'])
(train_examples, validation_examples, test_examples) = splits