如何在使用tf.py_function的映射函数中将Tensor转换为numpy数组

Question

我正在尝试为对象检测网络创建图像增强管道，在此过程中，我的训练示例进入网络时将得到增强。图像和边界框需要增强，但是标准的tf.image方法不适用于边界框数据。

所有与边界框一起使用的简单扩充库都需要numpy数组，但是我不知道如何在.map（）函数中将张量转换为numpy数组。即使我将我的增强函数包装在tf.py_function调用中，当我尝试通过AttributeError: 'Tensor' object has no attribute 'numpy'转换图像时，仍然出现错误image = image.numpy()。

我的数据集是通过此方式加载的：

def load_tfrecord_dataset(file_pattern, class_file, size=416):
    LINE_NUMBER = -1
    class_table = tf.lookup.StaticHashTable(tf.lookup.TextFileInitializer(
        class_file, tf.string, 0, tf.int64, LINE_NUMBER, delimiter="\n"), -1)

    files = tf.data.Dataset.list_files(file_pattern)
    dataset = files.flat_map(tf.data.TFRecordDataset)
    return dataset.map(lambda x: tf.py_function(parse_tfrecord(x, class_table, size), [x], tf.float32))
    # return dataset.map(lambda x: parse_tfrecord(x, class_table, size))

这将调用我的解析函数：

def parse_tfrecord(tfrecord, class_table, size):
    x = tf.io.parse_single_example(tfrecord, IMAGE_FEATURE_MAP)
    x_train = tf.image.decode_jpeg(x['image/encoded'], channels=3)
    x_train = tf.image.resize(x_train, (size, size))

    class_text = tf.sparse.to_dense(
        x['image/object/class/text'], default_value='')
    labels = tf.cast(class_table.lookup(class_text), tf.float32)

    y_train = tf.stack([tf.sparse.to_dense(x['image/object/bbox/xmin']),
                        tf.sparse.to_dense(x['image/object/bbox/ymin']),
                        tf.sparse.to_dense(x['image/object/bbox/xmax']),
                        tf.sparse.to_dense(x['image/object/bbox/ymax']),
                        labels], axis=1)

    x_train, y_train = tf.py_function(augment_images(x_train, y_train), [], tf.uint8)

    paddings = [[0, FLAGS.yolo_max_boxes - tf.shape(y_train)[0]], [0, 0]]
    y_train = tf.pad(y_train, paddings)

    return x_train, y_train

它调用了我的增强函数：

def augment_images(image, boxes):

    image = image.numpy()

    seq = iaa.Sequential([
        iaa.Fliplr(0.5),
        iaa.Flipud(0.5)
    ])

    image, label = seq(image=image, bounding_boxes=boxes)

    return image, label

但是无论我将代码的哪些部分包装在tf.py_function中还是尝试在哪里转换为numpy数组，我总是会遇到相同的错误。

我在做什么错？

Answer 1

在下面的简单程序中，我正在读取鸟的图像，然后再进行tf.image.central_crop裁剪图像的中央部分。

重新创建问题的代码-

%tensorflow_version 2.x
import tensorflow as tf
print(tf.__version__)
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.preprocessing.image import img_to_array, array_to_img
from matplotlib import pyplot as plt
import numpy as np

def load_file_and_process(path):
    image = load_img(bytes.decode(path.numpy()), target_size=(224, 224))
    image = image.numpy()
    image = tf.image.central_crop(image, np.random.uniform(0.50, 1.00))
    return image

train_dataset = tf.data.Dataset.list_files('/content/bird.jpg')
train_dataset = train_dataset.map(lambda x: tf.py_function(load_file_and_process, [x], [tf.float32]))

for f in train_dataset:
  for l in f:
    image = np.array(array_to_img(l))
    plt.imshow(image)

输出-

2.2.0
---------------------------------------------------------------------------
UnknownError                              Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/context.py in execution_mode(mode)
   1985       ctx.executor = executor_new
-> 1986       yield
   1987     finally:

10 frames
UnknownError: AttributeError: 'Image' object has no attribute 'numpy'
Traceback (most recent call last):

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/script_ops.py", line 241, in __call__
    return func(device, token, args)

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/script_ops.py", line 130, in __call__
    ret = self._func(*args)

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/autograph/impl/api.py", line 309, in wrapper
    return func(*args, **kwargs)

  File "<ipython-input-22-2aab1a57781b>", line 11, in load_file_and_process
    image = image.numpy()

AttributeError: 'Image' object has no attribute 'numpy'


     [[{{node EagerPyFunc}}]] [Op:IteratorGetNext]

During handling of the above exception, another exception occurred:

UnknownError                              Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/executor.py in wait(self)
     65   def wait(self):
     66     """Waits for ops dispatched in this executor to finish."""
---> 67     pywrap_tfe.TFE_ExecutorWaitForAllPendingNodes(self._handle)
     68 
     69   def clear_error(self):

UnknownError: AttributeError: 'Image' object has no attribute 'numpy'
Traceback (most recent call last):

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/script_ops.py", line 241, in __call__
    return func(device, token, args)

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/script_ops.py", line 130, in __call__
    ret = self._func(*args)

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/autograph/impl/api.py", line 309, in wrapper
    return func(*args, **kwargs)

  File "<ipython-input-22-2aab1a57781b>", line 11, in load_file_and_process
    image = image.numpy()

AttributeError: 'Image' object has no attribute 'numpy'


     [[{{node EagerPyFunc}}]]

解决方案

-使用img_to_array和array_to_img分别将图像转换为数组，并将数组转换为图像。您需要在程序中包含from tensorflow.keras.preprocessing.image import img_to_array, array_to_img。

固定代码-

%tensorflow_version 2.x
import tensorflow as tf
print(tf.__version__)
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.preprocessing.image import img_to_array, array_to_img
from matplotlib import pyplot as plt
import numpy as np

def load_file_and_process(path):
    image = load_img(bytes.decode(path.numpy()), target_size=(224, 224))
    image = img_to_array(image)
    image = tf.image.central_crop(image, np.random.uniform(0.50, 1.00))
    return image

train_dataset = tf.data.Dataset.list_files('/content/bird.jpg')
train_dataset = train_dataset.map(lambda x: tf.py_function(load_file_and_process, [x], [tf.float32]))

for f in train_dataset:
  for l in f:
    image = np.array(array_to_img(l))
    plt.imshow(image)

输出-

2.2.0

希望这能回答您的问题。祝您学习愉快。