如何对超过1类进行目标检测模型训练？

Question

链接：https://github.com/tensorflow/models/blob/master/research/object_detection/colab_tutorials/eager_few_shot_od_training_tf2_colab.ipynb

我已尝试使用上述 google colab 来训练具有 1 个类别的对象检测模型，如示例所示。

我正在尝试了解如何修改此代码以便能够训练 2 个类别。

在上面的示例中，在我用框注释图像后，它运行以下代码来创建

category_index

和图像/框张量。假设我修改了

num_classes = 2

并向

category_index

添加了另一个类，那么从这里开始如何进行呢？例如 - 我相信 one-hot 编码仅适用于 1 类。如何修改代码以使其适用于 2 个类？

# By convention, our non-background classes start counting at 1.  Given
# that we will be predicting just one class, we will therefore assign it a
# `class id` of 1.
duck_class_id = 1
num_classes = 1

category_index = {duck_class_id: {'id': duck_class_id, 'name': 'rubber_ducky'}}

# Convert class labels to one-hot; convert everything to tensors.
# The `label_id_offset` here shifts all classes by a certain number of indices;
# we do this here so that the model receives one-hot labels where non-background
# classes start counting at the zeroth index.  This is ordinarily just handled
# automatically in our training binaries, but we need to reproduce it here.
label_id_offset = 1
train_image_tensors = []
gt_classes_one_hot_tensors = []
gt_box_tensors = []
for (train_image_np, gt_box_np) in zip(
    train_images_np, gt_boxes):
  train_image_tensors.append(tf.expand_dims(tf.convert_to_tensor(
      train_image_np, dtype=tf.float32), axis=0))
  gt_box_tensors.append(tf.convert_to_tensor(gt_box_np, dtype=tf.float32))
  zero_indexed_groundtruth_classes = tf.convert_to_tensor(
      np.ones(shape=[gt_box_np.shape[0]], dtype=np.int32) - label_id_offset)
  gt_classes_one_hot_tensors.append(tf.one_hot(
      zero_indexed_groundtruth_classes, num_classes))
print('Done prepping data.')

Answer 1

为了单级检测教程：Rubber Ducky detector或Zombie detector。将其更改为与 multi-class 一起使用，需要进行如下更改：

• category_index

  变量必须看起来像这样。 （获取非常好的示例：category_index 示例）

    # Array of paths to the images
    train_image_filenames = [
          './datasets/train_images/train_image0001.jpg',
          './datasets/train_images/train_image0002.jpg'
          ]
    # Label map ids start from "1"
    category_index = {
         1: {'id': 1, 'name': 'cat'},
         2: {'id': 2, 'name': 'dog'}
         }
    # Array of IDs
    Gt_labels = [
        np.array([1,1]),
        np.array([1,2,2])
        ]   
    # Bounding boxes. Numpy array of [ miny, minx, maxy, maxx ] 
    Gt_boxes = [
         np.array([[0.436, 0.591, 0.629, 0.712],[0.539, 0.583, 0.73, 0.71]], dtype=np.float32),
         np.array([[0.464, 0.414, 0.626, 0.548],[0.313, 0.308, 0.648, 0.526],[0.256, 0.444, 0.484, 0.629]], dtype=np.float32)
        ]
    NUM_CLASSES = len(category_index)

• 这里

  np.ones(shape=[gt_box_np.shape[0]], dtype=np.int32)

  是无意义的（也在 Rubber Ducky detector 中），作者发现将 grunt true 类变量格式化为张量是一种非常尴尬的方式。 GT_classes条目必须采用格式Tensor("Const:0", shape=(1, NUM_CLASES), dtype=float32)和

  one_hot编码器

  （float32很重要）. 为此，必须同时替换为：
tf.one_hot
• 和

  tf.reshape

  。示例创建正确

  gt_classes_one_hot_tensors

  :

LABEL_ID_OFFSET = 1
train_image_tensors = []
gt_classes_one_hot_tensors = []
gt_box_tensors = []
# One memory-costly process is the internal conversion from (usually) numpy to tf.tensor. If you are sure that your GPU should be able to handle the batches:
# Manually convert the data to tf tensors using the CPU RAM, and only then pass it to your model (eventually using GPU).
print("\nUse CPU, GPU dont have space to locate one_hot_tensors. Use for hot_encoders:", str(tf.config.experimental.list_physical_devices('CPU')[0]))
with tf.device('/cpu:0'): # Recomended for this step Use CPU, if GPU dont have space
    for (train_image_np, gt_box_np, gt_label_np) in zip(train_images_np, Gt_boxes, Gt_labels):
        print("|", end="")
        train_image_tensors.append(tf.expand_dims(tf.convert_to_tensor(train_image_np, dtype=tf.float32), axis=0)) # image in np format train_image_np 
        gt_box_tensors.append(tf.convert_to_tensor(gt_box_np, dtype=tf.float32)) # put box in Tensor
        zero_indexed_groundtruth_classes = tf.convert_to_tensor(gt_label_np - LABEL_ID_OFFSET) # put labels in Numpy array (min:0)
        gt_classes_one_hot_tensors.append(tf.one_hot(zero_indexed_groundtruth_classes, num_classes)) # label Tensor to one hot
print('Done prepping data. Num Data: ',len(train_images_np), "\n" )

更多信息和 

完整示例 完整示例额外 github 示例链接 提示：如果您开始使用这些教程

Rubber Ducky detector

，我建议您阅读：tensorflow 的对象检测 api 支持多类多标签检测吗？