QAT 使用 TF2 和 MobileNetV3-large 进行全整数量化

Question

我有一个 Google Colab 文件，我正在尝试迁移学习和微调 MobileNetV3-Large 模型以进行二元分类，然后进行完整整数量化。我的目标是在我已连接到 Raspberry Pi 的 Coral TPU USB 加速器上运行该模型。

最初，我尝试了训练后量化，我能够运行它，但由于模型的硬激活，在此过程后准确性大幅下降。现在，我正在尝试使用量化感知训练，但 tfmot 遇到很多问题。 在顺序模型中，定义基本模型 (mobilenetv3) 后，我遵循 GlobalAvgPooling2D、dropout 和密集层。但是，当尝试从 tfmot 运行 quantize_model 时，这会导致错误“不支持在另一个 tf.keras 模型内量化 tf.keras 模型”。

我使用的代码如下。感谢您的帮助。

import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, regularizers, callbacks
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import GlobalAveragePooling2D, Dropout, Dense
from tensorflow.keras.applications import MobileNetV3Large
import tensorflow_model_optimization as tfmot

print("#### Import GDrive ####")
from google.colab import drive
drive.mount('/content/drive')

# Define model name
model_name = ""

# Declare the training, validation, and testing directories
train_dir = r""
val_dir = r""
test_dir = r""

# Load the training, validation, and testing datasets
print("#### Dataset Information ####\nTraining Dataset:")
train_ds = tf.keras.utils.image_dataset_from_directory(
    train_dir,
    label_mode='binary',
    image_size=(224, 224),
    batch_size=32)

print("Validation Dataset:")
val_ds = tf.keras.utils.image_dataset_from_directory(
    val_dir,
    label_mode='binary',
    image_size=(224, 224),
    batch_size=32)

print("Testing Dataset:")
test_ds = tf.keras.utils.image_dataset_from_directory(
    test_dir,
    label_mode='binary',
    image_size=(224, 224),
    batch_size=32)

# Instantiate the base model
print("#### Download Model ####")
base_model = MobileNetV3Large(input_shape=(224, 224, 3),
                              alpha=1.0,
                              minimalistic=False,
                              include_top=False,
                              weights='imagenet',
                              include_preprocessing=True)
base_model.trainable = False

# Define the model
model = tf.keras.Sequential([
    base_model,
    GlobalAveragePooling2D(),
    Dropout(0.5),
    Dense(1, activation='sigmoid', kernel_regularizer=regularizers.l2(0.01))
])

# Compile the model
model.compile(optimizer=keras.optimizers.Adam(),
              loss=keras.losses.BinaryCrossentropy(),
              metrics=[keras.metrics.BinaryAccuracy()])

# Add early stopping
early_stopping = callbacks.EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)

# Train the model
print("\n#### Transfer Learning ####")
model.fit(train_ds,
          epochs=25,
          validation_data=val_ds,
          callbacks=[early_stopping])

# Save the model
model.save(f"{model_name}_initial_raw")
print("\nInitial raw model saved.")

# Unfreezing the top layers of the base model
base_model.trainable = True
for layer in base_model.layers[:50]:
    layer.trainable = False

# Implement QAT for fine-tuning
with tfmot.quantization.keras.quantize_scope():
  q_aware_model = tfmot.quantization.keras.quantize_model(model)

  # Re-compile the QAT model
  q_aware_model.compile(optimizer=keras.optimizers.Adam(1e-5),
                        loss=keras.losses.BinaryCrossentropy(),
                        metrics=[keras.metrics.BinaryAccuracy()])

  # Train the fine-tuned model
  print("#### Fine Tuning ####")
  model.fit(train_ds,
            epochs=25,
            validation_data=val_ds,
            callbacks=[early_stopping],
            class_weight=class_weight)

  # Evaluate the fine-tuned model on the test dataset
  print("\n#### QAT Model Evaluation ####")
  model.evaluate(test_ds)

  # Save raw model
  model.save(f"{model_name}_fine_raw")
  print("\nFine-tuned raw model saved.")

# Convert the QAT fine-tuned model into a full int quantised TFLite model
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]

quantized_tflite_model = converter.convert()

Answer 1

和你有同样的问题。

这里的评论帮助了我：

https://github.com/tensorflow/tensorflow/issues/57034#issuecomment-1207889502

基本上带有子模型的顺序模型会产生这个问题。

因此，您需要在功能模型的帮助下以不同的方式链接模型。

遗憾的是，您可能会遇到 MobileNetV3Large-Model 的另一个问题。

例如，您可能会收到以下错误：

 Exception encountered when calling layer "tf.__operators__.add_364" (type TFOpLambda).

'list' object has no attribute 'dtype'

我仍在弄清楚出了什么问题，但我目前的猜测是，这是 quantize_model-method 内部的一个错误。