RGB到灰度，但像素值似乎错误

Question

我有一个使用tensorflow 2.0 / Keras构建的模型。输入的图像为28x28，带有1个通道。该模型已保存并转换为.tflite，并在我的Swift ios应用程序中使用。不幸的是，当调用解释器时，我得到的预测与预期大不相同。当我进一步调查时，似乎我的图像准备可能是错误的。在将像素阵列输入模型中之前，请按照以下步骤进行操作。

加载图像。
将图像转换为灰度
通过除以255归一化像素值。

这是我的代码

let im =  UIImage(named: "dotsgray")!
let i = (im.pixelBufferGray(width: 28, height: 28))!
i.normalize()



extension UIImage {
  public func pixelBufferGray(width: Int, height: Int) -> CVPixelBuffer? {
        return pixelBuffer(width: width, height: height,
                           pixelFormatType: kCVPixelFormatType_OneComponent8,
                           colorSpace: CGColorSpaceCreateDeviceGray(),
                           alphaInfo: .none)
    }


    func pixelBuffer(width: Int, height: Int, pixelFormatType: OSType,
                     colorSpace: CGColorSpace, alphaInfo: CGImageAlphaInfo) -> CVPixelBuffer? {
        var maybePixelBuffer: CVPixelBuffer?
        let attrs = [kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue,
                     kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue]
        let status = CVPixelBufferCreate(kCFAllocatorDefault,
                                         width,
                                         height,
                                         pixelFormatType,
                                         attrs as CFDictionary,
                                         &maybePixelBuffer)

        guard status == kCVReturnSuccess, let pixelBuffer = maybePixelBuffer else {
            return nil
        }

        CVPixelBufferLockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0))
        let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer)

        guard let context = CGContext(data: pixelData,
                                      width: width,
                                      height: height,
                                      bitsPerComponent: 8,
                                      bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer),
                                      space: colorSpace,
                                      bitmapInfo: alphaInfo.rawValue)
            else {
                return nil
        }

        UIGraphicsPushContext(context)
        context.translateBy(x: 0, y: CGFloat(height))
        context.scaleBy(x: 1, y: -1)
        self.draw(in: CGRect(x: 0, y: 0, width: width, height: height))
        UIGraphicsPopContext()

        CVPixelBufferUnlockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0))
        return pixelBuffer
    }
}

extension CVPixelBuffer {

func normalize() {
  // 1
  let bytesPerRow = CVPixelBufferGetBytesPerRow(self)
  let totalBytes = CVPixelBufferGetDataSize(self)

  let width = bytesPerRow / MemoryLayout<UInt8>.size
  let height = totalBytes / bytesPerRow

  // 2
  CVPixelBufferLockBaseAddress(self, CVPixelBufferLockFlags(rawValue: 0))

  // 3
  let floatBuffer = unsafeBitCast(
    CVPixelBufferGetBaseAddress(self),
    to: UnsafeMutablePointer<Double>.self)

  // 4
  var minPixel: Double = 1.0
  var maxPixel: Double = 0.0

  // 5
  for i in 0 ..< width * height {
    let pixel = floatBuffer[i]
    minPixel = min(pixel, minPixel)
    maxPixel = max(pixel, maxPixel)
  }

  // 6
  let range = maxPixel - minPixel

  // 7
  for i in 0 ..< width * height {
    let pixel = floatBuffer[i]
    floatBuffer[i] = (pixel - minPixel) / range
  }

  // 8
  CVPixelBufferUnlockBaseAddress(self, CVPixelBufferLockFlags(rawValue: 0))
}

Answer 1

您的normalize()完全不符合您的目的。

将基于Double的像素缓冲区标准化为0.0 ... 1.0，但是您没有创建Double的像素缓冲区。

[您的pixelBufferGray(width:height:)创建UInt8的像素缓冲区，因为您为kCVPixelFormatType_OneComponent8提供了pixelFormatType。

删除i.normalize()，然后检查像素缓冲区`。您将看到您的期望。

您可能需要pack像素缓冲区，因为它在每个64字节行中仅使用28个字节，但这是另一个问题。

RGB到灰度，但像素值似乎错误

问题描述投票：0回答：1

1个回答

最新问题

RGB到灰度，但像素值似乎错误

问题描述 投票：0回答：1

1个回答

最新问题

问题描述投票：0回答：1