如何从AVCapturePhoto生成具有正确方向的UIImage？

问题描述 投票：12回答：5

我正在调用AVFoundation的委托方法来处理照片捕获，但是我很难将它生成的AVCapturePhoto转换为具有正确方向的UIImage。虽然下面的例程是成功的，但我总是得到一个面向右侧的UIImage（UIImage.imageOrientation = 3）。使用UIImage（数据：图像）并尝试首先使用photo.cgImageRepresentation（）时，我无法提供方向？takeRetainedValue（）也没有帮助。请协助。

在此处，图像方向至关重要，因为生成的图像将被输入Vision框架工作流程。

func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
    // capture image finished
    print("Image captured.")
    if let imageData = photo.fileDataRepresentation() {
        if let uiImage = UIImage(data: imageData){
            // do stuff to UIImage
        }
    }
}

更新1：阅读Apple的Photo Capture Programming Guide（过时的iOS11），我确实找到了一件我做错的事情：

  1. 在每次捕获调用（self.capturePhotoOutput.capturePhoto）时，必须与PhotoOutput对象建立连接并更新其方向以匹配拍摄照片时设备的方向。为此，我创建了UIDeviceOrientation的扩展，并在我创建的snapPhoto（）函数上使用它来调用捕获例程并等待执行didFinishProcessingPhoto委托方法。我添加了代码的快照，因为这里的代码示例分隔符似乎没有正确显示它们。 enter image description here enter image description here

更新2链接到GitHub上的完整项目：https://github.com/agu3rra/Out-Loud

avfoundation ios11 avcapturesession swift4
5个回答
17
投票

最后更新：我使用该应用程序进行了一些实验并得出以下结论：

  1. kCGImagePropertyOrientation似乎不会影响应用程序内捕获图像的方向，只有在每次要调用capturePhoto方法时更新photoOutput连接时，它才会因设备方向而异。所以： func snapPhoto() { // prepare and initiate image capture routine // if I leave the next 4 lines commented, the intented orientation of the image on display will be 6 (right top) - kCGImagePropertyOrientation let deviceOrientation = UIDevice.current.orientation // retrieve current orientation from the device guard let photoOutputConnection = capturePhotoOutput.connection(with: AVMediaType.video) else {fatalError("Unable to establish input>output connection")}// setup a connection that manages input > output guard let videoOrientation = deviceOrientation.getAVCaptureVideoOrientationFromDevice() else {return} photoOutputConnection.videoOrientation = videoOrientation // update photo's output connection to match device's orientation let photoSettings = AVCapturePhotoSettings() photoSettings.isAutoStillImageStabilizationEnabled = true photoSettings.isHighResolutionPhotoEnabled = true photoSettings.flashMode = .auto self.capturePhotoOutput.capturePhoto(with: photoSettings, delegate: self) // trigger image capture. It appears to work only if the capture session is running. }
  2. 在调试器上查看生成的图像向我展示了它们是如何生成的，因此我可以推断出所需的旋转（UIImageOrientation）以使其直立显示。换句话说：更新UIImageOrientation会告诉您应该如何旋转图像，以便您以正确的方向查看图像。所以我来到下表：Which UIImageOrientation to apply according to how the device was at the time of capture
  3. 我不得不将我的UIDeviceOrientation扩展更新为一个相当不直观的形式： extension UIDeviceOrientation { func getUIImageOrientationFromDevice() -> UIImageOrientation { // return CGImagePropertyOrientation based on Device Orientation // This extented function has been determined based on experimentation with how an UIImage gets displayed. switch self { case UIDeviceOrientation.portrait, .faceUp: return UIImageOrientation.right case UIDeviceOrientation.portraitUpsideDown, .faceDown: return UIImageOrientation.left case UIDeviceOrientation.landscapeLeft: return UIImageOrientation.up // this is the base orientation case UIDeviceOrientation.landscapeRight: return UIImageOrientation.down case UIDeviceOrientation.unknown: return UIImageOrientation.up } } }
  4. 这就是我的最终委托方法现在的样子。它以预期的方向显示图像。 func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) { // capture image finished print("Image captured.") let photoMetadata = photo.metadata // Returns corresponting NSCFNumber. It seems to specify the origin of the image // print("Metadata orientation: ",photoMetadata["Orientation"]) // Returns corresponting NSCFNumber. It seems to specify the origin of the image print("Metadata orientation with key: ",photoMetadata[String(kCGImagePropertyOrientation)] as Any) guard let imageData = photo.fileDataRepresentation() else { print("Error while generating image from photo capture data."); self.lastPhoto = nil; self.controller.goToProcessing(); return } guard let uiImage = UIImage(data: imageData) else { print("Unable to generate UIImage from image data."); self.lastPhoto = nil; self.controller.goToProcessing(); return } // generate a corresponding CGImage guard let cgImage = uiImage.cgImage else { print("Error generating CGImage");self.lastPhoto=nil;return } guard let deviceOrientationOnCapture = self.deviceOrientationOnCapture else { print("Error retrieving orientation on capture");self.lastPhoto=nil; return } self.lastPhoto = UIImage(cgImage: cgImage, scale: 1.0, orientation: deviceOrientationOnCapture.getUIImageOrientationFromDevice()) print(self.lastPhoto) print("UIImage generated. Orientation:(self.lastPhoto.imageOrientation.rawValue)") self.controller.goToProcessing() } func photoOutput(_ output: AVCapturePhotoOutput, willBeginCaptureFor resolvedSettings: AVCaptureResolvedPhotoSettings) { print("Just about to take a photo.") // get device orientation on capture self.deviceOrientationOnCapture = UIDevice.current.orientation print("Device orientation: \(self.deviceOrientationOnCapture.rawValue)") }
3
投票

我成功地做到了这一点：

func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {

        let cgImage = photo.cgImageRepresentation()!.takeRetainedValue()
        let orientation = photo.metadata[kCGImagePropertyOrientation as String] as! NSNumber
        let uiOrientation = UIImage.Orientation(rawValue: orientation.intValue)!
        let image = UIImage(cgImage: cgImage, scale: 1, orientation: uiOrientation)

}

它基于Apple在他们的文档中提到的内容：

每次访问此方法时，AVCapturePhoto都会生成一个新的CGImageRef。当由压缩容器（例如HEIC）支持时，CGImageRepresentation根据需要被懒惰地解码。当由诸如BGRA的未压缩格式支持时，它被复制到单独的后备缓冲区中，其生存期与AVCapturePhoto的生命周期无关。对于1200万像素的图像，BGRA CGImage代表每次呼叫约48兆字节。如果您只打算使用CGImage进行屏幕渲染，请改用previewCGImageRepresentation。请注意，CGImageRef的物理旋转与主图像的物理旋转匹配。 Exif方向尚未应用。如果您希望在使用UIImage时应用旋转，可以通过查询照片的元数据[kCGImagePropertyOrientation]值并将其作为orientation参数传递给+ [UIImage imageWithCGImage：scale：orientation：]来实现。 RAW图像始终返回CGImageRepresentation为零。如果您希望从RAW图像制作CGImageRef，请在CoreImage框架中使用CIRAWFilter。

1
投票

在AVCapturePhoto内部，我很确定你会发现一个metadata对象，也被称为CGImageProperties。 在它内部，你会发现EXIF词典的方向，下一步就是采取方向并根据它创建一个图像。 我没有使用AVCapturePhotoOutput的经验，但我有一些使用旧的方式。 请注意，在UIImageOrientation中EXIF字典的映射方式不同。 这是我很久以前写过的article，但主要原则仍然有效。 这个question会指出你的一些实现，它也很老，我很确定在最新版本中他们发布了更简单的API，但它仍然会指导你解决这个问题。

1
投票

Andre提供的更新扩展程序适用于Swift 4.2：

import Foundation
import UIKit

extension UIDeviceOrientation {
    var imageOrientation: UIImage.Orientation {
        switch self {
        case .portrait, .faceUp:                return .right
        case .portraitUpsideDown, .faceDown:    return .left
        case .landscapeLeft:                    return .up
        case .landscapeRight:                   return .down
        case .unknown:                          return .up
        }
    }
}
1
投票

要以正确的方向创建图像，我们需要在初始化图像时输入正确的UIImage.Orientation。

最好使用从photoOutput代表返回的CGImagePropertyOrientation来获取拍摄照片时相机会话的确切方向。这里唯一的问题是，虽然UIImage.Orientation和CGImagePropertyOrientation之间的枚举值是相同的，但原始值不是。 Apple提出了一个简单的映射来解决这个问题

https://developer.apple.com/documentation/imageio/cgimagepropertyorientation

这是我的实现：

AVCapturePhotoCaptureDelegate

func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
        if let _ = error {
            // Handle Error
        } else if let cgImageRepresentation = photo.cgImageRepresentation(),
            let orientationInt = photo.metadata[String(kCGImagePropertyOrientation)] as? UInt32,
            let imageOrientation = UIImage.Orientation.orientation(fromCGOrientationRaw: orientationInt) {

            // Create image with proper orientation
            let cgImage = cgImageRepresentation.takeUnretainedValue()
            let image = UIImage(cgImage: cgImage,
                                scale: 1,
                                orientation: imageOrientation)
        }
    }

映射扩展

extension UIImage.Orientation {

    init(_ cgOrientation: CGImagePropertyOrientation) {
        // we need to map with enum values becuase raw values do not match
        switch cgOrientation {
        case .up: self = .up
        case .upMirrored: self = .upMirrored
        case .down: self = .down
        case .downMirrored: self = .downMirrored
        case .left: self = .left
        case .leftMirrored: self = .leftMirrored
        case .right: self = .right
        case .rightMirrored: self = .rightMirrored
        }
    }


    /// Returns a UIImage.Orientation based on the matching cgOrientation raw value
    static func orientation(fromCGOrientationRaw cgOrientationRaw: UInt32) -> UIImage.Orientation? {
        var orientation: UIImage.Orientation?
        if let cgOrientation = CGImagePropertyOrientation(rawValue: cgOrientationRaw) {
            orientation = UIImage.Orientation(cgOrientation)
        } else {
            orientation = nil // only hit if improper cgOrientation is passed
        }
        return orientation
    }
}
最新问题
© www.soinside.com 2019 - 2024. All rights reserved.