SwiftUI：用Firebase ML Kit识别的元素周围绘制矩形

Question

我目前正在尝试在图像上方绘制用Firebase ML Kit识别的文本框。目前，我还没有成功，因为它们全部显示在屏幕外，所以我完全看不到任何框。我正在查看本文以供参考：https://medium.com/swlh/how-to-draw-bounding-boxes-with-swiftui-d93d1414eb00

这是显示框的视图：

struct ImageScanned: View {
var image: UIImage
@Binding var rectangles: [CGRect]
@State var viewSize: CGSize = .zero

var body: some View {
    // TODO: fix scaling
       ZStack {
           Image(uiImage: image)
               .resizable()
               .scaledToFit()
               .overlay(
                   GeometryReader { geometry in
                    ZStack {
                        ForEach(self.transformRectangles(geometry: geometry)) { rect in
                            Rectangle()
                            .path(in: CGRect(
                                x: rect.x,
                                y: rect.y,
                                width: rect.width,
                                height: rect.height))
                            .stroke(Color.red, lineWidth: 2.0)
                        }
                    }
                }
           )
       }
}

private func transformRectangles(geometry: GeometryProxy) -> [DetectedRectangle] {
    var rectangles: [DetectedRectangle] = []

    for rect in self.rectangles {
        let x = rect.minX * geometry.size.width
        let y = rect.minY * geometry.size.height
        let width = rect.width * geometry.size.width
        let height = rect.height * geometry.size.height
        rectangles.append(DetectedRectangle(width: width, height: height, x: x, y: y))
    }
    return rectangles
}

}

struct DetectedRectangle: Identifiable {
    var id = UUID()
    var width: CGFloat = 0
    var height: CGFloat = 0
    var x: CGFloat = 0
    var y: CGFloat = 0
}

---

这是嵌套此视图的视图：

struct StartScanView: View {
@State var showCaptureImageView: Bool = false
@State var image: UIImage? = nil
@State var rectangles: [CGRect] = []

var body: some View {
    ZStack {
        if showCaptureImageView {
            CaptureImageView(isShown: $showCaptureImageView, image: $image)
        } else {
            VStack {

                Button(action: {
                    self.showCaptureImageView.toggle()
                }) {
                    Text("Start Scanning")
                }

                // show here View with rectangles on top of image
                if self.image != nil {
                    ImageScanned(image: self.image ?? UIImage(), rectangles: $rectangles)
                }


                Button(action: {
                    self.processImage()
                }) {
                    Text("Process Image")
                }
            }
        }
    }
}

func processImage() {
    let scaledImageProcessor = ScaledElementProcessor()
    if image != nil {
        scaledImageProcessor.process(in: image!) { text in
            for block in text.blocks {
                for line in block.lines {
                    for element in line.elements {
                        self.rectangles.append(element.frame)
                    }
                }
            }
        }
    }
}

}

我认为，本教程的计算没有太大意义，因为结果框很大，但是如果我将宽度（例如）更改为：

rect.width * geometry.size.width / image.size.width

矩形变得太小了。（高度类似）不幸的是，我找不到Firebase决定元素大小的尺寸。这是调试时看到的尺寸：

 - image.size.width = 3024.0
 - image.size.height = 4032.0
 - geometry.size.width = 495.0
 - geometry.size.height = 660.5
 - first rectangle as example: width 149.0 & height: 103.0

[根本不计算宽度和高度，矩形似乎具有它们应该具有的尺寸（不完全是），因此这给了我一个假设，即ML Kit的尺寸计算与图像不成比例。 size.height / width。

Answer 1

ML Kit有一个QuickStart应用程序，可以准确显示您要执行的操作：识别文本并在文本周围绘制一个矩形。这是Swift代码：

https://github.com/firebase/quickstart-ios/tree/master/mlvision/MLVisionExample