如何处理麦克风中的音频并实时播放？

Question

我正在开发一个使用 AVFoundation 框架进行实时音频处理的 iOS 应用程序。该应用程序从麦克风捕获音频，获取每个缓冲区，将其传递给返回修改后的缓冲区的函数，然后回放。

但是，我在对着麦克风讲话和听到播放之间存在显着的延迟（大约 500 毫秒）。起初，我认为延迟是因为处理花费了太多时间而引起的，但即使我删除处理并只播放原始缓冲区，也会发生同样的事情。

这是我的设置：

import SwiftUI
import AVFoundation

class MicTestAudioKitService: MicTestRepository {
    private let engine = AVAudioEngine()
    private let playerNode = AVAudioPlayerNode()
    
    // ------------------
    
    @Injected private var pitchCorrectionService: PitchCorrectionRepository
    
    // ------------------
    
    private var isInitialized = false
    private var pitchCorrectionIntensity: Float = 0.5
    
    // ------------------
    
    private func initialize() {
        guard !isInitialized else { return }
        isInitialized.toggle()
        
        let inputNode = engine.inputNode
        let format = inputNode.inputFormat(forBus: 0)
        
        // Attach and connect the playerNode
        engine.attach(playerNode)
        engine.connect(playerNode, to: engine.mainMixerNode, format: format)
    }
    
    func start() {
        initialize()
        
        let inputNode = engine.inputNode
        let format = inputNode.inputFormat(forBus: 0)
        
        inputNode.installTap(onBus: 0, bufferSize: 1024, format: format) { [weak self] buffer, _ in
            guard let self else { return }

            let processedBuffer = pitchCorrectionService.pitchCorrect(buffer: buffer, intensity: pitchCorrectionIntensity) ?? buffer
            outputAudioBuffer(processedBuffer)
        }
        
        do {
            try engine.start()
        } catch {
            print("Audio Engine failed to start: \(error)")
        }
    }
    
    func stop() {
        engine.stop()
    }
    
    func setPitchCorrectionIntensity(_ intensity: Float) {
        pitchCorrectionIntensity = intensity
    }
}

extension MicTestService {
    private func outputAudioBuffer(_ buffer: AVAudioPCMBuffer) {
        playerNode.scheduleBuffer(buffer, completionHandler: nil)
        
        if !playerNode.isPlaying {
            playerNode.play()
        }
    }
}

即使使用有线耳机也会发生同样的情况。

有什么想法吗？

Answer 1

这就是我对类似应用程序所做的事情。效果很好。有延迟，但我认为不到 100 毫秒。

监控输入：

var mixer: AVAudioMixerNode=AVAudioMixerNode()
let audioInputNode=engine.inputNode
let inputFormat = audioInputNode.outputFormat(forBus: 0)
engine.attach(mixer)
engine.connect(mixer, to: engine.outputNode, format: nil)
engine.connect(audioInputNode, to: mixer, format: inputFormat)
engine.prepare()

录制音频（我使用文件，你可以使用缓冲区代替）：

let format = audioInputNode.outputFormat(forBus: 0)
let documentURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0]
 var file:AVAudioFile?
 do {
            file = try AVAudioFile(forWriting: documentURL.appendingPathComponent(recordingFileName), settings: format.settings)
        } catch _ {
            print("Could not open file for writing")
            handleErrors(theError: "Could not open recording file for writing")
        }
        
 audioInputNode.installTap(onBus: 0, bufferSize: 4096, format: format, block: {
            (buffer, time) in
            try? file!.write(from: buffer)
          })