Android 的 Codec2 实现

Question

我正在寻找适用于 Android 的 Codec2 impl。找到了THIS库，看起来很有前途

我尝试实现如下所示的编码解码逻辑，但我的音频输出非常失真，难以理解。当我尝试使用内置解码器时，输出根本无法识别（但适用于资产中的示例文件）。我缺少什么？

companion object {
    public const val codec2mode = Codec2.CODEC2_MODE_3200

    private const val sampleRate = 8000 // enough
    private const val audioFormat = AudioFormat.ENCODING_PCM_16BIT
    private const val monoStereoIn = AudioFormat.CHANNEL_IN_MONO
    private const val monoStereoOut = AudioFormat.CHANNEL_OUT_MONO
}

private val minBufSize = AudioRecord.getMinBufferSize(sampleRate, monoStereoIn, audioFormat)

private val c2instance = Codec2.create(codec2mode)
private val bits = Codec2.getBitsSize(c2instance) // bytes?
private val samples = Codec2.getSamplesPerFrame(c2instance)

fun recordData(): CharArray {
    Timber.i("recordData bits:$bits, samples:$samples, minBufSize:$minBufSize")
    val recorder = getRecorder()

    val recorderBuffer = ShortArray(samples.coerceAtLeast(minBufSize))
    val framesNum = recorderBuffer.size / samples

    Timber.i("startRecording recorderBuffer size:${recorderBuffer.size} framesNum:$framesNum")
    recorder.startRecording()

    val start = System.currentTimeMillis()

    val encodedBuffer = CharArray(bits)
    var encodedBufferSum = CharArray(0)

    var it = 50
    while (it > 0) {
        recorder.read(recorderBuffer, 0, recorderBuffer.size)
        for (i in 0 until framesNum) {
            Codec2.encode(c2instance, recorderBuffer, encodedBuffer)
            encodedBufferSum += encodedBuffer
        }
        it--
    }

    Timber.i(
        "encode finished in " + (System.currentTimeMillis() - start) +
            "ms, data length:${encodedBufferSum.size}",
    )

    return encodedBufferSum
}

fun playData(encodedBuffer: CharArray) {
    Timber.i("playData bits:$bits, samples:$samples, minBufSize:$minBufSize")
    val start = System.currentTimeMillis()

    val audioTrack = getAudioTrack().apply { play() }

    var workingCopy = encodedBuffer.clone()
    var it = 0
    while (workingCopy.isNotEmpty()) {
        val frame = workingCopy.slice(0 until bits).toCharArray()

        var codec2Buffer = ByteArray(0)
        frame.forEach { codec2Buffer += it.code.toByte() }

        val playbackAudioBuffer = ShortArray(samples)
        Codec2.decode(c2instance, playbackAudioBuffer, codec2Buffer)

        audioTrack.write(playbackAudioBuffer, 0, playbackAudioBuffer.size)

        workingCopy = workingCopy.slice(bits until workingCopy.size).toCharArray()

        it++
    }

    Timber.i(
        "decode and play finished in " + (System.currentTimeMillis() - start) +
            "ms, iterations:$it",
    )
}

输出日志：

  recordData bits:8, samples:160, minBufSize:640
  startRecording recorderBuffer size:640 framesNum:4
  encode finished in 4138ms, data length:1600
  playData bits:8, samples:160, minBufSize:640
  decode and play finished in 3999ms, iterations:200

顺便说一句。我发现实现上述库的THIS项目可能有效，但我没有设备可以尝试...并且这里我们已经构建了AAR（个人使用

0.8-SNAPSHOT64-2

）

Answer 1

不幸的是，我已经有一段时间没有积极致力于这个工作了。我已经根据我的记忆尽可能地回答了在 GitHub 上发布的问题：我认为示例文件的标题与帧大小相同（或其倍数）。当帧大小发生变化而情况并非如此时，会导致数据帧不对齐。

不幸的是我现在没有时间解决这个问题。我可以通过 Github 问题提供提示/建议，并且我很高兴审查/合并修复该问题的拉取请求。

Answer 2

回答当前版本

1.2.0

bug 被隐藏在 JNI 端的

encode

方法中，其中发生了一些不必要的划分/下采样。而不是

short v = (short) jbuf[i * 2]; 应该是

short v = (short) jbuf[i]; 对于当前版本，有一个简单的解决方法 - 只需添加一些额外的虚拟字节（每秒/偶数位置），这些字节将被 buggy

for

 循环忽略

val frameBuffer: ShortArray = ... e.g. from AudioRecorder
var bugAround = ShortArray(0)
frameBuffer.forEach {
    bugAround += it // one short  single sample
    bugAround += 0x00
}
// bugAround now have 2x size as frameBuffer
Codec2.encode(c2instance, bugAround/*frameBuffer */, encodedBuffer)

更多描述和解决方法请参见 github 上的

issue