所以我正在像这样打开一个与elevenlabs api 的 websocket 连接并获取音频数据流：
final socketChannel = IOWebSocketChannel.connect(
'wss://api.elevenlabs.io/v1/text-to-speech/${widget.voiceId}/stream-input?model_id=$model');
final AudioPlayer audioPlayer = AudioPlayer();
final streamController = StreamController<List<int>>();
final myCustomSource = MyCustomSource(streamController.stream);
final bosMessage = {
"text": " ",
"voice_settings": {"stability": 0.5, "similarity_boost": true},
"xi_api_key": "api_key",
};
socketChannel.sink.add(jsonEncode(bosMessage));
socketChannel.stream.listen((message) async {
final response = jsonDecode(message);
if (response['audio'] != null) {
try {
final audioChunk = response['audio'] as String;
final uint8List = Uint8List.fromList(base64.decode(audioChunk));
streamController.add(uint8List);
} catch (e) {
print("Error setting audio source and playing: $e");
}
}
}, onDone: () async {
print('we are done here');
await audioPlayer.setAudioSource(myCustomSource);
print('source is done');
try {
audioPlayer.play();
} catch (e) {
print('error is $e');
}
MyCustomSource 设置如下：
class MyCustomSource extends StreamAudioSource {
final Stream<List<int>> byteStream;
final List<int> bytes = [];
MyCustomSource(this.byteStream) {
byteStream.listen((data) {
bytes.addAll(data);
});
}
@override
Future<StreamAudioResponse> request([int? start, int? end]) async {
start ??= 0;
end ??= bytes.length;
return StreamAudioResponse(
sourceLength: bytes.length,
contentLength: end - start,
offset: start,
stream: Stream.value(bytes.sublist(start, end)),
contentType: 'audio/mpeg',
);
}
}
现在的问题是，当我设置源并在 onDone(){} 内播放音频时，音频播放流畅。但我预期的行为是在流媒体块到达时播放音频并继续播放。现在的问题是，如果我将 .setAudioSource 和 .play 移到 onDone 之前并在监听中移动，每次新的音频块进入时，都会设置一个新的音频源，该音频源从头开始一直到新的流媒体块。因此，每当新的流媒体块到达时，它就会从头开始播放，本质上是在完成之前重复自身。
我该如何修复它，以便它立即开始播放并继续播放，而不是从头开始
通过此更改，您可以创建 MyCustomSource 并设置初始音频源 在监听 WebSocket 流之前。当新的音频块到达时，您添加 将它们添加到streamController中，MyCustomSource将自动更新 包含这些新块。音频将继续播放而无需重新启动 当新数据从 WebSocket 流到达时从头开始。
// Define MyCustomSource outside the widget build method to maintain its state.
final streamController = StreamController<List<int>>();
final myCustomSource = MyCustomSource(streamController.stream);
// Inside your widget's build method or wherever you are creating the WebSocket connection:
final socketChannel = IOWebSocketChannel.connect(
'wss://api.elevenlabs.io/v1/text-to-speech/${widget.voiceId}/stream-input?model_id=$model',
);
final AudioPlayer audioPlayer = AudioPlayer();
final bosMessage = {
"text": " ",
"voice_settings": {"stability": 0.5, "similarity_boost": true},
"xi_api_key": "api_key",
};
socketChannel.sink.add(jsonEncode(bosMessage));
socketChannel.stream.listen((message) async {
final response = jsonDecode(message);
if (response['audio'] != null) {
try {
final audioChunk = response['audio'] as String;
final uint8List = Uint8List.fromList(base64.decode(audioChunk));
// Add the new audio chunk to the stream
streamController.add(uint8List);
} catch (e) {
print("Error setting audio source and playing: $e");
}
}
}, onDone: () {
print('WebSocket connection closed.');
});
// Set the initial audio source
await audioPlayer.setAudioSource(myCustomSource);
// Start playing the audio
audioPlayer.play();