使用 Flask 构建的 API 在 Safari 上直播 Elevenlabs 音频

Question

我正在尝试通过我的 Flask API 将音频流返回到我的网站。 api 调用将生成一些内容，然后调用 Elevenlabs API 来生成音频。然后我希望将此音频作为流发送回我的网站。我最初创建的代码在 Chrome 上运行良好，但在桌面 Safari 和 iOS 上则无法运行。我没有从 api 调用中得到响应标头。

这是初始代码：

@application.route('/get-audio', methods=['GET'])
def get_audio():
    # Get data from request (you might be getting a text or some parameters based on which audio is generated)
    data = request.args.get('data')

    def generate():
        url = "https://api.elevenlabs.io/v1/text-to-speech/EXAVITQu4vr4xnSDxMaL/stream"
    
        headers = {
            "Accept": "audio/mpeg",
            "Content-Type": "application/json",
            "xi-api-key": "api_key"
        }
    
        data = {
            "text": data,
            "model_id": "eleven_monolingual_v1",
            "voice_settings": {
                "stability": 0.5,
                "similarity_boost": 0.5
            }
        }
    
        try:
            response = requests.post(url, json=data, headers=headers, stream=True)
            for chunk in response.iter_content(chunk_size=8192):
                yield chunk
        except requests.RequestException as e:
            print(f"Error fetching audio: {e}")

    return Response(stream_with_context(generate()), content_type='audio/mpeg')

然后我尝试使用 ffmpeg 返回音频流，但仍然没有成功。我能做的最好的事情就是保存文件，然后让 safari 从该文件中读取。

@application.route('/get-audio', methods=['GET'])
def get_audio():    
    audio_generator = generate_track(data)
    return Response(audio_generator, content_type="audio/aac")
   
def generate_track(data)
[........]
    try:
        response = requests.post(url, json=data, headers=headers, stream=True)

        ffmpeg_process = subprocess.Popen(
            ["ffmpeg", "-i", "pipe:0", "-f", "adts", "-"],
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            bufsize=10**8 
        )

        for audio_chunk in response.iter_content(chunk_size=4096):
            aac_audio, _ = ffmpeg_process.communicate(input=audio_chunk)
            yield aac_audio

    except requests.RequestException as e:
        print(f"Error fetching audios: {e}")

我遇到了一些潜在的解决方案，但从未设法将其中一个解决方案适应我的代码，这里的特殊性是尝试通过我的 api 传输传入流。

任何帮助将不胜感激！

Answer 1

这对我来说是使用elevenlabs 0.22

from flask import Flask
from flask_cors import CORS
from elevenlabs import generate, set_api_key, stream
import key
app = Flask(__name__)
CORS(app)

set_api_key(key.ELEVENLABS_API_KEY)

def generate_text():
    yield "Hi there, I'm Eleven "
    yield "I'm a text to speech API "

@app.route("/speak", methods=["POST"])
def speak():
    audio = generate(
        text=generate_text(),
        voice="Antoni",
        model="eleven_multilingual_v2",
        stream=True
    )
    stream(audio)