无法从 fastify 服务器传输 openAI 响应

Question

我用头撞墙，本该是非常简单的事情！只是为了调用 openAI API 并从 fastify 服务器返回流。

我看到文档和流看起来非常简单：

fastify.get('/streams', async function (request, reply) {
  const fs = require('node:fs')
  const stream = fs.createReadStream('some-file', 'utf8')
  reply.header('Content-Type', 'application/octet-stream')
  return reply.send(stream)
})

但是，当尝试将其连接到 openAI 调用时：

async function main() {
  const stream = await openai.chat.completions.create({
    model: 'gpt-4',
    messages: [{ role: 'user', content: 'Say this is a test' }],
    stream: true,
  });
  for await (const part of stream) {
    process.stdout.write(part.choices[0]?.delta?.content || '');
  }
}

我就是无法让它工作......

我已经尝试过：

• 直接传递流
• 在异步循环中传递流
• 在 fastify 中使用原始 HTTP 请求
• 转换为可读流

似乎没什么作用😦

这是我最新的尝试：

app.get<{ Querystring: GetSearchParams }>('/stream', async (req, reply) => {
  try {
    const { q } = req.query;
    const stream = await openai.chat.completions.create({
      model: 'gpt-3.5-turbo',
      messages: [{ role: 'user', content: q }],
      stream: true,
    });

    reply.raw.writeHead(200, { 'Content-Type': 'text/plain' });

    for await (const part of stream) {
      console.log(part.choices[0]?.delta?.content || '');
      reply.raw.write(part.choices[0]?.delta?.content || '');
    }
    reply.raw.end();
  } catch (err) {
    reply.raw.writeHead(500, { 'Content-Type': 'text/plain' });
    reply.raw.end('Ooops');
  }
});

Answer 1

有效：

app.get<{ Querystring: GetSearchParams }>('/stream', async (req, reply) => {
  try {
    const { q } = req.query;
    const stream = await openai.chat.completions.create({
      model: 'gpt-3.5-turbo',
      messages: [{ role: 'user', content: q }],
      stream: true,
    });

    reply.raw.writeHead(200, { 'Content-Type': 'text/plain' });
    for await (const part of stream) {
      reply.raw.write(part.choices[0]?.delta?.content || '');
    }
    return reply.raw.end();
  } catch (err) {
    console.log(err);
    reply.raw.end('Error sending chat stream.');
  }
});