使用Kinesis Firehose将数据写入S3的问题，所有记录均为404条消息

Question

我已经建立了一个Kinesis Firehose流，该流在AWS控制台中成功运行了“测试”。我写到流中的代码看起来像这样：

return graphqlHTTP(
(request: Object, response: Object, params: Object): Object => {
  const firehoseConfig: Object = {
    region: config.get('awsRegion'),
    credentials = {
      accessKeyId: config.get('awsAccessKey'),
      secretAccessKey: config.get('secretAccessKey'),
    }
  }
  // Init Kinesis Firehose
  const firehose = new AWS.Firehose(firehoseConfig)
  // Prep response object for sending
  const stringifiedResponse = JSON.stringify(response)
  // Send to Kinesis Firehose stream
  const firehoseParams = {
    DeliveryStreamName: 'test-stream',
    Record: {
      Data: Buffer.from(stringifiedResponse),
    },
  }
  firehose.putRecord(firehoseParams, (err: Object, data: Object) => {
    // eslint-disable-next-line no-console
    if (err) console.log('FIREHOSE ERROR: ', err, err.stack)
    // eslint-disable-next-line no-console
    else console.log(data)
  })

当端点被命中时，记录正在写入流中，但是所有记录看起来都像这样：

 {
  "status": 404,
  "message": "Not Found",
  "header": {
    "x-frame-options": "SAMEORIGIN",
    "strict-transport-security": "max-age=86400",
    "x-download-options": "noopen",
    "x-content-type-options": "nosniff",
    "x-xss-protection": "1; mode=block",
    "vary": "Accept-Encoding, Origin",
    "cache-control": "max-age=60, s-maxage=60",
    "access-control-allow-origin": "<redacted internal site>",
    "access-control-allow-credentials": "true",
    "access-control-expose-headers": "content-length,etag",
    "x-ratelimit-remaining": "996",
    "x-ratelimit-reset": "1571372307",
    "x-ratelimit-limit": "1000"
  }
} {
  "status": 404,
  "message": "Not Found",
  "header": {
    "x-frame-options": "SAMEORIGIN",
    "strict-transport-security": "max-age=86400",
    "x-download-options": "noopen",
    "x-content-type-options": "nosniff",
    "x-xss-protection": "1; mode=block",
    "vary": "Accept-Encoding, Origin",
    "cache-control": "max-age=60, s-maxage=60",
    "access-control-allow-origin": "<redacted internal site>",
    "access-control-allow-credentials": "true",
    "access-control-expose-headers": "content-length,etag",
    "x-ratelimit-remaining": "995",
    "x-ratelimit-reset": "1571372307",
    "x-ratelimit-limit": "1000"
  }
}

[我想做的是让Firehose将response发送到S3，以便以后可以使用Athena在其上运行查询。

Answer 1

事实证明，根据docs，来自Kinesis的404指示格式错误的查询。在这种情况下，这意味着Data中提供的Record不是有效的JSON。由Koa / response提供的koa-graphql对象是带有循环引用的大对象，并且其中的某个地方是无效的JSON或AWS认为无效的JSON。