使用 Jest 和 AWS SDK V3 模拟 AWS S3 Select 响应

Question

我正在使用 S3 Select 查询 CSV 文件并在我的 Nodejs 应用程序中从中提取行。如何编写单元测试来模拟

SelectObjectContentCommand

的响应。 响应类型为

SelectObjectContentCommandOutput

import { SelectObjectContentCommand, S3Client, ExpressionType, FileHeaderInfo } from '@aws-sdk/client-s3';

const s3Client = new S3Client({});

export const getDataUsingS3Select = async (bucketName: string, fileName: string, query: string, params: any) => {
  
  const command = new SelectObjectContentCommand(params);
  const response = await s3Client.send(command);

  if (response.$metadata.httpStatusCode === 200) {
    if (response.Payload) {
      const data = convertDataToJson(response.Payload);
      return data;
    } else {
      console.log(`S3Select did not have payload for ${bucketName}/${fileName}`);
      return [];
    }
  } else {
    console.log(`S3Select did not receive 200 for ${bucketName}/${fileName}. 
    Metadata is ${response.$metadata}`);
    return [];
  }
}

Answer 1

我通过使用 Symbol.asyncIterator 属性来实现此功能：

class MockAsyncIterator {
  constructor(readonly chunks: string[]) {
  }

  async* [Symbol.asyncIterator]() {
    for (const c of this.chunks) {
      yield { Records: { Payload: Buffer.from(c, "utf8") } }
    }
  }
}

然后，使用 aws-sdk-client-mock 我们可以像这样测试您的函数：

import { mockClient } from "aws-sdk-client-mock";
const s3Mock = mockClient(S3Client);

test("s3 query", async () => {
  s3Mock.on(SelectObjectContentCommand).resolves({
    Payload: new MockAsyncIterator(["column1,column2", "value1,value2"]),
    "$metadata": {
      httpStatusCode: 200
    }
  })

  const query = {
    // query params
  };

  const result = await getDataUsingS3Select("my-bucket", "my.csv", "SELECT * FROM S3Object", query);
  expect(result).toBeDefined();
  // additional checks
});