AWS CLI：JSON 加载到 DynamoDB 中

Question

您好，我正在尝试通过 AWS CLI 命令将以下 JSON 文件结构（> 8k 事务）加载到 DynamoDB 中。

{
    "transactions": [
        {
            "customerId": "abc",
            "transactionId": "123",
            "transactionDate": "2020-09-01",
            "merchantId": "1234",
            "categoryId": "3",
            "amount": "5",
            "description": "McDonalds"
        },
        {
            "customerId": "def",
            "transactionId": "456",
            "transactionDate": "2020-09-01",
            "merchantId": "45678",
            "categoryId": "2",
            "amount": "-11.70",
            "description": "Tescos"
        },
        {
            "customerId": "jkl",
            "transactionId": "gah",
            "transactionDate": "2020-09-01",
            "merchantId": "9081",
            "categoryId": "3",
            "amount": "-139.00",
            "description": "Amazon"
        },
    ...

任何人都可以帮助我解决这个问题应该使用的命令吗？我还需要转换这个 JSON 文件才能使其工作吗？

Answer 1

您可以使用batch-write-item来完成。最多可以进行 25 次放置或删除。

aws dynamodb batch-write-item  --request-items file://request-items.json 

将您的 request-items.json 设置为，

{
    your-table-name: [
        {
            "PutRequest": {
                "Item": {
                    "customerId": {"S": "abc"},
                    "transactionId": {"S": "5651"}
                    ....
                }
            }
        },
        {
            "PutRequest": {
                "Item": {
                    "customerId": {"S": "def"},
                    "transactionId": {"S": "125"},
                    ....
                }
            }
        },
        {
            "PutRequest": {
                "Item": {
                    "customerId": {"S": "jkl"},
                    "transactionId": {"S": "4466"},
                    ....
                }
            }
        },
        .....
        .....
    ]
}

https://docs.aws.amazon.com/cli/latest/reference/dynamodb/batch-write-item.html

Answer 2

下面使用

jq

将“普通”json 转换为“DynamoDB”json：

json="$(cat <<EOF
  {
    "mynull": null,
    "myboolean": false,
    "mynumber": 123,
    "mystring": "abc",
    "myarray": [
      null,
      false,
      123,
      "abc",
      {
        "key": "value"
      }
    ]
  }
EOF
)"

echo "$json" | jq "$(cat <<EOF
  walk(
    if type == "null" then
      {"NULL": true}
    elif type == "boolean" then
      {"BOOL": .}
    elif type == "number" then
      {"N": . | tostring}
    elif type == "string" then
      {"S": .}
    elif type == "array" then
      {"L": .}
    elif type == "object" then
      {"M": .}
    else
      .
    end
  )
EOF
)"

输出：

{
  "M": {
    "mynull": {
      "NULL": true
    },
    "myboolean": {
      "BOOL": false
    },
    "mynumber": {
      "N": "123"
    },
    "mystring": {
      "S": "abc"
    },
    "myarray": {
      "L": [
        {
          "NULL": true
        },
        {
          "BOOL": false
        },
        {
          "N": "123"
        },
        {
          "S": "abc"
        },
        {
          "M": {
            "key": {
              "S": "value"
            }
          }
        }
      ]
    }
  }
}