使用java.Without使用作业加载数据,将json数据流式传输到Bigquery

问题描述 投票:0回答:1

我是bigquery的新手,所以我不完全理解如何将数据流传输到bigquery,这是我的问题,我有jsonInString,从这样的对象映射

    String customerJsonInString = mapper.writeValueAsString(customer);

   {
  "id": "1",
  "first_name": "John",
  "last_name": "Doe",
  "dob": "1968-01-22",
  "addresses": [
    {
      "status": "current",
      "address": "123 First Avenue",
      "city": "Seattle",
      "state": "WA",
      "zip": "11111",
      "numberOfYears": "1"
    },
    {
      "status": "previous",
      "address": "456 Main Street",
      "city": "Portland",
      "state": "OR",
      "zip": "22222",
      "numberOfYears": "5"
    }
  ]
}

该表已使用正确的架构创建。现在我想将这些数据流式传输到bigquery(插入行),我正在使用(https://cloud.google.com/bigquery/streaming-data-into-bigquery#bigquery-stream-data-java)上的示例来使它适应我的,这是我试过的,

TableId tableId = TableId.of(DATASET_NAME,TABLE_NAME);
        Map<String, Object> recordsContent = new HashMap<>();
        recordsContent.put("Customer", customerJsonInString);
        InsertAllResponse response = bigquery.insertAll(InsertAllRequest.newBuilder(tableId)
                        .addRow("rowId", recordsContent)
                        .build());
        if (response.hasErrors()) {
            for (Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
            }
        }
java json google-bigquery
1个回答
0
投票

我猜这张桌子的架构是id, first_name, last_name, ...而不是customer?在这种情况下,您需要逐个设置字段而不是整个json字符串,例如recordsContent.put("id", 1)。 检查example with comments

© www.soinside.com 2019 - 2024. All rights reserved.