如何在R中使用tidyjson处理嵌套的空JSON数组

问题描述 投票：1回答：1

我正在从Salesforce中读取JSON对象。从某种意义上说，该对象是不规则的，因为有些嵌套数组是空的，有些不是。如何在tidyjson中处理此问题？

我正在使用R中的Salesforce设置API。目标是从Salesforce中获取有意义的数据以在R中进行处理。

json <- '
{
  "totalSize": [
    355710
  ],
  "done": [
    false
  ],
  "nextRecordsUrl": [
    "/services/data/v45.0/query/01gc000001L8zdkAAB-749"
  ],
  "records": [
    {
      "attributes": {
        "type": "Order_Line__c",
        "url": "/services/data/v45.0/sobjects/Order_Line__c/a0T1N000009aZ9lUAE"
      },
      "Id": "a0T1N000009aZ9lUAE",
      "Name": "OrderLine-1099369",
      "SO_Number_Formula__c": "548402-2.3",
      "Ship_From_Inventory__c": "XXX",
      "RMA_Number__c": "548402",
      "Part_Number__c": "01t1N00000JNeAQQA1",
      "Marketing_Part__c": "XXXXXXXXXXX",
      "Family__c": "XXXXXXXX",
      "Serial_Numbers__r": {
        "records": {}
      }
    },
    {
      "attributes": {
        "type": "Order_Line__c",
        "url": "/services/data/v45.0/sobjects/Order_Line__c/a0T1N000009aZ9mUAE"
      },
      "Id": "a0T1N000009aZ9mUAE",
      "Name": "OrderLine-1099370",
      "SO_Number_Formula__c": "962816-1.1",
      "Ship_From_Inventory__c": "XXX",
      "RMA_Number__c": "962816",
      "Part_Number__c": "01t1N00000JNc3qQAD",
      "Marketing_Part__c": "XXXXXXXXXX",
      "Family__c": "XXXXXXX",
      "RMA_Received_Date__c": "2019-02-18",
      "Serial_Numbers__r": {
        "totalSize": 1,
        "done": true,
        "records": [
          {
            "attributes": {
              "type": "Serial_Number__c",
              "url": "/services/data/v45.0/sobjects/Serial_Number__c/a0X1N00000NoyAjUAJ"
            },
            "Id": "a0X1N00000NoyAjUAJ",
            "Name": "SN217426",
            "Legacy_Line_Id__c": "962816SN217426",
            "Customer_Name__c": "XXXXXX",
            "Original_Shipment_Date__c": "2018-06-26",
            "Disposition__c": "Pending",
            "Status__c": "FailureVerification"
          }
        ]
      }
    }
  ]
}
'

mydata <- json %>% 
    as.tbl_json %>%
    enter_object("records") %>%
    gather_array() %>%
    spread_values(
      Id = jstring("Id"),
      Name = jstring("Name"),
      SO_Number_Formula = jstring("SO_Number_Formula__c"),
      Ship_From_Inventory = jstring("Ship_From_Inventory__c"),
      RMA_Number = jstring("RMA_Number__c"),
      Part_Number = jstring("Part_Number__c"),
      Marketing_Part = jstring("Marketing_Part__c"),
      Family = jstring("Family__c")) %>%
    enter_object("Serial_Numbers__r") %>%
    enter_object("records") %>%
    gather_ %>%
      spread_values(
    Id = jstring("Id"))

[该不规则性出现在[记录] [Serial_Numbers__r] [记录]中。在此示例中，第一次出现为空{}，第二次出现不为空。代码在执行collect_keys或collect _array时会产生以下错误：collect_keys（。）中的错误：1个记录是值而不是对象collect_array（。）中的错误：1个记录是值而不是数组

我认为这是由空数组[records]引起的。 Salesforce输出中存在很多此类不规则性：某些记录具有详细的嵌套数据，而有些则没有。我该如何处理？

r salesforce-lightning jsonlite
1个回答
0
投票

这是一个奇妙的问题，我们应该真正采用一种更清洁的方式来处理。在此类情况下，enter_object()被证明是有问题的，这些情况下您会基于不规则的JSON做法丢失记录。

我提交了一个问题来跟踪此处的改进：https://github.com/colearendt/tidyjson/issues/121

同时，我通常这样做的方法是根据描绘记录的特征来分割记录。在这种情况下，可以在父对象上使用gather_object()获得与enter_object()相同的效果，然后使用filter / bind_rows区别对待行。

理想情况下，bind_rows()在此处的管道中会更好......我想将其作为dplyr (Issue here)的改进！我很好奇这是否能解决您的问题！ （此外，请记住spread_all()来简化指定的某些列，但要花一些“猜测”的代价！）。

  json <- '{
  "totalSize": [
    355710
  ],
  "done": [
    false
  ],
  "nextRecordsUrl": [
    "/services/data/v45.0/query/01gc000001L8zdkAAB-749"
  ],
  "records": [
    {
      "attributes": {
        "type": "Order_Line__c",
        "url": "/services/data/v45.0/sobjects/Order_Line__c/a0T1N000009aZ9lUAE"
      },
      "Id": "a0T1N000009aZ9lUAE",
      "Name": "OrderLine-1099369",
      "SO_Number_Formula__c": "548402-2.3",
      "Ship_From_Inventory__c": "XXX",
      "RMA_Number__c": "548402",
      "Part_Number__c": "01t1N00000JNeAQQA1",
      "Marketing_Part__c": "XXXXXXXXXXX",
      "Family__c": "XXXXXXXX",
      "Serial_Numbers__r": {
        "records": {}
      }
    },
    {
      "attributes": {
        "type": "Order_Line__c",
        "url": "/services/data/v45.0/sobjects/Order_Line__c/a0T1N000009aZ9mUAE"
      },
      "Id": "a0T1N000009aZ9mUAE",
      "Name": "OrderLine-1099370",
      "SO_Number_Formula__c": "962816-1.1",
      "Ship_From_Inventory__c": "XXX",
      "RMA_Number__c": "962816",
      "Part_Number__c": "01t1N00000JNc3qQAD",
      "Marketing_Part__c": "XXXXXXXXXX",
      "Family__c": "XXXXXXX",
      "RMA_Received_Date__c": "2019-02-18",
      "Serial_Numbers__r": {
        "totalSize": 1,
        "done": true,
        "records": [
          {
            "attributes": {
              "type": "Serial_Number__c",
              "url": "/services/data/v45.0/sobjects/Serial_Number__c/a0X1N00000NoyAjUAJ"
            },
            "Id": "a0X1N00000NoyAjUAJ",
            "Name": "SN217426",
            "Legacy_Line_Id__c": "962816SN217426",
            "Customer_Name__c": "XXXXXX",
            "Original_Shipment_Date__c": "2018-06-26",
            "Disposition__c": "Pending",
            "Status__c": "FailureVerification"
          }
        ]
      }
    }
  ]
}
'

  library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
  library(tidyr)
  library(tidyjson)
#> 
#> Attaching package: 'tidyjson'
#> The following object is masked from 'package:dplyr':
#> 
#>     bind_rows
#> The following object is masked from 'package:stats':
#> 
#>     filter

  prep_data <- json %>%
    as.tbl_json %>%
    enter_object("records") %>%
    gather_array() %>%
    spread_values(
      Id = jstring("Id"),
      Name = jstring("Name"),
      SO_Number_Formula = jstring("SO_Number_Formula__c"),
      Ship_From_Inventory = jstring("Ship_From_Inventory__c"),
      RMA_Number = jstring("RMA_Number__c"),
      Part_Number = jstring("Part_Number__c"),
      Marketing_Part = jstring("Marketing_Part__c"),
      Family = jstring("Family__c")) %>%
    enter_object("Serial_Numbers__r")

  # show that types are different
  prep_data %>%
    gather_object("key") %>%
    json_types() %>%
    select(key, type) %>%
    filter(key == "records")
#> # A tbl_json: 2 x 2 tibble with a "JSON" attribute
#>   `attr(., "JSON")`      key     type  
#>   <chr>                  <chr>   <fct> 
#> 1 "{}"                   records object
#> 2 "[{\"attributes\":..." records array

  # handle
  taller <- prep_data %>%
    gather_object("key") %>%
    json_types("type") %>%
    filter(key == "records")

  final <- tidyjson::bind_rows(
    taller %>% filter(type == "object"),
    taller %>% filter(type == "array") %>%
      gather_array("record_row") %>%
      spread_values(
        RecordId = jstring("Id")
      )
  )

  final %>% select(key, type, record_row, RecordId)
#> # A tbl_json: 2 x 4 tibble with a "JSON" attribute
#>   `attr(., "JSON")`      key     type   record_row RecordId          
#>   <chr>                  <chr>   <fct>       <int> <chr>             
#> 1 "{}"                   records object         NA <NA>              
#> 2 "{\"attributes\":{..." records array           1 a0X1N00000NoyAjUAJ

由reprex package（v0.3.0）在2020-03-15创建

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.