Azure DataFactory DataFlow：需要将 JSON 文档从每行文档转换为单个复杂文档

Question

当前使用 ADF 尝试将 Datalake 中的 JSON 文档架构转换为需要将项目加载到 HubSpot 的架构。我遇到的问题是输出模式始终是每行文档而不是单个文档。我使用带有 DeriveColumn 的 DataFlow 来定义架构。

需要获取此源格式：

{"Id":"1000000000003919138","Version":0,"ContactKey":"148101","IsIndividual":true,"Name":"Margerie Bouvier Simpson","FirstName":"Margerie","LastName":"Simpson","MiddleName":"Bouvier","EmailAddresses":"","OtherAddresses":"","PhoneNumbers":"","Roles":122,"FaxNumbers":"","WebSites":"","RecordDate":"2024-04-12T18:34:07.019505+00:00"}
{"Id":"1000000000003919134","Version":0,"ContactKey":"148100","IsIndividual":true,"Name":"Homer J Simpson","FirstName":"Homer","LastName":"Simpson","MiddleName":"J","EmailAddresses":"","OtherAddresses":"","PhoneNumbers":"","Roles":122,"FaxNumbers":"","WebSites":"","RecordDate":"2024-04-12T18:34:07.007973+00:00"}

对于此输出格式：

{
    "inputs": [
        {
            "properties": {
                "email": "[email protected]",
                "firstname": "Brenda",
                "lastname": "Wooster",
                "phone": "(123) 1232332",
                "company": "",
                "website": ""
            }
        },
        {
            "properties": {
                "email": "",
                "firstname": "James",
                "lastname": "Kline",
                "phone": "",
                "company": "",
                "website": ""
            }
        }
    ]
}

来源设置：

接收器设置：

Answer 1

• 使用源数据集进行源转换。

• 采用派生列转换并给出表达式以将源中的所有列添加为

  properties

  列下的子列。

派生列转换中的示例表达式：

@(properties=@(Email=EmailAddresses,
        FirstName=FirstName,
        LastName=LastName,
        Phone=PhoneNumbers,
        WebSites=WebSites))

• 采取聚合转换，不按任何列分组。添加新列

  input

  进行聚合，并将表达式指定为

  collect(properties)

• 进行接收器转换并将格式设置为 JSON，并将文件名设置为所需的输出文件名。

此方法应提供所需的输出 JSON 格式。