ADF 管道将数据从位于 Teams 通道中的 Excel 文件加载到 SQL Server

问题描述 投票:0回答:1

我在 Teams 频道中填充了一个 Excel 文件。这是一份每天手动更新的报告。我想创建一个管道来将此文件加载到 SQL Server 数据库。

可行吗?我尝试了网络活动,但不确定这是否是正确的开始方式。

sql-server excel azure-data-factory microsoft-teams
1个回答
0
投票

如果我转到“复制链接”,我会看到 SharePoint 位置。

要将 Excel 文件从 SharePoint 复制到 Azure SQL 数据库,应首先将该文件存储在 Blob 存储中。您可以使用逻辑应用程序将 Excel 文件从 SharePoint 复制到 Blob 存储。成功复制文件后,可以使用 ADF 将其复制到 Azure SQL 数据库。如果您的 Excel 中只有一张工作表,请手动选择它,否则请按照以下步骤操作:

  • 添加三个名为

    c1
    的变量,默认值为0、
    tempc
    iserror
    ,如下所示:

    Variables in Pipeline

  • 将 Until 活动添加到管道,并使用

    tempc
    动态表达式设置变量
    @string(add(int(variables('c1')),1))
    的变量活动。

  • 将复制活动添加到设置的变量中。选择带有

    index
    和动态值
    @int(variables('c1'))
    的 Excel 数据集,然后选择 Excel 文件作为源,选择带有
    tableName
    和动态值
    excel@{variables('c1')}
    参数的 SQL 数据集作为接收器。

  • 复制数据成功后,添加带有

    c1
    变量和动态值
    @variables('tempc')
    的“设置变量”活动。复制数据活动失败时,添加带有
    iserror
    和动态值
    @bool(1)
    的设置变量活动。将
    @variables('iserror')
    添加到 Until 活动。

这是动态 Excel 工作表副本的管道 JSON:

{
    "name": "excel",
    "properties": {
        "activities": [
            {
                "name": "Until1",
                "type": "Until",
                "dependsOn": [],
                "userProperties": [],
                "typeProperties": {
                    "expression": {
                        "value": "@variables('iserror')",
                        "type": "Expression"
                    },
                    "activities": [
                        {
                            "name": "tempc",
                            "type": "SetVariable",
                            "dependsOn": [],
                            "policy": {
                                "secureOutput": false,
                                "secureInput": false
                            },
                            "userProperties": [],
                            "typeProperties": {
                                "variableName": "tempc",
                                "value": {
                                    "value": "@string(add(int(variables('c1')),1))",
                                    "type": "Expression"
                                }
                            }
                        },
                        {
                            "name": "Copy data1",
                            "type": "Copy",
                            "dependsOn": [
                                {
                                    "activity": "tempc",
                                    "dependencyConditions": [
                                        "Succeeded"
                                    ]
                                }
                            ],
                            "policy": {
                                "timeout": "0.12:00:00",
                                "retry": 0,
                                "retryIntervalInSeconds": 30,
                                "secureOutput": false,
                                "secureInput": false
                            },
                            "userProperties": [],
                            "typeProperties": {
                                "source": {
                                    "type": "ExcelSource",
                                    "storeSettings": {
                                        "type": "AzureBlobStorageReadSettings",
                                        "recursive": true,
                                        "enablePartitionDiscovery": false
                                    }
                                },
                                "sink": {
                                    "type": "AzureSqlSink",
                                    "writeBehavior": "insert",
                                    "sqlWriterUseTableLock": false,
                                    "tableOption": "autoCreate",
                                    "disableMetricsCollection": false
                                },
                                "enableStaging": false,
                                "translator": {
                                    "type": "TabularTranslator",
                                    "typeConversion": true,
                                    "typeConversionSettings": {
                                        "allowDataTruncation": true,
                                        "treatBooleanAsNumber": false
                                    }
                                }
                            },
                            "inputs": [
                                {
                                    "referenceName": "Excel1",
                                    "type": "DatasetReference"
                                }
                            ],
                            "outputs": [
                                {
                                    "referenceName": "AzureSqlTable1",
                                    "type": "DatasetReference"
                                }
                            ]
                        },
                        {
                            "name": "c1",
                            "type": "SetVariable",
                            "dependsOn": [
                                {
                                    "activity": "Copy data1",
                                    "dependencyConditions": [
                                        "Succeeded"
                                    ]
                                }
                            ],
                            "policy": {
                                "secureOutput": false,
                                "secureInput": false
                            },
                            "userProperties": [],
                            "typeProperties": {
                                "variableName": "c1",
                                "value": {
                                    "value": "@variables('tempc')",
                                    "type": "Expression"
                                }
                            }
                        },
                        {
                            "name": "iserror",
                            "type": "SetVariable",
                            "dependsOn": [
                                {
                                    "activity": "Copy data1",
                                    "dependencyConditions": [
                                        "Failed"
                                    ]
                                }
                            ],
                            "policy": {
                                "secureOutput": false,
                                "secureInput": false
                            },
                            "userProperties": [],
                            "typeProperties": {
                                "variableName": "iserror",
                                "value": {
                                    "value": "@bool(1)",
                                    "type": "Expression"
                                }
                            }
                        }
                    ],
                    "timeout": "0.12:00:00"
                }
            }
        ],
        "variables": {
            "c1": {
                "type": "String",
                "defaultValue": "0"
            },
            "iserror": {
                "type": "Boolean"
            },
            "tempc": {
                "type": "String"
            }
        },
        "annotations": [],
        "lastPublishTime": "2024-02-23T08:00:17Z"
    },
    "type": "Microsoft.DataFactory/factories/pipelines"
}

调试它将运行的管道,如下所示:

enter image description here

Excel 工作表将成功复制为 SQL 数据库中的表。运行以下脚本以显示复制的表:

SELECT TABLE_SCHEMA,TABLE_NAME FROM information_schema.TABLES
WHERE TABLE_TYPE =  'BASE TABLE'  and TABLE_SCHEMA =  'dbo'

脚本将显示如下所示的表格:

表_架构 表名称
dbo excel0
dbo excel11
dbo 输入
dbo 输入1
© www.soinside.com 2019 - 2024. All rights reserved.