我有一组数据,我想使用 Azure 数据资源管理器使用 KQL 进行查询。我每隔几秒钟就有一个连续的传入数据源。我想将此数据放入 Azure 集群以运行查询。
我探索了几个使用 python 库的选项,但它只支持使用文件或 blob。
https://learn.microsoft.com/en-us/azure/data-explorer/python-ingest-data
如何使用 python 将单个记录放入,以便我可以使用 Azure 数据资源管理器进行查询。
这里有示例代码,用于使用
azure-kusto-ingest
库从数据框(包含一条或多条记录)摄取here
cluster = "https://ingest-{cluster_name}.kusto.windows.net/"
# In case you want to authenticate with AAD application.
client_id = "<insert here your AAD application id>"
client_secret = "<insert here your AAD application key>"
# read more at https://docs.microsoft.com/en-us/onedrive/find-your-office-365-tenant-id
authority_id = "<insert here your tenant id>"
kcsb = KustoConnectionStringBuilder.with_aad_application_key_authentication(cluster, client_id, client_secret, authority_id)
client = QueuedIngestClient(kcsb)
# there are a lot of useful properties, make sure to go over docs and check them out
ingestion_props = IngestionProperties(
database="{database_name}",
table="{table_name}",
data_format=DataFormat.CSV,
# in case status update for success are also required (remember to import ReportLevel from azure.kusto.ingest)
# report_level=ReportLevel.FailuresAndSuccesses,
# in case a mapping is required (remember to import IngestionMappingKind from azure.kusto.data.data_format)
# ingestion_mapping_reference="{json_mapping_that_already_exists_on_table}",
# ingestion_mapping_kind= IngestionMappingKind.JSON,
)
###########################
## ingest from dataframe ##
###########################
import pandas
fields = ["id", "name", "value"]
rows = [[1, "abc", 15.3], [2, "cde", 99.9]]
df = pandas.DataFrame(data=rows, columns=fields)
client.ingest_from_dataframe(df, ingestion_properties=ingestion_props)