Azure流分析作业的运行时错误

Question

我试图通过添加更多临时结果集来修改现有的Azure流分析作业。但是，当我运行SA作业时，它会引发运行时错误，并且水印延迟会增加。

下面是Stream Analytics作业中的现有SAQL：

-- Reading from Event Hub
WITH INPUTDATASET AS (
    SELECT 
        udf.udf01([signals2]) AS flat
    FROM [signals2]
    PARTITION BY PartitionId
    WHERE [signals2].ABC IS NOT NULL
),

INPUT1 AS (
    SELECT
        ID,
        SIG1,
        SIG2
    FROM [signals2] as input
    WHERE GetArrayLength(input.XYZ) >=1
)

--Dump the data from above result set into cosmosDB

我试图将以下临时结果集添加到SAQL：

INPUT2 AS (
    SELECT
        ID,
        SIG3,
        SIG4
    FROM [signals2] as input
    WHERE GetArrayLength(input.XYZ) =0
)

现在，当我开始执行SA作业时，它将引发运行时错误。

[当我获取以下日志时，是我收到的错误日志。

TimeGenerated,Resource,"Region_s",OperationName,"properties_s",Level
"2020-01-01T01:10:10.085Z",SAJOB01,"Japan West","Diagnostic: Diagnostic Error","{""Error"":null,""Message"":""First Occurred: 01\/01\/2020 01:10:10 | Resource Name: signals2 | Message: Maximum Event Hub receivers exceeded. Only 5 receivers per partition are allowed.\r\nPlease use dedicated consumer group(s) for this input. If there are multiple queries using same input, share your input using WITH clause. \r\n "",""Type"":""DiagnosticMessage"",""Correlation ID"":""xxxx""}",Error
"2020-01-01T01:10:10.754Z",SAJOB01,"Japan West","Receive Events: ","{""Error"":null,""Message"":""We cannot connect to Event Hub partition [25] because the maximum number of allowed receivers per partition in a consumer group has been reached. Ensure that other Stream Analytics jobs or Service Bus Explorer are not using the same consumer group. The following information may be helpful in identifying the connected receivers: Exceeded the maximum number of allowed receivers per partition in a consumer group which is 5. List of connected receivers - AzureStreamAnalytics_xxxx_25, AzureStreamAnalytics_xxxx_25, AzureStreamAnalytics_zzz_25, AzureStreamAnalytics_xxx_25, AzureStreamAnalytics_xxx_25. TrackingId:xxx_B7S2, SystemTracker:eventhub001-ns:eventhub:ehub01~26|consumergrp01, Timestamp:2020-01-01T01:10:10 Reference:xxx, TrackingId:xxx_B7S2, SystemTracker:eventhub001-ns:eventhub:ehub01~26|consumergrp01, Timestamp:2020-01-01T01:10:10, referenceId: xxx_B7S2"",""Type"":""EventHubBasedInputQuotaExceededError"",""Correlation ID"":""xxxx""}",Error...

对于SA作业，输入signals2具有专用的使用者组（consumergrp01）

对于此Stream Analytics作业，可以使用专用的使用者组。该使用者组的分区上有3个读取器，但由于最大事件中心接收器超过，它仍然会引发错误。为什么会这样？

Answer 1

消息：事件集线器接收器已超过最大数量。每个仅5个接收器允许分区。

我认为错误消息在运行时错误的根本原因上是明确的。请参阅此doc中的语句：

每个使用者在一个分区上最多可以有5个并发读取器组;但是，建议只有一个活动每个消费者组的分区上的接收方。在一个分区内，每个阅读器都会收到所有消息。如果您有多个读者在同一分区上，然后您处理重复的消息。你需要在您的代码中处理此问题，这可能并非易事。但是，这是一个在某些情况下有效的方法

您可以遵循错误消息中的建议：If there are multiple queries using same input, share your input using WITH clause。尝试将input2隔离为一个临时结果集，然后在其他临时结果中使用它。