将文件上传到存储的Python函数失败并出现403 TooManyDevicesError

问题描述 投票:0回答:1

我有一个在嵌入式 Linux 设备上运行的 python 应用程序。该应用程序运行连接 iothub 设备客户端的异步代码。我有一个名为 device_client_wrapper 的类,它使用信号量来确保一次只有一个函数调用设备客户端方法,因为这之前给我带来了问题。

我刚刚向此类添加了一个新函数,用于将文件从设备上传到 azure blob 存储。它运行良好一段时间,并且经常第一次运行,但是如果我执行太多次,我会收到以下错误:

ERROR upload_file Exception: Unexpected failure
ERROR Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/azure/iot/device/iothub/aio/async_clients.py", line 33, in handle_result
    return await callback.completion()
  File "/usr/lib/python3.10/site-packages/azure/iot/device/common/async_adapter.py", line 91, in completion
    return await self.future
azure.iot.device.exceptions.ServiceError: HTTP operation returned: 403 TooManyDevicesError(Error: Forbidden)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/cogo_ig60/cloud/device_client_wrapper.py", line 99, in upload_file
    blob_info = await self.device_client.get_storage_info_for_blob(blob_name)
  File "/usr/lib/python3.10/site-packages/azure/iot/device/iothub/aio/async_clients.py", line 571, in get_storage_info_for_blob
    storage_info = await handle_result(callback)
  File "/usr/lib/python3.10/site-packages/azure/iot/device/iothub/aio/async_clients.py", line 57, in handle_result
    raise exceptions.ClientError("Unexpected failure") from e
azure.iot.device.exceptions.ClientError: Unexpected failure

这是导致错误的函数:

async def upload_file(self, path_to_file: str, blob_name: str):
    """Upload a file to Azure Blob Storage."""
    try:
        async with self.semaphore:
            blob_info = await self.device_client.get_storage_info_for_blob(blob_name)
            sas_url = "https://{}/{}/{}{}".format(
                blob_info["hostName"],
                blob_info["containerName"], 
                blob_info["blobName"],
                blob_info["sasToken"]
            )
            logger.info(
                f"Uploading file {path_to_file} to blob storage as {blob_name}")
            with BlobClient.from_blob_url(sas_url) as blob_client:
                with open(path_to_file, "rb") as f:
                    result = blob_client.upload_blob(f, overwrite=True)
                    return (True, result)
    except FileNotFoundError as e:
        # catch file not found and add an HTTP status code to return in notification to IoT Hub
        logger.error(f'upload_file FileNotFoundError: {e}')
        logger.error(traceback.format_exc())
        e.status_code = 404
        return (False, e)
    except AzureError as e:
        # catch Azure errors that might result from the upload operation
        logger.error(f'upload_file AzureError: {e}')
        logger.error(traceback.format_exc())
        return (False, e)
    except Exception as e:
        # catch all other errors
        logger.error(f'upload_file Exception: {e}')
        logger.error(traceback.format_exc())
        return (False, e)

这似乎是 device_client.get_storage_info_for_blob 的问题。我知道每个设备最多有 10 个并发文件上传,但我正在上传 7 个文件,然后它们在我重试之前成功上传。即便如此,使用信号量应该可以防止任何并发文件上传。除非我遗漏了有关我的代码如何工作的信息。

是否有某些内容未正确发布或无法正确完成?

顺便说一句,这是 python 3.10.4。

python azure azure-blob-storage azure-iot-hub azure-iot-sdk
1个回答
0
投票

找到了指示此问题的潜在原因的其他信息。在此发布,以防其他社区成员遇到此行为。

每台设备一次最多只能上传 10 个并发活动文件。除此之外,还会根据 SKU 和可用设备数量对 IoT 中心应用额外的限制。例如,S1 层的限制为 1.67 个文件上传启动/秒/单位。假设您有 S1 层的单元,您将达到每秒 6.67 个文件上传的限制。

请参阅以下文档以供参考。

© www.soinside.com 2019 - 2024. All rights reserved.