google.api_core.exceptions.ServiceUnavailable：超过503个截止日期

Question

google.api_core.exceptions.Service不可用：超过503个截止日期使用python 3.7，google-cloud-pubsub == 1.1.0发布有关主题的数据。在我的本地计算机上，它运行良好，可以发布有关该主题的数据，还可以通过订阅者从该主题中提取数据。但是当我在服务器上部署代码并且以INLINE ERROR失败时，它不起作用，但是当我在服务器上显式调用Publisher方法时，它也可以在服务器框上正常发布。发布时：

future = publisher.publish(topic_path, data=data)

**ERROR:2020-02-20 14:24:42,714 ERROR Failed to publish 1 messages.**
Trackback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 826, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 729, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "Deadline Exceeded"
        debug_error_string = "{"created":"@1582208682.711481693","description":"Deadline Exceeded","file":"src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":14}"
The above exception was the direct cause of the following exception: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/google/api_core/retry.py", line 184, in retry_target
    return target()
  File "/usr/local/lib/python3.7/site-packages/google/api_core/timeout.py", line 214, in func_with_timeout
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
    six.raise_from(exceptions.from_grpc_error(exc), exc)
  File "<string>", line 3, in raise_from
google.api_core.exceptions.ServiceUnavailable: 503 Deadline Exceeded

上述异常是以下异常的直接原因：

追踪（最近通话）：_commit中的文件“ /usr/local/lib/python3.7/site-packages/google/cloud/pubsub_v1/publisher/_batch/thread.py”，第219行响应= self._client.api.publish（self._topic，self._messages）在发布文件“ /usr/local/lib/python3.7/site-packages/google/cloud/pubsub_v1/gapic/publisher_client.py”中，第498行请求，重试=重试，超时=超时，元数据=元数据call中的文件“ /usr/local/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py”，第143行返回wrapd_func（* args，** kwargs）retry_wrapped_func中的文件“ /usr/local/lib/python3.7/site-packages/google/api_core/retry.py”，行286on_error = on_error，文件“ /usr/local/lib/python3.7/site-packages/google/api_core/retry.py”，行206，位于retry_target中last_exc，在第三行的“ _”文件“”中google.api_core.exceptions.RetryError：在调用functools.partial（.error_remapped_callable at 0x7f67d064e950>

时，截止日期超过了60.0s

Answer 1

您应该尝试将数据划分为合理大小的块（max_messages），并且不要忘记添加完成的回调。

# Loop over json containing records/rows
for idx, row in enumerate(rows_json):
    publish_json(row, idx, rowmax=len(rows_json), topic_name)

# Publish messages asynchronous 
def publish_json(msg, rowcount, rowmax, topic_project_id, topic_name):
    batch_settings = pubsub_v1.types.BatchSettings(max_messages=100)
    publisher = pubsub_v1.PublisherClient(batch_settings)
    topic_path = publisher.topic_path(topic_project_id, topic_name)
    future = publisher.publish(
        topic_path, bytes(json.dumps(msg).encode('utf-8')))
    future.add_done_callback(
        lambda x: logging.info(
            'Published msg with ID {} ({}/{} rows).'.format(
                future.result(), rowcount, rowmax))
    )