我正在尝试使用 celery 应用程序设置 SQS 代理,并在 django 项目中配置。 这是我的设置:
celery.py:
import os
from celery import Celery
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "core.settings")
app = Celery("my-app")
app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()
我的芹菜相关的django设置:
CELERY_BROKER_TRANSPORT_OPTIONS = {
"region": "eu-west-1",
"queue_name_prefix": f"celery-",
}
CELERY_BROKER_URL = "sqs://"
CELERY_TASK_DEFAULT_QUEUE = "default"
这个配置使用rabbitmq代理工作得很好,这让我认为整体配置是正确的,但是当我使用SQS作为代理时,消息被发送到SQS(“消息可用”计数器增加),但是一旦被我的接收到工作人员,他们会转到“飞行中的消息”并似乎永远呆在那里(至少几个小时)。 另外,我可以看到工作人员记录正在执行的操作,但由于某种原因从未实际执行该任务。
以下是一些芹菜工人日志:
host;x-amz-date
786cb490d758593ebf5a6e0c0b34cf025b3309bb0d777891344fd32bd01cb61b
[2023-11-19 19:52:05,766: DEBUG/MainProcess] StringToSign:
AWS4-HMAC-SHA256
20231119T195205Z
20231119/eu-west-1/sqs/aws4_request
32a2b642b24ffcd3f42216826611c27799efb1cf756b26e7119d5553009f527c
[2023-11-19 19:52:05,766: DEBUG/MainProcess] Signature:
93bcf47703548603fa0787fbbd2ae8aa4b54460db91695f4f6c818b9141b620e
[2023-11-19 19:52:10,959: DEBUG/MainProcess] Response headers: {}
[2023-11-19 19:52:10,960: DEBUG/MainProcess] Response body:
b'<?xml version="1.0"?><ReceiveMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/"><ReceiveMessageResult><Message><MessageId>c4e1b404-.....-c046dc4aeb60</MessageId><ReceiptHandle>AQEBy+4G0x9BF8su10zFWJQuyEXJOFSxF........cRPUdk/vjWzirMiBw97ZSG44M=</ReceiptHandle><MD5OfBody>483202aa2938b...da1b024b63</MD5OfBody><Body>eyJib2R5IjogIlcxdGRMQ0I3...CJyZXBseV90byI6ICJmOWZjY2FmDIyMTBkYTAyMiJ9fQ==</Body><Attribute><Name>ApproximateReceiveCount</Name><Value>1</Value></Attribute></Message></ReceiveMessageResult><ResponseMetadata><RequestId>28a76707-9a01...d3f3a42ad7</RequestId></ResponseMetadata></ReceiveMessageResponse>'
[2023-11-19 19:52:10,960: DEBUG/MainProcess] Event choose-signer.sqs.ReceiveMessage: calling handler <function set_operation_specific_signer at 0xffff9b901750>
[2023-11-19 19:52:10,960: DEBUG/MainProcess] Calculating signature using v4 auth.
[2023-11-19 19:52:10,960: DEBUG/MainProcess] CanonicalRequest:
POST
/ACCOUNT_ID/celery-default
host:sqs.eu-west-1.amazonaws.com
x-amz-date:20231119T195210Z
host;x-amz-date
786cb490d758593ebf5a6e...d777891344fd32bd01cb61b
[2023-11-19 19:52:10,961: DEBUG/MainProcess] StringToSign:
AWS4-HMAC-SHA256
20231119T195210Z
20231119/eu-west-1/sqs/aws4_request
a6b5c2bb4cb9eef89...4e16bc164d0a0d9b1a1
[2023-11-19 19:52:10,961: DEBUG/MainProcess] Signature:
81bd2621f3ce88b5a76...79dc18951c3bc0c8a
[2023-11-19 19:52:21,000: DEBUG/MainProcess] Response headers: {}
[2023-11-19 19:52:21,000: DEBUG/MainProcess] Response body:
b'<?xml version="1.0"?><ReceiveMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/"><ReceiveMessageResult/><ResponseMetadata><RequestId>802bb8a8-...-e57c9a538959</RequestId></ResponseMetadata></ReceiveMessageResponse>'
[2023-11-19 19:52:21,000: DEBUG/MainProcess] Event choose-signer.sqs.ReceiveMessage: calling handler <function set_operation_specific_signer at 0xffff9b901750>
[2023-11-19 19:52:21,001: DEBUG/MainProcess] Calculating signature using v4 auth.
[2023-11-19 19:52:21,001: DEBUG/MainProcess] CanonicalRequest:
POST
/ACCOUNT_ID/celery-default
我不认为这是权限问题,因为我可以使用执行角色在 AWS (fargate) 上重现该错误,也可以在本地使用管理员用户重现该错误,这两种情况下的结果相同。
请问您知道出了什么问题吗?
最后,更新芹菜(和昆布)解决了这个问题!