Celery立刻超过了Heroku的内存

Question

我正在向Heroku部署一个Celery进程，每次启动它都会立即开始占用内存使用量，并在超过最大值后崩溃。

我只有一个名为“test_task”的任务，每分钟打印一次。

这是使用Celery的Django应用程序，以及在Heroku上托管的Redis后端。

Proc文件：

web: daphne chatbot.asgi:channel_layer --port $PORT --bind 0.0.0.0 --verbosity 1
chatworker: python manage.py runworker --verbosity 1
celeryworker: celery -A chatbot worker -l info

Heroku日志：

app[celeryworker.1]: [INFO] 2018-09-11 23:06:36,710 : Scheduling celery jobs...
app[celeryworker.1]: [INFO] 2018-09-11 23:06:36,880 : adding minute task
app[celeryworker.1]: 
app[celeryworker.1]: -------------- celery@f46a12fb-1666-449a-b287-2ce90f95cf2c v4.2.0 (windowlicker)
app[celeryworker.1]: --- * ***  * -- Linux-4.4.0-1027-aws-x86_64-with-debian-jessie-sid 2018-09-11 23:06:37
app[celeryworker.1]: -- * - **** ---
app[celeryworker.1]: ---- **** -----
app[celeryworker.1]: - ** ---------- [config]
app[celeryworker.1]: - ** ---------- .> app:         chatbot:0x7f1c2a5ad5d0
app[celeryworker.1]: - ** ---------- .> transport:   redis://east-1-4.ec2.cloud.redislabs.com:13275//
app[celeryworker.1]: - ** ---------- .> results:     disabled://
app[celeryworker.1]: - *** --- * --- .> concurrency: 8 (prefork)
app[celeryworker.1]: -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
app[celeryworker.1]: --- ***** -----
app[celeryworker.1]: -------------- [queues]
app[celeryworker.1]: .> celery           exchange=celery(direct) key=celery
app[celeryworker.1]: 
app[celeryworker.1]: 
app[celeryworker.1]: [tasks]
app[celeryworker.1]: . companies.tasks.test_task
app[celeryworker.1]: 
heroku[celeryworker.1]: Process running mem=616M(120.4%)
heroku[celeryworker.1]: Error R14 (Memory quota exceeded)
app[api]: Build succeeded
app[celeryworker.1]: [2018-09-11 23:06:55,033: INFO/MainProcess] Connected to redis://rediscloud:**@redis-13275.c12.us-east-1-4.ec2.cloud.redislabs.com:13275//
app[celeryworker.1]: [2018-09-11 23:06:55,201: INFO/MainProcess] mingle: searching for neighbors
app[celeryworker.1]: [2018-09-11 23:06:56,462: INFO/MainProcess] mingle: all alone
heroku[celeryworker.1]: Process running mem=1063M(207.8%)
heroku[celeryworker.1]: Error R15 (Memory quota vastly exceeded)
heroku[celeryworker.1]: Stopping process with SIGKILL
heroku[celeryworker.1]: State changed from up to crashed
heroku[celeryworker.1]: State changed from crashed to starting
heroku[celeryworker.1]: Process exited with status 137

有什么明显的东西我做错了吗？

Answer 1

结果是将芹菜工人流程中的--concurrency参数更改为较低的工作数。

我把它设置为celery -A chatbot worker -l info --concurrency 2，它似乎正在工作！