django celery [celery.worker] 调试：定时器唤醒！运行 scrapy 进程时的下一个预计到达时间

问题描述 投票：0回答：1

我正在 Django 中使用 celery 在 task.py 文件中运行一个简单的蜘蛛。运行我的蜘蛛的代码如下：

文件task.py：

@shared_task(bind=True) 
def scrape_amazon_products(self,products_urls,current_user,task_id,current_site):

    process = CrawlerProcess()
    process.crawl(my_spider) #Here the name of the spider
    process.start()
    process.stop()

效果很好，蜘蛛开始并完成，但是：

问题： 芹菜永不停歇，不断地给出下面提到的这些日志：

[2022-03-12 19:44:57,512: WARNING/MainProcess] 2022-03-12 19:44:57 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:44:58,535: DEBUG/MainProcess] Timer wake-up! Next ETA 0.9690000000118744 secs.
[2022-03-12 19:44:58,536: WARNING/MainProcess] 2022-03-12 19:44:58 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.9690000000118744 secs.
[2022-03-12 19:44:59,527: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:44:59,528: WARNING/MainProcess] 2022-03-12 19:44:59 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:00,543: DEBUG/MainProcess] Timer wake-up! Next ETA 0.9839999999967404 secs.
[2022-03-12 19:45:00,544: WARNING/MainProcess] 2022-03-12 19:45:00 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.9839999999967404 secs.
[2022-03-12 19:45:01,546: DEBUG/MainProcess] Timer wake-up! Next ETA 0.062000000005355105 secs.
[2022-03-12 19:45:01,547: WARNING/MainProcess] 2022-03-12 19:45:01 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.062000000005355105 secs.
[2022-03-12 19:45:01,619: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:01,620: WARNING/MainProcess] 2022-03-12 19:45:01 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:02,627: DEBUG/MainProcess] Timer wake-up! Next ETA 0.9219999999913853 secs.
[2022-03-12 19:45:02,628: WARNING/MainProcess] 2022-03-12 19:45:02 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.9219999999913853 secs.
[2022-03-12 19:45:03,556: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:03,556: WARNING/MainProcess] 2022-03-12 19:45:03 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:04,571: DEBUG/MainProcess] Timer wake-up! Next ETA 0.9849999999860302 secs.
[2022-03-12 19:45:04,571: WARNING/MainProcess] 2022-03-12 19:45:04 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.9849999999860302 secs.
[2022-03-12 19:45:05,577: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:05,577: WARNING/MainProcess] 2022-03-12 19:45:05 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:06,588: DEBUG/MainProcess] Timer wake-up! Next ETA 0.031000000017229468 secs.
[2022-03-12 19:45:06,590: WARNING/MainProcess] 2022-03-12 19:45:06 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.031000000017229468 secs.
[2022-03-12 19:45:06,649: DEBUG/MainProcess] Timer wake-up! Next ETA 0.9379999999946449 secs.
[2022-03-12 19:45:06,650: WARNING/MainProcess] 2022-03-12 19:45:06 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.9379999999946449 secs.
[2022-03-12 19:45:07,605: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:07,606: WARNING/MainProcess] 2022-03-12 19:45:07 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:08,635: DEBUG/MainProcess] Timer wake-up! Next ETA 0.9679999999934807 secs.
[2022-03-12 19:45:08,636: WARNING/MainProcess] 2022-03-12 19:45:08 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.9679999999934807 secs.
[2022-03-12 19:45:09,629: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:09,630: WARNING/MainProcess] 2022-03-12 19:45:09 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:10,643: DEBUG/MainProcess] Timer wake-up! Next ETA 0.985000000015134 secs.
[2022-03-12 19:45:10,643: WARNING/MainProcess] 2022-03-12 19:45:10 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.985000000015134 secs.
[2022-03-12 19:45:11,644: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:11,645: WARNING/MainProcess] 2022-03-12 19:45:11 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:12,660: DEBUG/MainProcess] Timer wake-up! Next ETA 0.9839999999967404 secs.
[2022-03-12 19:45:12,660: WARNING/MainProcess] 2022-03-12 19:45:12 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.9839999999967404 secs.
[2022-03-12 19:45:13,652: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:13,652: WARNING/MainProcess] 2022-03-12 19:45:13 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:14,666: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:14,666: WARNING/MainProcess] 2022-03-12 19:45:14 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:15,687: DEBUG/MainProcess] Timer wake-up! Next ETA 0.9529999999795109 secs.
[2022-03-12 19:45:15,687: WARNING/MainProcess] 2022-03-12 19:45:15 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.9529999999795109 secs.
[2022-03-12 19:45:16,658: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:16,667: WARNING/MainProcess] 2022-03-12 19:45:16 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:17,678: DEBUG/MainProcess] Timer wake-up! Next ETA 0.015000000013969839 secs.
[2022-03-12 19:45:17,679: WARNING/MainProcess] 2022-03-12 19:45:17 [celery.worker] DEBUG: Timer wake-up! Next ETA 0.015000000013969839 secs.
[2022-03-12 19:45:17,714: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2022-03-12 19:45:17,716: WARNING/MainProcess] 2022-03-12 19:45:17 [celery.worker] DEBUG: Timer wake-up! Next ETA 1.0 secs. ```


Please anyone can help regarding it! thanks in advance
python django scrapy celery
1个回答
0
投票

这是预期的行为。这不是一个错误。日志只是打印工作人员将要检查是否分配了任何新任务。

解决方案： 不要将日志级别保留为调试。

celery -tavant_webapp 工作者 --loglevel=info

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.