在线程中运行时，Python记录器消息冲突

Question

使用模块threading和logger进行并发文件下载。似乎正在按预期工作。唯一的问题是我每次下载的登录状态行都相互冲突

代码段

import sys, os, re
import threading
import requests
import queue
import logging
import time
import argparse

def download(session, url, filename):
    connection_pool.acquire()

    url = re.sub(r'/+$', '', url)
    filename = re.sub(r'^/+', '', filename)
    localpath = args.basepath + "/" + filename
    os.makedirs(os.path.dirname(localpath), exist_ok=True)
    full_url = url + "/" + filename + "?B"
    starttime = time.time()
    logger.info("Downloading {}".format(full_url))

    response = session.get(full_url, allow_redirects=True, stream=True)
    filesize = response.headers.get('Content-Length')

    with open(localpath, 'wb') as f:
        for chunk in response.iter_content(1024):
            f.write(chunk)

    local_filesize = os.path.getsize(localpath)
    if (int(filesize) != int(local_filesize)):
        logger.error("File sizes do not match remote={} local={}".format(filesize, local_filesize))
        exit(1)
    endtime = time.time()
    elapsedtime = (endtime - starttime)
    bitrate = int(filesize) / elapsedtime
    logger.info("STATUS - file={} size={} time={} rate={} region={}".format(filename, filesize, elapsedtime, int(bitrate), args.region))
    connection_pool.release()

def setup_logger():
    hdlr = logging.StreamHandler()
    hdlr.flush = sys.stdout.flush
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    hdlr.setFormatter(formatter)
    logger.addHandler(hdlr)
    logger.setLevel(logging.DEBUG)

... bunch of other functions ...

输出：

895 2020-05-30 14:29:32,992 - get_files.py - INFO - Downloading https://10.12.134.17/3010A120200528.z
p?B 20-05-30 14:29:28,564 - get_files.py - INFO - STATUS - file=3066A120200528.zip size=9427150 time=8.5233154296875 rate=110604
2020-05-30 14:29:28,565 - get_files.py - INFO - Downloading https://10.12.134.17/3033A120200528.zip?
2020-05-30 14:29:29,610 - get_files.py - INFO - STATUS - file=3038A120200528.zip size=2472569 time=2.0577292442321777 rate=1201
00 2020-05-30 14:29:29,610 - get_files.py - INFO - Downloading https://10.12.134.17/3031A120200528.zi
?B 2020-05-30 14:29:29,836 - get_files.py - INFO - STATUS - file=3034A120200528.zip size=1441683 time=1.5025453567504883 rate=95
493 2020-05-30 14:29:29,836 - get_files.py - INFO - Downloading https://10.12.134.17/3030A120200528.z
p?B 2020-05-30 14:29:30,065 - get_files.py - INFO - STATUS - file=3037A120200528.zip size=3431121 time=2.3450839519500732 rate=14
3112 2020-05-30 14:29:30,065 - get_files.py - INFO - Downloading https://10.12.134.17/3029A120200528.
p?B 20-05-30 14:29:28,564 - get_files.py - INFO - STATUS - file=3066A120200528.zip size=9427150 time=8.5233154296875 rate=110604
2020-05-30 14:29:28,565 - get_files.py - INFO - Downloading https://10.12.134.17/3033A120200528.zip?

如上所述，来自某个线程的记录器消息正在与其他线程冲突。

• 旁注，此文件正在运行amazonlinux：2映像的AWS ECS Fargate v1.4.0容器中运行，并使用python 3.6登录到AWS Cloudwatch日志流中

Answer 1

虽然它对我来说不起作用，但解决方案来自this帖子。在完全删除该处理程序并仅使用默认处理程序之后，不再发生任何行冲突。