Python在日志记录中向基于structlog的格式化程序添加额外的字段

Question

我正在尝试将日志的多余字段添加为键值，同时继续使用标准日志记录库和structlog ProcessorFormatter。

这里是一个例子：

如果使用JsonFormatter，则会将其他字段作为键值添加到日志中。

使用python-json-logger：

formatter = jsonlogger.JsonFormatter()
handler = logging.StreamHandler()
handler.setFormatter(formatter)

logging.basicConfig(
    level=logging.DEBUG,
    handlers=[handler]
)

logger = logging.getLogger('testnameloger1')
logger.debug('testmsg2', extra={'extra2': 2})

控制台：

{"message": "testmsg2", "extra2": 2}

在日志记录中使用基于structlog的格式化程序进行渲染：

shared_processors = [
    structlog.stdlib.add_logger_name,
    structlog.stdlib.add_log_level,
    structlog.processors.TimeStamper(fmt='iso'),
]

formatter = structlog.stdlib.ProcessorFormatter(
    processor=structlog.processors.JSONRenderer(),
    foreign_pre_chain=shared_processors,
)
handler = logging.StreamHandler()
handler.setFormatter(formatter)

logging.basicConfig(
    level=logging.DEBUG,
    handlers=[handler]
)

logger = logging.getLogger('testnameloger1')
logger.debug('testmsg1', extra={'extra1': 1})

控制台：

{"event": "testmsg1", "logger": "testnameloger1", "level": "debug", "timestamp": "2020-06-16T13:33:50.881221Z"}

如何在日志输出中添加额外的字段？

我想要的日志格式如下：

{"event": "testmsg1", "extra1": 1, "logger": "testnameloger1", "level": "debug", "timestamp": "2020-06-16T13:33:50.881221Z"}

谢谢您的见解

Answer 1

这里是一个答案。其他可能会更好。

LOG_RECORD_ATTRIBUTES = {
    'args', 'asctime', 'created', 'exc_info', 'exc_text', 'filename',
    'funcName', 'levelname', 'levelno', 'lineno', 'message', 'module',
    'msecs', 'msg', 'name', 'pathname', 'process', 'processName',
    'relativeCreated', 'stack_info', 'thread', 'threadName',
}

def extract_stdlib_extra(logger, method_name, event_dict):
    """
    Extract the `extra` key-values from the standard logger record
    and populate the `event_dict` with them.
    """
    record_extra = {k: v for k, v in vars(event_dict['_record']).items()
                    if k not in LOG_RECORD_ATTRIBUTES}
    event_dict.update(record_extra)
    return event_dict

processors = [
    structlog.stdlib.add_logger_name,
    structlog.stdlib.add_log_level,
    structlog.processors.TimeStamper(fmt='iso'),
    extract_stdlib_extra,
]

formatter = structlog.stdlib.ProcessorFormatter(
    processor=structlog.processors.JSONRenderer(),
    foreign_pre_chain=processors,
)
handler = logging.StreamHandler()
handler.setFormatter(formatter)

logging.basicConfig(
    level=logging.DEBUG,
    handlers=[handler]
)

logger = logging.getLogger('testnameloger1')
logger.debug('testmsg1', extra={'t': 1, 'z': {'r': 2}, 'k': []})

控制台：

{"event": "testmsg1", "logger": "testnameloger1", "level": "debug", "timestamp": "2020-06-17T08:14:56.465786Z", "t": 1, "z": {"r": 2}, "k": []}