这里至少是您第一个问题的部分答案。您会在每个print
上获得这些空行,因为Python 2.x中的print
语句可能会调用stdout.write()
twice
更新:向下滚动至编辑部分(4),即可使用几乎完全可用的版本。还删除了EDIT SECTION(1),因为该帖子过长,否则可能是最没有帮助的部分。原来在编辑部分(1)中的链接在下面。
How do I duplicate sys.stdout to a log file in python?
这里是长时间的搜索者,但第一次提出问题。
EXPLANATION:
我需要使打印重定向到日志,因为我正在通过使用打印语句记录消息的系统调用来合并C代码。也有很多我的同事编写的旧Python代码被调用,它们也使用打印语句进行调试。
最终,我希望它能够为更新后的代码处理logging.info('message'),而且还能够通过内置的日志记录模块重定向打印语句,以获取我无法更改或仅更改的代码还没有更新。
下面是我想出的一些示例代码,以简洁地演示我的问题。
问题:
我的代码:
编辑:最初是setuplog.Tee。init包含if os.path.exists(LOGNAME): os.remove(LOGNAME)
。它已被删除并放入base.py。
setuplog.py:
#!/usr/bin/python
import sys
import os
import logging
import logging.config
LOGNAME = 'log.txt'
CONFIG = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'simple': {
'format': '%(module)s:%(thread)d: %(message)s'
},
},
'handlers': {
'console': {
'level': 'NOTSET',
'class': 'logging.StreamHandler',
'formatter': 'simple'
},
'file': {
'level': 'NOTSET',
'class': 'logging.FileHandler',
'formatter': 'simple',
'filename': LOGNAME
},
},
'root': {
'level': 'NOTSET',
'handlers': ['console', 'file']
},
}
class Tee(object):
def __init__(self):
logging.config.dictConfig(CONFIG)
self.logger = logging.getLogger(__name__)
self.stdout = sys.stdout
sys.stdout = self
def __del__(self):
sys.stdout = self.stdout
def write(self, data):
self.logger.info(data)
base.py:
#!/usr/bin/python
import sys
import os
import logging
# My modules.
import setuplog
#import aux2
LOGNAME = 'log.txt'
if os.path.exists(LOGNAME):
os.remove(LOGNAME)
not_sure_what_to_call_this = setuplog.Tee()
print '1 base'
logging.info('2 base')
print '3 base'
os.system('./aux1.py')
logging.info('4 base')
#aux2.aux2Function()
#logging.info('5 base')
aux1.py:
#!/usr/bin/python
import sys
import os
import logging
import setuplog
not_sure_what_to_call_this = setuplog.Tee()
def main():
print '1 aux1'
logging.info('2 aux1')
print '3 aux1'
logging.info('4 aux1')
if __name__ == '__main__':
main()
aux2.py:
#!/usr/bin/python
import sys
import os
import logging
import setuplog
not_sure_what_to_call_this = setuplog.Tee()
def aux2Function():
print '1 aux2'
logging.info('2 aux2')
print '3 aux2'
然后我从外壳程序“ ./base.py”运行并产生以下输出(在控制台和log.txt中:)]
setuplog:139833296844608: 1 aux1 setuplog:139833296844608: aux1:139833296844608: 2 aux1 setuplog:139833296844608: 3 aux1 setuplog:139833296844608: aux1:139833296844608: 4 aux1
如您所见,将重复打印生成的条目(问题1)。另外,我需要提出一个更好的约定来显示模块名称(问题2)。
支持问题3的信息
从base.py,如果我取消注释“ import aux2”,“ aux2.aux2Function()”和“ logging.info('5 base')”,则这是新的输出(直接从控制台输入,因为这是仅将Python错误转到):
base:140425995155264: 2 base setuplog:140360687101760: 1 aux1 setuplog:140360687101760: aux1:140360687101760: 2 aux1 setuplog:140360687101760: 3 aux1 setuplog:140360687101760: aux1:140360687101760: 4 aux1 base:140425995155264: 4 base aux2:140425995155264: 2 aux2 base:140425995155264: 5 base Exception AttributeError: "'NoneType' object has no attribute 'stdout'" in <bound method Tee.__del__ of <setuplog.Tee object at 0x7fb772f58f10>> ignored
编辑部分(2):
我一直在玩,并且有这种作品。这里是(再次)示例代码的更新版本。
之所以“有效”的原因是:
setuplog.py:
#!/usr/bin/python
import sys
import os
import logging
import logging.config
def startLog(name):
logname = 'log.txt'
config = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'simple': {
'format': '%(name)s:%(module)s:%(thread)s: %(message)s'
},
},
'handlers': {
'console': {
'level': 'NOTSET',
'class': 'logging.StreamHandler',
'formatter': 'simple'
},
'file': {
'level': 'NOTSET',
'class': 'logging.FileHandler',
'formatter': 'simple',
'filename': logname
},
},
'root': {
'level': 'NOTSET',
'handlers': ['console', 'file'],
},
}
logging.config.dictConfig(config)
return logging.getLogger(name)
class Tee():
def __init__(self, logger):
self.stdout = sys.stdout
self.data = ''
self.logger = logger
sys.stdout = self
def __del__(self):
try:
sys.stdout = self.stdout
except AttributeError:
pass
def write(self, data):
self.data += data
self.data = str(self.data)
if '\x0a' in self.data or '\x0d' in self.data:
self.data = self.data.rstrip('\x0a\x0d')
self.logger.info(self.data)
self.data = ''
base.py:
#!/usr/bin/python
import sys
import os
import logging
# My modules.
import setuplog
import aux2
LOGNAME = 'log.txt'
if os.path.exists(LOGNAME):
os.remove(LOGNAME)
logger = setuplog.startLog('base')
setuplog.Tee(logger)
print '1 base'
logger.info('2 base')
print '3 base'
os.system('./aux1.py')
logger.info('4 base')
aux2.aux2Function()
logger.info('5 base')
aux1.py:
#!/usr/bin/python
import sys
import os
import logging
import setuplog
def main():
logger = setuplog.startLog('aux1')
setuplog.Tee(logger)
print '1 aux1'
logger.info('2 aux1')
print '3 aux1'
logger.info('4 aux1')
if __name__ == '__main__':
main()
aux2.py:
#!/usr/bin/python
import sys
import os
import logging
import setuplog
def aux2Function():
logger = setuplog.startLog('aux2')
setuplog.Tee(logger)
print '1 aux2'
logger.info('2 aux2')
print '3 aux2'
和输出:
base:setuplog:139712687740736: 1 base
base:base:139712687740736: 2 base
base:setuplog:139712687740736: 3 base
aux1:setuplog:140408798721856: 1 aux1
aux1:aux1:140408798721856: 2 aux1
aux1:setuplog:140408798721856: 3 aux1
aux1:aux1:140408798721856: 4 aux1
base:base:139712687740736: 4 base
aux2:setuplog:139712687740736: 1 aux2
aux2:aux2:139712687740736: 2 aux2
aux2:setuplog:139712687740736: 3 aux2
编辑部分(3):
感谢对reddit(http://www.reddit.com/r/learnpython/comments/1kaduo/python_logging_module_for_print_statements/cbn2lef)的精彩答复,我能够针对AttributeError进行开发。我没有使用异常,而是将类转换为单例。
这里是更新的Tee类的代码:
class Tee(object): _instance = None def __init__(self, logger): self.stdout = sys.stdout self.data = '' self.logger = logger sys.stdout = self def __new__(cls, *args, **kwargs): if not cls._instance: cls._instance = super(Tee, cls).__new__(cls, *args, **kwargs) return cls._instance def __del__(self): sys.stdout = self.stdout def write(self, data): self.data += data self.data = str(self.data) if '\x0a' in self.data or '\x0d' in self.data: self.data = self.data.rstrip('\x0a\x0d') self.logger.info(self.data) self.data = ''
编辑部分(4):
这几乎完全有效!它足以让我实现它。现在唯一的问题是使输出格式更有用。例如,对于所有重新路由的打印语句,%(filename)s是setuplog.py。如果%(filename)s是源自print语句的文件,那将更加有益。有什么想法吗?
而且,我不得不放弃字典方法。我可以完成所有工作的唯一方法是通过Python代码设置记录器。
最后一点,查看aux3.py。如果使用os.system而不是子进程,则日志记录的顺序会混乱。有谁知道解决方法仍然可以使用os.system并获得正确的顺序(因此我不必将每个最后的os.system都更改为subprocess.Popen)?
setuplog.py(您可以忽略功能startDictLog和startFileLog,因为它们不起作用。但是,startCodeLog可以!):]]
#!/usr/bin/python import sys import os import logging import logging.config def startLog(name, propagate): '''The only point of this function was to enable me to quickly and easily switch how I wanted to configure the logging. So far, I have only been able to get the last way working (startCodeLog).''' #return startDictLog(name) #return startFileLog(name) return startCodeLog(name, propagate) def startDictLog(name): '''Configure logging usinga dictionary.''' LOGNAME = 'loop.log' DEBUGNAME = 'debug.log' config = { 'version': 1, 'disable_existing_loggers': True, 'formatters': { 'bare': { # Added the BARE to distinguish between normal prints # and those that get rerouted. In use, I would format it # such that only the message is printed. 'format': 'BARE: %(message)s' }, 'simple': { 'format': '%(module)s-%(name)s: %(message)s' }, 'time': { 'format': '%(asctime)s-%(filename)s-%(module)s-%(name)s: %(message)s', 'datefmt': '%H:%M:%S' }, }, 'handlers': { 'console': { 'level': 'NOTSET', 'class': 'logging.StreamHandler', 'formatter': 'bare' }, 'normal': { 'level': 'INFO', 'class': 'logging.FileHandler', 'formatter': 'simple', 'filename': LOGNAME }, 'debug': { 'level': 'NOTSET', 'class': 'logging.FileHandler', 'formatter': 'time', 'filename': DEBUGNAME }, }, 'root': { 'level': 'NOTSET', 'handlers': ['console', 'normal', 'debug'], }, } logging.config.dictConfig(config) return logging.getLogger(name) def startFileLog(name): '''Configure logging using a configuration file.''' CONFIGFILE = 'logging.conf' logging.config.fileConfig(CONFIGFILE) return logging.getLogger(name) def startCodeLog(name, propagate): '''Configure logging using this code.''' LOGFILE = 'loop.log' DEBUGFILE = 'debug.log' _logger = logging.getLogger(name) _logger.setLevel(logging.NOTSET) if propagate in [False, 'n', 'no', 0]: _logger.propagate = False _console = logging.StreamHandler() _normal = logging.FileHandler(LOGFILE) _debug = logging.FileHandler(DEBUGFILE) _bare = logging.Formatter('BARE: %(message)s') _simple = logging.Formatter('%(module)s-%(name)s: %(message)s') _time = logging.Formatter('%(asctime)s-%(module)s-%(name)s: %(message)s', datefmt = '%H:%M:%S') # I added _complex only here, to the working way of setting up the configuration, # in hopes that this data may help someone figure out how to get the printout # to be more useful. For example, it's not helpful that the filename is # setuplog.py for every print statement. It would be more beneficial to somehow # get the filename of where the print statement originated. _complex = logging.Formatter('%(filename)s-%(funcName)s-%(name)s: %(message)s') # Normally this is set to _bare to imitate the output of the old version of the # scripts I am updating, but for our purposes, _complex is more convenient. _console.setLevel(logging.NOTSET) _console.setFormatter(_complex) # This imitates the format of the logs from versions before I applied this update. _normal.setLevel(logging.INFO) _normal.setFormatter(_bare) # This is a new log file I created to help me debug other aspects of the scipts # I am updating. _debug.setLevel(logging.DEBUG) _debug.setFormatter(_time) _logger.addHandler(_console) _logger.addHandler(_normal) _logger.addHandler(_debug) return _logger class Tee(object): '''Creates a singleton class that tees print statements to the handlers above.''' _instance = None def __init__(self, logger): self.stdout = sys.stdout self.logger = logger sys.stdout = self self._buf = '' # Part of old method in the write function. #self.data = '' def __new__(cls, *args, **kwargs): '''This is the singleton implementation. This avoids errors with multiple instances trying to access the same standard output.''' if not cls._instance: cls._instance = super(Tee, cls).__new__(cls, *args, **kwargs) return cls._instance def write(self, data): # This method doesn't work with how I had to implement subprocess. #self.data = data.rstrip('\r\n') #if self.data: # self.logger.info(self.data) # Also doesn't seem to work with my subprocess implementation. #self.data += data #self.data = str(self.data) #if '\x0a' in self.data or '\x0d' in self.data: # self.data = self.data.rstrip('\x0a\x0d') # self.logger.info(self.data) # self.data = '' # Only way I could get it working with my subprocess implementation. self._buf = ''.join([self._buf, data]) while '\n' in self._buf: line, _, tail = self._buf.partition('\n') if line.strip(): self.logger.info(line) self._buf = tail
base.py:
#!/usr/bin/python import sys import os import logging import subprocess # My modules below. import setuplog import aux2 # It is assumed that this script will be executed via the command line, # and each time we run it, we want new log files. LOGNAME = 'loop.log' DEBUGNAME ='debug.log' if os.path.exists(LOGNAME): os.remove(LOGNAME) if os.path.exists(DEBUGNAME): os.remove(DEBUGNAME) # It seems more convenient to store the logging configuration elsewhere, # hence my module setuplog. logger = setuplog.startLog('', True) logger = setuplog.startLog('base', False) # This initializes sys.stdout being redirected through logging. Can anyone # explain how calling this class achieves this? I am a bit fuzzy on my # understanding here. setuplog.Tee(logger) # Test to see how it works in this module. print '1 base' logger.info('2 base') print '3 base' # Below shows how to get logging to work with scripts that can # not be modified. In my case, I have C code that I don't want to modify, # but I still need to log it's output. # I will have to go through the old code and change all os.system calls # to instead utilize subprocess. Too bad because this will take some time # and be "busy work", but at least it works. What can ya do, os.system is # depreciated anyway. The positive side is that only the root application # needs to change its system calls to use subprocess. The scipts that it # calls upon can remain untouched. aux1_py_path = '"%s"' % os.path.join(os.path.dirname(__file__), 'aux1.py') #os.system(aux1_py_path) # Example to show how os.system doesn't work. print 'aux1_py_path:', aux1_py_path sys_call = subprocess.Popen(aux1_py_path, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) sys_stdout, sys_stderr = sys_call.communicate() print sys_stdout print sys_stderr # This is to ensure that the order of the logging events is correct. With an # old method I used (not with logging, simply redirecting stdout into an # opened file), the order would often get messed up. logger.info('4 base') # This example is to show how to get logging to work with Python scripts I am # willing to modify, at least partially. See aux2.py. It is simply a print # statement, followed by a logging statement, followed by another print # statement. They all output properly with this method. aux2.aux2Function() # Again to ensure the order of events is correct. logger.info('5 base')
aux1.py:
#!/usr/bin/python import logging import subprocess import os def main(): '''We expect the print statements to go through, as they are being sent to logging. However, these logging statements do nothing as no logger has been instantiated. This is the behavior we should expect, as this script mimics a script that we would not modify, so it would not have logging calls anyway.''' print '1 aux1' logging.info('2 aux1') print '3 aux1' logging.info('4 aux1') # Here, neither option works unless the root of all these calls # was made with subprocess and in the script that called the # tee class. If both conditions are met, subprocess works as it # should. However, os.system returns the print out from the call # out of order. aux3_py_path = '"%s"' % os.path.join(os.path.dirname(__file__), 'aux3.py') #os.system(aux3_py_path) sys_call = subprocess.Popen(aux3_py_path, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) sys_stdout, sys_stderr = sys_call.communicate() print sys_stdout print sys_stderr if __name__ == '__main__': main()
aux2.py:
#!/usr/bin/python import sys import os import logging import setuplog def aux2Function(): # The following two lines are not necessary for most functionality. # However, if the following two lines are uncommented, then %(name)s # in the format descriptor of logging will correctly be 'aux2' rather # than base. Both lines are required for this to work. logger = setuplog.startLog('aux2', False) setuplog.Tee(logger) print '1 aux2' logging.info('2 aux2') print '3 aux2'
aux3.py:
#!/usr/bin/python import logging def main(): '''See __doc__ for aux1.py. Again, we don't expect the logging.info to work, but that's okay because theoretically, this is some script we can't modify that simply generates output with print or print like functions.''' print '1 aux3' logging.info('2 aux3') print '3 aux3' if __name__ == '__main__': main()
logging.conf(无效):
[loggers] keys=root [handlers] keys=console,normal,debug [formatters] keys=bare,simple,time [logger_root] level=NOTSET handlers=console,normal,debug [handler_console] level=NOTSET class=StreamHandler formatter=bare args=(sys.stdout,) [handler_normal] level=INFO class=FileHandler formatter=simple args=('loop.log',) [handler_debug] level=DEBUG class=FileHandler formatter=time args=('debug.log',) [formatter_bare] format=%(message)s [formatter_simple] format=%(module)s-%(name)s: %(message)s [formatter_time] format=%(asctime)s-%(filename)s-%(module)s-%(name)s: %(message)s datefmt=%H:%M:%S
更新:向下滚动至编辑部分(4),即可使用几乎完全可用的版本。还删除了EDIT SECTION(1),因为该帖子过长,否则可能是最没有帮助的部分。 ...
这里至少是您第一个问题的部分答案。您会在每个print
上获得这些空行,因为Python 2.x中的print
语句可能会调用stdout.write()
twice
此外,如果将INFO
级消息发送到设置为level
为'NOTSET'
的记录器,则默认情况下,根据记录器文档,消息也会在sys.stderr
上回显,这就是为什么即使Tee
有效,也可以查看控制台输出。我看不到重复的日志条目。
为了防止出现空白行,请尝试使用此Tee
类定义。请注意write()
方法的修改(已更新为单例以匹配您的“编辑部分(3)”):
class Tee(object): _instance = None def __init__(self, logger): self.stdout = sys.stdout self.logger = logger sys.stdout = self def __new__(cls, *args, **kwargs): if not cls._instance: cls._instance = super(Tee, cls).__new__(cls, *args, **kwargs) return cls._instance def __del__(self): sys.stdout = self.stdout def write(self, data): data = data.rstrip('\r\n') if data: # anything left? self.logger.info(data)
有了此更新以及您所有其他的更新,对于我来说似乎很有效(包括未注释的
aux2
内容。从您的角度来看是否还有其他问题?如果是这样,那么您最初的长期问题就变得更加如此了。并且应该彻底清除,只保留最新的代码,并专注于所有剩余的问题。
这里至少是您第一个问题的部分答案。您会在每个print
上获得这些空行,因为Python 2.x中的print
语句可能会调用stdout.write()
twice