在python中运行子进程时泄漏

Question

[运行时，我漏了水。我不确定这是怎么回事。我猜管道没有关闭或可能正在发生其他事情。

def deactivateMetadataDevice(input_dmd_lun_wwn):
    #print('pvremove /dev/mapper/' + input_dmd_lun_wwn)
    status_cmd = False
    ps = subprocess.Popen('/sbin/pvremove /dev/mapper/' + input_dmd_lun_wwn, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    for line in iter(ps.stdout.readline, ''):
        print line
        if re.search('wiped', line):
           status_cmd = True
        else:
           # Cleaning metadata and removing from LVM if ok then return true
           status_cmd = False
           raise Warning('\t\t PV /dev/mapper/'+ input_dmd_lun_wwn +' belongs to Volume Group')

    return status_cmd

当我运行上面的代码时遇到此问题：

File descriptor 4 (pipe:[323948]) leaked on pvremove invocation. Parent PID 15380: python

Answer 1

问题是您在读取管道中的所有数据之前先返回，并且没有发出等待以获取返回代码并从操作系统pid表中删除该进程的等待。我认为可以做一些调整（我还删除了一些我认为多余的内容）。

def deactivateMetadataDevice(input_dmd_lun_wwn):
    #print('pvremove /dev/mapper/' + input_dmd_lun_wwn)
    status_cmd = False
    ps = subprocess.Popen('/sbin/pvremove /dev/mapper/' + input_dmd_lun_wwn, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    for line in ps.stdout:
        print line
        if 'wiped' in line:
           status_cmd = True
    ps.wait()
    # need to handle ps.returncode != 0
    if status_cmd is False:
       # Cleaning metadata and removing from LVM if ok then return true
       raise Warning('\t\t PV /dev/mapper/'+ input_dmd_lun_wwn +' belongs to Volume Group')
    return status_cmd # likely not needed because you are using exceptions for errors

Answer 2

我知道这已经很老了，但我遇到了类似的问题，并认为我的回答可能会帮助其他在寻求帮助时绊脚石的人。

subprocess.Popen的默认值为close_fds = False（ETA：在Python 3.2中，默认值在POSIX上更改为True）。但是lvm（8）手册页指出：

在调用时，lvm要求仅标准文件描述符stdin，stdout和stderr可用。如果找到其他人，他们将被关闭并显示消息发出关于泄漏的警告。

因此，对于像pvremove这样的lvm命令，可以通过在subprocess.Popen调用中设置close_fds = True来避免泄漏。

Answer 3

我最终这样做了。不幸的是，env中的大多数服务器都具有python 2.6。 Check_output随附python 2.7版本。

# check_output in subprocess unfortunately comes by default in 2.7 version
def check_output(*popenargs, **kwargs):
    # Passing all arguments to process
    process = subprocess.Popen(stdout=subprocess.PIPE, *popenargs, **kwargs)
    output, unused_err = process.communicate()
    retcode = process.poll()

    if retcode:
       cmd = kwargs.get("args")
       if cmd is None:
          cmd = popenargs[0]
       error = subprocess.CalledProcessError(retcode, cmd)
       error.output = output
       raise error

    return output

# Function to remove metadata from phisical device
def deactivateMetadataDevice(input_dmd_lun_wwn):
    #print('pvremove /dev/mapper/' + input_dmd_lun_wwn)
    status_cmd = False
    if 'wiped' in check_output(["/sbin/pvremove", "/dev/mapper/" + input_dmd_lun_wwn]):
       status_cmd = True

    return status_cmd