如何异步运行Matplolib服务器端并超时？进程随机挂起

Question

我正在尝试重现 ChatGPT 代码解释器功能，其中法学硕士通过执行代码按需创建图形。

不幸的是，Matplotlib 有 20% 的时间挂起，我还没明白为什么。

我想要实施：

• 对于服务器的其余部分来说是非阻塞的
• 设置超时，以防代码太长而无法执行

我做了第一个实现：

import asyncio
import platform
import psutil


TIMEOUT = 5


async def exec_python(code: str) -> str:
    """Execute Python code.

    Args:
        code (str): Python code to execute.

    Returns:
        dict: A dictionary containing the stdout and the stderr from executing the code.
    """
    code = preprocess_code(code)
    stdout = ""
    stderr = ""
    try:
        stdout, stderr = await run_with_timeout(code, TIMEOUT)
    except asyncio.TimeoutError:
        stderr = "Execution timed out."
    return {"stdout": stdout, "stderr": stderr}


async def run_with_timeout(code: str, timeout: int) -> str:
    proc = await run(code)

    try:
        stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
        return stdout.decode().strip(), stderr.decode().strip()
    except asyncio.TimeoutError:
        kill_process(proc.pid)
        raise


async def run(code: str):
    timeout_command = 'gtimeout' if platform.system() == 'Darwin' else 'timeout'
    sanity_timeout = TIMEOUT + 1
    command = f'{timeout_command} {sanity_timeout} python -c "{code}"'
    return await asyncio.create_subprocess_shell(
        command,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )


def kill_process(pid: int):
    try:
        parent = psutil.Process(pid)
        for child in parent.children(recursive=True):
            child.kill()
        parent.kill()
        print(f"Killing Process {pid} (timed out)")
    except psutil.NoSuchProcess:
        print("Process already killed.")


PLT_OVERRIDE_PREFIX = """
import matplotlib
import asyncio
matplotlib.use('Agg') # non-interactive backend
import matplotlib.pyplot as plt
import io
import base64

def custom_show():
    buf = io.BytesIO()
    plt.gcf().savefig(buf, format='png')
    buf.seek(0)
    image_base64 = base64.b64encode(buf.getvalue()).decode('utf-8')
    print('[BASE_64_IMG]', image_base64)
    plt.clf()

plt.show = custom_show
"""


def preprocess_code(code: str) -> str:
    override_prefix = ""
    code_lines = code.strip().split("\n")
    if not code_lines:
        return code  # Return original code if it's empty
    if "import matplotlib.pyplot as plt" in code:
        override_prefix = PLT_OVERRIDE_PREFIX + "\n"
        code_lines = [
            line for line in code_lines if line != "import matplotlib.pyplot as plt"
        ]

    last_line = code_lines[-1]
    # Check if the last line is already a print statement
    if last_line.strip().startswith("print"):
        return "\n".join(code_lines)

    try:
        compile(last_line, "<string>", "eval")
        # If it's a valid expression, wrap it with print
        code_lines[-1] = f"print({last_line})"
    except SyntaxError:
        # If it's not an expression, check if it's an assignment
        if "=" in last_line:
            variable_name = last_line.split("=")[0].strip()
            code_lines.append(f"print({variable_name})")

    return override_prefix + "\n".join(code_lines)

我已经尝试过但没有成功：

• ipython 而不是 python
• 使用线程而不是进程
• 将图像保存在磁盘上而不是缓冲区上

最奇怪的是我无法使用上面的代码重现该错误。它在产品和我的机器上经常看到错误。

Answer 1

经过几个小时的调试，我终于修复了它。

更换

plt.clf()

与

plt.close('全部')

第一个仅仅清除了数字。