无法使用 Docker 和 xvfb 在 AWS Lambda 上运行 Playwright

Question

几天来，我一直在尝试使用 xvfb 在 AWS Lambda 上运行 Playwright。我能够构建一些在本地工作的 Docker 镜像变体，但是当推送到 AWS 时总是遇到错误。

Docker文件：

FROM mcr.microsoft.com/playwright/python:v1.24.0-focal 
COPY index.py /
COPY start.sh /
RUN apt install xvfb
RUN chmod 755 /start.sh
CMD ["/bin/bash", "./start.sh"]

开始.sh

#!/bin/bash
xvfb-run --auto-servernum --server-num 1 --server-args "-screen 0, 1920x1080x24" python index.py

索引.py

from playwright.sync_api import sync_playwright

def lambda_handler(event, context):
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=False)
        page = browser.new_page()
        page.set_default_timeout(0)
        page.goto("http://example.com")
        title = page.title()
        browser.close()
        return title

看起来由于某些与权限相关的原因，浏览器无法启动 - 任何可能导致它的想法以及如何解决它？

2022-08-30T09:50:28.053+02:00   Traceback (most recent call last):
2022-08-30T09:50:28.053+02:00   File "index.py", line 8, in lambda_handler
2022-08-30T09:50:28.053+02:00   page = browser.new_page()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/sync_api/_generated.py", line 11188, in new_page
2022-08-30T09:50:28.053+02:00   self._sync(
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_sync_base.py", line 89, in _sync
2022-08-30T09:50:28.053+02:00   return task.result()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_browser.py", line 174, in new_page
2022-08-30T09:50:28.053+02:00   page = await context.new_page()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_browser_context.py", line 230, in new_page
2022-08-30T09:50:28.053+02:00   return from_channel(await self._channel.send("newPage"))
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_connection.py", line 43, in send
2022-08-30T09:50:28.053+02:00   return await self._connection.wrap_api_call(
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_connection.py", line 369, in wrap_api_call
2022-08-30T09:50:28.053+02:00   return await cb()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_connection.py", line 78, in inner_send
2022-08-30T09:50:28.053+02:00   result = next(iter(done)).result()
2022-08-30T09:50:28.053+02:00   playwright._impl._api_types.Error: Browser closed.
2022-08-30T09:50:28.053+02:00   ==================== Browser output: ====================
2022-08-30T09:50:28.053+02:00    <launching> /ms-playwright/chromium-1015/chrome-linux/chrome --disable-field-trial-config --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --no-sandbox --user-data-dir=/tmp/playwright_chromiumdev_profile-gHIdYn --remote-debugging-pipe --no-startup-window
<launching> /ms-playwright/chromium-1015/chrome-linux/chrome --disable-field-trial-config --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --no-sandbox --user-data-dir=/tmp/playwright_chromiumdev_profile-gHIdYn --remote-debugging-pipe --no-startup-window
2022-08-30T09:50:28.053+02:00   <launched> pid=129
2022-08-30T09:50:28.053+02:00
[pid=129][err] [0830/075021.572233:WARNING:crashpad_client_linux.cc(362)] prctl: Operation not permitted (1)
[pid=129][err] [0830/075021.572233:WARNING:crashpad_client_linux.cc(362)] prctl: Operation not permitted (1)
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00
[pid=129][err] [129:147:0830/075023.537911:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[pid=129][err] [129:147:0830/075023.537911:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:147:0830/075023.538017:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075023.774216:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075023.774254:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.183157:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.183204:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.449390:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.453692:ERROR:network_service_instance_impl.cc(461)] Network service crashed, restarting service.
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.510964:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.511033:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.511061:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.511109:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.512161:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.512820:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.570748:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.632383:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.671409:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.674327:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.854785:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.854822:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075025.701411:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075025.712194:ERROR:network_service_instance_impl.cc(461)] Network service crashed, restarting service.
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075025.793276:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [0830/075025.850673:ERROR:scoped_ptrace_attach.cc(27)] ptrace: Operation not permitted (1)
2022-08-30T09:50:28.053+02:00   [pid=129][err] Received signal 11 SEGV_MAPERR 000000000000
2022-08-30T09:50:28.053+02:00   [pid=129][err] #0 0x55fa8ab9efb9 base::debug::CollectStackTrace()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #1 0x55fa8ab11853 base::debug::StackTrace::StackTrace()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #2 0x55fa8ab9eac1 base::debug::(anonymous namespace)::StackDumpSignalHandler()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #3 0x7f93d4dc4420 (/usr/lib/x86_64-linux-gnu/libpthread-2.31.so+0x1441f)
2022-08-30T09:50:28.053+02:00   [pid=129][err] #4 0x55fa88ae4589 content::WebContentsImpl::SetDeviceEmulationSize()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #5 0x55fa8864714b content::protocol::EmulationHandler::SetDeviceMetricsOverride()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #6 0x55fa886476e8 content::protocol::EmulationHandler::SetDeviceMetricsOverride()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #7 0x55fa884a52af content::protocol::Emulation::DomainDispatcherImpl::setDeviceMetricsOverride()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #8 0x55fa89b0c0c3 v8_crdtp::UberDispatcher::DispatchResult::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #9 0x55fa886286fd content::DevToolsSession::HandleCommandInternal()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #10 0x55fa8862866b content::DevToolsSession::HandleCommand()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #11 0x55fa8862ac6e base::internal::Invoker<>::RunOnce()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #12 0x55fa8d741a18 ChromeDevToolsSession::HandleCommand()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #13 0x55fa8d740048 ChromeDevToolsManagerDelegate::HandleCommand()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #14 0x55fa8862858e content::DevToolsSession::DispatchProtocolMessageInternal()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #15 0x55fa8862823f content::DevToolsSession::DispatchProtocolMessage()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #16 0x55fa88624162 base::internal::FunctorTraits<>::Invoke<>()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #17 0x55fa8ab5b010 base::TaskAnnotator::RunTaskImpl()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #18 0x55fa8ab6ebfc base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWorkImpl()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #19 0x55fa8ab6e7ad base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #20 0x55fa8ab6f172 base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #21 0x55fa8ab1834b base::(anonymous namespace)::WorkSourceDispatch()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #22 0x7f93d4c7917d g_main_context_dispatch
2022-08-30T09:50:28.053+02:00   [pid=129][err] #23 0x7f93d4c79400 (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.6400.6+0x523ff)
2022-08-30T09:50:28.053+02:00   [pid=129][err] #24 0x7f93d4c794a3 g_main_context_iteration
2022-08-30T09:50:28.053+02:00   [pid=129][err] #25 0x55fa8ab18177 base::MessagePumpGlib::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #26 0x55fa8ab6f3e3 base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #27 0x55fa8ab37aed base::RunLoop::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #28 0x55fa8859dcad content::BrowserMainLoop::RunMainMessageLoop()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #29 0x55fa8859f4f2 content::BrowserMainRunnerImpl::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #30 0x55fa8859b7dc content::BrowserMain()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #31 0x55fa8a67ab88 content::RunBrowserProcessMain()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #32 0x55fa8a67c0da content::ContentMainRunnerImpl::RunBrowser()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #33 0x55fa8a67bc70 content::ContentMainRunnerImpl::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #34 0x55fa8a678e1e content::RunContentProcess()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #35 0x55fa8a6797de content::ContentMain()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #36 0x55fa86f8f227 ChromeMain
2022-08-30T09:50:28.053+02:00   [pid=129][err] #37 0x7f93d3eef083 __libc_start_main
2022-08-30T09:50:28.053+02:00   [pid=129][err] #38 0x55fa86f8f02a _start
2022-08-30T09:50:28.053+02:00   [pid=129][err] r8: aaaaaaaaaaaaaaaa r9: 00007fff828b2460 r10: 0000000000000003 r11: 00000000000002d0
2022-08-30T09:50:28.053+02:00   [pid=129][err] r12: 00007fff828b2730 r13: 00007fff828b2460 r14: 0000000000000000 r15: 00000000000002d0
2022-08-30T09:50:28.053+02:00   [pid=129][err] di: 00003ac400a8e070 si: 00007fff828b2460 bp: 00007fff828b2280 bx: 00003ac40108cc00
2022-08-30T09:50:28.053+02:00   [pid=129][err] dx: aaaaaaaaaaaaaaaa ax: 0000000000000000 cx: 000055fa90552dd8 sp: 00007fff828b2230
2022-08-30T09:50:28.053+02:00   [pid=129][err] ip: 000055fa88ae4589 efl: 0000000000010206 cgf: 002b000000000033 erf: 0000000000000004
2022-08-30T09:50:28.053+02:00   [pid=129][err] trp: 000000000000000e msk: 0000000000000000 cr2: 0000000000000000
2022-08-30T09:50:28.053+02:00   [pid=129][err] [end of stack trace] 

Answer 1

这是显示 Playwright lambda 执行的屏幕截图：

我使用 AWS SAM 在本地测试您的 lambda

sam build && sam local invoke PlaywrightFunction

template.yaml文件包含以下代码：

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  python3.9
  Sample SAM Template for sam-app
Globals:
  Function:
    Timeout: 30
    MemorySize: 128

Resources:
  PlaywrightFunction:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      Architectures:
        - x86_64
      Events:
        Playwright:
          Type: Api
          Properties:
            Path: /hello
            Method: get
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: ./my_playwright_runner
      DockerTag: python3.9-v1

我创建了

my_playwright_runner

文件夹并将您的 Dockerfile 更改为使用多阶段构建并在 Playwright 图像中安装 AWS lambda 运行时接口：

# Define function directory
ARG FUNCTION_DIR="/function"

FROM python:buster as build-image

# Install aws-lambda-cpp build dependencies
RUN apt-get update && \
  apt-get install -y \
  g++ \
  make \
  cmake \
  unzip \
  libcurl4-openssl-dev

# Include global arg in this stage of the build
ARG FUNCTION_DIR
# Create function directory
RUN mkdir -p ${FUNCTION_DIR}

# Copy function code
COPY app/* ${FUNCTION_DIR}

# Install the runtime interface client
RUN pip install \
        --target ${FUNCTION_DIR} \
        awslambdaric

# Multi-stage build: grab a fresh copy of the base image
FROM mcr.microsoft.com/playwright/python:v1.27.0-focal 
ARG FUNCTION_DIR

RUN pip install \
        --target ${FUNCTION_DIR} \
        awslambdaric

COPY index.py /
COPY start.sh /
RUN apt-get update && apt-get install xvfb -y

# Include global arg in this stage of the build

# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}


RUN chmod 755 /start.sh

# Copy in the build image dependencies
COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}

ENTRYPOINT ["/bin/bash", "/start.sh"]

# ENTRYPOINT [ "/usr/local/bin/python", "-m", "awslambdaric" ]
CMD [ "app.lambda_handler" ]

我修改了你的入口点脚本：

#!/bin/sh
if [ -z "${AWS_LAMBDA_RUNTIME_API}" ]; then
  exec xvfb-run --auto-servernum --server-num 1 --server-args "-screen 0, 1920x1080x24"  /usr/local/bin/aws-lambda-rie python -m awslambdaric $@
else
  exec xvfb-run --auto-servernum --server-num 1 --server-args "-screen 0, 1920x1080x24"  python -m awslambdaric $@
fi

我还重新使用了您的一些代码来创建 lambda 处理函数：

import json
from playwright.sync_api import sync_playwright

def lambda_handler(event, context):
    """Lambda handler"""
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=False)
        page = browser.new_page()
        page.set_default_timeout(0)
        page.goto("http://www.google.com")
        title = page.title()
        print(f"Page title: {title}")
        browser.close()
    return {
        "statusCode": 200,
        "body": json.dumps(
            {
                "page title": title,
            }
        )
    }

我还建议使用 x11vnc 将 VNC 连接到虚拟帧缓冲区，否则您可以在没有

xvfb-run

的情况下以无头模式运行 Playwright。我没有尝试部署 CloudFormation 堆栈，但我假设您还需要创建一个安全组，其中包含允许端口 5900 上的入站流量用于 VNC 连接的规则。