使用libvirt直通GPU时出现vfio_dma_map错误

Question

这是qemu日志

2016-08-28T02:52:05.699498Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio_dma_map(0x7f67ea3b3e10, 0x100000, 0xbff00000, 0x7f6651d00000) = -12 (Cannot allocate memory)
2016-08-28T02:52:05.847310Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio_dma_map(0x7f67ea3b3e10, 0x100000000, 0xae400000, 0x7f6711c00000) = -12 (Cannot allocate memory)
2016-08-28T02:52:05.847323Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: memory listener initialization failed for container
2016-08-28T02:52:05.847330Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to setup container for group 1
2016-08-28T02:52:05.847440Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 1

dmesg的：

[  356.004278] vfio_pin_pages: RLIMIT_MEMLOCK (7218397184) exceeded
[  356.004446] vfio_pin_pages: RLIMIT_MEMLOCK (7218397184) exceeded
[  356.371582] vfio_pin_pages: RLIMIT_MEMLOCK (7218397184) exceeded
[  356.371698] vfio_pin_pages: RLIMIT_MEMLOCK (7218397184) exceeded

libvirt版本：

Compiled against library: libvirt 1.2.9
Using library: libvirt 1.2.9
Using API: QEMU 1.2.9
Running hypervisor: QEMU 2.1.2

尝试设置ulimit -l限制没用

root@debian:/home/deb# ulimit -l
unlimited

Answer 1

在以非root用户身份运行VFIO时遇到同样的问题，因为这里似乎没有解决方案。我通过在/etc/security/limits.d/99-memlock.conf中为用户“user”添加1G内存“1048576”来增加锁定内存的限制（无论如何我的系统）。

   user hard memlock 1048576
   user soft memlock 1048576

Answer 2

你也应该发布你的libvirt的xml，一旦我遇到这个问题，我修改了我的libvirt的xml以使用一个条目而不是 QEMU-ARGS。

Answer 3

XML：

    <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
    <name>host2</name>
    <uuid>d35811f6-7865-4144-b78b-a0366ca30888</uuid>
    <metadata>
      <vmtemplate name="Custom" icon="windows7.png" os="windows7"/>
    </metadata>
    <memory unit='KiB'>6291456</memory>
    <currentMemory unit='KiB'>6291456</currentMemory>
    <memoryBacking>
      <nosharepages/>
      <locked/>
    </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='1'/>
    <vcpupin vcpu='2' cpuset='2'/>
    <vcpupin vcpu='3' cpuset='3'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.1'>hvm</type>
    <boot dev='network'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='4' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='direct'>
      <mac address='52:54:00:33:ac:18'/>
      <source dev='eth0' mode='bridge'/>
      <model type='e1000'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=01:00.0,x-vga=on'/>
  </qemu:commandline>
</domain>