在对 ACC100 5G BLER 任务运行 BBdev 测试验证时,我们发现我们的程序卡住了
== test: validation
dev:cb:00.0, burst size: 32, num ops: 64, op type: RTE_BBDEV_OP_LDPC_DEC
首先我们按照 pf-bb-config 中的步骤操作,我们遇到以下问题:
Baseband devices using DPDK-compatible driver
=============================================
0000:ca:00.0 'Device 0d5c' drv=vfio-pci unused=
0000:cb:00.0 'Device 0d5d' drv=vfio-pci unused=
0000:cb:00.1 'Device 0d5d' drv=vfio-pci unused=
zq40@tc319-srv2:/opt/dpdk-stable-21.11.2-older/app/test-bbdev$ dmesg | grep "DMAR: IOMMU"
[ 0.457583] DMAR: IOMMU enabled
[ 0.457977] DMAR: IOMMU enabled
zq40@tc319-srv2:/opt/dpdk-stable-21.11.2/app/test-bbdev/test_vectors$ sudo dpdk-test-bbdev -c F0 -a cb:00.0 --vfio-vf-token=00112233-4455-6677-8899-aabbccddeeff -- -c validation -v ldpc_dec_v2342_drop.data
[sudo] password for zq40:
EAL: Detected 56 lcore(s)
EAL: Detected 2 NUMA nodes
Option -w, --pci-whitelist is deprecated, use -a, --allow option instead
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: using IOMMU type 1 (Type 1)
EAL: Probe PCI driver: intel_acc100_vf (8086:d5d) device: 0000:cb:00.0 (socket 1)
WARNING: Num of operations was not provided or was set 0. Set to default (64)
WARNING: Burst size was not provided or was set 0. Set to default (32)
WARNING: Num of lcores was not provided or was set 0. Set to value from RTE config (4)
===========================================================
Starting Test Suite : BBdev Validation Tests
Test vector file = ldpc_dec_v2342_drop.data
Device 0 supports 0 < N <= 0 queues, not 0
rte_bbdev_setup_queues(0, 0, 1) ret -22
Adding active bbdev cb:00.0 skipped
No suitable devices found!
+ Test suite setup BBdev Validation Tests skipped!
+ ------------------------------------------------------- +
我们按照 pf_bb_config README 并按照教程的指示配置了 ACC100,但在尝试设置队列时该功能似乎失败了。
我们尝试的另一件事是使用 igb_uio 来驱动我们的 acc100 卡,
sudo dpdk-devbind.py -b igb_uio ca:00.0
zq40@tc319-srv2:/opt/dpdk-stable-20.11.5/app/test-bbdev$ cat /sys/bus/pci/devices/0000\:ca\:00.0/sriov_totalvfs
16
echo 16 | sudo tee /sys/bus/pci/devices/0000\:ca\:00.0/max_vfs
echo 16 | sudo tee /sys/bus/pci/devices/0000\:ca\:00.0/sriov_numvfs
我尝试通过 PF 来配置 VF,方法是尝试 pf_bb_config 中的 vf/pf.cfg。我们能够避免之前在设置队列时看到的错误:
zq40@tc319-srv2:/opt/dpdk-stable-20.11.5/app/test-bbdev$ sudo dpdk-test-bbdev -c F0 -a cb:00.0 --vfio-vf-token=00112233-4455-6677-8899-aabbccddeeff -- -c validation -v ./ldpc_dec_default.data
EAL: Detected 56 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Probe PCI driver: intel_acc100_vf (8086:d5d) device: 0000:cb:00.0 (socket 1)
WARNING: Num of operations was not provided or was set 0. Set to default (64)
WARNING: Burst size was not provided or was set 0. Set to default (32)
WARNING: Num of lcores was not provided or was set 0. Set to value from RTE config (4)
===========================================================
Starting Test Suite : BBdev Validation Tests
Test vector file = ./ldpc_dec_default.data
+ ------------------------------------------------------- +
== test: validation
dev:cb:00.0, burst size: 32, num ops: 64, op type: RTE_BBDEV_OP_LDPC_DEC
测试程序就挂在那里,没有任何进展。
这种情况发生在之前配置错误时(比如PF配置为VF模式,但试图在PF上运行测试而不是在VF上运行),即使我们尝试在右侧运行测试,也会立即影响后续测试VF 设备。
解决方案:
echo 1 >> /sys/bus/pci/devices/{PCI}/reset
然后必须使用 pf_bb_config 重新配置 PF。对于 VF,则不需要。