[ovs][libvirt][virtio][qemu] vhost user client 排障

时间：2023-03-10 03:16:00
[ovs][libvirt][virtio][qemu] vhost user client 排障

ovs-dpdk + qemu 使用vhostuserclient通信，连路不通，libvirt报错如下：

  -- ::37.265+: starting up libvirt version: 3.2., package: .el7_4. (CentOS BuildSystem <http://bugs.centos.org>, 2018-03-07-13:51:24, x86-01.bsys.centos.org), qemu version: 2.9.0, hostna

  LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /export/Datapath/tong/qemu_home/bin/qemu-system-x86_64 -name guest=centos7.,debug-threads=on -S -object secret,id=master

  --04T13::.295614Z qemu-system-x86_64: -chardev socket,id=charnet1,path=/tmp/nlb_vm0.sock,server: QEMU waiting for connection on: disconnected:unix:/tmp/nlb_vm0.sock,server

  --04T13::.808975Z qemu-system-x86_64: -chardev socket,id=charnet2,path=/tmp/nlb_vm1.sock,server: QEMU waiting for connection on: disconnected:unix:/tmp/nlb_vm1.sock,server

  --04T13::.809417Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/ (label charserial0)

  --04T13::.706248Z qemu-system-x86_64: Failed initializing vhost-user memory map, consider using -object memory-backend-file share=on

  --04T13::.706284Z qemu-system-x86_64: vhost_set_mem_table failed: Interrupted system call ()

  --04T13::.706421Z qemu-system-x86_64: unable to start vhost net: : falling back on userspace virtio

  --04T13::.708417Z qemu-system-x86_64: Failed initializing vhost-user memory map, consider using -object memory-backend-file share=on

  --04T13::.708436Z qemu-system-x86_64: vhost_set_mem_table failed: Resource temporarily unavailable ()

  --04T13::.708558Z qemu-system-x86_64: unable to start vhost net: : falling back on userspace virtio

  --04T13::.734457Z qemu-system-x86_64: terminating on signal  from pid  (/usr/sbin/libvirtd)

在libvirt的xml里增加一行：

        <cpu mode='host-passthrough' check='none'>

          <topology sockets='' cores='' threads=''/>

          <numa>

            <cell id='' cpus='0-7' memory='' unit='KiB' memAccess='shared'/>

          </numa>

        </cpu>

对应的qemu命令增加了：

 -object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu/-centos7.,share=yes,size= -

排障成功。

参考：

http://docs.openvswitch.org/en/latest/topics/dpdk/vhost-user/

In addition, QEMU must allocate the VM’s memory on hugetlbfs. vhost-user ports access a virtio-net device’s 
virtual rings and packet buffers mapping the VM’s physical memory on hugetlbfs. To enable vhost-user ports 
to map the VM’s memory into their process address space, pass the following parameters to QEMU:
-object memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on

-numa node,memdev=mem -mem-prealloc

https://libvirt.org/formatdomain.html#elementsNUMATuning

相关文章