Nvidia GPU passthrough fail with code 43
I'm currently trying to pass a nvidia GPU to Windows 10 guest using qemu 2.5 and libvirt 1.3.5.
I see "Error 43" on Nvidia GPU in Device Manager.
I had tried to hide the hypervisor by adding "kvm=off" and "hv_vendor_id=123456780ab", but it does not work for me. I searched in google and people solved the problem in this way.
And I also saw Virtual Machine : Yes in task manager.
Did I use in the wrong way? I can pass a AMD gpu to windows guest(AMD does not check the kvm virtualization).
Can I spoof nvidia in other way?
My system information:
#uname -a
Linux ns.mqcache.net 4.2.0-1.el7.elrepo.x86_64 #1 SMP Sun Aug 30 21:25:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
#/root/qemu25/qemu/x86_64-softmmu/qemu-system-x86_64 --version
QEMU emulator version 2.5.1.1, Copyright (c) 2003-2008 Fabrice Bellard
GPU:
02:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 620 OEM] (rev a1)
02:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)
libvirt.xml
<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
<name>win10</name>
<os>
<type machine="q35">hvm</type>
<boot dev="hd"/>
<boot dev="cdrom"/>
</os>
<features>
<acpi/>
<apic/>
<hyperv>
<vendor_id state='on' value='1234567890ab'/>
</hyperv>
<kvm>
<hidden state='on'/>
</kvm>
</features>
<clock offset="localtime">
<timer name="rtc" tickpolicy="catchup"/>
<timer name="pit" tickpolicy="delay"/>
<timer name="hpet" present="no"/>
<timer name='hypervclock' present='yes'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<vcpu current="4">4</vcpu>
<cpu mode="host-passthrough">
<topology sockets="1" cores="4" threads="1"/>
</cpu>
<memory>8388608</memory>
<currentMemory>8388608</currentMemory>
<devices>
<emulator>/root/qemu25/qemu/x86_64-softmmu/qemu-system-x86_64</emulator>
<disk device="disk" type="file">
<driver name="qemu" type="qcow2"/>
<source file="/root/vm/win10/image.qcow2"/>
<target bus="virtio" dev="vda"/>
</disk>
<sound model="ac97"/>
<interface type="bridge">
<mac address="fa:16:3e:81:00:03"/>
<source bridge="eucabr"/>
<model type="virtio"/>
<driver name="qemu"/>
<alias name="net0"/>
</interface>
<hostdev mode="subsystem" type="pci" managed="yes">
<source>
<address domain="0x0000" bus="0x02" slot="0x00" function="0x1"/>
</source>
</hostdev>
</devices>
<qemu:commandline>
<qemu:arg value="-machine"/>
<qemu:arg value="smm=off"/>
<qemu:arg value="-device"/>
<qemu:arg value="ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"/>
<qemu:arg value="-device"/>
<qemu:arg value="vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on"/>
<qemu:arg value="-vga"/>
<qemu:arg value="none"/>
</qemu:commandline>
</domain>
qemu command
/root/qemu25/qemu/x86_64-softmmu/qemu-system-x86_64 \
-name win10 \
-machine q35,accel=kvm,usb=off \
-cpu host,kvm=off,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_vendor_id=blah \
-m 2048 \
-realtime mlock=off \
-smp 2,sockets=1,cores=2,threads=1 \
-no-user-config \
-nodefaults \
-rtc base=localtime \
-no-shutdown \
-boot strict=on \
-device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \
-device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x1 \
-drive file=/root/vm/win10/snap.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 \
-device virtio-blk-pci,scsi=off,bus=pci.2,addr=0x2,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \
-k en-us \
-device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x4 \
-machine smm=off \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=02:00.1,bus=root.1,addr=00.1 \
-msg timestamp=on \
-vga none
Look forward to your help!
Answer
You need to pass copy of unmodified videocard ROM to VM.
- You need a secondary GPU that you can use as the primary for this
process. You cannot dump a clean copy of the BIOS without having the passthrough GPU as a secondary card - Put the extra card in the primary slot and the intended passthrough card in another pci-e port and bootup.
- Find your intended GPU again via lspci -v. In my case it had about the same address.
Now you can dump the ROM to a file:
# echo "0000:05:00.0" > /sys/bus/pci/drivers/vfio-pci/unbind # cd /sys/bus/pci/devices/0000\:05\:00.0 # echo 1 > rom # cat rom > /home/username/KVM/evga_gtx970.dump # echo 0 > rom # echo "0000:05:00.0" > /sys/bus/pci/drivers/vfio-pci/bindIn this case, 0000:05:00.0 is my PCI card address. You don't really need the bind step at the bottom since you'll be rebooting anyways.
You can check the integrity of the ROM dump with this handy utility at https://github.com/awilliam/rom-parser. My rom looks like:
# ./rom-parser evga_gtx970.dump Valid ROM signature found @0h, PCIR offset 1a0h PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 13c2, class: 030000 PCIR: revision 0, vendor revision: 1 Valid ROM signature found @f400h, PCIR offset 1ch PCIR: type 3 (EFI), vendor: 10de, device: 13c2, class: 030000 PCIR: revision 3, vendor revision: 0 EFI: Signature Valid, Subsystem: Boot, Machine: X64 Last imageYou should have both an EFI and a non-EFI x86 ROM in the dump ( I think most cards have both)
- Turn off the machine and put your GTX 1070 back in the primary slot.
After booting, edit your VM xml and in the section for your GPU (if you have already assigned the GPU to the VM) there should be a section. Add a file='path/to/dump/here' statement to it. My full section looks like:
<hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <rom bar='on' file='/home/username/KVM/evga_gtx970.dump'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </hostdev>This will have the VM start the card with that BIOS instead of whatever the kernel gives it.
Please note that you have to use OVMF (EFI) because SeaBIOS does not use card ROM properly.