IRQL_NOT_LESS_OR_EQUAL at .iso boot | Problems with Windows 10 virtualisation

Question

Since my gaming hardware supports IOMMU, I decided to migrate the machine to Linux.

The setup:

AMD FX-8350
8 GB 1600
Asus Sabertooth 990FX R1.0
ATI HD 5450 (for base system)
nvidia GTX970 (for VM)

I installed a clean Void Linux and set up IOMMU and fixed my IVRS table. Afterwards, I bound my nVidia GTX 970 to the vfio-pci kernel module. Then I set up libvirtd daemon with its dependencies, where I had to add virtlogd and dbus daemon by hand. Then I installed virt-manager and set up a new VM based on the recommendations from the answers.

Update (overcome the BSOD)

after trying the prebuild arch OVMF images with the same results i started to change ALL the options. u know just in case.

One of the first things i tried was switching from host-passthrough/Copy host CPU configuration in CPU to Hypervisor Default. Now the installer starts without any issues. So the IRQL error is caused by the CPU passtrough!

Problem: I will have performance impacts if i emulate a CPU instead of just handing it over. So the goal will be to get host-passthrough/Copy host CPU configuration to work...

The original issue:

When i try to boot from a current and tested Win10.iso it gives me the BSOD with IRQL_NOT_LESS_OR_EQUAL.

So how can I fix the BSOD?

I see that the error is related to memory allocation, but I have no idea what to change to make it work.

I already had the VM up and running (so it is possible to get around that error), except for the error 43 thing from nVidia, but then i realized that my base system was corrupted, so I decided to do a clean reinstallation, which worked perfectly until the first VM boot up.

Heres the current XML. Its lacking the vfio-drivers.iso and the Bootdrive, because the problem is that the Win10.iso won't boot.

<domain type='kvm'>
  <name>win10</name>
  <uuid>6f957f60-c90c-4d33-8821-01348fa531a9</uuid>
  <memory unit='KiB'>6291456</memory>
  <currentMemory unit='KiB'>6291456</currentMemory>
  <vcpu placement='static'>8</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-2.12'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10_VARS.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
    <vmport state='off'/>
  </features>
  <cpu mode='host-passthrough' check='none'/>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/pitch/Downloads/Win10_1803_EnglishInternational_x64.iso'/>
      <target dev='sdb' bus='sata'/>
      <readonly/>
      <boot order='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-to-pci-bridge'>
      <model name='pcie-pci-bridge'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:36:19:70'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <sound model='ich6'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x01' function='0x0'/>
    </sound>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc069'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc316'/>
      </source>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </memballoon>
  </devices>
</domain>

OLD XMLs

This was my original libvirt XML:

<domain type='kvm'>
  <name>win10</name>
  <uuid>502f32c7-01fc-4690-98b7-85c1c82e392b</uuid>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.12'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10_VARS.fd</nvram>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
    <vmport state='off'/>
  </features>
  <cpu mode='host-passthrough' check='none'/>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/win10.qcow2'/>
      <target dev='sdb' bus='sata'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/pitch/Downloads/Win10_1803_EnglishInternational_x64.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='floppy'>
      <driver name='qemu' type='raw'/>
      <source file='/home/pitch/Downloads/virtio-win-0.1.149.iso'/>
      <target dev='fda' bus='fdc'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='fdc' index='0'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </controller>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:04:2a:7a'/>
      <source network='default'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' autoport='yes'>
      <listen type='address'/>
      <image compression='off'/>
    </graphics>
    <sound model='ich6'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='2'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='3'/>
    </redirdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </memballoon>
  </devices>
</domain>

Just to test, I tried stripping the VM configuration. Here's the XML dump:

<domain type='kvm'>
  <name>win10</name>
  <uuid>502f32c7-01fc-4690-98b7-85c1c82e392b</uuid>
  <memory unit='KiB'>6291456</memory>
  <currentMemory unit='KiB'>5242880</currentMemory>
  <vcpu placement='static'>8</vcpu>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.12'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10_VARS.fd</nvram>
    <bootmenu enable='no'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
    <vmport state='off'/>
  </features>
  <cpu mode='host-passthrough' check='partial'/>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/pitch/Downloads/Win10_1803_EnglishInternational_x64.iso'/>
      <target dev='sda' bus='scsi'/>
      <readonly/>
      <boot order='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc069'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc316'/>
      </source>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </memballoon>
  </devices>
</domain>

Deltik · Answer 1 · 2018-07-30T13:47:21.047

Updated Answer

It sounds like you were able to reinstall Windows, but the first boot blue-screened. This updated answer addresses that.

Windows is fragile when it comes to hardware changes, virtualised or bare metal.

If you changed something in your libvirt domain XML after installation, the IRQL_NOT_LESS_OR_EQUAL blue screen of death could easily happen.

It's especially hard to figure out what caused the blue screen if you do not remember what you changed. For example, if I installed Windows on the scsi (virtio-scsi) disk bus and then changed the disk bus to virtio (virtio-blk), I'll get this far before the guest crashes with no details:

The easiest way to resolve this is to configure the libvirt domain XML how you want it beforehand and then reinstall Windows for that hardware.

A harder way would be to start up the Windows install ISO, add virtio-win drivers with DISM, and hope that one of them fixes the BSOD. You would have to adapt the instructions from this Super User answer. It's an annoying process with no guarantee of success.

^{_{(And in the case of changing the boot disk bus after installation, it does not appear to be possible for Windows 10, as I found out.)}}

So spare yourself the headache of troubleshooting and reinstall Windows once more.

Here's how I customised my libvirt domain XML for Windows:

The installed boot disk uses scsi as its disk bus with discard='unmap' in the <driver> for TRIM support.
The installer CD/ISO drive uses the sata bus so that the Windows installer can boot.
The virtio-win drivers ISO drive also uses sata so that the Windows installer can read it.
I use q35 as the machine attribute of the <type> under <os> instead of pc (i440fx) for better support of PCIe passthrough because i440FX only supports PCI. Note that q35 requires a different set of <controller> elements. You can set this in virt-manager if you delete (undefine) your current guest, create a new one, check "Customize configuration before install", and then change the chipset from i440FX to Q35.
The network adapter model is virtio for best performance.
The video model is QXL for a better-performing display.

In the Windows installer, install all the compatible drivers from the virtio-win ISO before completing the disk partitioning step:

Hopefully after the reinstall, you should no longer be getting the blue screen of death.

Old Answer

This answer went on the assumption that you were trying to reinstall Windows.

According to the libvirt XML file you provided, you are not booting from the ISO file (cdrom). You seem to be booting directly from your virtual machine image, which likely contains Windows installed on different hardware and is thus incompatible with the virtualised hardware.

You can boot from cdrom by adding

    <boot dev='cdrom'/>

right above

    <boot dev='hd'/>

and then starting the virtual machine normally. Note that the Windows boot disk may give you a time-limited prompt to press any key to start the installer.

Alternatively, you can enable booting from cdrom through virt-manager like so:

lonedd · Answer 2 · 2020-10-28T20:59:20.490

  <cpu mode='host-passthrough' check='none'/>

I soved this issue on my side with setting the CPU-Type from "host-passthrough/host-model" to a pre-defined "Opteron G4"-template - I have an Threadripper-Processor. Windows can't handle too much possibilities.

Hopefully this is helpful.

Edit 25.03.2019

Further Details on my solution: I've encountered the same issues as above, which I could "solve" by setting the CPU-Type in Virt-Manager (not editing the XML-File) to an "Opteron G4".

I solved the Windows Issue with that but was still fighting with other issues and GPU-Passthrough didn't work properly. I could start the VM but the GPU was not correctly listed in the VM and couldn't install the driver, the GPU was not detected.

I found this tutorial: https://heiko-sieger.info/running-windows-10-on-linux-using-kvm-with-vga-passthrough/ which from my pov covers a lot of topics on GPU-Passthrough. Especially on the "Troubleshooting" section, whih at the end helped me to solve my problem.

Due the fact that I was using an X399 Taichi Motherboard in combination with Threadripper CPU which at that time weren't capable to work, I encountered several problems including the one with the CPU-host-passthrough which actually didn't really make sense to me because I already have successfully done this on other hardware.

For me at the end updating the BIOS did the trick. There was an update for especially those functionalities.

After that everything just worked like a charm.

From my experience, it needs:

-Hardware needs to be ready + look for BIOS update which are related.

-Settings on the Motherboard like SVM/xVTd, IOMMU, ACS and other Motherboard-related options needs to be activated.

-Using the newest packages provides probably the best experience.

-It needs two GPU's. The first one should be either onboard or dedicated. Otherwise it probably won't really work smoothly...

-nVidia doesn't like it, AMD is a friend of linux.

Hope this helps. For futher question, don't hesitate.

IRQL_NOT_LESS_OR_EQUAL at .iso boot | Problems with Windows 10 virtualisation

2 Answers2

Updated Answer

Old Answer