3

What happens if a CPU attempts to execute a binary that has been compiled with some instructions that your CPU doesn't support. I'm specifically wondering about some of the new AVX instructions running on older processors.

I'm assuming this can be tested for, and a friendly message could in theory be displayed to a user. Presumably most low level libraries will check this on your behalf. Assuming you didn't make this check, what would you expect to happen? What signal would your process receive?

Kellen
  • 621
  • 1
  • 5
  • 15
  • 1
    [What causes signal 'SIGILL'?](https://stackoverflow.com/q/7901867/608639) and friends. As far as I know, `SIGILL` is raised on all Linux platforms, including i386, x86_64, ARM, Aarch32, Aarch64 and MIPS. – jww Jun 25 '17 at 20:44
  • *"Presumably most low level libraries will check this on your behalf."* Uh, I don't think so. Most low-level libraries that use potentially unsupported instructions are doing so for speed above all else. It is the responsibility of high-level code to verify whether those instructions are supported *before* calling down to the low-level code. Use the `CPUID` instruction (exposed as an intrinsic in most compilers) to verify that the required instruction set is supported *before* executing them, thus avoiding any risk of an invalid instruction exception. Display the friendly error message then. – Cody Gray - on strike Jun 26 '17 at 13:02
  • 1
    Halts and catches fire. – Michael Petch Jun 27 '17 at 01:29

1 Answers1

7

A new instruction can be designed to be "legacy compatible" or it can not.
To the former class belong instructions like tzcnt or xacquire that have an encoding that produces valid instructions in older architecture: tzcnt is encoded as rep bsf and xacquire is just repne.
The semantic is different of course.

To the second class belong the majority of new instructions, AVX being one popular example.
When the CPU encounters an invalid or reserved encoding it generates the #UD (for UnDefined) exception - that's interrupt number 6.

The Linux kernel set the IDT entry for #UD early in entry_64.S:

idtentry invalid_op         do_invalid_op           has_error_code=0

the entry points to do_invalid_op that is generated with a macro in traps.c:

DO_ERROR(X86_TRAP_UD,     SIGILL,  "invalid opcode",        invalid_op)

the macro DO_ERROR generates a function that calls do_error_trap in the same file (here).

do_error_trap uses fill_trap_info (in the same file, here) to create a siginfo_t structure containing the Linux signal information:

case X86_TRAP_UD:
    sicode = ILL_ILLOPN;
    siaddr = uprobe_get_trap_addr(regs);
    break;

from there the following calls happen:

do_trap in traps.c force_sig_info in signal.c specific_send_sig_info in signal.c

that ultimately culminates in calling the signal handler for SIGILL of the offending process.

The following program is a very simple example that generates an #UD

BITS 64

GLOBAL _start

SECTION .text

_start:

 ud2

we can use strace to check the signal received by running that program

--- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPN, si_addr=0x400080} ---
+++ killed by SIGILL +++

as expected.

As Cody Gray commented, libraries don't usually rely on SIGILL, instead they use a CPU dispatcher or check the presence of an instruction explicitly.

Margaret Bloom
  • 41,768
  • 5
  • 78
  • 124