Compiler using local variables without adjusting RSP

Question

In question Compilers: Understanding assembly code generated from small programs the compiler uses two local variables without adjusting the stack pointer.

Not adjusting RSP for the use of local variables seems not interrupt safe and so the compiler seems to rely on the hardware automatically switching to a system stack when interrupts occur. Otherwise, the first interrupt that came along would push the instruction pointer onto the stack and would overwrite the local variable.

The code from that question is:

#include <stdio.h>

int main()
{
    for(int i=0;i<10;i++){
        int k=0;
    }
}

The assembly code generated by that compiler is:

00000000004004d6 <main>:
  4004d6:       55                      push   rbp
  4004d7:       48 89 e5                mov    rbp,rsp
  4004da:       c7 45 f8 00 00 00 00    mov    DWORD PTR [rbp-0x8],0x0
  4004e1:       eb 0b                   jmp    4004ee <main+0x18>
  4004e3:       c7 45 fc 00 00 00 00    mov    DWORD PTR [rbp-0x4],0x0
  4004ea:       83 45 f8 01             add    DWORD PTR [rbp-0x8],0x1
  4004ee:       83 7d f8 09             cmp    DWORD PTR [rbp-0x8],0x9
  4004f2:       7e ef                   jle    4004e3 <main+0xd>
  4004f4:       b8 00 00 00 00          mov    eax,0x0
  4004f9:       5d                      pop    rbp
  4004fa:       c3                      ret

The local variables are i at [rbp-0x8] and k at [rbp-0x4].

Can anyone shine light on this interrupt problem? Does the hardware indeed switch to a system stack? How? Am I wrong in my understanding?

Maybe this could be of interest: http://stackoverflow.com/questions/28759227/which-stack-is-used-by-interrupt-handler-linux — Support Ukraine, Mar 25 '17 at 07:26
Note that HW interrupts don't use the user stack. [kernel stacks *can't* use a red-zone because HW does clobber asynchronously](https://stackoverflow.com/a/38043511/224132). The user stack is only used asynchronously by software-driven things: *signal* handlers, and GDB calling a function for `print foo()`. — Peter Cordes, May 10 '20 at 10:50
Related: [Why is there no "sub rsp" instruction in this function prologue and why are function parameters stored at negative rbp offsets?](https://stackoverflow.com/q/28693863) — Peter Cordes, Nov 09 '22 at 21:28

score 15 · Accepted Answer · edited Apr 15 '17 at 21:01

15

This is the so called "red zone" of the x86-64 ABI. A summary from wikipedia:

In computing, a red zone is a fixed-size area in a function's stack frame beyond the current stack pointer which is not preserved by that function. The callee function may use the red zone for storing local variables without the extra overhead of modifying the stack pointer. This region of memory is not to be modified by interrupt/exception/signal handlers. The x86-64 ABI used by System V mandates a 128-byte red zone which begins directly under the current value of the stack pointer.

In 64-bit Linux user code it is OK, as long as no more than 128 bytes are used. It is an optimization used most prominently by leaf-functions, i.e. functions which don't call other functions,

If you were to compile the example program as a 64-bit Linux program with GCC (or compatible compiler) using the -mno-red-zone option you'd see code like this generated:

main:
        push    rbp
        mov     rbp, rsp
        sub     rsp, 16;     <<============  Observe RSP is now being adjusted.
        mov     DWORD PTR [rbp-4], 0
.L3:
        cmp     DWORD PTR [rbp-4], 9
        jg      .L2
        mov     DWORD PTR [rbp-8], 0
        add     DWORD PTR [rbp-4], 1
        jmp     .L3
.L2:
        mov     eax, 0
        leave
        ret

This code generation can be observed at this godbolt.org link.

For a 32-bit Linux user program it would be a bad thing not to adjust the stack pointer. If you were to compile the code in the question as 32-bit code (using -m32 option) main would appear something like the following code:

main:
        push    ebp
        mov     ebp, esp
        sub     esp, 16;     <<============  Observe ESP is being adjusted.
        mov     DWORD PTR [ebp-4], 0
.L3:
        cmp     DWORD PTR [ebp-4], 9
        jg      .L2
        mov     DWORD PTR [ebp-8], 0
        add     DWORD PTR [ebp-4], 1
        jmp     .L3
.L2:
        mov     eax, 0
        leave
        ret

This code generation can be observed at this gotbolt.org link.

edited Apr 15 '17 at 21:01

Michael Petch

46,082
8
107
198

answered Mar 25 '17 at 08:20

ead

32,758
6
90
153

Thanks for your answer. It explains everything. So upon an interrrupt, the hardware pushes the instruction pointer 128 bytes down the stack and so does not overwrite local variables. Should the function require more than 128 bytes for local storage, it must adjust the stack pointer. – Paul Ogilvie Mar 25 '17 at 08:34
2

@Paul - [The kernel uses a separate stack](http://stackoverflow.com/questions/15168822/intel-x86-vs-x64-system-call/15169141#15169141), so a privileged interrupt will not touch the stack of the user code. – Bo Persson Mar 25 '17 at 09:21
@Bo Persson, I understand the stacks are switched based on priviledge level of the interrupt. So an interrupt of the same priviledge level will use the current stack _and_ must not use the first 128 bytes. Is that correct? – Paul Ogilvie Mar 25 '17 at 14:10
1

@Paul - I'm not an expert on System V, but believe that the OS will not allow you to install an interrupt handler in user level code. So it never happens. – Bo Persson Mar 25 '17 at 15:54
@Bo Persson, so I must understand that the hardware (CPU) must have been set up to push at sp+128 when an interrupt occurrs? The code produced by the compiler must be able to runboth in kernel mode and in user mode. – Paul Ogilvie Mar 25 '17 at 17:17
3

The kernel code uses a different ABI where there's isn't a red zone.The adjustment of the stack by 128 bytes comes into play during signal processing. When a signal "interrupts" a process the kernel adjusts the stack pointer by 128 bytes downwards before executing the signal handler. – Ross Ridge Mar 25 '17 at 17:22
1

I altered the answer to provided the updated Wiki entry (I recently modified the Wiki article). The old one incorrectly suggested that the redzone starts below the location where the return address is stored. This is inaccurate - it is the 128 bytes directly under the current value of _RSP_ . _RSP_ may or may not be pointing at the return address. – Michael Petch Apr 15 '17 at 19:14

Compiler using local variables without adjusting RSP

1 Answers1

Linked

Related