2

I have disassembled a C file. At the start of what used to be a function I see

endbr64
push     %rax
pop      %rax

as the first instructions.

Why is the content of %rax pushed and then popped without any changes to the stack or the register in between? I would expect %rax to be unchanged by that operation.

Linus Wagner
  • 151
  • 1
  • 1
  • 8
  • 1
    `rax` is indeed unchanged by that operation. Could be either trash by an unoptimized build, not code at all interpreted as such, or more subtly a way to set the top of stack entry to the value of `rax`. – 500 - Internal Server Error Oct 05 '22 at 14:40
  • 2
    I assume that's the top of a function, given the `endbr64`. What compiler / obfuscator / weird human generated the machine code you're disassembling? I don't know any reason why you'd want to do this as a compact way to store RAX to `[rsp-8]`, which is the only net side-effect. of this, assuming no other threads or breakpoints are changing things asynchronously. ([rsp-8] is in the red-zone below RSP on x86-64 SysV, but that's not a thing on Windows x64). It's not something I've seen even in debug builds from mainstream compilers (GCC/clang/MSVC/ICC). – Peter Cordes Oct 05 '22 at 14:45
  • @Peter, possibly `int f() { int x; return x; }` ? ... although he did say "as the first instructions", implying that more follows. – prl Oct 05 '22 at 15:09
  • @prl: No compiler I'm aware of knows how to initialize local vars with `push`. Maybe this is for 16-byte stack alignment around an empty function body? – Peter Cordes Oct 05 '22 at 15:13
  • @Peter sadly, I don't have the compile script available, because the binary is from a security class I'm taking where it is about exploiting the program. The only thing I know is that the code is compiled using gcc with symbols, has partial RELRO and no PIE – Linus Wagner Oct 05 '22 at 21:32
  • I'm really surprised to see this in GCC output. I wouldn't expect that in GCC output unless the source used nline asm to put it there. What is the surrounding code? You're not maybe disassembling data that wasn't supposed to be code? – Peter Cordes Oct 06 '22 at 00:58
  • 1
    Apparently, it is due to the "-fstack-clash-protection" flag in GCC (see https://godbolt.org/z/Pvd1ox9d3). It is removed, however, if the call to exit() is removed as well – Linus Wagner Oct 06 '22 at 14:11
  • 1
    @Venturion Consider posting your last comment as an answer. – Sep Roland Oct 06 '22 at 17:05
  • @Peter, coincidentally, this answer has an example of clang allocating an uninitialized variable using push. https://stackoverflow.com/a/73984434/8422330 – prl Oct 07 '22 at 10:13
  • @prl: Allocating / deallocating space with dummy push/pop, yes that's a well known thing. I said *initializing*, which is a [missed optimization](https://stackoverflow.com/questions/49485395/what-c-c-compiler-can-use-push-pop-instructions-for-creating-local-variables) when GCC/clang do `push rcx` / `mov [rsp], edi` instead of `push rdi`. (Although since your C example didn't init, only read, I should have said *reading*.) In that linked Q&A, note that it's a silly `mov al, [rsp]` / `pop rcx`, not `pop rax`. – Peter Cordes Oct 07 '22 at 10:48
  • @Peter, so your previous response was a nonsequitur, then, since my example clearly showed an uninitialized variable? – prl Oct 07 '22 at 10:51
  • @prl: Agreed. The correct rebuttal is that compilers miss the optimization of using `pop` to load a variable and free its space at the end of a function. That's highly related to knowing how to allocate + init a variable with `push`, but a separate optimization. (Some compilers will decide the value of a C variable that's read uninitialized, e.g. sometimes forcing it to 0 instead of reading garbage from a register. But I mixed up some ideas like thinking about the compiler picking some garbage register to init with, or AL= # of XMM args, so the push could be an init.. But that's unlikely.) – Peter Cordes Oct 07 '22 at 11:01
  • I don't get why the compiler gets this wrong. Why doesn't it understand that the registers and flags are the same before and after? – puppydrum64 Dec 20 '22 at 16:35

1 Answers1

2

Apparently, it is due to the "-fstack-clash-protection" flag in GCC (see godbolt.org/z/Pvd1ox9d3). It is removed, however, if the call to exit() is removed.

In case the link goes down: The following code is compiled there with x86-64 gcc 12.2 using the flags "-fstack-clash-protection -O1":

#include <stdio.h>
#include <stdlib.h>


void fun(void) {
    printf("hello\n");

    exit(EXIT_SUCCESS);
}
Linus Wagner
  • 151
  • 1
  • 1
  • 8
  • 1
    Seems like a missed optimization, then. `push/pop/sub rsp,8` is super dumb when it could just do like clang (with some `-march=` settings) and use one `push` *instead* of `sub rsp,8` to align the stack for a call as well as touching it. https://godbolt.org/z/hd99v88T7 . It's a missed optimization anyway because `call` pushes a return address and `sub rsp,8` is less than 4096, so it can't skip a single guard page. – Peter Cordes Oct 06 '22 at 21:17
  • 1
    But yes, it seems you've correctly identified where this nonsense is coming from. – Peter Cordes Oct 06 '22 at 21:19