Retq instruction, where does it return

Question

I am unable to understand where the assembly instruction retq returns to.

I understand that when my normal code executes then it return to the address specified in the stack. But how does it know where in the stack is the return address located?

In short, does it use rbp or esp for finding the address on the stack?

If you want to experiment, try replacing `retq` with `addq $8, %rsp; jmpq -8(%rsp)`. Functionally equivalent (not performance-wise though). — FrankH., Sep 11 '13 at 15:07
`ret` doesn't affect FLAGS, so the actual drop-in replacement uses `lea 8(%rsp), %rsp` instead of `add`. And this is only safe in user-space in the x86-64 System V calling convention, where you have a red-zone below RSP that's safe from asynchronous clobbers (e.g. by signal handlers and debuggers). On Windows, an SEH exception could potentially be delivered between `lea` and `ret`. — Peter Cordes, Mar 27 '23 at 14:18

score 6 · Accepted Answer · edited Oct 10 '19 at 16:20

After studying assembly code, here are my thoughts, let's look at a sample:

fun:
push %rbp
mov %rsp,%rbp
...
...
pop %rbp
retq

main:
...
...
callq  "address" <fun>
...
...

We can see there is a instruction before retq. The pop %rbp (sometimes it is a leave instruction but they are similar) instruction will

save the content of current stack pointer %rsp to base stack pointer %rbp.
move the %rsp pointer to previous address on stack.

For example: before pop command, the %rsp pointed to 0x0000 0000 0000 00D0. After the pop command it points to 0x0000 0000 0000 00D8 (assume the stack grows from high address to low address).

After the pop command, now %rsp points to a new address and retq takes this address as return address.

score 4 · Answer 2 · answered Oct 10 '19 at 16:57

ret is how you spell pop rip on x86: a stack pop and an indirect branch to that value. https://www.felixcloutier.com/x86/ret documents exactly what it does and doesn't do.

It's effectively pop %tmp / jmp *%tmp where tmp is an internal temporary register.

ret depends only on RSP.

Using RBP as a frame pointer is a totally optional software convention that modern compilers don't even do when optimization is enabled.

Retq instruction, where does it return

2 Answers2

Linked