assembly subroutines get called twice without even being called from main

Question

I'm trying to define some subroutines that have calls to printf in them. A very trivial example is as follows:

extern printf
LINUX        equ     80H
EXIT         equ     60

section .data
    intfmt: db "%ld", 10, 0

segment .text
    global  main

main:
    call os_return      ; return to operating system

os_return:
    mov  rax, EXIT      ; Linux system call 60 i.e. exit ()
    mov  rdi, 0     ; Error code 0 i.e. no errors
    int  LINUX      ; Interrupt Linux kernel

test:
    push rdi
    push rsi
    mov rsi, 10
    mov rdi, intfmt
    xor rax, rax
    call printf
    pop rdi
    pop rsi
    ret

Here test just has a call to printf that outputs the number 10 to the screen. I would not expect this to get called as I have no call to it.

However when compiling and running:

nasm -f elf64 test.asm
gcc -m64 -o test test.o

I get the output:

10
10

I'm totally baffled and wondered if someone could explain why this is happening?

Matthew Slattery · Accepted Answer · 2011-12-11T17:49:09.700

3

int 80H invokes the 32-bit system call interface, which a) uses the 32-bit system call numbers and b) is intended for use by 32-bit code, not 64-bit code. Your code is actually performing a umask system call with random parameters.

For a 64-bit system call, use the syscall instruction instead:

...
os_return:
    mov  rax, EXIT      ; Linux system call 60 i.e. exit ()
    mov  rdi, 0     ; Error code 0 i.e. no errors
    syscall         ; Interrupt Linux kernel
...

edited Dec 11 '11 at 17:49

answered Dec 11 '11 at 16:25

Matthew Slattery

45,290
8
103
119

Thanks! Do I put the value 60 (EXIT) into rdi then instead of rax, similar to calling printf? – Sarah Tattersall Dec 11 '11 at 17:24
No, putting the syscall number in `rax` and the first argument in `rdi` is correct. See http://www.x86-64.org/documentation/abi.pdf (appendix A in particular) for some documentation of the kernel syscall ABI and the differences from the user-level calling conventions. – Matthew Slattery Dec 11 '11 at 17:30
Sorry to keep badgering you about this but I changed the line "int LINUX" to call syscall and added extern syscall to the top and still get the two ten's. Is there any chance you could show me a mini example of how to call syscall? Thanks so much :) – Sarah Tattersall Dec 11 '11 at 17:42
See edited answer: I'm talking about the `syscall` x86-64 instruction (which is the closest 64-bit equivalent to doing things with `int 80h`), not the `syscall()` library function. – Matthew Slattery Dec 11 '11 at 17:51
That was why I was getting confused. It works - thanks for all your help! – Sarah Tattersall Dec 11 '11 at 18:31

score 2 · Answer 2 · answered Dec 11 '11 at 14:27

I would say that your call to exit is failing, so when it returns, it falls through to the test function, that prints the first 10.

Then when you return with ret you go back to the instruction just after the call os_return, that is, well os_return. The call to exit fails again and falls through to the test function again. But this time the ret returns from the main function and the program ends.

About why is the exit call failing, I cannot tell as I don't have a 64-bit system available. But you could disassemble the exit function from libc and see how it is done there. My guess is that the int LINUX interface is 32-bit only, as it exists only for historic compatibility, and 64-bit linux in not so old.

`int 0x80` is the 32 system call interface, in which syscall 60 is `umask`. See [What happens if you use the 32-bit int 0x80 Linux ABI in 64-bit code?](https://stackoverflow.com/questions/46087730/what-happens-if-you-use-the-32-bit-int-0x80-linux-abi-in-64-bit-code) — Timothy Baldwin, Nov 28 '19 at 21:30

assembly subroutines get called twice without even being called from main

2 Answers2