How to write my own atoi function in x86 assembly

Question

im having some trouble writing my own atoi function in assembly. The instructions are

"Change the function so that it returns the integer equivalent of the C-string (pointer) that is passed into the function. You may assume that the first character is between ‘0’ and ‘9’, inclusive. atoishould consider all the characters from the first up until the first character that is not a decimal digit. As you can see, mainuses the value returned by atoias an exit code (this is just a cheap way of accessing the output from atoi, without writing an itoafunction.) As given to you, atoireturns 1234. The return value is ANDedwith 0xFF to reduce it to a byte. Thus 1234 & 255 becomes 210."

    # Useful constants 
    .equ    STDIN,0 
    .equ    STDOUT,1 
    .equ    READ,0 
    .equ    WRITE,1 
    .equ    EXIT,60 
# Stack frame 
    .equ    bufferSize, 32
    .equ    buffer,-bufferSize
    .equ    localSize,16 
    .equ    frameSize, bufferSize + localSize
# Read only data 
    .section    .rodata # the read-only data section 
prompt: 
    .string     "Enter an integer: " 
    .equ    promptSz,.-prompt-1 
msg: 
    .string     "You entered: " 
    .equ    msgSz,.-msg-1 

Code

    .text   # switch to text section 


    .globl  __start 
 __start: 
    pushq   %rbp    # save caller’s frame pointer 
    movq    %rsp, %rbp  # establish our frame pointer 
    subq    $frameSize, %rsp    # for local variables 

    movl    $promptSz, %edx # prompt size 
    movl    $prompt, %esi   # address of prompt text string 
    movl    $STDOUT, %edi   # standard out 
    movl    $WRITE, %eax 
    syscall     # request kernel service 

    movl    $bufferSize,%edx
    leaq    buffer(%rbp), %rsi  # load buffer address
    movl    $STDIN, %edi    # standard in 
    movl    $READ, %eax 
    syscall     # request kernel service 
    movl    %eax, (%rsp)    # store num chars read

    leaq    buffer(%rbp), %rsi  # load buffer address
    call    atoi    # our exit code will be the return from atoi

    movq    %rbp, %rsp  # delete local variables 
    popq    %rbp    # restore caller’s frame pointer 
    movl    %eax, %edi  # put exit status in %edi (will be ANDed with FF)
    movl    $EXIT, %eax # exit from this process 

    syscall

the base code looks like this where i just have to implement my own atoi. so far what i have for the atoi function is

atoi:
    pushq   %rbp    # save caller’s frame pointer 
    movq    %rsp, %rbp  # establish our frame pointer 
    subq    $16, %rsp   # for local variables

    movq    %rdi, -16(%rbp) #moving first argument to local variable
    movl    $0, -4(%rbp) #moving 0 to local variable
    movl    $10, -12(%rbp) #moving 10 to local variable

    movl    -16(%rbp), %rax
    movzbl  (%rax), %eax #getting value of rax
    movl    -4(%rbp), %eax

    imull   -12(%rbp), %eax
    movl    %eax,   -4(%rbp)

    movq    %rbp, %rsp  # delete local variables 
    popq    %rbp    # restore caller’s frame pointer 
    ret

im at a loss for where to go next. it seems anything i do just gives me segmentation faults

Answer 1

You're over-using local variables (and under-using registers); will need a loop that stops when it finds an invalid character; and probably using the wrong calling conventions (the syscalls look like Linux, which implies System V AMD64 ABI, which means parameters are passed in registers and not on the stack).

Note that this can all be done without any local variables at all. For example (NASM syntax because I don't do AT&T, untested):

;Convert string to integer
;
;Input
; rdi = first parameter (address of string)
;
;Output
; rax = result

atoi:
    xor rax,rax               ;rax = 0 (this will become the returned result)
.nextChar:
    movzx rcx,byte [rdi]      ;rcx = next character
    sub rcx,'0'               ;rcx = value of next digit
    jb .done                  ;Invalid character (too low to be a decimal digit)
    cmp rcx,9                 ;Was it too high to be a decimal digit?
    ja .done                  ; yes, invalid

    lea rax,[rax*4+rax]       ;rax = result*5
    lea rax,[rax*2+rcx]       ;rax = result*5*2 + digit = result*10 + digit
    inc rdi                   ;rdi = address of next character
    jmp .nextChar
.done:
     ret

Note: This code will not work for negative values (e.g. strings that begin with '-') and won't return an error condition if/when the result overflows. The result will also be 64-bit (while an int is probably supposed to be 32-bit). Mostly, it's a "convert to unsigned long long" (with error handling that's as bad as atoi()).