1

I am trying to understand how a variable sized static array work internally:

Following is a fixed size static array in C and its Assembly equivalent;

int main()
{
int arr[2] = {3};
}
================
main:
        push    rbp
        mov     rbp, rsp
        mov     QWORD PTR [rbp-8], 0
        mov     DWORD PTR [rbp-8], 2
        mov     eax, 0
        pop     rbp
        ret

However a variable sized array is shown below

int main()
{
int varSize ;
int Arr[varSize];
}
=================
main:
        push    rbp
        mov     rbp, rsp
        sub     rsp, 32
        mov     rax, rsp
        mov     rcx, rax
        mov     eax, DWORD PTR [rbp-4]
        movsx   rdx, eax
        sub     rdx, 1
        mov     QWORD PTR [rbp-16], rdx
        movsx   rdx, eax
        mov     r8, rdx
        mov     r9d, 0
        movsx   rdx, eax
        mov     rsi, rdx
        mov     edi, 0
        cdqe
        lea     rdx, [0+rax*4]
        mov     eax, 16
        sub     rax, 1
        add     rax, rdx
        mov     edi, 16
        mov     edx, 0
        div     rdi
        imul    rax, rax, 16
        sub     rsp, rax
        mov     rax, rsp
        add     rax, 3
        shr     rax, 2
        sal     rax, 2
        mov     QWORD PTR [rbp-24], rax
        mov     rsp, rcx
        mov     eax, 0
        leave
        ret

I am seeing a whole lot of assembly instructions if I declare a variable sized array. Can some one explain how is this flexibility of variable size achieved?

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
Franc
  • 319
  • 9
  • 28
  • Tactical mistakes: You cannot identify what these instructions are there for. Set varSize to some non-constant value, say varSize = f();. Store something in the array, say Arr[20] = 3; So you you get some idea what the instructions are doing. – gnasher729 Jun 21 '20 at 10:02
  • 3
    You apparently compiled without optimization so the code is unnecessarily complex. Anyway, what the variable version does is round up the array size to a multiple of 16 bytes to maintain stack alignment and then allocates that by adjusting the stack pointer. The static version uses the 128 byte red zone. With a bigger but still static array that would also adjust the stack pointer but with a compile time constant. – Jester Jun 21 '20 at 11:27
  • 1
    The code as it stands is nonsensical because the size of the array is taken from an uninitalized variable. – Nate Eldredge Jun 21 '20 at 15:58
  • 1
    [Here](https://godbolt.org/z/sYA6Jz) is a better example, that also survives enabling optimizations. You should find this much easier to understand what's going on. – Nate Eldredge Jun 21 '20 at 16:05

1 Answers1

1

Same mechanism as alloca() - allocate memory by decreasing the stack pointer, with the assumption that the stack is big enough and/or the OS will grow it as needed.

There might be a bit of an issue when the requested size is over a memory page and the stack is near its end. Normally, the OS grows the stack by setting up a guard page at the stack top and watching for faults in that area, but that assumes that the stack grows more or less sequentially (by pushes and function calls). If the decreased stack pointer overshoots the guard page, it might end up pointing at a bogus location. I'm not sure what does the compiler do about that possibility.

Seva Alekseyev
  • 59,826
  • 25
  • 160
  • 281
  • 1
    Only Windows requires that every intervening page be touched (stack probes). [Linux allows](//stackoverflow.com/a/46791370) stack growth anywhere in the 8MiB (by default) region of the initial thread stack. (Extra threads need thread-stacks fully pre-allocated). See [Linux process stack overrun by local variables (stack guarding)](https://stackoverflow.com/q/60058873) which also describes what `gcc -fstack-check` does: always stack-probes even on targets that don't need it, as defence against stack-clash attacks where a huge VLA / alloca could jump all the way into some other mapping. – Peter Cordes Jun 21 '20 at 15:29