By definition, a stack underflow is a type of undefined behaviour, and thus any code which triggers such a condition must be UB. Therefore, you can't reliably cause a stack underflow.
That said, the following abuse of variable-length arrays (VLAs) will cause a controllable stack underflow in many environments (tested with x86, x86-64, ARM and AArch64 with Clang and GCC), actually setting the stack pointer to point above its initial value:
#include <stdint.h>
#include <stdio.h>
#include <string.h>
int main(int argc, char **argv) {
    uintptr_t size = -((argc+1) * 0x10000);
    char oops[size];
    strcpy(oops, argv[0]);
    printf("oops: %s\n", oops);
}
This allocates a VLA with a "negative" (very very large) size, which will wrap the stack pointer around and result in the stack pointer moving upwards. argc and argv are used to prevent optimizations from taking out the array. Assuming that the stack grows down (default on the listed architectures), this will be a stack underflow.
strcpy will either trigger a write to an underflowed address when the call is made, or when the string is written if strcpy is inlined. The final printf should not be reachable.
Of course, this all assumes a compiler which doesn't just make the VLA some kind of temporary heap allocation - which a compiler is completely free to do. You should check the generated assembly to verify that the above code does what you actually expect it to do. For example, on ARM (gcc -O):
8428:   e92d4800    push    {fp, lr}
842c:   e28db004    add fp, sp, #4, 0
8430:   e1e00000    mvn r0, r0 ; -argc
8434:   e1a0300d    mov r3, sp
8438:   e0433800    sub r3, r3, r0, lsl #16 ; r3 = sp - (-argc) * 0x10000
843c:   e1a0d003    mov sp, r3 ; sp = r3
8440:   e1a0000d    mov r0, sp
8444:   e5911004    ldr r1, [r1]
8448:   ebffffc6    bl  8368 <strcpy@plt> ; strcpy(sp, argv[0])