Note that as commented, MASM is ignores the []. Instead MASM goes by the type for a label. In this case, the problem is that the : after table (table:) makes label of type "code" that is normally used as a branch or call target, so jmp [table] or jmp table, branches to table as if it were code.
Removing the : and putting the qword (or dq could be used) on the same line, changes table to type qword, so jmp [table] or jmp table, loads the qword address at table into RIP and does the branch as wanted.
table   qword   subroutine, subroutine2
However, if you want to index into table, you'll either need to use a register to hold the offset of the table (like lea r9,table), or in the case of Visual Studio, go to project / properties / linker / system / enable large addresses : no (sets linker parameter /LARGEADDRESSAWARE:NO). I posted examples for both cases below.
This example works with ML64.EXE (MASM) from Visual Studio. The table can be in code or data section. If table is first line in data, lea generates {4C 8D 0D 79 E5 00 00}, if table is first line in code, lea geneates {4C 8D 0D E1 FF FF FF}. I don't know which is better for performance. It would seem that if the data cache is not being fully utilized, then it would keep a copy of the table the data cache.
        .data
tbl     dq      fun1, fun2, fun3            ;table
        .code
main    proc
        lea     r9,tbl
        mov     rax,0
main0:  jmp     qword ptr [r9+rax*8]
main1:: inc     rax
        cmp     rax,3
        jb      main0
        xor     eax,eax
        ret
main    endp
fun1    proc
        mov     rdx,1
        jmp     main1
fun1    endp
fun2    proc
        mov     rdx,2
        jmp     main1
fun2    endp
fun3    proc
        mov     rdx,3
        jmp     main1
fun3    endp
        end
With Visual Studio linker parameter /LARGEADDRESSAWARE:NO, there no need to use a second register. The table can be in data or code section. If table is first line in data, jmp generates {FF 24 C5 00 00 3D 00}, if table is first line in code, jmp geneates {FF 24 C5 80 1A 2D 01}. I don't know which is better for performance. It would seem that if the data cache is not being fully utilized, then it would keep a copy of the table the data cache.
        .data
tbl     dq      fun1, fun2, fun3            ;table
        .code
main    proc
        mov     rax,0
main0:  jmp     qword ptr [tbl+rax*8]
main1:: inc     rax
        cmp     rax,3
        jb      main0
        xor     eax,eax
        ret
main    endp
fun1    proc
        mov     rdx,1
        jmp     main1
fun1    endp
fun2    proc
        mov     rdx,2
        jmp     main1
fun2    endp
fun3    proc
        mov     rdx,3
        jmp     main1
fun3    endp