First of all, your book has a mistake - it says that "a in %rsi, b in %rdi" - but this is not the standard x64 calling convention and it is inconsistent with the rest of the assembly.
What the book meant:
- %rdi->- a,- %rsi->- b,- %rdx->- c, and- %rcx->- dest
Moving on, let's understand what happens:
Default Code Block
The first two opcodes are:
cmpq $7, %rdi
ja .L2
ja is jump if above, i.e. if a > 7 then go to .L2 - which is at the end of the assembly. We can deduce that this is the default code block (it continues immediately to the end of the function) - and under .L2 we have:
movq %rsi, %rdi
movq %rdi, %(rcx) ; this corresponds to *dest = val in C
so we can conclude that %(rcx) gets %rsi's value in this case - in other words, in the default code block, val = b.
Switch Code Blocks
If our first ja above did not excute, then we jmp *.L4(,%rdi,8). Since a is not above 7, we have eight possibilities - we can see in the .L4 table that:
- If a == 0then we jump to.L3
- If a == 1,a == 3, ora == 6, we jump to.L2(our default code block, described above)
- If a == 2ora == 7we jump to.L5
- If a == 4we jump to.L6
- If a == 5we jump to.L7
.L3, or case 0
This block runs leaq 112(%rdx), %rdi, which simply has the effect of setting %rdi to %rdx + 112 - which is c + 112. We then jump to the end of the function - we can conclude that val = c + 112 in the case 0 code block.
.L7, or case 5
This block runs leaq (%rdx, %rsi), %rdi, which sets %rdi to %rdx + %rsi (which is c + b) - and then calls salq $2, %rdi, which just shifts this value left by 2 bits - for a total value of (c + b) << 2. We then jump to the end of the function - we can conclude that val = (c + b) << 2 in the case 5 code block.
.L6, or case 4
Here we have jumped immediately to the end of the function, just calling the movq %rdi, (%rcx) opcode - which is effectively equivalent to setting *dest = a. We can conclude that in this case, val = a.
.L7, or case 5
This block runs xorq $15, %rsi - which is equivalent to b ^= 15. It then runs movq %rsi, %rdx - setting c to this value. We then continue straight into .L3 described above - which sets val = c + 112. We can conclude that .L7 is our fall-through switch case.
In general, reversing switch cases can be very straightforward - it mostly involves understanding how the jump table corresponds to different values in the compared register (notice here how several possible values of a mapped to the same jump in the table) - and understanding the fall-throughs between different switch cases.