0

I was trying to decode the following hex numbers:

3b 5c 78 f6

I checked the manual of Intel, and 3b refers to the opcode of CMP Gv,Ev.

5c is 01 011 100in binary, meaning MOD=01, REG =011, R/M=100.

MOD =01 means this instruction has one-byte displacement. REG=011 means, the instruction compares %ebx with something. Here is the part I don't understand. When R/M =100 refers to a register, %esp, if I am correct, how do I use this register in this instruction? The reason why I am confused is that the SIB and displacement (7b f6)combined together is something like 0x-a(%exa,%edi,2) in ATT syntax, so I think the instruction should be something like cmp 0x-a(%eax,%edi,2),%ebx in att syntax.

I am not sure if my decoding process was correct. If the process was correct, what does R/M=100 refer to? What is the purpose of R/M here?

Sunderam Dubey
  • 1
  • 11
  • 20
  • 40
Chen Yao
  • 35
  • 5
  • 1
    https://defuse.ca/online-x86-assembler.htm#disassembly2 shows it's `cmp ebx,DWORD PTR [eax+edi*2-0xa]` aka AT&T `cmp -0xa(%eax, %edi, 2), %ebx`. Don't forget the disp8 = `f6` is a 2's complement 8-bit value to be sign-extended, not zero-extended. – Peter Cordes Feb 06 '23 at 16:19
  • 1
    base=ESP in ModRM is the escape code for a SIB where you can specify base=ESP just fine (but not index=ESP, that encoding actually means no index.) Since most code keeps the stack pointer in ESP, this isn't a problem, you wouldn't want to scale it anyway. If you want to use all 8 registers for something, allocate them so ESP doesn't have to be an index. See [rbp not allowed as SIB base?](https://stackoverflow.com/q/52522544) – Peter Cordes Feb 06 '23 at 16:26
  • 1
    If you're curious how to encode something like `cmp (%esp), %ebx`, ask an assembler. x86 isn't a new ISA; many well-tested assemblers exist that can make machine code for you that you can trust. If you're ever in doubt (e.g. about x87 `fsubr` where [AT&T syntax design bugs](https://sourceware.org/binutils/docs/as/i386_002dBugs.html) make things extra weird), assemble with one assembler and disassemble with something from a different software package, e.g. assemble with GAS and disassemble with `ndisasm` or Agner Fog's `objconv`. – Peter Cordes Feb 06 '23 at 16:31
  • @PeterCordes Yes you are right it should be -0xa. However, in this case, I thought base is ```%eax``` according to SIB = 78H here. 78H = 01 111 000 B. I think it means scale =01 index =111 base =000. The part I don't understand is ModRM byte. Specifically R/M part in ModRM. Did you mean R/M=100 is just a escape code? – Chen Yao Feb 06 '23 at 16:38
  • Yes, ModRM.r/m = ESP (with .mode=anything other than register) is the escape code for the presence of a SIB, as explained in the linked duplicate. And yes, the base of the addressing mode is indeed EAX. The "destination" of the `cmp` is the register EBX, from ModRM.reg – Peter Cordes Feb 06 '23 at 16:53

0 Answers0