You need to use pointer-width registers in addressing modes.  x86 doesn't have memory-indirect addressing modes, only register-indirect.  Referencing the contents of a memory location. (x86 addressing modes).  The limits of what you can do in one instruction come from what machine-code can represent.  An assembler isn't a compiler; each line has to work as a single machine instruction.
Ideally keep pixel_x in a register instead of memory at all; that's what registers are for.
Assuming 32-bit code,
   movzx eax, byte [pixel_x]
   movzx eax, byte [bytemap + eax]     ; AL = EAX = bytemap[pixel_x]
You can of course use a different reg like EBX if you want the pixel_x value around in a register for something else later.
Or in this case, imul eax,eax because the array entries are just index-squared; you don't need a lookup table.
In 64-bit code, you'd use default rel so movzx eax, byte [pixel_x] uses a RIP-relative addressing mode.  And you might need to get bytemap's address into a separate register in code where static addresses aren't guaranteed to fit in a 32-bit sign-extended displacement.
In 16-bit code (that can assume 386 compatible), you need to deal with the limitations of 16-bit addressing modes: only BX,BP, SI, and DI can be base or index registers.  NASM x86 16-bit addressing modes
   movzx bx, byte [pixel_x]
   movzx ax, byte [bytemap + bx]      ; AL = AX = bytemap[pixel_x]
If (unlikely) your code needs to run on 8086 to 286, you need to emulate movzx.
On modern x86 in 16-bit mode, using EBX and EAX as the destinations for movzx might help performance, but costs code size.  If you're writing 16-bit code, you probably don't care about speed, just code-size.  If performance mattered, you'd switch to protected mode or long mode.
movzx is the best / most-efficient way to load a single byte, zero-extending to a full register to avoid partial-register performance problems like false dependencies.  Only use mov al, [mem] when you actually want to merge into the low byte of EAX/RAX.
For byte stores, you still just read the partial register like mov [mem], al.
Reading partial regs is fine, just generally avoid writing them when you can use movzx instead.  Something like add al, [mem] is also efficient on most CPUs.  Why doesn't GCC use partial registers?
In general it's not a disaster to use partial registers, just avoid it when you easily can by using movzx or movsx instead of mov.