I am learning Intel X86 Assembly and reverse engineering as I want to create game cheats.
I’m trying to understand memory and how the stack works. I know it’s a LIFO and some people say think of it as a stack of books where you add to the top and take out from the top. But instead of the books going upwards do you have to visualise it going downwards?
For example:
push 1
push 2
push 3
pop x
push would equal:
1
2
3 <- esp
pop would equal:
1
2 <- esp
Vertical up/down are not the only possible visualizations — we can use horizontal as well:
lower addresses ... higher addresses
+----------------------------------------------+
| (free stack) | 3 | 2 | 1 | (in use) |
+----------------------------------------------+
^
|
esp
This visualization allows structs to be viewed in a "forwards" direction, left to right; argument lists that are pushed last to first as well. Push expands the stack leftward, which still feels pretty natural.
YMMV
You can basically visualize the stack however you like. The more often encountered way to represent stack is that a stack is said to grow down, from higher addresses to lower addresses.
To support this narrative, stack is often visualized as
+------------------+ <- top of memory = 0x10000
| used_stack |
| |
+------------------+
| local variables | <- top of stack = 0xff00
| [red zone] | <- red zone is guaranteed to be unused
| unused memory | by exception handlers
| |
| ---------------- |
| HEAP | <- dynamically allocated memory
+------------------+ address = 0x0000
Personally I found this unintuitive, since this reverses the order in which other memory structures or arrays are typically presented. There's no down in a CPU.
| offset description. hex dump
+-----------+----------------+------------------------
| 0 | char header[4] | 01 02 03 04
| 8 | int16 width | 08 00
| 10 | int16 height | 04 00
| 12 | int32 size | 20 00 00 00
vs.
struct FileFormat {
char header[4];
int16_t width{8};
int16_t height{4};
int32_t size{32};
};
When stack is visualized as a stalactite (hangs like an icicle) it physically grows down, but it loses the connection to real life stacks, as in stack of plates or books, which grow up.
A B
+-------------+ +--------------+
| ceil of stk | <-high addresses | unused | <-small addresses
| 20 00 00 00 | | |
| 08 00 04 00 | | |
| 01 02 03 04 | <-- top of stack -> | 01 02 03 04 |
| | | 08 00 04 00 |
| | | 20 00 00 00 |
| unused/heap | |bottom of stck|
+-------------+ +--------------+
The format A is more convenient at least when you have a written text like
push 1
push 2
push 3
In the style A, this is represented as is, in style B, the values are reversed.
The "growing down" is simply used to match how most operating systems set up x86 stacks. The hardware is capable of working with stacks that grow in either direction but the operating system sets the CPU to have the stack use lower and lower memory addresses as more items are pushed onto the stack.
Having stacks grow down was a particularly useful choice when systems were single-CPU and would generally be running single-threaded processes in control of the entire system. The stack was placed at the top of available memory and the heap at the bottom and they would each grow to meet the other. In modern multi-threaded processes it doesn't work quite so simply, each thread requires an independent stack, although the heap can be shared by the entire process. The thread will have some maximum size chosen when the thread is created and if that space is exhausted some kind of error condition will need to be generated. Note, however, that 'thread' and 'process' in this context is entirely up to the OS design, the CPU is simply informed what code to run.
