0
LEA          ECX, Perm[3000]

START:       MOV EAX, N           
             MOV EBX, EAX
             MOV Perm[3000 + EBX * 4], EBX
FACT :       DEC EBX
             MOV Perm[3000 + EBX * 4], EBX
             TEST EBX, EBX
             JZ FACTEND
             MUL EBX
             JMP FACT
FACTEND :    MOV Num, EAX

This is a snippet of a code where I found this problem.

The code runs fine, but if I put a breakpoint at TEST EBX, EBX or the second MOV Perm[3000 + EBX * 4], EBX I get errors, but not if I put it at the first MOV Perm

What is happening?

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
JustJohn
  • 59
  • 1
  • 7
  • How do you declare/define `Perm`? Does it have space for accessing a byte offset of `Perm + 3000 + factorial(N)*4`? (Both being plus seems strange, and the LEA result in ECX is unused). IDK why visual studio's debugger would map memory differently depending on breakpoint setting, but accessing memory outside of what you've reserved can fault or not depending on whether it happens to touch an unmapped page. If the kernel decided to use a 2MiB largepage that extends farther than the end of a 4k page past your array, that could possibly explain not faulting sometimes. – Peter Cordes Jun 11 '21 at 01:51
  • @PeterCordes Perm is defined as follows : int Perm[4326]; Now I'm thinking it might have to do with this warning Warning C6262 Function uses '17324' bytes of stack: exceeds /analyze:stacksize '16384'. Consider moving some data to heap. – JustJohn Jun 11 '21 at 02:17
  • Correction to my last comment, the highest byte offset you index in Perm is `3000 + N*4`. factorial(N) is only in the values you store. – Peter Cordes Jun 11 '21 at 02:28
  • Oh, you had tagged this inline asm, but since there wasn't any surrounding C code, I assumed it was actually masm. If `Perm` is a local var on the stack in a C function, then you could be running into Windows stack growth effects, maybe touching not touching the guard page to trigger stack growth if N isn't huge. Probably initializing Perm in C first like `int Perm[4326] = {0};` would get the compiler to touch all the memory safely but inefficiently. Obviously the surrounding C is a key part of the [mcve] here! – Peter Cordes Jun 11 '21 at 02:28
  • If that guess is right, IDK why it wouldn't touch pages to grow the stack safely when entering the function. That's just a warning, not an error; Windows uses I think a 1MiB stack for the main thread in a user-space process. But it can only grow by 1 page at a time, so functions with large arrays need to call `__chkstk()`. ([How Windows thread stack guard page mechanism works in case of uninitialized local variables?](https://stackoverflow.com/a/31086936)). The compiler should do this for you, but MSVC's support for inline asm is clunky and nasty so it's certainly possible it fails to do th – Peter Cordes Jun 11 '21 at 02:33
  • 1
    The relevant compiler is [msvc]. That tag is a synonym for [visual-c++], even when the question is actually about MS's compiler in C mode, C++ mode. – Peter Cordes Jun 14 '21 at 13:10
  • I see, I must've missclicked. I correctly flagged the question as C now – JustJohn Jun 14 '21 at 19:31
  • 1
    Stop deleting the [visual-c++] tag. I've had to re-add it twice now, not including when I originally added it. This is MSVC inline asm, and [visual-c++] is the right tag for that compiler, even in C mode. You apparently misread my previous comment, where I explained why I re-added it – Peter Cordes Jun 14 '21 at 19:35
  • That I did, my apologies – JustJohn Jun 14 '21 at 19:37

0 Answers0