0

Should an application calling MessageBoxA() always terminate by ExitProcess() instead of normal function epilogue?

I've trouble understanding why the "MessageBox"-program below hangs after executing the ret instruction on Windows 10 x64.

Debugging the program under GDB, it seems like the Win32 function NtUserSetCursor is the reason for the behavior, i.e., after executing the ret 0x4 instruction nothing happens.

0x779c2440 in ?? ()
=> 0x779c2440:  ff 25 18 12 a5 77       jmp    DWORD PTR ds:0x77a51218
(gdb)
0x6c417000 in ?? ()
=> 0x6c417000:  ea 09 70 41 6c 33 00    jmp    0x33:0x6c417009
(gdb)
0x7476262c in win32u!NtUserSetCursor () from C:\WINDOWS\SysWoW64\win32u.dll
=> 0x7476262c <win32u!NtUserSetCursor+12>:      c2 04 00        ret    0x4

However, if I exit the program using the ExitProcess there are no problems.

Should an application using the MessageBoxA function always terminate by ExitProcess?

Program:

BITS 32


extern _MessageBoxA@16

section .data

_szTitle: db "This is a test", 0x00
_szMsg: db "Hello World!", 0x00

section .text

global _start
_start:
    push ebp
    mov ebp, esp

    push 0x00
    push _szTitle
    push _szMsg
    push 0x00

    call _MessageBoxA@16

    leave
    ret

Assembled and linked:

C:\Users\nlykkei\Desktop>nasm -f win32 test.asm

C:\Users\nlykkei\Desktop>ld -o test.exe test.obj "C:\dev\lib\User32.Lib" --entry _start
Shuzheng
  • 11,288
  • 20
  • 88
  • 186
  • Please don't re-post questions. If you want to change something use the _edit_ link. – Jester Nov 03 '16 at 18:34
  • I deleted the old post - it was badly written and too many mistakes. Now, it should be "clear" and concise. – Shuzheng Nov 03 '16 at 18:35
  • FYI, it works fine using `wine`. You might want to add the windows version you have too. – Jester Nov 03 '16 at 18:37
  • The question also states I run Windows 10 x64 :-) – Shuzheng Nov 03 '16 at 18:39
  • It does, I missed that. – Jester Nov 03 '16 at 18:45
  • application always must call ExitProcess. unconditionally. because if simply exit from _start - ExitThread will be called. so terminated process or no depended from - are other thread exist in process at this time – RbMm Nov 03 '16 at 19:57
  • You are not the only victim, [this bug](http://stackoverflow.com/q/18036863/17034) is back again in Win10. Well, at least you can call ExitProcess(). – Hans Passant Nov 03 '16 at 22:39
  • @HansPassant - thank you for your comment. So one should not call ExitProcess, unconditionally, as the answer suggests? The program should terminate properly without? Also, does code compiled and linked under MS tools always call ExitProcess? – Shuzheng Nov 04 '16 at 07:40
  • I'm 95% sure that this is a bug, no one should *have* to call ExitProcess() just because they used MessageBox(). The issue got resolved pretty quickly back when I contacted Microsoft about it. It is taking too long in Win10 however, version 1607 didn't fix it. So this is starting to look like a hard-to-fix bug, calling ExitProcess() is the proper workaround. – Hans Passant Nov 04 '16 at 08:15
  • @HansPassant: What is the purpose of ExitProcess in general, if a process should terminate properly without? Is the function made to solve problems like this? – Shuzheng Nov 04 '16 at 09:43
  • You got plenty of feedback already about cases where it is expected to be used. – Hans Passant Nov 04 '16 at 09:48
  • @HansPassant - "no one should have to call ExitProcess() just because they used MessageBox()." - you mistaken. what if another threads exits in process at this time ? process not terminated ! win 10 create this aditional worked threads by self. i describe this in answer – RbMm Nov 04 '16 at 14:24
  • 1
    Hmya, who ever expects MessageBox() to create another thread? Or for it to work correctly if he *doesn't* use MessageBox(), even though Win10 still uses the threadpool to load DLLs. This issue has nothing to do with threads. The much more unpleasant side-effect of this issue is that it makes existing programs fail to operate correctly. Like that .NET program I linked to. – Hans Passant Nov 04 '16 at 14:36
  • @HansPassant MessageBox - load dlls. LoadLibrary now create another threads (in win 10) - you can check this by self. all exe already start with several threads (3 i looked) "This issue has nothing to do with threads. " - but if exist threads in process (except first thread which exit) - process not terminated – RbMm Nov 04 '16 at 16:10
  • @HansPassant i think this will be easy to you build app which only `MessageBoxW(0,0,0,0)` called (even without c++ crt for clear effect) and run it on win 10(1607) and look how many and which threads in app. simply test this – RbMm Nov 04 '16 at 16:25
  • @HansPassant, I don't think this is the same problem. That's .NET and this is native. Environment.Exit() should exit the process because that's what it is documented to do, and the same applies if you exit the C runtime's main() but the documentation makes no such promise about the OPs scenario. – Harry Johnston Nov 06 '16 at 21:23
  • 1
    @Shuzheng, your program should *always* call ExitProcess, either directly or indirectly. If you use Microsoft's C runtime library, you can call ExitProcess indirectly by exiting from the main function. In .NET you can call ExitProcess indirectly by exiting from all the foreground threads. But if you are writing in assembler, and don't explicitly link in the C runtime, you have to do it yourself. – Harry Johnston Nov 06 '16 at 21:28
  • 1
    ExitProcess() *should* probably be called as it is the safest all around solution in light of the "new behavior" in Windows 10. With that said, I would still argue this behavior is indeed a bug as @HansPassant pointed out unless someone can show documentation that states native non-C applications must exit with ExitProcess(). The Windows loader should clearly stick to the "safer" behavior that has been present in previous OS versions allowing a process to be cleaned up correctly in both scenarios. – byteptr Nov 07 '16 at 06:34
  • I'm aware that a process won't exit until all of its threads exit and that various unsolicited threads can become active at any time. 1> I would be interested in confirmation that one of these extra threads is causing the OP's process to not exit. 2> I have a hard time believing the OS will start unsolicited threads ultimately preventing the process from exiting AND that the OS will not check back in a reasonable amount of time to terminate that thread. There was a discussion recently about the OS making an internal distinction between these "silent" worker threads to prevent this very thing. – byteptr Nov 07 '16 at 06:44
  • @byteptr - "I have a hard time believing the OS will start unsolicited threads" - in win10 this is usual behavior - worked thread in EVERY process. so now no more single-thread applications. all apps multi-threaded because now DLL load - multi-thread operation – RbMm Nov 07 '16 at 18:13
  • 1
    @byteptr: [this has been happening since at least 2007,](https://blogs.msdn.microsoft.com/oldnewthing/20100827-00/?p=13023/) Windows 10 just makes it more common. See also [If you return from the main thread, does the process exit?](https://blogs.msdn.microsoft.com/oldnewthing/20100827-00/?p=13023/). Programmers are very definitely expected to call ExitProcess if their language doesn't do it for them. – Harry Johnston Nov 07 '16 at 20:47
  • If there's a fault, it is in the documentation, and it would certainly have been nice if they'd mentioned this in the documentation for `/ENTRY` or for ExitProcess(). Still, from a strict documentation-lawyer perspective it never says anywhere that you're allowed to return from the entry point in the first place, so by doing so you're arguably invoking undefined behaviour. :-) – Harry Johnston Nov 07 '16 at 20:49
  • @RbMm - will an application also have multiple threads if no DLLs are loaded? – Shuzheng Nov 07 '16 at 20:50
  • @Shuzheng: there might or might not be API calls that don't load new DLLs but do create extra threads. But in practice the question is moot, because even if your code doesn't load a DLL or create any threads, code injected into your process by a third-party application might do either or both. – Harry Johnston Nov 07 '16 at 20:56
  • @Shuzheng - in every process DLLs loaded. how minimum kernel32.dll and kernelbase.dll - this already causes working thread start. :) we can not assume are system create new working threads – RbMm Nov 07 '16 at 20:56
  • @HarryJohnston - are you know/listen about "Parallel loader" in windows 10 ? i send you mail – RbMm Nov 07 '16 at 20:57
  • @RbMm: yes, I'm aware of that. But the OPs question seems to imply that the problem only occurs if he calls MessageBox. – Harry Johnston Nov 07 '16 at 20:58
  • @HarryJohnston - no this is not true. even if program containing single `ret` instruction - at start point will be several threads active in process. so process not terminated just but after 30 sec – RbMm Nov 07 '16 at 21:02
  • Ive have not tested other calls. – Shuzheng Nov 07 '16 at 21:03
  • @RbMm: OK. I just wanted to make it clear that even if you somehow took care of the threads started by the parallel loader (perhaps just by waiting for them to exit naturally) you still can't safely assume that there won't be any other threads that you don't know about. – Harry Johnston Nov 07 '16 at 21:04
  • @HarryJohnston - of course. can be and additional working threads. so we always must direct or indirect call ExitProcess but not assume about we last thread – RbMm Nov 07 '16 at 21:06
  • i mistake - if application have no imports at all. only single `ret` instruction. loader not create additional threads. and app just exit. only 3 dll in process (ntdll, kernel32 and kernelbase) but if exist import in exe - already worked threads created – RbMm Nov 07 '16 at 21:58

1 Answers1

2

application terminated in three cases:

  1. all thread ended
  2. application call ExitProcess
  3. somebody call TerminateProcess

about 1 case - frequently windows by self created worked threads in your app. this is very depended from what you doing and windows version. so even if you have single thread application - you can not be sure that no other thread started in your process. in your case after MessageBoxA you returned to kernel32.dll code (say BaseThreadInitThunk in win 8.1) wich call ExitThread. but not ExitProcess. so very possible your application not terminated. we always must call ExitProcess. if we use c/c++ crt - runtime called ExitProcess when we return from main or WinMain. you not use crt - so must by self direct call ExitProcess

-------------- EDIT -------------------------------

i do some research om win 10 (1607). at first now ntdll in initialize phase call EtwEventRegister(UserDiagnosticGuid , UserDiagnosticProviderCallback);

internally this call TpAllocWait and 2 worked thread (TppWorkerThread) created.
enter image description here so every, even single-thread designed application started how minimum with 3 threads (2 worked threads wait in ZwWaitForWorkViaWorkerFactory with WrQueue KWAIT_REASON)

also now LoadLibrary sometimes create worker threads. LdrpMapAndSnapDependency -> LdrpQueueWork -> TpPostWork(when exist cyclic module dependencies i guess - like user32 <-> gdi32 ).

also when MessageBox exited - ole32.dll loaded in process and working thread is created - here call stack:

enter image description here

as result, after MessageBox returned - we have 2 thread in process - your and woking thread, which called LdrpWorkCallback and then wait near 1 minute in ZwWaitForWorkViaWorkerFactory before exit

enter image description here

as result when return from _Start -ExitThread (but not ExitProcess) was called. but process not terminated - another thread live in this process. and ~ 1 min late - when worked thread is exit - process finally terminated

RbMm
  • 31,280
  • 3
  • 35
  • 56
  • Thank you for great answer. Do you have a source I can read, or is your knowledge from personal experience? – Shuzheng Nov 03 '16 at 20:07
  • 1
    @Shuzheng https://msdn.microsoft.com/en-us/library/windows/desktop/ms686722(v=vs.85).aspx - in your case you assume that your thread - single in process - "The last thread of the process terminates" - but this can be not true. kernel32 on return from your EP call ExitThread but not ExitProcess. so result depended - are you last thread – RbMm Nov 03 '16 at 20:14
  • What is "EP"? :-) – Shuzheng Nov 03 '16 at 20:37
  • @Shuzheng - EP - entry point i mean. _start in your code – RbMm Nov 03 '16 at 20:58
  • can you say why GDB stops in NtUserSetCursor? – Shuzheng Nov 04 '16 at 07:35
  • @Shuzheng - don`t know about NtUserSetCursor - in my test on win 10 - your thread is exit correctly but process not exit, because another thread live in it. i describe this - edit answer. – RbMm Nov 04 '16 at 14:22
  • What tool do you use to see which threads live and which threads exit correctly? – Shuzheng Nov 04 '16 at 14:24
  • @Shuzheng - my own ( – RbMm Nov 04 '16 at 14:25
  • @Shuzheng - if you have good debugger and experience in this - no any problem to see theads, callstack,which thread exit and which alive.. i think you incorrect look in GDB (or he lie) - your GUI thread exit but working thread executed – RbMm Nov 04 '16 at 14:29
  • @Shuzheng - and try wait 1 minute, after you close MessageBox - process exit after this period ? – RbMm Nov 04 '16 at 14:31
  • Yes it does, but with wrong exit code. – Shuzheng Nov 07 '16 at 06:33
  • @Shuzheng - of course. because in current case exit code of process - is exit code of last exited thread. but not what you return from _start. look like almost nobody know that in win10 added new feature - loading dlls in parallel. now several threads loading dll. as result singlethread application no more exist in win10. all aps multi-threaded. however loader worker threads (`LdrpWorkCallback`) must exit after 30 second if no jobs. – RbMm Nov 07 '16 at 08:58
  • 30 secs? Don't you mean 1 min? – Shuzheng Nov 07 '16 at 14:37
  • @Shuzheng - function `NTSTATUS LdrpEnableParallelLoading (ULONG LoaderThreads)` which init Parallel loader create thread pool `LdrpThreadPool` - and then set timeout for this pool - `TpSetPoolWorkerThreadIdleTimeout(LdrpThreadPool, -300000000);// 30 second` - so if no 30 second jobs - thread exit – RbMm Nov 07 '16 at 14:56