How can i make "stack oveflow" exception on x64

Question

I tried to make a "Stack overflow" exception.

This code throws an exception on x64 Debug.

void DumpSystem::makeStackOverflow()
{
    static int callCount = 0;
    ++callCount;
    makeStackOverflow();
}

But, This code don't throws an exception on x64 Release The x64 Release xxx.exe was LOOP without causing a "Stack Overflow" exception.

Build Option : "SEH(/EHa)"

I want to create a "Dump File" using "SetUnhandledExceptionFilter".

This is the code I used

LONG saveDumpfile(EXCEPTION_POINTERS* ex);

unsigned __stdcall saveDumpFileForStackOverflow(void* arg)
{
    EXCEPTION_POINTERS* ex = static_cast<EXCEPTION_POINTERS*>(arg);
    return saveDumpfile(ex);
}

LONG exceptionHandling(EXCEPTION_POINTERS* ex)
{

    if (ex &&
        ex->ExceptionRecord &&
        ex->ExceptionRecord->ExceptionCode == EXCEPTION_STACK_OVERFLOW)
    {

        HANDLE hThread = (HANDLE)_beginthreadex(NULL, 0,
            saveDumpFileForStackOverflow, ex, NULL, NULL);
        WaitForSingleObject(hThread, INFINITE);
        CloseHandle(hThread);
        return EXCEPTION_EXECUTE_HANDLER;
    }

    return saveDumpfile(ex);
}

void registrationDumpSystem()
{
    ::SetUnhandledExceptionFilter(exceptionHandling);
}

LONG saveDumpfile(EXCEPTION_POINTERS* ex)
{
    if (ex == NULL)
        return EXCEPTION_EXECUTE_HANDLER;

    LONG result = EXCEPTION_EXECUTE_HANDLER;

    //%APPDATA% : C:\Users\[user name]\AppData\Roaming
    wstring filePath = getAppDataFolderPath();


    SHCreateDirectoryEx(NULL, filePath.c_str(), NULL);


    filePath.append(TEXT("\\Dump.dmp"));
    HANDLE file = CreateFile(filePath.c_str(),
        GENERIC_WRITE,
        FILE_SHARE_WRITE,
        NULL,
        CREATE_ALWAYS,
        FILE_ATTRIBUTE_NORMAL, NULL);

    if (file == INVALID_HANDLE_VALUE)
    {
        DWORD lerr = GetLastError();
        return lerr;
    }

    HANDLE processHandle = GetCurrentProcess();
    DWORD processId = GetCurrentProcessId();

    MINIDUMP_EXCEPTION_INFORMATION mei;
    mei.ThreadId = GetCurrentThreadId();
    mei.ExceptionPointers = ex;
    mei.ClientPointers = false;

    MiniDumpWriteDump(processHandle, processId, file,
        MiniDumpNormal, &mei, NULL, NULL);

    CloseHandle(file);

    return result;
}

main.cpp

int APIENTRY _tWinMain(HINSTANCE hInstance,
    HINSTANCE hPrevInstance,
    LPTSTR    lpCmdLine,
    int       nCmdShow)
{
    registrationDumpSystem();
    //to do
    return 0;
}

"x64 Debug exe" it is work. so i can make "dump.dmp" file.

but, "x64 release exe" is't work. i can't make "dump.dmp" file

enter image description here I want to know why the program does not exit on x64 release.

please, could you tell me this reason?

Answer 1

You're making a common mistake, which is to think of your C++ source code as a one-to-one mapping of human instructions to machine instructions. It isn't. It's a description of a program. The process of converting this description into an actual program that the computer can execute is very complicated; naively we say that modern compilers "optimize" the code, but that's really a backwards way to look at it.

In fact, the compiler will try to create a program that does what you wanted it to do, producing the best code possible given those constraints (or slightly worse code, if you asked for a "low optimisation level", resulting in code that more closely matches your source and thus permits more convenient debugging). In this case, you simply asked to perform the behaviour of that function (which is effectively nothing) repeatedly and infinitely.

Now, if the code were directly converted into a recursive sequence of jumps and whatnot, you'd end up with a stack overflow because you'd run out of stack space for all the function contexts.

But! "Tail call optimisation" exists. This is a thing that compilers can do, in certain circumstances, to produce something more resembling a loop than a nest of recursive calls. In the resulting code, there is no infinite stack required and thus no exception.

As explored above, dropping optimisation levels down (which is what a debug build tends to involve), will result in "worse" code more closely matching the specific source that you wrote, and it seems that you are seeing the effect of this in your debug build: an actual recursion is produced even though it doesn't need to be. Since the recursion is infinite, your program crashes.

Answer 2

void DumpSystem::makeStackOverflow()
{
    static int callCount = 0;
    ++callCount;
    makeStackOverflow();
}

Can be optimized to something like

void DumpSystem::makeStackOverflow()
{
    static int callCount = 0;
    while (true) ++callCount;
}

with tail call optimization and if that is what your compiler is doing then you will never get a stack overflow. You wouldn't see that in debug mode because debug mode normally does no optimizations.

If you want to force a crash there are many ways presented in: What is the easiest way to make a C++ program crash?