With the code in your question you are hiding the default parameters - including the optimization level - that CMake does apply.
Please try appending your options with
SET ( CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /MDd" )
or with the use of generator expressions and add_compile_options():
add_compile_options("$<$<CONFIG:Debug>:/MDd>")
But you may not need this particular option, because /MDd is already part of CMake's default MSVC debug flag's setting.
Background
If you look at CMake's Windows-MSVC.cmake you'll see the following initialization settings:
set(CMAKE_${lang}_FLAGS_DEBUG_INIT "/D_DEBUG /MDd /Zi /Ob0 /Od ${_RTC1}")
Without changing any flags in your CMakeLists.txt you will see in your CMakeCache.txt:
//Flags used by the compiler during debug builds.
CMAKE_CXX_FLAGS_DEBUG:STRING=/D_DEBUG /MDd /Zi /Ob0 /Od /RTC1
With your code you are hiding this cached variable and you will end up with just /MDd.
References