I think I was complicating things unnecessarily. The reason I wanted to check for specific headers is because in Ubuntu 18.04 I had the mpfr.hpp header but not the mpc.hpp one. Later I found out that this is because the latter is included since version 1.69 of Boost, so adding VERSION 1.69 to find_package(...) would've been enough for me.
Nevertheless, a solution using check_include_file_cxx is possible, as @Tsyvarev pointed out.
The macro check_include_file_cxx not only checks if the required header is available, but also tries to compile a small test program. For that, it also needs to link to libraries (-lmpfr for boost/multiprecision/mpfr.hpp and -lmpfr -lmpc for boost/multiprecision/mpc.hpp).
This did the trick for me.
find_package(Boost REQUIRED
VERSION 1.69)
if ( Boost_FOUND )
include_directories(${Boost_INCLUDE_DIRS})
endif()
include(CheckIncludeFileCXX)
cmake_policy(SET CMP0075 NEW) # Required for specifying CMAKE_REQUIRED_LIBRARIES
set(CMAKE_REQUIRED_INCLUDES "${Boost_INCLUDE_DIRS}")
set(CMAKE_REQUIRED_LIBRARIES "mpfr" "mpc" "gmp")
set(REQUIRED_BOOST_HEADERS
"boost/multiprecision/mpfr.hpp;boost/multiprecision/mpc.hpp"
)
foreach(HEADER ${REQUIRED_BOOST_HEADERS})
check_include_file_cxx("${HEADER}" ${HEADER}_FOUND)
if (NOT ${HEADER}_FOUND)
message(FATAL_ERROR "HEADER ${HEADER} NOT FOUND")
endif()
endforeach()