You don't show your current makefile, but my suspicion is that it's wrong.  However as we can't see it, we'll leave that alone.
The compiler does not support writing multiple object files to a different directory.  If you pass multiple source files along with the -c flag then it will write out multiple object files but only to the current directory... as you've discovered the -o flag can't be specified on compile lines which generate multiple output files.
You can change your recipe to look like this:
cd lib && clang++ $(CC_FLAGS) -c -I../include ../source/*.cpp
this will cause all the object files to be written to the lib directory because it's now the current directory.
However, putting this into a makefile is not simple, because make itself is designed to have a single recipe create a single target.  However you have this problem with your existing makefile which you don't show, as well.