How to avoid forgetting dependencies in make/CMake?

Question

I am new to C++ and am trying to get the hang of build systems like make/CMake. Coming from Go, it seems that there is a constant risk that if you forget to do a little thing, your binaries will become stale. In particular, I can't find a best practice for remembering to keep dependencies/prerequisites updated in make/CMake. I'm hoping I am missing something obvious.

For example, suppose I have a basic makefile that just compiles main.cpp:

CFLAGS = -stdlib=libc++ -std=c++17

main: main.o
    clang++ $(CFLAGS) main.o -o main

main.o: main.cpp
    clang++ $(CFLAGS) -c main.cpp -o main.o

main.cpp:

#include <iostream>

int main() {
    std::cout << "Hello, world\n";
}

So far so good; make works as expected. But suppose I have some other header-only library called cow.cpp:

#include <iostream>

namespace cow {
    void moo() {
        std::cout << "Moo!\n";
    }
}

And I decide to call moo() from within main.cpp via `include "cow.cpp":

#include <iostream>
#include "cow.cpp"

int main() {
    std::cout << "Hello, world\n";
    cow::moo();
}

However, I forget to update the dependencies for main.o in makefile. This mistake is not revealed during the obvious testing period of running make and rerunning the binary ./main, because the whole cow.cpp library is directly included in main.cpp. So everything seems fine, and Moo! is printed out as expected.

But when I change cow.cpp to print Bark! instead of Moo!, then running make doesn't do anything and now my ./main binary is out of date, and Moo! is still printed from ./main.

I'm very curious to hear how experienced C++ devs avoid this problem with much more complicated codebases. Perhaps if you force yourself to split every file into a header and an implementation file, you'll at least be able to quickly correct all such errors? This doesn't seem bulletproof either; since header files sometimes contain some inline implementations.

My example uses make instead of CMake, but it looks like CMake has the same dependency listing problem in target_link_libraries (though transitivity helps a bit).

As a related question: it seems like the obvious solution is for the build system to just look at the source files and infer dependencies (it can just go one level in and rely on CMake to handle transitivity). Is there a reason this doesn't work? Is there a build system that actually does this, or should I write my own?

Thanks!

Answer 1

First of all you will need to reference the dependencies file in your Makefile.

This can be done with the function

SOURCES := $(wildcard *.cpp)
DEPENDS := $(patsubst %.cpp,%.d,$(SOURCES))

wich will take the name of all *.cpp files and substitute and append the extension *.d to name your dependency.

Then in your code

-include $(DEPENDS)

- tells the Makefile to not complain if the files do not exist. If they exist they will be included and recompile your sources properly according to the dependencies.

Finally the dependencies can be created automatically with the options: -MMD -MP for the rules to create the objects file. Here you can find a complete explanation. What generates the dependencies is MMD; MP is to avoid some errors. If you want to recompile when system libraries are updated use MD instead of MMD.

In your case you can try:

main.o: main.cpp
    clang++ $(CFLAGS) -MMD -MP -c main.cpp -o main.o

If you have more files it is better to have a single rule to create object files. Something like:

%.o: %.cpp Makefile 
    clang++ $(CFLAGS) -MMD -MP -c $< -o $@

You can take a look also at this 2 great answers:

• one
• two

In your case a more suitable Makefile should look like the following (there might be some errors but let me know):

CXX = clang++
CXXFLAGS = -stdlib=libc++ -std=c++17
WARNING := -Wall -Wextra

PROJDIR   := .
SOURCEDIR := $(PROJDIR)/
SOURCES   := $(wildcard $(SOURCEDIR)/*.cpp)
OBJDIR    := $(PROJDIR)/

OBJECTS := $(patsubst $(SOURCEDIR)/%.cpp,$(OBJDIR)/%.o,$(SOURCES))
DEPENDS := $(patsubst $(SOURCEDIR)/%.cpp,$(OBJDIR)/%.d,$(SOURCES))

# .PHONY means these rules get executed even if
# files of those names exist.
.PHONY: all clean

all: main

clean:
    $(RM) $(OBJECTS) $(DEPENDS) main

# Linking the executable from the object files
main: $(OBJECTS)
    $(CXX) $(WARNING) $(CXXFLAGS) $^ -o $@

#include your dependencies
-include $(DEPENDS)

#create OBJDIR if not existin (you should not need this)
$(OBJDIR):
    mkdir -p $(OBJDIR)

$(OBJDIR)/%.o: $(SOURCEDIR)/%.cpp Makefile | $(OBJDIR)
    $(CXX) $(WARNING) $(CXXFLAGS) -MMD -MP -c $< -o $@

EDIT to answer comments As another question, is there any problem with rewriting the DEPENDS definition as just DEPENDS := $(wildcard $(OBJDIR)/*.d)?

Nice question, it took me a while to see your point

From here

$(wildcard pattern…) This string, used anywhere in a makefile, is replaced by a space-separated list of names of existing files that match one of the given file name patterns. If no existing file name matches a pattern, then that pattern is omitted from the output of the wildcard function.

So wildcard return a list of the file names matching the pattern. patsubst acts on strings, it does not care about what are those strings: it is used as a way to create the file names of the dependencies, not the files themselves. In the Makefile example that I posted DEPENDS is actually use in two cases: when cleaning with make clean and with include so in this case they both work because you are not using DEPENDS in any rule. There are some differences (I tried to run and you should too to confirm). With DEPENDS := $(patsubst $(SOURCEDIR)/%.cpp,$(OBJDIR)/%.d,$(SOURCES)) if you run make clean dependencies *.d that do not have a correspondent *.cpp file will not be removed while they will with your change. On the contrary you might include dependencies not relevant to your *.cpp file.

I asked this questions: let's see the answers.

If the .d files get deleted from carelessness but the .o files remain, then we are in trouble. In the original example, if main.d is deleted and then cow.cpp is subsequently changed, make won't realize it needs to recompile main.o and thus it will never recreate the dependency file. Is there a way to cheaply create the .d files without recompiling the object files? If so then we could probably recreate all the /.d files on every make command?

Nice question again.

Yes, you are right. Actually it was an error of mine. This happens because of the rule

main: main.o
    $(CXX) $(WARNING) $(CFLAGS) main.o -o main 

actually should have been:

main: $(OBJECTS)
    $(CXX) $(WARNING) $(CXXFLAGS) $^ -o $@

so that it got relinked (the executable is updated) whenever one of the objects change and they will change whenever one their cpp file change.

One problem remains: if you delete your dependencies but not the objects, and change only one or more header files (but not the sources) then your program is not updated.

I corrected also the previous part of the answer.

EDIT 2 To create the dependencies you can also add a new rule to your Makefile: here is an example.