Don’t Lie To Your Build System¶
Yesterday, in Make Concepts, I talked about what Make does. Today, I’m going to talk about how Make gets abused.
.PHONY Targets¶
That last post ended with a short summary of the common features you
need to be able to understand the majority of most Makefiles, but it
lied a bit. There’s one more thing: .PHONY. It’s a target that
tells Make that its dependencies aren’t actually files. It’s also not
actually a file. You’ll see this pretty frequently in real Makefiles:
.PHONY: clean all
all: things you actually care about building
# ...
clean:
rm -f things you actually care about building
There are no files actually named .PHONY, clean, or
all. Make knows this, so when it goes to build clean, it doesn’t
skip the recipe if there happens to be a file named clean. You
can read more about why phony targets exist in the GNU Make manual
but I’m here to talk about times when they’re not used but should be,
or when they are used but shouldn’t be.
That all target is very convenient. Remembering the names of all
the things I want to build is hard, particularly when I’m running a
complex build system whose ultimate output is a file called
out/lib/my_project_v1.7.328.a and maybe another file called
out/bin/fancy_program and a bunch of tests whose results are in
files called things like out/test_results/test_invert_frob.xml. I just want to write make lib or make exe or make test
and have Make build everything it needs for me, rerun tests as
appropriate, and put the files where they should be.
This is all good and proper. I can write some rules like
.PHONY: test
test: $(TEST_RESULTS)
out/test_results/test_%.xml: out/tests/test_%
$^ > $@
out/tests/test_%: tests/test_%.c
$(CC) $(CFLAGS) -o $@ $^ $(TEST_LFLAGS)
If I build my $(TEST_RESULTS) variable sensibly, I can add tests
easily and any time I update my test source code, I can just run
make test and I’ll automatically get new values in my test result
files for any affected tests. There’s even some weird magic
built into GCC that will parse my test .c file and generate a bunch of
dependencies for any headers it includes.
Lying With .PHONY¶
But what if I get lazy, or have the wrong model of what Make is doing, and instead just do something like this:
.PHONY: test
test:
$(CC) $(CFLAGS) -o test1 test1.c $(TEST_LFLAGS)
$(CC) $(CFLAGS) -o test2 test2.c $(TEST_LFLAGS)
$(CC) $(CFLAGS) -o test3 test3.c $(TEST_LFLAGS)
./test1 > test1.xml
./test2 > test2.xml
./test3 > test3.xml
Or maybe instead something like this:
.PHONY: gen all
all: gen final_exe
gen:
generate_code > generated.c
final_exe: $(SOURCE_FILES)
$(CC) $(CFLAGS) $(IFLAGS) -o $@ $^ $(LFLAGS)
Now, Make will happily run the test recipe for you, and when you run
make all you’ll get final_exe out. But it will be a
lie. Make’s whole thing is making files from other files, and these
examples completely circumvent that thing. These would be better off
as shell scripts (or properly written Makefiles) because when someone
else comes along to try to extend them, they will fail in frustrating
and unexpected ways.
For example, if someone runs make -j all instead of make all
in the second example, sometimes it will fail to build. Parallelizing
Make is a common tactic for speeding up the build. Its ability to do
this is arguably Make’s single killer feature. I once worked at a
company whose build took around 30 minutes. Fixing up the Makefiles
and flipping on -j 9 dropped that to around 2 minutes for a full
build, and incremental rebuilds took 1 to 2 seconds unless they
touched widely used and rarely modified header files.
I don’t want to argue that you shouldn’t lie to Make because it will run faster, though. It will, but some builds are either so fast that it doesn’t matter or so slow that it makes no difference. I’m not really even arguing that you shouldn’t lie to Make because it will lead to subtle bugs. It will though, because having a file you modified not get rebuilt is a frustrating problem to have.
You shouldn’t lie to Make because it will simplify your life. Adding tests that depend on your build artifacts is simple when your build artifacts are correctly specified in your Makefile. I sometimes have to do something like the following:
Modify some code.
Rebuild the firmware.
Flash the firmware onto a dev board.
Run a particular test against the recently flashed dev board.
Make can simplify all of this into a single command. If I only care
about the one test that’s failing, I can just run make
test_that_fails.log and it will do steps 2-4 for me.
Signs That Your Build System Is Being Deceived¶
Alright, so you’re convinced that you shouldn’t lie to your build system, but you’ve inherited it and you’re not sure whether it needs rejiggering or not. Here are some indicators.
People routinely run make clean before re-building.¶
If it’s normal to run make clean before building code, you might be
lying to your build system. Either that, or you should ask your IT
people to get the fileservers synchronized with NTP.
Phony targets have recipes.¶
If your convenience shortcut targets actually have stuff being done in
them, you’re probably lying to your build system. The only thing an
all target should generally have is a list of dependencies.
.PHONY: all
all: thing1 thing2
Big red flag that anything is here
More than one thing is getting built per recipe.¶
This isn’t a guaranteed indication that something’s wrong; some tools generate lots of outputs (like compile stages where you keep intermediate files to be used by other tooling) but it’s definitely suspicious.
In particular, if you’re running one command to generate one file and another command to generate another file in the same recipe, they should be broken up.
main_thing: $(SOURCE_THINGS)
$(COMPILE) -o $@ $^
also do something here > Shiny_Red_Flag
Conclusion¶
That’s it. Tell your Makefile about the files it’s making and your life will be better. Treat it like a fancy shell script with different options to run different functions and your life will be worse. More importantly, if I work with you, my life will be worse.