Mike Bland

Instigator

Makefile Refactoring

I'm experimenting with refactoring OpenSSL's existing recursive Make structure into a top-Makefile-with-includes structure.

- Boston
Tags: OpenSSL testing, programming, technical
Discuss: Discuss "Makefile Refactoring" on Google+

In the makefiles branch of my OpenSSL fork, I’m experimenting with refactoring the existing recursive make structure into a Makefile-with-includes structure. I believe this is a critical first step towards improving automated test coverage and instilling a testing culture.

Motivation
Refactoring Steps
Side Effects

Motivation

I started down this path when writing a small test in my encode-test branch. Not the most important test to write, but it was something to sink my teeth into to get better acquainted with the code and try out more of my testing ideas. When I tried to change crypto/evp/encode.c to make the test fail—and it didn’t—I realized that make test didn’t rebuild the crypto library on which the test depends. The paper Recursive Make Considered Harmful explains why: The top-level Makefile recursively invokes make in the test/ directory. The make process parsing test/Makefile doesn’t have the full dependency graph information, therefore it doesn’t know that it needs to rebuild libcrypto when rebuilding a test.

The first, most obvious workaround is to always invoke make && make test. The second, less-obvious way is to use the top-level GitConfigure and GitMake scripts instead; GitConfigure will compile a single Makefile out of all of the sub-Makefiles with the correct dependency info (mostly).

When trying to encourage testing as a regular habit, any friction in the build process works against that goal. The make && make test solution is inefficient, increasing the length of the feedback loop due to unnecessary work being done (i.e. tons of unnecessary recursive make processes), and error-prone. The GitConfigure/GitMake solution works, but in both cases, the need to resort to anything other than a straightforward make test violates the principle of least surprise. This is especially problematic right now, as I’ve got a good number of volunteers ready and willing to help improve OpenSSL’s testing, but who are struggling to understand the code and how they can best contribute. I feel like they’re waiting for me to clear a path, and the first obstacle isn’t the code itself, but the recursive make structure.

Refactoring Steps

The end result I’m aiming for is to have a top-level Makefile include all of the other Makefiles, as well as dependency files for each object file generated as a side-effect of compilation, so that rebuilding and executing a test is fast and correct. It will remain compatible with GitConfigure and GitMake for those who prefer to use them.

I’m not going to go into excruciating detail here, but suffice it to say this will happen over a long series of changes, and I’ll constantly rebase them on updates to the master branch. Each change will maintain compatibility with the existing recursive make structure until the final switch is flipped. I’ll take care to support both GNU and BSD make, and ensure that everything works on Mac OS X, Linux, and FreeBSD at the very least. Now that I’m also running Windows 8.1, I’ll be able to check my changes for Windows compatibility, too.

I wrote a Python script to automate the changes now evident in my makefiles branch. I developed it to apply the changes one step at a time, and made sure that, after each step, running the script multiple times didn’t alter the Makefiles after the first run. As I’m now going back to redo many of those changes in a different order, I’m going to be building the script back up again, this time checking it into my personal Google Code repository as update_makefiles.py. Since this is a one-off script that only I’m using, I don’t imagine it’ll go into OpenSSL proper—plus, I’m breaking tradition by using Python instead of Perl for this—but this way folks can take a look at how I’m going about the process if they’re curious.

Side Effects

Even if the core team ultimately decides to reject my experiment, it’s already paid off enormously. First of all, now I know many of the problems that could frustrate people helping with the testing effort. If we can’t fix these issues, we can at least make sure they’re clearly documented. Second, as a result of all this digging into the build system, I’ve already issued a pull request to fix several build issues, particularly those that prevented successful OS X builds, since that’s my home platform. These fixes are worth committing independently of all the other experimentation I’ve been doing; they’re small, noncontroversial, and eliminate a great deal of friction in and of themselves.

Third, I’m having way too much fun with this.