Mike Bland

Call for OpenSSL Testing Help

The OpenSSL team has accepted my offer to help improve its unit/automated testing, and now I'm recruiting people to help in the effort.

- Boston
Tags: Heartbleed, OpenSSL testing, goto fail

This formal recruiting letter is also available in a Google Doc version.

My name is Mike Bland. During my time at Google from 2005-2011, I helped change the development culture to one in which unit testing and other forms of automated testing became a widespread practice. This experience motivated me to write articles exploring how unit testing could’ve prevented the “goto fail” and Heartbleed bugs, including “Finding More Than One Worm in the Apple” for ACM Queue and “Goto Fail, Heartbleed, and Unit Testing Culture” for Martin Fowler.

I recently contributed ssl/heartbeat_test.c, a unit test that acts as a regression test against the Heartbleed bug, to the OpenSSL code base. I’m now working with the OpenSSL team on a proposal to help grow a robust suite of unit/integration/automated tests, and I need your help.

I’ve been collecting suggestions from the team into an OpenSSL testing tasklist in order to identify both short-term and long-term goals. There’s plenty of ways for people to help in addition to the actual writing of tests, including code review, mailing list advice, tool setup and documentation work. My proposal (current draft) includes well-defined roles and responsibilities, so you would have a clear idea of how to start helping right away. If there are ways you’d like to contribute that aren’t captured by any existing role, we can always define a new one for you. The ultimate goal would be to help everyone learn effective unit testing strategies so that, over time, test coverage and code quality steadily improves. This will be a lengthy, imperfect process, but one that I believe will ultimately make a positive difference in the code base if people are willing to try it, and set a precedent for other Open Source projects to help avoid defects like “goto fail” and Heartbleed in the future.

In other words, I want to help everyone learn to fish, to use the tired cliché. Doing all the heavy lifting by myself, acting as the lone “testing guy” would not be the best use of my time or in the best interests of OpenSSL. Together, however, I think we can help the OpenSSL community as a whole adopt a healthy set of automated testing habits.

If you’re interested, please submit your information in the OpenSSL Testing Survey form or email me directly at mbland@acm.org, indicating the capacity in which you’d like to help.

Thank you,

Mike Bland