- What’s a
- Is this related to the Go programming language?
- Why write a framework?
- Why Bash?
- Will this work on Windows?
- Why not use tool X instead?
- Where can I run it?
- How is it tested?
- So what’s next?
./go script idea came from Pete Hodgson’s blog posts In Praise of the
./go Script: Part
I and Part
paraphrase Pete’s original idea, rather than dump project setup, development,
testing, and installation/deployment commands into a
README that tends to get
stale, or rely on oral tradition to transmit project maintenance knowledge,
automate these tasks by encapsulating them all inside a single script in the
root directory of your project source tree, conventionally named
go". Then the interface to these tasks becomes something
./go test, and
./go deploy. Not only would this script
save time for people already familiar with the project, but it smooths the
learning curve, prevents common mistakes, and lowers friction for new
contributors. This is as desirable a state for Open Source projects as it is for
Is this related to the Go programming language?
./go script convention in general and this framework in particular are
completely unrelated to the “Go programming language”:https://golang.org. In
fact, the actual
./go script can be named anything. However, the “
from the Go language distribution”:https://golang.org/cmd/go/ encapsulates many
common project functions in a similar fashion.
Why write a framework?
Of course, the danger is that this
./go script may become as unwieldy as the
README it’s intended to replace, depending on the project’s complexity. Even
if it’s heavily used and kept up-to-date, maintenance may become an intensive,
frightening chore, especially if not covered by automated tests. Knowing what
the script does, why it does it, and how to run it may become more and more
challenging—resulting in the same friction, confusion, and fear the script was
trying to avoid.
./go script framework makes it easy to provide a uniform and easy-to-use
project maintenance interface that fits your project perfectly regardless of the
mix of tools and languages, then it gets out of the way as fast as possible. The
hope is that by “making the right thing the easy
scripts using the framework will evolve and stay healthy along with the rest of
your project sources, which makes everyone working with the code less frustrated
and more productive all-around.
This framework accomplishes this by:
- encouraging modular, composable
./gocommands implemented as individual scripts—in the language of your choice!
- providing a set of builtin utility commands and shell command aliases—see
./go help builtinsand
./go help aliases
- supporting automatic tab-completion of commands and arguments through a
./go help envand
./go help complete
- implementing a quick, flexible, robust, and convenient documentation
system—document your script in the header, and help shows up automatically as
./go help my-command! See
./go help help.
Plus, its own tests serve as a model for testing command scripts of all shapes and sizes.
The inspiration for this model (and initial implementation hints) came from Sam
rbenv Ruby version manager.
It’s the ultimate backstage pass! It’s the default shell for most mainstream UNIX-based operating systems, easily installed on other UNIX-based operating systems, and is readily available even on Windows.
Will this work on Windows?
Yes. It is an explicit goal to make it as easy to use the framework on Windows as possible. Since Git for Windows in particular ships with Bash as part of its environment, and Bash is available within Windows 10 as part of the Windows Subsystem for Linux (Ubuntu on Windows), it’s more likely than not that Bash is already available on a Windows developer’s system. It’s also available from the MSYS2 and Cygwin environments.
Why not use tool X instead?
Of course there are many common tools that may be used for managing project
tasks. For example: Make,
Bazel, and the Go programming language’s
There are certainly more powerful scripting languages:
Ruby, and even Node.js
is a possibility. There are even more powerful shells, such as the
Z-Shell and the fish shell.
./go script framework isn’t intended to replace all those other tools and
languages, but to make it easier to use each of them for what they’re good for.
It makes it easier to write good, testable, maintainable, and extensible shell
scripts so you don’t have to push any of those other tools beyond their natural
Bash scripting is really good for automating a lot of traditional command line tasks, and it can be pretty awkward to achieve the same effect using other tools—especially if your project uses a mix of languages, where using a tool common to one language environment to automate tasks in another can get weird. (Which is part of the reason why there are so many build tools tailored to different languages in the first place, to say nothing of the different languages themselves.)
If you want to incorporate different scripting languages or shells into your
project maintenance, this framework makes it easy to do so. However, by starting
with Bash, you can implement a
./go init command to check that these other
languages or shells are installed and either install them automatically or
prompt the user on how to proceed. Since Bash is (almost certainly) already
present, users can run your
./go script right away and get the setup or hints
that they need, rather than wading through system requirements and documentation
before being able to do anything.
./go init tells the user “go to this website and install this other
thing”, that’s still an immediate, tactile experience that triggers a reward
response and invites further exploration. (Think of
Zork and the first "open
Where can I run it?
The real question is: Where can’t you run it?
The core framework is written 100% in Bash and it’s been tested under Bash 3.2, 4.2, 4.3, and 4.4 across OS X, Ubuntu Linux, Arch Linux, Alpine Linux, FreeBSD 9.3, FreeBSD 10.3, and Windows 10 (using all the environments described in the "Will this work on Windows?" section).
How is it tested?
The project’s own
./go test command does it all. Combined with automatic
tab-completion enabled by
./go env and pattern-matching via
./go glob, the
./go test command provides a convenient means of selecting subsets of test
cases while focusing on a particular piece of behavior. (See
./go help test.)
Again, Sam Stephenson’s Bash Automated Testing System (BATS) was a huge revelation. During the course of writing 260+ Bats test cases, I learned a few effective ways of setting up test harnesses, found an easy way to emit test suite info in the Bats output, and developed an assertion library I may break out into a new repository.
I also discovered Simon Kagstrom’s
kcov code coverage
tool which not only provides code
coverage for Bash scripts (!!!) but can push the results to Coveralls! See for
Note that the coverage shows some lines as uncovered that clearly are; it seems
kcov doesn’t recognize commands continued across multiple lines using
It’s a fixable problem, and certainly not a major one. I’m just astonished it
works so well and so quickly as it does!
kcov currently isn’t available to Travis CI via
apt-get, so I had
to write the
scripts/lib/kcov library to build it on-the-fly. Not an ideal
solution, but I may break it out into a plugin for reuse across projects until
an up-to-date version is available to Travis. (And I learned even more about
how to effectively test Bash scripts that execute various binaries!)
So what’s next?
Eventually I’d like to run ShellCheck on the
code, but my weird choice of prefixing internal functions with
\_@go. apparently doesn’t jive well with it because of the
. characters, even though Bash considers them valid. I may
clone the ShellCheck repo and try to resolve this myself.
There’s a plugin model in-place (
./go help plugins), but I haven’t actually
used it yet. As I break the assertion library and other pieces into their own
repos, I’ll put this model to use and see how well it works in practice. It’s
possible the plugin interface may change somewhat based on that experience.
I know there’s more I want to add, some I can probably take away, and there’s no
doubt the documentation could use a lot of work. (Though I’m proud that most of
the documentation resides in the command scripts themselves, accessible via
./go help!) I plan to actively develop this framework as I use it on my own
projects in the future—including, as of today, this blog!
This has been a labor of love for the past few weeks, and I’ve learned a lot
about Bash—what you can do with it, what to watch out for (the commit
history may provide
entertainment for the curious—for example, Bash 4.4 was released last night, and
it caused a couple of breakages that I fixed this morning), and how you can
actually test it. It’s been an absolute blast diving into this challenge, and if
anyone reading this is inclined to give it a try, I’d love to hear your
experiences with writing your own
./go scripts using this framework. If you
like it so much you’d care to contribute to its improvement, even better!
In other news, today is the fifth anniversary of my last day at Google.