TDD a CLI Caching Script - Part One

This is the first in a series about writing a general-purpose script to cache CLI output. In this series we'll learn about using bats to test CLI programs, level up our Bash skills, and hopefully end up with a useful tool we can use every day.

Goal

The end result script should work something like this:

cache cache-key script-name arg1 arg2 <additional args...>

1. On first run, it invokes some-script-name with the arguments and caches the STDOUT result.
2. On subsequent runs, it returns the cached content from the prior run.

Future versions of the cache script can incorporate a TTL, async refreshes, etc.

Why is this useful?

Caching allows us to do expensive work once and use the result until it is no longer timely. Some program results can be cached permanently because the content is easily fingerprinted to a unique cache key.

A real-world example is the rake routes (or rails routes) command. This command generates a list of available routes (think urls) in your application. Unfortunately, Rails has to essentially boot your entire app to generate this list. This takes longer and longer to do as your app grows.

If your Rails' route setup is traditional (single file, no surprising metaprogramming) then you can trust that your routes will only change if the config/routes.rb file changes. We can use md5 to get a simple string fingerprint of the file contents. We can use that fingerprint as a permanent cache key for the result of running rake routes because any changes to the routes will change the md5 and invalidate the cache.

This means that cache $(md5 config/routes.rb) rake routes can reliably cache the output and cut the time down from >10 seconds on a large app to essentially zero. This a huge difference if you're using this output for something like route-completion with fzf in Vim.

Writing our first test

Following TDD, we'll describe the behavior we wish our script had with tests. These tests will fail because the behavior doesn't exist yet. Then we'll implement just-enough functionality to make the test pass. We repeat this loop until our script is feature-complete.

First we install bats (from source or via brew install bats) and make a new directory for our script. Make a new directory cli-cache and give it a subdirectory of test.

Within the test directory, we'll make a new file named cache.bats and add our initial test:

@test "initial run is uncached" {
  run ./cache some-test-key echo hello
  [ "$status" -eq 0 ]
  [ $output = "hello" ]
}

run executes a given command, sets $output to the STDERR and STDOUT of the command, and sets $status to the status code of the command.

Our test is really just showing that the cache script can successfully execute the command we provide it and return the output. That's a small but important first step.

We can run the test with bats test from the cache directory.

 ✗ initial run is uncached
   (in test file test/cache.bats, line 3)
     `[ "$status" -eq 0 ]' failed

1 test, 1 failure

Hooray, our first failing test! The status code didn't match the expected code. If we put echo $status before our comparison, we'll see that $status is 127 which means the command is not found. That makes sense because we haven't made our cache script yet. Let's create an empty file named cache in the cli-cache folder and try again.

The test still fails, but now $status is 126 because the command isn't executable. chmod +x cache and try again.

 ✗ initial run is uncached
   (in test file test/cache.bats, line 4)
     `[ $output = "hello" ]' failed with status 2
   /var/folders/40/y21j3fw13432jk6z_y08mnbm0000gn/T/bats.59924.src: line 4: [: =: unary operator expected

1 test, 1 failure

The status code is fine now but our $output isn't what we want since our cache script doesn't do anything. Let's modify the cache script to run the command provided so the test will pass.

#!/usr/bin/env bash

set -e

cache_key=$1
shift

$@

We have a shebang line. We set -e so our script will fail at the first invalid command (this is generally a best practice).

Then we assign our $cache_key to the first argument. Next we shift to remove the $cache_key from our argument list. Now we can execute the provided command.

Rerunning bats test shows success. Nice work!

Add more tests to flesh out the implementation

Let's add a new test to verify that it works for quoted arguments to the provided command:

@test "works for quoted arguments" {
  run ./cache some-test-key printf "%s - %s\n" flounder fish
  [ "$status" -eq 0 ]
  [ $output = "flounder - fish" ]
}

Hrm. That didn't work. If we echo $output, we see -%s\nflounderfish -- all our arguments to printf smushed together. To preserve the arguments, we can update our cache script by changing $@ to the quoted form "$@".

With that passing, there's one more useful fundamental to get right: the cache command should return the exit code of the underlying command.

@test "preserves the status code of the original command" {
  run ./cache some-test-key exit 1
  [ "$status" -eq 1 ]
}

That one already passes for free by virtue of the "$@" being the last line of our script.

Now we have three passing tests, but we're not actually caching anything yet. We add a new test for the caching behavior.

@test "subsequent runs are cached" {
  run ./cache some-test-key echo initial-value
  [ "$status" -eq 0 ]
  [ $output = "initial-value" ]

  run ./cache some-test-key echo new-value
  [ "$status" -eq 0 ]
  [ $output = "initial-value" ]
}

Here we call echo twice with two different strings. Since our cache-key remains the same, the second echo should never get evaluated and our script should instead return the cached value from the first echo call.

With that test failing, let's update our script to do some caching.

#!/usr/bin/env bash

set -e

cache_key=$1
shift

cache_dir=${CACHE_DIR:-$TMPDIR}
cache_file="$cache_dir$cache_key"

if test -f $cache_file; then
    cat $cache_file
else
    "$@" | tee $cache_file
fi

Looks easy enough, right? If the cache file exists, we read it. Otherwise we execute the command and pipe it to tee. tee prints the output to STDOUT and also writes the output to our $cache_file.

You can specify the cache directory by setting the environment variable CACHE_DIR or we'll default to $TMPDIR.

Running our tests shows (perhaps) unexpected results:

 ✓ initial run is uncached
 ✗ works for quoted arguments
   (in test file test/cache.bats, line 18)
     `[ $output = "flounder - fish" ]' failed
 ✗ preserves the status code of the original command
   (in test file test/cache.bats, line 23)
     `[ "$status" -eq 1 ]' failed
 ✗ subsequent runs are cached
   (in test file test/cache.bats, line 29)
     `[ $output = "initial-value" ]' failed

4 tests, 3 failures

Wait, why is everything broken but the first test? Oh yeah, we're caching now and all the tests use the same cache-key. We could give each test a unique cache key, but instead let's use bats' setup function to ensure we delete cached content between tests.

setup() {
  export TEST_KEY="cache-tests-key"

  # clean up any old cache file (-f because we don't care if it exists or not)
  rm -f "$TMPDIR$TEST_KEY"
}

We'll replace anywhere we're using some-test-key in the tests with $TEST_KEY.

bats test now shows everything passing except the "preserves the status code of the original command" test. This is a side-effect of piping our command to tee. tee exits with a status code of 0 because tee worked fine (even though the preceding command did not). Fortunately we can use $PIPESTATUS to get the status of the any command in the pipe chain. We just need to add the line exit ${PIPESTATUS[0]} after our "$@" | tee $cache_file line.

 ✓ initial run is uncached
 ✓ works for quoted arguments
 ✓ preserves the status code of the original command
 ✓ subsequent runs are cached

4 tests, 0 failures

Closing

Here's the final version of the script:

#!/usr/bin/env bash

set -e

cache_key=$1
shift

cache_dir=${CACHE_DIR:-$TMPDIR}
cache_file="$cache_dir$cache_key"

if test -f $cache_file; then
    cat $cache_file
else
    "$@" | tee $cache_file
    exit ${PIPESTATUS[0]}
fi

You can add this to your $PATH to invoke cache from anywhere.

Let's compare timings of ways to invoke rake routes on a large app:

With a small update to the source of our fzf route completion, things are super speedy!

inoremap <expr> <c-x><c-r> fzf#complete({
  \ 'source':  'cache $(md5 -q config/routes.rb) rake routes',
  \ 'reducer': '<sid>parse_route'})

If this all feels like a lot of work to save 12 seconds, you're right. From my experience, the value is rarely in the actual time saved, but in the preservation of flow. Any time I spend waiting on the computer is time when I can get distracted or otherwise lose my flow. In my career, I've observed that disruptions compound in the negative. I've found that eliminating them (where possible) can compound in the positive as well.

Now we have a new trick to eliminate disruptions and help preserve flow.

Up next

Check out part two where we add a TTL option to specify when cached content should expire. We'll also update the script to only cache successful runs of the provided command.

You can always find the most up-to-date version of the cache script on GitHub.

It may surprise you to hear that there isn't a standard unix utility to cache CLI script output. Thankfully, there's a number of community-provided examples to choose from. e.g. cachecmd, runcached, bash-cache, etc.