CI Results in Your tmux Status Line

2020-02-13

I use a custom status line in tmux to show the CI test results of my current branch. This post will talk a little about tmux status lines, hub's ci-status command, and my ci-status-indicator scripts.

Example status line:

full tmux status preview

Tmux status lines

The tmux status line lets you show information about the current tmux session. You can also use the #() syntax to pull in content from external programs. For example, you might want to show the currently playing track from Spotify.

I recommend The Tao of tmux for a further introduction to the tmux status line (also known as the "status bar").

There are four ways the status line is updated:

  1. Automatically every 15 seconds (this frequency can be changed with status-interval in your tmux configuration).
  2. When you move between panes or windows.
  3. After a command finishes in a tmux-owned shell pane.
  4. On-demand when you invoke refresh-client (Note: passing -S to refresh-client will update only the status line).
Tmux appears to have an internal throttle that limits updating the status line to once per second (this throttle is ignored by `refresh-client`). To test this, I set my status-right in my tmux conf to be set -g status-right '#(date >> /tmp/debug.txt)'. This appends the current time to a temporary file whenever the status line is updated. Moving from one tmux window to another slowly, you can see that you get a new line written to the debug file for each window change. I then ran ruby -e '100.times { system("tmux next-window") }' to cycle through tmux windows 100 times. Only one line was written to the debug file per second.

The status line runs #() commands asynchronously, but because your status line can potentially be updated every second, you should still keep the contents speedy. Avoid content that makes network calls or is cpu intensive. Where network calls or expensive computation are unavoidable, consider throttling or caching.

hub ci-status

hub lets you "do everyday GitHub tasks without leaving the terminal."

Running hub ci-status with no options will return pending, failure, success, etc. to indicate the status of the GitHub checks associated with your branch.

Running hub ci-status -v will show each current check with the associated status.

✔︎       unit-tests              https://url-to-see-unit-test-details/
●       e2e-tests               https://url-to-see-e2e-test-details/

You can see additional options with hub ci-status --help.

ci-status-indicator

Showing the current GitHub checks results in the tmux status line requires a little massaging. The tmux status line doesn't support ANSI colors. If it did, we could reach for hub ci-status -f "%sC%t " which prints out the names of each check color-coded depending on state (failure = red, pending = yellow, success = green).

We could do some clever work to translate the ANSI color output of hub to tmux colors, but, since the number of colors here is limited, I'm going to use a lazy sed command.

hub ci-status -f "%S, %t " | sed 's/failure, /#[fg=red]/g; s/success, /#[fg=green]/g; s/pending, /#[fg=yellow]/g'

initial colored status output

That gets us close, but we can do better. The -tests part of the GitHub check name isn't adding any value, so we can remove it. Finally, I'll use a white | to delimit the checks rather than a space.

Since this command is getting long, let's extract it to a script.

#!/usr/bin/env bash

hub ci-status -f "%S, %t|" | \
  gsed -r '
    s/failure, /#[fg=red]/g;
    s/success, /#[fg=green]/g;
    s/pending, /#[fg=yellow]/g;
    s/\|$//;
    s/\|/#[fg=white]|/g;
    s/-tests//g;'

This outputs #[fg=yellow]e2e#[fg=white]|#[fg=green]unit which looks like this:

final colored status output

Much better.

I'm using gsed here rather than OSX's built-in sed. This isn't required, but I find gsed's extended regular expressions more useful than the built-in as these regular expressions grow. brew install gnu-sed and you're good to go.

You'll likely want to customize this script to make the output more succinct as we did by removing -tests. Perhaps replacing codecov with cc or similar could help you conserve real estate.

For customization's sake, since different projects can have different checks, I like to have a copy of this script local to each project. I name it .ci-status-indicator, chmod +x it, and add .ci-status-indicator to my global gitignore. The final tmux configuration line looks like this:

set -g status-right ' #(cd #{pane_current_path} && [ -f ".ci-status-indicator" ] && ./.ci-status-indicator) #[fg=white]%Y-%m-%d %H:%M'

This runs .ci-status-indicator if it exists and also shows the time. You need to use pane_current_path because the default current working directory of the status line process is whatever directory you initially started tmux from.

Caching / Throttling

Remember what I said earlier about keeping the status line speedy and avoiding network calls? hub ci-status is pretty fast, but we can be avoid hammering the GitHub API with a little caching. I've recently been writing about a general purpose cli caching script and that script will work great here.

Since the tmux config line is already a bit long, I put the cache usage in the .ci-status-indicator script itself.

#!/usr/bin/env bash

cache tmux-ci-status --ttl 30 \
  hub ci-status -f "%S, %t|" | \
    gsed -r '
      s/failure, /#[fg=red]/g;
      s/success, /#[fg=green]/g;
      s/pending, /#[fg=yellow]/g;
      s/\|$//;
      s/\|/#[fg=white]|/g;
      s/-tests//g;'

You might also be asking yourself "Won't this get stale when I change branches?" Yes, until the cache TTL expires and the status line updates. Think for a moment about ways to avoid stale content.

Avoiding stale results

We're working against two caches here:

  1. tmux's status-interval (15 seconds by default)
  2. cache's 30 second TTL (customizable to whatever we want)

We'll need to solve for both.

Git hooks are the best place to start. Issuing tmux refresh-client -S in the post-checkout hook handles the status-interval cache. We could use this same git hook to first purge the cache key on every branch change. That would work fine but is wastefully throwing away our previous information.

We can be a little more clever and use the current git SHA as part of our cache key. This has the advantage of not needing to re-query GitHub if you change between known commits within cache's TTL.

#!/usr/bin/env bash

cache_key="tmux-ci-status-$(git rev-parse HEAD)"

cache $cache_key --ttl 30 --cache-status "*" \
  hub ci-status -f "%S, %t|" | \
    gsed -r '
      s/failure, /#[fg=red]/g;
      s/success, /#[fg=green]/g;
      s/pending, /#[fg=yellow]/g;
      s/\|$//;
      s/\|/#[fg=white]|/g;
      s/-tests//g;'

With a per-SHA cache and a hook-triggered instant status line refresh, we'll never see stale content from another branch/SHA.

Next steps

This works great, and hopefully you've learned several new things along the way.

There's always room for improvement, so here's something to ponder and work on if you wish: Under what circumstances can the GitHub checks results change for a single SHA? Put another way: are there scenarios where you can cache the results indefinitely and avoid pinging hub ci-status again for the SHA?

If the check's result is pending then it makes sense to query again until the check finishes (the pending state will transition to failure or success). If the results are all success then querying hub ci-status again feels wasteful. If the result is failure then you might want to re-check hub ci-status if you manually re-trigger a build (e.g. for flaky tests) but perhaps it is otherwise a final state.

I leave this as an exercise to the reader. I've not yet added this optimization since I've found the cache TTL sufficient.

Shaving that yak is tempting though. It could lead to some fun side paths (e.g. it might be neat to annotate your git history with git notes regarding the CI status of each commit).


entr

2020-02-07

There's no shortage of ways to automatically run your tests when files change (e.g. jest has jest --watch). I find these mechanisms useful for maintaining flow. Having Vim open on one screen and a test-runner open on another screen is fantastic.

There's not a custom runner for everything, but you might not need one. I've found that entr often gets the job done. entr lets you "run arbitrary commands when files change."

When I was writing the cli caching script, I kept git ls-files | entr bats test running. Every time I saved my script or the test, the tests would run.

The entr homepage specifies several good use-cases and how to watch for new files, etc. Give it a try.


TDD a CLI Caching Script - Part Six - Utility

2020-02-05

This is part six in a series about writing a general-purpose script to cache CLI output. In this brief post we'll make our script more useful as a utility to other scripts by adding some new options.

If you want to try your hand at implementing these features, checkout 780d577 locally. I'll omit my implementation here but will link to the appropriate diffs.

--check

We can implement --check to allow checking to see if the cache is fresh per the provided --ttl and --stale-while-revalidate options. The best way to communicate this freshness is with exit status, IMHO. This will allow a shell script to use cache --check as a conditional. We'll return 0 (truthy) if the cache is fresh and 1 (falsy) if it is not.

Example usage:

fresh() {
    cache --check --ttl 1 test-cache-key
}

if fresh; then
    echo "FRESH"
else
    echo "NOT FRESH"
fi

Here's the tests for the feature:

@test "--check returns 0 if the content exists and is inside the TTL/SWR" {
  run ./cache $TEST_KEY echo 1
  [ "$status" -eq 0 ]
  [ "$output" = "1" ]

  run ./cache --stale-while-revalidate 1 --ttl 1 --check $TEST_KEY
  [ "$status" -eq 0 ]
  [ "$output" = "" ]
}

@test "--check returns 1 if the content exists but is outside the TTL/SWR" {
  run ./cache $TEST_KEY echo 1
  [ "$status" -eq 0 ]
  [ "$output" = "1" ]

  wait_for_second_to_pass

  run ./cache --stale-while-revalidate 1 --ttl 0 --check $TEST_KEY
  [ "$status" -eq 1 ]
  [ "$output" = "" ]
}

@test "--check returns 1 if the content is not yet cached" {
  run ./cache --check $TEST_KEY
  [ "$status" -eq 1 ]
  [ "$output" = "" ]
}

Between the feature description and the tests, you can probably guess the implementation. You can check out the --check implementation as you wish.

--purge

Every cache needs a good mechanism for clearing the cached content on-demand. Maybe your cache key wasn't specific enough or outside forces have made the content stale before the TTL naturally expires. Whatever your reason, we'll support cache --purge <cache-key> to remove the content currently cached at the provided key.

Here's the tests for --purge:

@test "--purge exits with 0 if the content is not yet cached" {
  [ ! -f "$CACHE_DIR$TEST_KEY" ]

  run ./cache --purge $TEST_KEY
  [ "$status" -eq 0 ]
  [ "$output" = "" ]

  [ ! -f "$CACHE_DIR$TEST_KEY" ]
}

@test "--purge exits with 0 and removes the file if the content is cached" {
  touch "$CACHE_DIR$TEST_KEY"

  [ -f "$CACHE_DIR$TEST_KEY" ]

  run ./cache --purge $TEST_KEY
  [ "$status" -eq 0 ]
  [ "$output" = "" ]

  [ ! -f "$CACHE_DIR$TEST_KEY" ]
}

Try implementing this feature and then check out my implementation afterwards.

Closing

With a few small tweaks we've made our cache script more useful. This will wrap up the series for now, but I'm sure you'll see the cache script appear as a background character in future posts.

My goofy face

Hi, I'm Jeffrey Chupp.
I solve problems, often with code.