TDD a CLI Caching Script - Part Four - GitHub Actions, Linux-compatibility, and shellcheck

This is part four in a series about writing a general-purpose script to cache CLI output. Up to this point, our script has only worked in OS X. In this post, we'll add Linux compatibility, show how you can setup a GitHub action for running your bats tests, and talk about shellcheck for linting shell scripts.

GitHub Action / Linux compatibility

We've been TDD-ing our cache script development with bats tests, but they're only running locally and manually. I wanted to setup a GitHub Action to run the tests automatically on every push. I didn't know anything about GitHub Actions so this took awhile to get right.

Here's the final .github/workflows/tests.yml file:

name: CI

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Check out code
        uses: actions/checkout@v1

      - name: Setup bats
        run: git clone https://github.com/sstephenson/bats.git

      - name: Test
        run: CACHE_DIR="$GITHUB_WORKSPACE/" ./bats/bin/bats test

This is mostly self-explanatory, but making the tests pass required some changes to the tests and cache script for Linux compatibility (since the Action is running in Ubuntu).

Here's the full diff. The changes break down into three types:

1. Quote every variable. I was getting [: too many arguments errors on lines like [ $output = "flounder - fish" ] because of the $output variable has spaces in it.
2. Allow configuring the $CACHE_DIR in the tests because I wasn't able to write to $TMPDIR in the Action.
3. GNU stat and grep behave differently than their OS X (BSD) counterparts. stat needs different arguments and grep --help exits with a different status code and has different copy.

With these small tweaks, the cache script now works fine in Linux and our tests pass in our GitHub Action (example).

shellcheck

Quoting variables made our script more compatible. What other best practices are we missing? How can we learn more? shellcheck is a fantastic utility for linting and finding bugs in your shell scripts.

Running shellcheck cache alerts us to a number of issues with our script. The most prevalent issue is SC2086: Double quote to prevent globbing and word splitting. At the bottom of the output there's a link to SC2086 on the shellcheck wiki to learn more. This is the same issue we saw above where we needed to quote variables in our test file.

Shellcheck also alerts us to two best practices $/${} is unnecessary on arithmetic variables and Don't quote rhs of =~, it'll match literally rather than as a regex.

Linters give us a common language to talk about issues/best-practices and further reading to make the potential problems clear and to explain the alternatives in a user-friendly way. Instead of internalizing the naive "quote everything, I guess" we can refine our understanding to get to the details of why.

Like most linters, you can add magic comments to ignore lines you don't want to change. In this case, we like all the suggestions, so we can use shellcheck cache -f diff | patch -p1 cache to have shellcheck generate a patch that we immediately pass to patch to apply the fixes to our script.

Not all issues can be auto-fixed. We have to clean up $/${} is unnecessary on arithmetic variables manually, but that's trivial to do.

As a neat bonus, you can run shellcheck against .bats files to find potential issues in your bats tests.

Here's the diff that applies all the shellcheck suggestions to our script and tests.