Executable Note-Taking — Writing Automated Tests as a Learning Method

2017-11-27

I'm trying a new (to me) approach to learning programming languages: writing automated tests as a learning method. This surely isn't a new idea but I wanted to share my experience.

The concept is pretty straightforward:

Discover how something works
Write a test to illustrate your understanding (aim for concise examples)
Add more tests as-needed to probe the boundaries of your understanding

These tests are effectively executable note-taking. You should try it.

Benefits

As with traditional note-taking, the process of writing the tests help commit the concept to memory.
The tests are there to reference in the future if I forget how something works.
The tests' passing shows that my understanding is correct.
I can update the tests when my understanding improves or I find a better approach.
I can re-run the tests when a major version of the language is released to see what is deprecated.

You might worry that this effort is duplicating existing tests from the language itself. Right! But the purpose here is to solidify and exercise your understanding. (Do feel encouraged to consult the official tests afterwards as you can learn even more.)

An example

I needed to write a method in Elixir that could take an optional block as an argument (this is a common thing in Ruby/Rails). I did some digging and found the do: block approach (example). Next I wrote a couple tests in my local elixir_learning app to document my understanding:

describe "methods that take an optional block" do
  defmodule OptionalBlock do
    @moduledoc """
    This works by specifying a method with and without the `do: block`
    argument.
    """

    def foo(arg1, arg2) do
      [arg1, arg2]
    end

    def foo(arg1, arg2, do: block) do
      [arg1, arg2, block]
    end
  end

  test "without a block" do
    assert OptionalBlock.foo("test", %{a: 1}) == ["test", %{a: 1}]
  end

  test "with a block" do
    x = :example_result_of_block

    result =
      OptionalBlock.foo "test", %{a: 1} do
        x
      end

    assert result == ["test", %{a: 1}, :example_result_of_block]
  end
end

Now I understand the behavior. Whenever I forget how this works, I can come back to it for reference.

Further exploration

After writing the initial examples, I try to probe the edges of my understanding.

Looking at the code now, it isn't clear when block execution happens. Is it lazy (like in Ruby) or does it happen at function call time?

Let's write another test to find out:

# higher in the test file
import ExUnit.CaptureIO

test "time of block execution" do
  defmodule UnusedBlock do
    def foo(do: _block) do
      # intentionally not using _block
    end
  end

  assert capture_io(fn ->
           IO.puts("Before")

           UnusedBlock.foo do
             IO.puts("Block")
           end

           IO.puts("After")
         end) == "Before\nAfter\n"
end

Here we're making a guess at what the output might be. UnusedBlock.foo never uses block. If the do/end block is lazily-evaluated then we will only see the "Before" and "After" lines.

Running the test shows:

1) test methods that take an optional block time of block execution (OptionalBlockTest)
   test/optional_block_test.exs:36
   Assertion with == failed
   code:  assert capture_io(fn ->
            IO.puts("Before")
            UnusedBlock.foo() do
              IO.puts("Block")
            end
            IO.puts("After")
          end) == "Before\nAfter\n"
   left:  "Before\nBlock\nAfter\n"
   right: "Before\nAfter\n"
   stacktrace:
     test/optional_block_test.exs:43: (test)

Our guess was incorrect: the "Block" line is present. This means the block is not lazily-evaluated. It is evaluated at the time the function is invoked.

This behavior is good to remember for the future and worth codifying in the test. We update our assertion to include the "Block" output line (we do expect it now) and update the test name to mention how the block is always invoked even if it goes unused.

Now I know a little more about how Elixir handles block arguments by default and I have executable notes to look back on.

If you want to learn more, Henrik Nyh has helpful background info and discusses using macros for lazy evaluation in his excellent Elixir block keywords article.

I've only been using this approach for a short while so I can't comment on the long-term benefits or downsides. I can say that I've really been enjoying the benefits mentioned above — particularly being able to experiment with my understanding in a reproducible way.

semantic art

code should say something

Executable Note-Taking — Writing Automated Tests as a Learning Method

Benefits

An example

Further exploration