distributed systems in a band

Watching the symphony last night: the conductor provides common ground for the joint activity of playing the music. She is a single point of coordination, that canonical clock that everyone can see at the same time. There is no information delay from the conductor to any member of the orchestra. They are playing from the same script, they have situational awareness of each other, plus they have a common source of timing.

Maybe a jazz band, playing improvisationally, is more like a distributed system without a central coordination point. All listening to each other, all sharing enough knowledge of the music and each other to respond and build the song. They have common snippets of scripts, and lots of awareness of each other.

For distributed systems in computing, imagine a jazz band with information delay. They’re trying to improvise but they can only hear what the other musicians played a few measures ago. Worse: usually it’s one measure ago but sometimes it’s several measures ago and no one knows which. They have to decide what to play in realtime based on out-of-date information. Everything they play will affect their peers, but they don’t know when.

Yeesh.

Add a central point of coordination, and a strict script, and we’re back to being in an orchestra – except with information delay, they can’t play in sync. If it takes the conductor’s movements a different amount of time to reach each musician, they aren’t playing together anymore.

We want our software systems to sing. But we can’t expect them to play a perfect song, not with central coordination, not with peer-to-peer. I don’t know what the solution is, but more and more I appreciate that this is hard

Good news, bad news

The big news stories are almost always about disasters and terrible things. The good news in life is not nearly so dramatic. The good parts are people helping each other, one-on-one. One building going up, one life improved by science. Each child who grows and learns for another day.

This is a property of complex systems. Improvement happens in “lots of tiny, measured steps in the same direction performed by many people over a long time.“ (@jackdanger) Catastrophic failures are sudden.

It is the same with software. Once critical systems are in production, and reach a level of scale and complexity that makes them useful and hard to change, improvements come in small incremental steps. Disaster comes suddenly. Success is in keeping them alive to grow and learn another day.

Property Testing in Elm

Elm is perfectly suited to property testing, with its delightful data-in–data-out functions. Testing in Elm should super easy.

The tooling isn’t there yet, though. This post documents what was necessary today to get a property to run in Elm.

Step 1: elm-test

This includes an Elm library and a node module for a command-line runner. The library alone will let you create a web page of test results and look at it, but I want to run them in my build script and see results in my terminal.

Installation in an existing project:

elm package install deadfoxygrandpa/elm-test
npm install -g elm-test

The node module offers an “elm test init” functionality to put some test files in the current directory: TestRunner (which is the Main module for test runs[1]) and Tests.elm which holds actual tests. Personally, I found it necessary to follow the following steps as well.

  • create a test directory (I don’t want tests in my project home), and move the TestRunner.elm and Tests.elm files there.
  • add that test directory to the source directories in elm-package.json

Step 2: elm-check

The first thing to know is: which elm-check to install. You need the one from NoRedInk:

elm package install NoRedInk/elm-check

The next thing is: what to import. Where do all those methods used in the README live?

Here is a full program that lets elm-test execute the properties from the elm-check readme.
TL;DR: You need to import stuff from Check and Check.Producer for all properties; and  for the runner program, ElmTest and Check.Test and Signal, Console, and Task.

Name it test/Properties.elm and run it with

elm test test/Properties.elm

The output looks like

Successfully compiled test/Properties.elm
Running tests…
  1 suites run, containing 2 tests
  All tests passed

Here’s the full text just in case.

module Main (..) where
import ElmTest
import Check exposing (Evidence, Claim, that, is, for)
import Check.Test
import Check.Producer as Producer
import List
import Signal exposing (Signal)
import Console exposing (IO)
import Task

console : IO ()
console =
  ElmTest.consoleRunner (Check.Test.evidenceToTest evidence)

port runner : Signal (Task.Task x ())
port runner =
  Console.run console

myClaims : Claim
myClaims =
  Check.suite
    “List Reverse”
    [ Check.claim
        “Reversing a list twice yields the original list”
        `that` (\list -> List.reverse (List.reverse list))
        `is` identity
        `for` Producer.list Producer.int
    , Check.claim
        “Reversing a list does not modify its length”
        `that` (\list -> List.length (List.reverse list))
        `is` (\list -> List.length list)
        `for` Producer.list Producer.int
    ]

evidence : Evidence
evidence =
  Check.quickCheck myClaims

How to write properties is a post for another day. For now, at least this will get something running.

See also: a helpful post for running elm-check in phantom.js

[1] How does that even work? I thought modules needed the same name as their file name. Apparently this is not true of Main. You must name the module Main. You do not have to have a ‘main’ function in there (as of this writing). The command-line runner needs the ‘console’ function instead.