Friday, November 28, 2014

A victory for abstraction, re-use, and small libraries

The other day at Outpace, while breaking some coupling, Eli and I decided to retain some information from one run of our program to another. We need to bookmark how far we read in each input data table. How can we persist this small piece of data?

Let's put it in a file. Sure, that'll work.[1] 

Next step, make an abstraction. Each of three configurations needs its own "how to read the bookmark" and "how to write the bookmark."[2] What can we name it?

After some discussion we notice this is basically a Clojure atom - a "place" to store data that can change - except persistent between runs.

Eli googles "clojure persist atom to disk" and bam! He finds a library. Enduro, by @alandipert. Persistent atoms for Clojure, backed by a file or Postgres. Complete with an in-memory implementation for testing. And thread safety, which we would not have bothered with. Hey, come to think of it, Postgres is a better place to store our bookmarks.

From a need to an abstraction to an existing implementation! with better ideas! win!

Enduro has no commits in the last year, but who cares? When a library is small enough, it reaches feature-completion. For a solid abstraction, there is such a thing as "done."

Now, it happens that the library isn't as complete as we hoped. There are no tests for the Postgres implementation. The release! method mentioned in the README doesn't exist.

But hey, we can add these to the library faster and with less risk than implementing it all ourselves. Alan's design is better than ours. Building on a solid foundation from an expert is more satisfying that building from scratch. And with pull requests, everybody wins!

This is re-use at its best. We paused to concentrate on abstraction before implementation, and it paid off.

[1] If something happens to the file, our program will require a command-line argument to tell it where to start.

[2] In OO, I'd put that in an object, implementing two single-method interfaces for ISP, since each function is needed in a different part of the program. In Clojure, I'm more inclined to create a pair of functions. Without types, though, it's hard to see the meaning of the two disparate elements of the pair. The best we come up with is JavaScript-object-style: a map containing :read-fn and :write-fn. At least that gives them names.


  1. Awesome! You've contributed the fix back, right?

  2. Regarding footnote #2: for the sake of clarity of meaning, why not create two single-method protocols and then implement them in the context of extend-type on the appropriate built-in or library-supplied type?