It’s Atomist Time!

I’m hella excited to get to work on Atomist full-time starting now (January 2017). Why? What do they do? Oh let me tell you!
I love developing software, not least because we (as an industry) have not yet figured out how to software. We know it’s powerful, but not yet how powerful. Software is like engineering except that the constraints aren’t in what we can build, but what we can design and specify. Atomist is expanding this capacity.
Atomist builds tooling to smooth some bumps in software development. There are three components that I’m excited about, three components that open new options in how we develop software.

Component 1: Code that changes code

First, there are code editors, called Rugs. On the surface, these automate the typing part. Like code generators, except they continue to work with the code after you modify it. Like refactorings in an IDE, except they appear as a pull request, and then you can continue development on that branch. If you have some consistent code structure (and if you use a framework, you do), Rugs can perform common feature-adding or upgrading or refactoring operations. Use standard Rugs to, say, add graph database support to an existing Spring Boot project. Customize Rugs to set up your Travis build uniformly in your projects. Create your own Rugs to implement metrics integration according to your company’s standards — and to upgrade existing code when those standards change.
On the surface this is an incremental improvement over existing code generation and IDE refactoring tools. Yet, I see it as something more. I see it as a whole new answer to the question of “indirection or repetition?” in code. Take for instance: adding a field to a Rails app makes us change the controller, the model, and four other places. Or creating a new service means changing deployment configuration, provisioning, and service discovery. Whenever a single conceptual change requires code changes in multiple spots, we complain about the work and we make mistakes. Then we start to get clever with it: we introduce some form of indirection that localizes that change to one place. Configuration files get generated in the build, Ruby metaprogramming introduces syntax that I can’t even figure out how it’s executable — magic happens. The code gets less explicit, so that we can enforce consistency and make changing it … well, I’m not gonna say “easier” because learning to cast the spell is tricky, but it is less typing.
Atomist introduces a third alternative: express that single intention (“create a new service” or “add this field”) as a Rug editor. This makes writing it one step, and then the editor makes all those code changes in a single commit in a branch. From there, customize your field or your new service; each commit that you make shows how your feature is special. The code remains explicit, without additional magic. When I come back and read it, I have some hope of understanding what it’s doing. When I realize that I forgot something (“oops! I also need to add that service to the list of log sources”) then I fix it once, in the NewService.rug editor. Now I never forget, and I never have to remember.
I love this about developing with Rugs: as I code, I’m asking myself, “how could I automate this?” and then what I learn is encoded in the Rug, for the benefit of future-me and (if I publish it) of future-everyone-else. That is when I feel productive.

Component 2: Coordination between projects

Editors are cute when applied to one project. When applied across an organization, they start to look seriously useful. Imagine: A library released a security update, and we need to upgrade it across the organization. Atomist creates a pull request on every project that uses that library. The build runs, maybe we even auto-merge it when the build passes. Or perhaps there are breaking changes; the editor can sometimes be taught how to make those changes in our code.
And if a Rug can change the way we use a library, then it can change the way we use ours. This is cross-repository refactoring: I publish an internal library, and I want to rename this function in the next version. Here’s my game: I publish not only the new version of my library, but an editor – and then I ask Atomist to create pull requests across the organization. Now it is a quick code review and “accept” for teams to upgrade to the new version.
Atomist coordinates with teams in GitHub and in Slack. Ask Atomist in Slack to start that new feature for you, or to check all repositories in the organization and create pull requests. Atomist can also coordinate with continuous integration. It ties these pieces together across repositories, and including humans. It can react to issues, to build results, to merges; and it can ping you in Slack if it needs more information to act appropriately. I have plans to use this functionality to link libraries to the services that use them: when the build passes on my branch, go build the app that uses my library with this new version, and tell me whether those tests pass.
This is cross-repository refactoring and cross-repository build coordination. This gives companies an alternative to the monorepo, to loading all their libraries and services into one giant repository in order to test them together. The monorepo is a lie: our deployments are heterogenous, so while the monorepo is like “look at this lovely snapshot of a bunch of code that works together” the production environment is something different. The monorepo is also painful because git gets slow when the repository gets large; because it’s hard to tell which commits affect which deployed units; and because application owners lose control over when library upgrades are integrated. Atomist will provide a layer on top of many repositories, letting us coordinate change while our repositories reflect production realities.
Atomist tooling will make multirepo development grow with our codebases.

Component 3: is still a secret

I’m not sure I can talk about the third piece of possibility-expanding tooling yet. So have this instead:
Automated coordination among systems and people who interact with code — this is useful everywhere, but it’s a lot of work to create our own bots for this. Some companies put the resources into creating enough automation for their own needs. No one business-software-building organization has a reason to develop, refine, and publish a general solution for this kind of development-process automation. Atomist does.
When it becomes easy for any developer to script this coordination and the reactions just happen — “Tell me when an issue I reported was closed” “Create a new issue for this commit and then mark it closed as soon as this branch is merged” — then we can all find breakages earlier and we can all keep good records. This automates my work at a higher level than coding. This way whenever I feel annoyed by writing a status report, or when I forget to update the version in one place to match the version in another, my job is not to add an item to a checklist. My job is to create an Atomist handler script to make that happen with no attention from me.

My secret

I love shaving yaks. Shaving them deeply, tenderly, finding the hidden wisdom under their hair. I love adding a useful feature, and then asking “How could that be easier?” and then “How could making that easier be easier?” This is Atomist’s level of meta: We are making software to make it easier for you to make your work easier, as you work to make software to make your customers’ lives easier.
I think we’re doing this in depths and ways other development tools don’t approach. At this level of meta (software for building software for building software for doing work), there’s a lot of leverage, a lot of potential. This level of meta is where orders-of-magnitude changes happen. Software changes the world. I want to be part of changing the software world again, so we can change the real world even faster.
With Atomist, I get to design and specify my own reality, the reality of my team’s work. (Atomist does the operations bit.) Without spending tons of time on it! Well, I get to spend tons of time on it because I get to work for Atomist, because that’s my thing. But you don’t have to spend tons of time on it! You get to specify what you want to happen, in the simplest language we can devise.
We’re looking for teams to work with us on alpha-testing, if you’re interested now. (join our slack, or email me) Let’s learn together the next level of productivity and focus in software development.

Key signatures in piano music: the underlying technology

Today, sheet music (at least, the major keys) went from magic to technology for me. Magic is spellcasting, is memorization. Technology is understanding, is derivation. Technology takes up so much less space in my head!

If you can read music enough to pick out a simple song but wonder why chords and their weird names seem so obvious to some people, this post is for you.

Those markings at the beginning of the line that show which notes are played as sharps (called a key signature) – I have been trying to memorize their names. This one is called D Major, and it means that all Fs and Cs are to be played as sharps, hit the black key to the right instead of the white key.

D Major signature

I know how to play music with C# and F#, but why on earth is this called D Major? And why is it a thing?

Today I read a few pages in a book about scales, and now I get it. It takes some history to understand.

See, a long time ago the Greeks made a lyre with four strings, and they tuned the strings into a tetrachord, four notes that sound good. On a piano, a tetrachord is made out of four notes with the following separations: a whole step, then a whole step, then a half step. A whole step goes two keys, counting white and black keys; a half step is one key. Like walking — right foot, left foot makes a whole step. One easy tetrachord starts with middle C:

From C to D is a whole step because there is a black key in between. From E to F is a half-step because there is no key in between. The C tetrachord is C, D, E, F.

The same formula makes a tetrachord starting from any key. The tetrachord starting with D goes D, E, F#, G.

A whole step from D is easy, skip the black key and hit E, a white key. A whole step from E means skipping the next key (F, a white key) and hitting the key after that (F#, a black key). Then a half step means the very next key (G, a white key). This is where the F# in the key signature is coming from.

But wait! There’s more!

Put two tetrachords together to get an Ionian scale, also called a major scale. The tetrachords are separated by a whole step. In C, the first tetrachord is C, D, E, F. Take a whole step to G and start the next tetrachord. It goes G, A, B, C.

Eight notes, the last one the same as the first (one octave higher), make a major scale. The keys have this pattern of separations between them, made up of the Ionian scale and tetrachord patterns. Each scale uses a little over half the keys on the keyboard, and ignores the rest. Songs in C major use all white keys and none of the black keys. You want anything else, gotta put a sharp or flat symbol in front of the note.

What does this mean for D? The D major scale starts with the D tetrachord and adds a second tetrachord: D, E, F#, G; A, B, C#, D.

C# and F#! There they are, the two black keys with numbers on them! The “normal” keys to play in the D scale include C# and F# (black keys), but never C or F (white keys). Putting the D Major key signature in front of the music means that all the keys in the D scale look like ordinary notes.

With the C# and F# handled by the key signature, any special marking (sharp, flat, or natural) points out a note that is unexpected, that does not fit in with the rest.

The same pattern works for other key signatures; try constructing the Ionian scale for G out of two tetrachords separated by a whole step. You’ll find that the only black key used is F#, so this is G major:

G major key signature

These are historical explanations for the structure of major scales, of seven different notes (plus the same notes in other octaves) that sound good together. There are scientific explanations too, even ratios of wavelengths.

On the piano, this means only 7/12 of the keys are used in any song, ordinarily. Why have the other keys? They bring equality to the different notes: any key can start a scale, because the whole-steps and half-steps are there no matter where you start. C is not special, just convenient. The circle is complete. Actually 12 circles for the 12 keys in an octave. So many patterns are visible to me now!

Now I can name the key signatures and say why they have those names. I don’t have to memorize them, because I can derive them. Next I can learn chords and why certain ones like C and F and G7 appear together frequently. All of this from two pieces of vocabulary and some counting. Goodbye magic, hello math.