Great software teaches

Great software solves a problem that you have — plus problems you didn’t know you had.

Here’s an example: today on Twitter, a friend let me know about a broken link to one of my old posts:

The broken link lives in someone else’s blog post, so I can’t update the source. It looks like the link has been broken since I migrated from Blogger to some months ago. Darn!

Ideally, the old link would redirect to the correct one, the one Gary found after a bunch of looking. This would fix the internet (just a tiny bit).

How hard is it to make that work?

Look at this beautiful plugin that appears right at the top when I search for “redirect”:

Redirection plugin. It has a million zillion installations

Perfect. I installed it (thanks to $23/month to for my business site) and entered the two URLs and poof, the redirect worked. That took under 10 minutes.

But wait! There’s more!

During the installation it asked me whether I wanted logs of 301s (requests to my site that got redirected to the right place) and 404s (requests to my site for a page that is not found). Yeah, sure.

After entering the one redirect I knew about, I saw it work in the log of 301s. Then I clicked on the 404 report, and in that couple minutes it had already noticed two more broken requests!

part of the 404 report. It shows the link I just fixed plus two others I did not expect

So I fixed those too! Hovering over the link in the report even gives an “Add redirect” option that populates half of it for me. Amazing.

This Redirection plugin is great software. It worked to solve my specific problem. It teaches me more about the wider problems I’m having, and helps me solve those too. Brilliant.

Reading: Why Information Grows

Yesterday in a zine, I read an in-process book review of The Rise and Fall of the Roman Empire. Reading that was a project for the review author; they wrote the review while still in the thralls of the book. That seems like the best time, not after I’ve closed the book. So quick! before I listen to the epilogue:

César Hidalgo is a statistical physicist. His book, Why Information Grows, is about biology and economics. It scales from the quantum to molecules to life to ecosystems to human systems. The common theme at every level is: information, embodied in matter. Processed by systems of increasing complexity to produce even more information. This is how our planet keeps being interesting, while the universe as a whole marches toward the boringness of maximum entropy.

Traditional thermodynamics says entropy always increases, OK, and then it uses this principle to predict the behavior of closed systems near equilibrium.

Which is no system ever.

There is only one closed system: the universe. And it is nowhere near equilibrium. When you get into a planet like Earth, which then develops feedback loops like self-replicating proteins, which develop into feedback loops like ecosystems, and eventually into human systems — we’re only moving farther from equilibrium. Energy is constantly injected into the system, and we use that to process and create higher and higher levels of information. More and more interesting stuff.

Hidalgo extends this into economics, and uses it to explain geographic income disparity.

The trick is: information by itself is useless. To use it, a system/entity/person needs enough knowledge to know what it means, and enough know-how to do something with it.

His key metaphor is that products are embodied information. To create any product takes knowledge and know-how. From a little, like garments, to a lot, like airplanes. That knowledge and know-how must exist inside a country for that country to make such products. And knowledge and know-how are way, way harder to move around than raw materials. They exist inside people, and networks of people, and networks of networks of people. So a person can make a rug, a firm can make a jersey, and many firms together can make a satellite.

Furthermore, Knowledge and know-how create knowledge and know-how. There’s a feedback loop: I know how some aspects of how to build software, so I build it, and I work with other people, and we learn from each other and the knowledge in our particular piece of the world increases. We also create new k&kh, both in ourselves, plus embodied in our software, which can give us and other people new tools to build new things. It spirals.

So software concentrates in Silicon Valley. Movies in Hollywood. Financial instruments in New York. Cars in Germany and Japan.

I’m also reading a book now, Cybernetic Revolutionaries, about a sophisticated cybernetic computer system built in Chile in 1971–72, a collaboration between many Chileans and a few key foreigners. Stafford Beer and some other British computer scientists worked with them on cybernetics, economic modelling, and programming. Guy Bonsiepe (German) worked with them on industrial design. The project and the whole country were greatly held back by US embargos which deprived them of parts and computers that Chile didn’t have the knowledge and know-how to build for themselves. The story illustrates the value of acquiring knowledge in a nation, and the limitations of not already having it.

Personally, this concept of knowledge and know-how being embodied in people — not in text! text only helps if you already have the knowledge to attach meaning to the text. And the way it transfers is by working together. By pair programming, not by documenting your code.

I can build a thing; we build the thing together; now you can build the thing (and better). We know this is how we learn best. Why do we pretend that text is sufficient?

There’s also the part where, he looks at cars or pillows as embodied information: products have the value that you can get benefit from the information without having to understand all of it. Software is very obviously that. The value in the software is what people can now do, that they couldn’t before without possessing a whole lot of knowledge and know-how that the creators of the software put into it. Note that here, software is a medium, and the value is in the business, in whatever users can do with it.

In the Three Body Problem series, one of the characters describes inhabited planets as “low-entropy systems,” and humans and aliens as “low-entropy beings.” We get there by using and creating information, by means of the knowledge and know-how embodied in the organization of our components — proteins, cells, neurons, employees, firms, nations. This book has me thinking hard about how knowledge is created and moved. Because it is not through books or the internet — printed language is the feeblest form of knowledge transfer. Working alongside each other is the richest. Can we do more of that?

Get happy developers to do what you want

(originally published in the SD Times)

How do we influence the operation of a complex dynamical system like a team? How do we get a development team to function closer to our preferences, and still let people operate as self-actualized humans?

In Dynamics in Action, Alicia Juarrero describes complex systems such as people in terms of the constraints they operate within. Constraints make things interesting, and constraints allow for emergence.

Constraints here aren’t hard rules. They aren’t restraints, holding people still. Rather, constraints are changes in the probability of particular actions.
They’re like … an incentive landscape.

For instance: if you force people to change their password frequently, they’ll use a terrible password. If coffee is free and juice costs money, people will drink more coffee. If the stairs are pretty and easy to access, people will take them more often than the elevator.

These are all ambient (context-free) constraints. They don’t restrain the team from taking the elevator, but they do change the probability of whether each person, on each occasion, will do so.

restrain: one allowable path. constrain: one prettiest path

As a software organization — maybe as an architect, or technical lead, or SRE, or infrastructure team — how do you get developers to code in a certain way? to be consistent with the rest of the organization, to follow your 
preferred practices? Getting developers to do anything is supposed to be like herding cats.

Change the cost, change the behavior.

Development is a creative undertaking. Restraints don’t sit well. But if you make something easier for me, that’s a constraint I can live with. It will change my probabilities.

I do this for myself: a script to make submitting a single change as a PR into one step, and now I less frequently lump that function rename in with other unrelated changes.

My team expects me to look at every change coming in to the repositories we work on together, in many time zones. That seemed onerous, until I made my “git fetch and move my default branch pointer” script also open a GitHub compare window. Now this feels normal, and I wouldn’t work without this routine.

When we work in computers, a new automation changes our constraints, and this changes the probabilities of our various behaviors.

I can affect other people in my organization: I want people to improve the descriptions in our GraphQL schema, so I made an automation that deploys the schema when they push a button in Slack. I made the button appear whenever someone changes the schema in GitHub. They use this all the time.

When my coworker wants the dependencies alphabetized: sorry, I don’t care enough. He can add an autofix — a TypeScript function in our software delivery machine — to do that.

I can change the incentive landscape by making the way I want people to make software, the easy way to make software. Sometimes by giving them a button, other times having a robot come behind and clean up every push with an autofix. The autofix has the added benefit of keeping the change I want in the system, which is often harder than making the change.

Contrast this with code metrics as an incentive: metrics distract from our collective goal of making this particular software great. They draw our attention to numbers, instead of customers. Metrics are great for information, but as soon as they’re used to alter the incentive landscape, they distort it in unintended ways.

Whether for yourself or the other developers in your organization, consider automation a tool to constrain (influence) the way developers work, without curtailing freedom. Increase developer happiness and consistency at the same time!

Horizonal goals

Video version here

There’s this great, short book by John Kay called Obliquity. It’s about goals that you can’t achieve by aiming for them directly; you have to look for an oblique goal that will happen to get you there. Like, you can’t aim for “happiness;” you have to find something such that aiming for it makes you happy, like raising children or writing or helping people who are hurting.

This book gives a name to some parts of my seamaps. The star at the top is the “high-level objective,” the unquantifiable goal which can never be achieved. Aiming for it sends us in a direction which happens to obliquely fill a goal such as “happiness” or “profit.” Goals such as “change the way development is done” or “find the optimal combination of music and words” or “address the observability needs of modern architectures” These are horizonal goals; as we make progress, the state of the art moves. We can never reach the horizon, but aiming for it takes us interesting places.

The mountains in the seamap are milestones. They’re achievable, measurable goals that we work toward because they’re in the direction of our high-level objective. Periodically we climb up and look around, take stock of whether our current direction is still going toward our star, and if not, change our milestone goals.

There are many smaller milestones on the way to the bigger one. Each offers an opportunity to take stock and possibly shift direction. There are actions that we take to move toward these goals. This is us in the boat, rowing.

Obliquity adds another element: necessary states. A necessary state to moving toward the next feature is: tests are passing. A necessary state for teamwork is that we are getting along with each other. Many of the actions we take are aimed at maintaining or restoring necessary states. These are like the whirlpools in my seamap; we have to smooth them out before we can row in the direction of our choice.

For example, here is a seamap for my current activity:

high-level objective: change the way people think about programming. Goal: explain Symmathecist. Subgoal: explain Horizonal. Necessary state: don't be too drunk. Action: type this post before opening wine.  

I will now hit “publish” and go open a bottle of wine.