Subtasks vs Capabilities

There exists a myth about software: that we can start with what we want it to do, break that into subtasks and subtasks of subtasks that we can each specify completely, implement all those subtasks, and then assemble them into something useful.

Harold Abelson explains it in this classic lecture (h/t @pesterhazy for the link). He also explains the problem, and a solution.

He draws the hierarchy of subtasks. If we learn something and need to make a change at a spot in the middle, the work below it is all invalidated. And changes to that specification may ripple out to the sides, ruining everything.

Look at all these elegant subtasks, suddenly rendered worthless

That hierarchy of subtasks seems rational, but it is not practical. Instead, Abelson points out, the way things really get done is in levels of capabilities: fleshed-out subdomains of the problem. If we want to build graphics, then we need a capability of making shapes, and for that we need a capability of locating points. Each of these has its own vocabulary and implementation. Each is coherent and broadly useful, specialized to a problem domain but not a single task.

Abelson is working in a Lisp, and Lisp makes it easy to write mini-languages to solve each problem. So his capabilities work out to levels of language. Build a language to describe points and locations, then geometry on top of that, then combinations on top of that.

When something changes, chances are you can express the new need at one of the higher levels without changing anything further down.

The capabilities (his “language levels”) use each other flexibly.

These are capabilities. We need the capability to describe points, then a capability to assemble them into geometries, and on that we can build a capability of combining those into useful images.

The capabilities depend on each other, and they have specifications, but each is broadly useful. They are built for a range of closely related purposes, not one specific place in the whole.

This myth also exists in our companies and societies.

A culture is like a big organization which assigns each of its members a place where he can work in the spirit if the whole…. In an age without culture… forces become fragmented and the power of an individual man is used up in overcoming opposing forces and frictional resistances.

Wittgenstein, Culture and Value

Here, Wittgenstein is very wrong. (To be fair, he didn’t publish this; the book is a posthumous collection of notes.) A culture certainly doesn’t break work into subtasks. But our companies do!

And it doesn’t work, either. You get rigidity, you get people held back by needing to go through others for specific subtasks. When infrastructure, purchasing, and marketing are elsewhere and we are required to delegate to them, this creates friction. The subtasks are never independent, so this doesn’t work even in theory.

Instead, create capabilities. For instance, to make useful software, we need infrastructure. Don’t create a subtask, assign it to a team, and make every software development team wait on them. Instead, create infrastructure as a service. Form a capability, not a fiefdom.

Build teams and software as capabilities, not subtasks.

“Software” isn’t a thing

You can’t make Software. There is no “software”–there is only “software that…” or “software for…”.

That’s like having a business around “writing.” Writing what?? Your writers are going to have to learn every topic they’re asked to write about.

That’s either going to take a long time, involve very close collaboration with someone who does know the topic, or be vapid and unimpressive.

If your core business is “software” — which of these is yours?

Certainty as blindness

How long before it dawns on us that the world we see no longer reflects the world we inhabit, that we are blind?

Blindsight, by Peter Watts

This quote is about the rare cases of people who are blind but insist they can see. And it is about all of us sometimes.

There was a model of the world, and we didn’t look outward at all; our conscious selves saw only the simulation in our heads, an interpretation of reality, endlessly refreshed by the senses.

Blindsight
painting by Cyndi Taylor

Until it isn’t refreshed anymore, until we retreat deep into our model and no longer perceive the differences in the physical world, or perceive them only as wrongness.

After decades in a waterfall world, where information officially passed through tangible documents, can a person see the casual flows of information in a product team working closely with a customer? Can they see the tests that happen in pairing with the developers? Can they see the frustrations cut short by pairing together?

They see wrongness, they see wasted time, they see information lost when it is not stored in a file cabinet. It takes releasing an old model to see what new things are happening.

Our model of how the world works is precious to us. It is precious because we have a place in it: we belong in that world, the world we grew into, the world that needs us.

Loosening that model hurts, it is disconnection, it pulls us away from what we know and love. But that world is gone. We can grow into the new one, if we can see it. To see it, we have to loosen our current model. It helps to seek being wrong.

Whether we want to change the world or find our place in it, it helps to see it. To see it as it is, right now, not as it was or as it ought to be.

That’s hard; by adulthood, we each have a model of the world that has worked for us. It is normal to go deeper into that model rather than shake out of it. Seeking out the places where our model is wrong can push us back into uncertainty, into vulnerability, into a breadth of possibility we haven’t owned since childhood.

We don’t grow up blind. We grow into blindness. We can grow out of it.

I will not install your language tools

I have a principle now: no installing languages or tools or any project-specific stuff on my laptop. Only Docker. Every project can happily have its own version of Ruby or Python and their dependency managers. When my laptop gets lost again, I won’t spend (as many) weeks being surprised by what I need to install again.

Here is my tiniest intro to using Docker for this purpose. Not for production, but for my purposes. And for making projects work-on-able by other people on every operating system.

Find it on RubyTapas:

Complication for the win

“How can we make our programs simpler?”
I don’t want to make them simpler. I want to them make them complicated instead of complex.

Simplicity is not the goal.

Simpler implies they do less, they hold less rich domain knowledge. I want them to do a lot, to take work and thinking away from the user. Yet I still want them to be understandable by my team.

Like, say I’ve been to the grocery store and I’m stocking up on a lot of stuff. There’s a lot of stuff to bring in from the car.

How can I make this task simpler? Easy. Buy less stuff.

But I don’t want to buy less stuff. I’m gonna carry all this in. It feels like the simplest thing is to make one trip, load it all up in my arms. Head for the house and… shoot. Maybe I can kick the front door until someone opens it.

What seems “simple” might be the most complex.

Carrying all the groceries at once is a complex task. Every item I’m already carrying affects how I pick up the next one. Every extra pound feels significant, since I’m already encumbered. Every step is careful. And then I get to the kitchen and try to put it all down and crack the laundry detergent just came down hard on a carton of eggs.

Now that I’m older and wiser and have studied this phenomenon in software, I don’t try this anymore. I carry a few items in, then carry a few items in, etc etc until all the groceries are in the kitchen. This is more complicated because there are way more trips. It is less complex because each independent trip is simple.

Oodles of simple tasks are collectively complicated, but not complex.

I can carry two bags and a laundry detergent. One bag and the giant package of TP, easy. I can open the door for myself. I can put them safely on the counter.

I can take the detergent straight to the basement; every task doesn’t have to be the same as long as each one is simple.

As a bonus, my partner might start putting groceries away while I’m still carrying. The kids might join in and grab a few bags. When I’m carrying the whole mass they couldn’t grab one from me, because that would destabilize it all. But they can grab some from the car. Many simple tasks can be shared; one complex one cannot.

I’ll take complication over complexity any day.

In programs, it is tempting to see something as simpler if it has smaller parts. If one Order class has all the behaviors we need for an Order, that’s sounds simple. Then all those behaviors get intertwined and the Order class needs to change for many reasons and yet it’s scary to change because there are so many things we might break!

If we have a frobozz.purchasing.Order class that has behavior for building the order, and a frobozz.fulfillment.Order class with behavior around shipping the order, and a frobozz.billing.Order class with behavior around getting paid — gah! Three classes for one thing? Too complicated!

Yes, complicated. But far less complex.

Complication grows linearly, complexity… ugh.

twelve piles of groceries
twelve trips might be twelve times as complicated

Complicatedness can be measured (sort of) by a count of parts. Complicated systems can be broken down, broken down and each part understood alone, although that may take a lot of work because there are so many of them.

TP balanced on cases of beer held by arms barnacled in plastic bags — each part is leaning on another, interdependent.

Complexity is different. It exists in the interdependence between parts, which stops you from safely changing any of them independently. A complex system can only be broken down so far; you have to consider much of it in one lump. The most complex part determines the complexity of the system. It’s a maximum, not a sum.

Don’t be afraid of programs with a lot of parts. As long as those parts are independent — they communicate only with data, for instance, not interdependent database calls — then it doesn’t matter how many they are. Complication is not our enemy.

Aiming for “simplicity” by reducing complication leads us further into the morass.

Complexity bites us. Embrace complication, and keep your eggs safe.

PowerShell for `rm -rf`

TL;DR – Remove-Item -Recurse -Force <path>

On linuxy systems, rm -rf <path> means “remove this path and everything under it, dammit.” If the files aren’t writeable, but you own the files, it deletes them anyway.

In PowerShell, rm is aliased to Remove-Item, but it doesn’t accept -rf.

Windows Terminal> rm -rf foo
Remove-Item : A parameter cannot be found that matches parameter name 'rf'.

I can call Remove-Item without -rf, and it still works, but it asks me for confirmation.

Windows Terminal> rm foo

> rm foo

Confirm
The item at C:\Users\jessi\code\jessitron\injectify\foo has children and the Recurse parameter was not specified. If
you continue, all children will be removed with the item. Are you sure you want to continue?

I can say “A” for “Yes to all” and then it does what I want.

Unless! There are files in there with do-not-write permissions. There are zillions of these in the .git directory, for instance. Sometimes I want to wipe that out and start a fresh git history. Then I get a zillion errors:

Remove-Item : Cannot remove item
C:\Users\jessi\code\jessitron\injectify\.git\objects\bf\203afb5389983253213646fa165f749fdcaf09: You do not have
sufficient access rights to perform this operation.
At line:1 char:1
+ Remove-Item -Recurse .git
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : PermissionDenied: (203afb5389983253213646fa165f749fdcaf09:FileInfo) [Remove-Item],

Here’s the secret formula:

Remove-Item -Recurse -Force <path>

Recurse is for deleting all the directories and files underneath; Force is for overcoming those permissions errors. This will get rid of .git for me.

It’s a little sad that -rf doesn’t translate to -Recurse -Force. It is not an unreasonable abbreviation. Maybe in a future PowerShell release, it will.

How will you collaborate?

When there are decisions to be made, people have feelings about it. Some people want to make the decisions, others to influence it, others to hear about it.

Where does everyone in your group stand? Are you set up for conflict or for smooth collaboration?

For instance, say there’s a new product to start. We will need to decide how to market it, how to price it, and how to build it. Within building it, there will be all kinds of smaller decisions, from platform to architecture to every tiny piece of code.

Collaboration Contracts let us get explicit about how we will make decisions.

Doc Norton suggests: everyone pick your place in each decisionmaking. There are 6 options.

the six options as expressed by different colors of tooth people
which one are you?

Are you making the decision?

  1. Explainexplain person is orange“I will make this decision, and explain it to others.” No obligation to accept input.
  2. Consult: consult person is red“I will make this decision after consulting with others.” You can influence me, but I will decide.
  3. Agree: agree person is purple “We will make this decision by coming to agreement somehow.” Method of agreement to be determined later. You can influence us.

Otherwise, are you involved at all?

  1. Advise: advise person is green “You make the decision after I advise you.” I have thoughts; hear them and then I’ll respect the decision you make.
  2. Accept:accept person is blue“You make the decision. I’ll accept it.” Whatever it is, I’m in.
  3. Abstain:abstain person is grey “You do what you want. I abstain from this whole activity.” Why am I even in this meeting?

This lets us identify baked-in conflicts.

After everyone declares the role they expect to play in a decision, you can notice conflicts.

"explain" and "consult" and "agree" roles conflict with each other
“explain” and “consult” and “agree” conflict with each other.

If more than one person thinks they are making the decision, and they’re not all in the “agree” category, you have a problem.

"explain" and "advise" roles conflict with each other too

If someone thinks they’re making the decision unilaterally (the “explain” category), plus at least one person sees themself as “advise,” you have a problem.

Once you identify conflict, you can renegotiate. The person who thought they could make the decision without input can agree to ask others. The person who thought they would consult can seek agreement instead, or the people who wanted to agree can delegate the decision.

This model is useful at all levels of decisions. I find it especially interesting to apply it to low-level code decisions.

We can apply this model to collaboration on code.

Down at the nitty-gritty implementation level, someone is at the keyboard. Maybe its me. If I’m working alone, then I’m making the decisions. I am in the “explain” role.

After the code change is ready, what happens?

It’s OK to make unilateral decisions.

Maybe I push it to master, with an explanation of my decisions in a commit message. No conflict there.

Maybe I create a pull request, including an explanation of what I did and why. Then what?

If my colleage glances at it, sees that the build passed, and hits “merge,” maybe they’re taking the “accept” role and we’re fine.

There is conflict when we expect our decision to fly but it doesn’t.

But what if the reviewer has suggestions? Then they’re taking the “advise” role, and that is in conflict with my “explain” role. Looks like I actually have the “consult” role, except that I don’t get to consult until after I make the decision once and then I have to go make it again.

Pull request fight. Me: "Let's do this!" Them: "But what about..." Me: "Fine, they we'll..." Them: "That's not OK" Me (very sad): "Work with me here!"
the pull request slide, from decisionmaking to placation

And what if those comments aren’t suggestions, but blockers? Then we are both in the “agree” role. I thought it was my decision but it wasn’t; only I had to make the decision first and then someone else pops in and now we have to agree.

These are not smooth. I do not enjoy this.

We have ways of collaborating smoothly.

two "agree" people at one computer

If we were aiming for the “agree” role all along, why not start there? In pair programming, one of us is typing, but we make decisions together.

Or even better, mob programming: the person at the keyboard (the driver) is not allowed to make decisions, so every decision is spoken aloud. The driver is in the “accept” role.

mob programming with a navigator: an "accept" person with the keyboard; one "consult" person; several "advise" people

If your mob has a navigator, then each person takes a turn in “consult” and the rest are “advise.” This works.

mob programming: one "accept" person has the keyboard. Everyone else is "agree"

Without a navigator, everyone is “agree.” This works too.

Consider the design of your collaboration.

This model illuminates why pull requests hurt so much. They have conflict built in to their structure.

To make pull requests work, I have to lower my expectations.

I’ve learned to consider my pull requests to be suggestions. I’m proactively taking an “advise” role in a decision I hope some project committer will make. Success can look like: they implement the same functionality in a manner of their choosing. My PR functions as a very specific feature request. It doesn’t matter that my code didn’t make it into production, if my idea did.

The only exception to this is when I know the project maintainer (or am the project maintainer), and I come to agreement with them before submitting the PR. Then I expect it to be accepted, after some minor comments and changes are exchanged.

When you see conflict, consider your collective expectations.

Next time you feel resentment at someone else’s intrusion, or frustration at decisions made without you, try to picture the roles everyone thought they were playing. Are they compatible? Was conflict endemic to the structure of this interaction?

If so, don’t blame yourself, nor the other person.

Ask, can we be clear about our decisionmaking intentions? and then adjust them to remove conflict before it starts?

Be specific about your expectations. Set yourself up for smooth collaboration.

What makes a dependency dependable?

Lately I’ve been working in Node, and that means I have access to all┬áthe open-source modules on npm. It is easy to add a dependency on any of them. But is it a good idea?

Not every library is production-grade.

Not every library (or service) is something that I want my production code to depend on.

The first thing I look at when a module comes up in an npm search is: how many weekly downloads does it have? This is a cheat. “It’s production grade if other people use it in production” is a reasonable heuristic, but it doesn’t tell us how code becomes worthy of extensive re-use.

What does it take to make a production-grade library?

Some static qualities:

  • The code works as expected.
  • It is fast enough.
  • It has a well-designed API that lets me write readable code to call it.
  • The documentation is accurate and complete enough.

We look at:

  • What version is it on?
  • When was the last commit?
  • How many committers are there?
  • How many answered questions about it are on StackOverflow?

These touch on something deeper: how does the library change?

When I build my software on top of a library (or runtime, framework, toolchain etc), I want that library to be solid — and I want it to become more solid over time. I don’t want it to shift under my feet!

I want to know:

  • When there’s a security problem, a fix is released promptly.
  • Useful features are added (but not too quickly).
  • New features follow the pattern of existing APIs.
  • When I upgrade, my code will continue to work.

For bonus points:

  • Before I upgrade, I’ll still have access to the documentation for the version I use.
  • There is advance notice of upcoming features.
  • Someone from the core team responds to GitHub issues, StackOverflow questions, and tweets of despair.

I expect a production-grade library to practice responsible change.

Responsible change considers the larger world, all the people and software depending on the system that’s changing. It starts with backwards compatibility, regression testing, and careful API design. It continues with documentation, expectation-setting, and responsiveness to questions.

Very little of this happens in the code. Responsible change is a property of something larger. Whether in a company or a team of volunteers, a production-grade dependency is part of a symmathesy. Software resilience comes from people — people who have a mental model of how the software works. People who are actively involved in keeping it safe, and making it more appropriate for the shifting present.

That’s why we look for a team of committers, for recent attention, for a broad base of interest.

Most npm modules have a single author and haven’t changed in years. That’s fine! No open-source author is responsible to anyone for putting more work into that code than they darn well please. There are other ways to use this code than to depend on it.

If I can’t count on code to change responsibly, I don’t want it to change at all. I prefer to freeze the version of the library forever. If it needs to change, that’s my responsibility. I fork the library — or for similar effect, cut and paste the code into my project (with attribution).

This reduces the code that I have to write, but it does not reduce the code that my team is responsible for.

There are two kinds of open source: production-grade code with a community or company behind it, and code that I now own.

Next time you type “npm install,” make a guess as to which one you’re getting.

No Roadmaps for Worldbringers

Goals are for the smaller stuff,
specifics of this week, today
because we can’t see far enough
to guess what new surprises lay
beyond that. Top this nearest hill
and pause to find the next clear way.
A mountain in the distance still
calls to us, and yet we stay
grounded in the valley here
and set a goal not far away.

Tomorrow, top this next near hill —
the valley grows a different shape.
This landscape wasn’t formed until
we moved here. We create the place
we stand in; then we look for how
the world responds. This gives us play
to find where rock and rain allow
a level spot, where we can say:
We’re closer to that distant peak,
a taller state toward which we aim.
Because from there we just might reach
the star! And then the world will change
again.

Landing Zones, Long-term Desires, and Impossible Dreams

How do we get from here to where we want to be? Hint: don’t draw a roadmap. The road we’ll travel in six months doesn’t exist yet.

Landing zones

Landing zone: “an improvement that would feel like an accomplishment, as well as a pause point to catch breath, reassess, and plan how to achieve the next better thing.”

Esther Derby, 7 Rules of Positive, Productive Change

Big changes come from small changes. Good thing, since small changes are the ones we can make.

We create the path to where we will be one setpping-stone at a time. The smaller the step, the better.

Find the smallest step you can take that puts you one detectable bit closer to where you want to be — or, to where you have more information to know where you are and how to get where you need to go. Make it specific, so you know when you’re there and it’s time to look around.

Does your site need a redesign? First, improve the help text on a confusing button. Or, add events that show where people are using the interface in an unexpected way.

Small progress is the best progress, and information is also progress.

Long-term desires

Landing zones link “near-neighbor states” with long-term desires.

Esther Derby, 7 Rules of Positive, Productive Change

The bigger states we’re trying to achieve can be less specific than the landing zones we use day-to-day. They can tell us when it’s good enough, and it’s time reprioritize.

Maybe: 80% of customers who start make it all the way through enrollment.

We may still reprioritize before getting all the way up this mountain, but this gives us a direction to look in for day and week goals.

Esther suggests making a horizon map (sadly, this term does not google well) to get from long-term desires to landing zones. Map backwards from the desire by asking, “What conditions would have to exist for this to be true?” until you get to the current state. Then move forward one step, one landing zone, and check in.

Impossible Dream

You might achieve a long-term desire. Then what? You pick a new one, based on your organization’s purpose.

Everyone needs a direction to look in for the next mountain to climb. This purpose should not be achievable, because then your organization should close! We need a Quest, an Impossible Dream.

This is:

  • build better airplanes than the world has ever seen
  • help everyone in the world start their own business
  • advance systems thinking in software until the whole world works better

(It reminds me of John Cutler‘s North Star, and long-term desires are themes in that framework.)

We need a star over the horizon to point the direction, a long-term desire as a big step in that direction, and many landing zones along the way to detect progress and assess the landscape.

Keep commitment-style goals down to the landing zone level, so that we can keep our heads up and pointed toward our star.