Go toward uncertainty

It’s difficult for an executive to criticize a budget when most line items are for mysterious high technology activities. It’s easier to tackle the more understandable portions, like postage, janitorial services, and consulting.

Jerry Weinberg

We want to help. We want to do stuff. We look for matches between what needs done and what we know how to do, and then we do it.

But does that always help?

Today in his newsletter, Marcus Blankenship told the story of the three bricklayers. What are you doing? “Laying bricks.” “Making a wall.” “Building a cathedral.”

We want to do something. We want to lay some bricks. As programmers, we want to write some tests, make a class, throw out an API.

After we break down a feature implementation into tasks, it is tempting to get started on the parts we know how to do. Knock those out, advance that progress bar. Make the wall taller.

Those are the least helpful parts to start with! In software development, our job is making decisions. What we need most is knowledge.

The pieces of the task most amorphous, kinda vague, the ones our brains want to slide past with hand-waves — those are the tasks that will give us more than apparent progress.

Integrate with that new service. Get authorization working. Pick the database and get familiar with it.

Digging into the uncertain tasks gives us information. We will learn how the API needs to be different than we thought. The data we didn’t know we needed, the unhappy-paths we didn’t know we needed to pave.

Lay the bricks slowly. Consider, what is going to hold this wall up? and what will this wall hold up?

In the opening quote, an executive tries to manage costs. They are drawn to the little nitpicky items that look approachable. But the easy ones are already pretty good! The giant Megatechnology items with big dollars next to them, these provoke handwaves. What might the executive gain by digging in and learning more about these? Maybe not cost-cutting, but definitely better decision making.

To help with the cathedral: be okay with uncertainty, sit in the unknown. Feel confused, be uncomfortable. Observe things that don’t yet make sense.

It’s tempting to start with what you most know how to do. Start instead with what you least know how to do.

Code (especially unreleased code) is a fast material to change, far faster than bricks. The hard part is deciding what to change it to. Aim for understanding, and progress will come to you.

Certainty, Uncertainty, or the worst of both

Des Cartes looked for certainty because he wanted good grounds for knowledge, a place of fixity to build on, to make predictions.

Juarrero counters that uncertainty allows for novelty and individuation.

In software, we like to aim for certainty. Correctness. Except in machine learning or AI; we don’t ask or expect our algorithms to be “correct,” just useful.

The predictions made by algorithms reproduce the interpretations of the past. When we use these to make decisions, we are reinforcing those interpretations. Black people are more likely to be arrested. Women are less likely to be hired.

Machine learning based on the past, choosing the future — this reinforces bias. It suppresses novelty and individuation. It is the worst of both worlds!

This doesn’t mean we should eschew this technology. It means we should add to it. To combine the fluidity of the human world with the discreteness of machines, as Kevlin Henney puts it. We need humans working in symmathesy with the software, researching the factors that influence its decision and consciously altering them. We can tweak the algorithms toward the future we want, beyond the past they have observed.

Machine learning models come from empirical data. Logical deduction comes from theory. As Gregory Bateson insisted: progress happens in the interaction between the two. It takes a person to tack back and forth.

We can benefit from the reasoning ability we wanted from certainty, and still support novelty and individuation. It takes a symmathesy.

This post is based on Abeba Birhane’s talk at NCrafts this year. Video