Friday, May 5, 2017

Code and Coders: components of the sociotechnical system

TL;DR: Study all the interactions between people, code, and our mental models; gather data and we can make real improvements instead of guessing in our retros.

Software is hard to change. Even when it's clean, well-factored, and everyone working on it is sharp and nice. Why?

Consider a software team and its software. It's a sociotechnical system; people create the code and the code affects the people.
a blob of code and several people, with two-way arrows between the code and the people and the people

When we want to optimise this system to produce more useful code, what do we do? How do we make the developer->code interactions more productive?

the sociotechnical system, highlight on each person
As a culture, we started by focusing on the individual people: hire those 10x developers! As the software gets more complex, that doesn't go far. An individual can only do so much.

the sociotechnical system, highlight on the arrows between people The Agile software movement shifted the focus to the interactions between the people. This lets us make improvements at the team level.
the sociotechnical system, highlight on the blob of codeThe technical debt metaphor let us focus on how the code influences the developers. Some code is easier to change than other code.
We shape our tools, and thereafter our tools shape us. - McCluhan
the sociotechnical system, highlight on the arrows reaching the codeTest-driven development focuses on a specific aspect of the developer<->code interaction: tightening the feedback loop on "will this work as I expected?" Continuous Integration has a similar effect: tightening the feedback loop on "will this break anything else?"

All of these focuses are useful in optimizing this system. How can we do more?

Thereʼs a component in this system that we haven't explicitly called out yet. It lives in the heads of the coders. Itʼs the developerʼs mental model of the software.
a blob of code and two people. The people have small blobs in their heads. two-way arrows between the code and the small blobs, and between the people
Each developerʼs mental model of the software matches the code (or doesn't)
Every controller must contain a model of the process being controlled.
-
Nancy Leveson, Engineering a Safer World
When you write a program, you have a model of it in your head. When you come to modify someone else's code, you have to build a mental model of it first, through reading and experimenting. When someone else changes your code, your mental model loses accuracy. Depending on the completeness and accuracy of your mental model of the target software, adding features can be fun and productive or full of pain.

Janelle Klein models the developer⟺code interaction in her book Idea Flow.  We want to make a change, so we look around for a bit, then try something. If that works, we move forward (the Confirm loop). If it doesn't work, we shift into troubleshooting mode: we investigate, then experiment until we figure it out (the Conflict loop). We update our mental model. When weʼre familiar with the software, we make forward progress (Confirm). When weʼre not, pain! From the book:

to make a change, start with learn; modify; validate. If the validation works, Confirm! back to learn. If the validation is negative, Conflict! on to troubleshooting; rework; validate.

That 10x developer is the one with a strong mental model of this software. Probably they wrote it, and no one else understands it. Agile (especially pairing) lets us transfer our mental model to others on the team. Readable code makes it easier for others to construct an accurate mental model. TDD makes that Confirm loop happen many more times, so that Conflict loops are smaller.

We can optimize this developer⟺code interaction by studying it further. Which parts of the code cause a lot of conflict pain? Focus refactoring there. Who has a strong mental model of each part of the system, and who needs that model? Pair them up.

Idea Flow includes tools for measuring friction, for collecting data on the developer⟺code interaction so we can address these problems directly. Recording the switch from Confirm to Conflict tells us how much of our work is forward progress and how much is troubleshooting, so we can recognize when we're grinding.

Even better, we have data on the causes of the grinding.

We can reflect and choose actions based on what's causing the most pain, rather than on gut feel of what we remember on the day of the retrospective.

Picturing those internal models as part of the sociotechnical system changes my actions in subtle ways. For instance I now:
  • observe which of my coworkers are familiar with each part of the system.
  • refactor and then throw it away, because that improves my mental model without damaging anyone else's.
  • avoid writing flexible code if I don't need it yet, because alternatives inflate the mental model other people have to build.
  • spending more time reviewing PRs in order to keep my model up-to-date.

We can't do this by focusing on people or code alone. We have to optimize for learning. Well-factored code can help, but it isn't everything. Positive personal interactions help, but they aren't everything. Tests are only one way to minimize conflict. No individual skill or familiarity can overcome these challenges.

If we capture and optimize our conflict loops, consciously and with data, we can optimize the entire sociotechnical system. We can make collaborative decisions that let us change our software faster and faster.

4 comments:

  1. Very nice post!

    DDD and Specification by Example strive to deal with mental model issue. More tools to help optimize the system.

    ReplyDelete
  2. "Idea Flow" is a great book describing the process of a developer updating their mental model and how that can be a source of pain at times. I'd recommend also checking out "Domain Driven Design," by Eric Evans. It talks about making the domain model a first class citizen of the system.

    For example, one technique from domain driven design is the ubiquitous language, a practice that helps avoid the need to constantly "translate" between the business and the developers. Non-technical owners are also stuck with some of the pain in the model, to the degree that it doesn't express their intended design or is hard to change. By bringing non-technical owners and technical people to the table around a shared language will go a long way toward eliminating the "wall of ignorance" that Klein refers to.

    ReplyDelete
  3. Enjoyed reading this one - specifically the conflict/confirm cycle - from Idea Flow !. Could you detail on the last line with some examples - in another post : "If we capture and optimize our conflict loops, consciously and with data, we can optimize the entire sociotechnical system"

    ReplyDelete
  4. Some programming paradigm supports simpler execution model (thus part of the mental model) of the application system, such as functional programming encouraging immutability would especially help. It also helps by providing higher level abstractions such as dictionary, list, and higher order functions, map, filter, reduce, etc.

    I personally feel "literate programming" would also help by putting software code into explicit integrated narrative context. It may provide more unified mental model of the system among team members.

    ReplyDelete