Today in standup, a colleague reported that a GitHub isn’t sending the hooks it did last week, after we wrote code to handle it, and it’s a problem for us.
“Did you contact GitHub?”
“I asked around, and I heard they’re very unhelpful.”
So no, he didn’t contact GitHub. The people writing that hook will not hear from us about how it makes their integrations less useful to everyone. Customer service, people. It’s so important!
Today T-mobile did something very wrong with my very straightforward order. I’m gonna have to call customer service and be like, “Hey, when I order a new phone and add a new line, I want the new phone connected to the new line.” Will this request ever make it back to the people who specify their software? If I had confidence it would, I would feel good about making the call. But I’ve worked at a telecom before, so I expect that data to stay with the rep, with no power to change the system.
You know what we need in order to make great software? Great feedback.
Feedback loops are the soul of any ongoing system. What is fed back, gets sustained, it gets built into the very structure of the system, that this is perpetuated. Only what is fed back.
So we need more metrics, right? 
Metrics sustain numbers. Not whatever you thought you were measuring.
We need broad feedback loops. We need information that we didn’t know we needed. We need to know what our customers experienced, what they expected and didn’t get, what surprised them, what pieces of our software make the larger system better. We need all the information we can get.
Except! all the information is the same as no information. We could look at log dumps that show all the usage. But we’d never see any of it, because it’s too much. Our attention is precious.
What we need is some of the information, and a different “some” each day. Little enough that we can listen, random enough that we sometimes encounter that precious bit that provides insight.
Breadth and surprises come from talking to people. So talk to people! Different people each day, and get their stories about today and whatever sticks in their heads.
This is qualitative data. For real feedback about the real systems we contribute to, this should be most of what we look for.
Quantitative data, metrics, should be the exception. Every metric has its dark side. Use them with caution, because they distract us from everything else that we aren’t measuring.
All kinds of harm, all kinds of opportunities are obscured by a few shining metrics.
Seek out feedback loops that are incomplete and random and broad. Cherish the ones that bring you unexpected information. And please, please, meet your company’s customer service people and beg them for their stories.
hat tip to John Ohno for inspiring this post with a tweet thread.