Correctness

How important is correctness?

This is a raging debate in our industry today. I think the answer depends strongly on the kind of problem a developer is trying to solve: is the problem contracting or expanding? A contracting problem is well-defined, or has the potential to be well-defined with enough rigorous thought. An expanding problem cannot; as soon as you’ve defined “correct,” you’re wrong, because the context has changed.

A contracting problem: the more you think about it, the clearer it becomes. This includes anything you can define with math, or a stable specification: image conversion, what do you call it when you make files smaller for storage. There are others: ones we’ve solved so many times or used so many ways that they stabilize: web servers, grep. The problem space is inherently specified, or it has become well-defined over time.
Correctness is possible here, because there is such a thing as “correct.” Programs are useful to many people, so correctness is worth effort. Use of such a program or library is freeing, it scales up the capacity of the industry as a whole, as this becomes something we don’t have to think about.

An expanding problem: the more you think about it, the more ways it can go. This includes pretty much all business software; we want our businesses to grow, so we want our software to do more and different things with time. It includes almost all software that interacts directly with humans. People change, culture changes, expectations get higher. I want my software to drive change in people, so it will need to change with us.
There is no complete specification here. No amount of thought and care can get this software perfect. It needs to be good enough, it needs to be safe enough, and it needs to be amenable to change. It needs to give us the chance to learn what the next definition of “good” might be.

Safety
I propose we change our aim for correctness to an aim for safety. Safety means, nothing terrible happens (for your business’s definition of terrible). Correctness is an extreme form of safety. Performance is a component of safety. Security is part of safety.

Tests don’t provide correctness, yet they do provide safety. They tell us that certain things aren’t broken yet. Process boundaries provide safety. Error handling, monitoring, everything we do to compensate for the inherent uncertainty of running software in production, all of these help enforce safety constraints.

In an expanding software system, business matters (like profit) determine what is “good enough” in an expanding system. Risk tolerance goes into what is “safe enough.” Optimizing for the future means optimizing our ability to change.

In a contracting solution, we can progress through degrees of safety toward correctness, optimal performance. Break out the formal specification, write great documentation.

Any piece of our expanding system that we can break out into a contracting problem space, win. We can solve it with rigor, even make it eligible for reuse.

For the rest of it – embrace uncertainty, keep the important parts working, and make the code readable so we can change it. In an expanding system, where tests are limited and limiting, documentation becomes more wrong every day, the code is the specification. Aim for change.

The Quality Wheel

“Quality software.” It means something different to everyone who hears it.

You know quality when you see it, right? Or maybe when you smell it. Like a good perfume. Perfume preferences are different for everyone, and quality means something different for every application.

In perfume, we can discover and describe our preferences using the Fragrance Wheel. This is a spectrum of scent categories, providing a vocabulary for describing each perfume, the attributes of a scent.

Floral notes (Floral, Soft Floral); Oriental notes (Floral Oriental, Soft Oriental, Woody Oriental); Woody notes (Mossy woods, dry woods); Fresh notes (citrus, green, water)

Perhaps a similar construction could help with software quality?

When a developer talks about quality, we often mean code consistency and readability, plus automated testing. A tester means lack of bugs. A designer means a great UI, a user means great experience and exactly the right features and lack of errors or waiting. An analyst means insightful reporting and the right integrations, a system administrator means low CPU usage and consistent uptime and informative logging. Our partners mean well-documented, discoverable APIs and testing tools.

Usability (Features, Discoverability, User Experience); Performance (Responsiveness, Availability, Scalability); Flexibility (Speed of Evolution, Configurability); Correctness (Visibilty, Automated Tests, Accuracy)

Each of these are attributes of quality. For any given software system and for each component, different quality attributes matter most. What’s more, some aspects of quality compliment each other, each makes the other easier – for instance, a good design facilitates a great user experience. Readable code facilitates lack of bugs. Consistent uptime facilitates lack of waiting. Beautiful (consistent, modular, readable) code facilitates all the externally-visible aspects of quality.

However, other aspects of quality are in conflict. Quantity of features hurts code readability. More integrations leads to more error messages. Logging can increase response time.

If we add nuance to our vocabulary, we can discuss quality with more detail, less ambiguity. We can decide which attributes are essential to our software system, and to each piece of our system. Make the tradeoffs explicit, and allocate time and attention to carefully chosen quality attributes. This gets our system closer to something even greater: usefulness.

The quality wheel pictured above is oversimplified; it’s designed to parallel the original version of the Fragrance Wheel. I have a lot more quality attributes in mind. I’d love to have definitions of each piece, along with Chinese-Zodiac-style “compatible with/poor match” analysis. If this concept seems useful to you, please contribute your opinions in the comments, and we can expand this together.