Studying biology or economics, one finds organisms, ecosystems, and economies that are more than the sum of their parts. Somehow many interacting agents with limited information produce increasing organization, creating amazing complexity out of relatively simple components.
A fact of software development:
For every 25 percent increase in problem complexity, there is a 100 percent increase in complexity of the software solution. That’s not a condition to try to change (even though reducing complexity is always a desirable thing to do); that’s just the way it is. [2]
This is true because every feature we add potentially impacts every feature we already support. The increase in solution complexity is combinatorial, not linear. The way to combat this is to reduce the complectedness of our software. Stop intertwining all the bits — a coherent architecture is more efficient in lines of code, but it is inherently limiting. It is limited by what the God Architect can hold in his head.
Break the software into simpler components. Some of these may be open-source components, which someone has to learn well enough to configure and implement. Others will be custom. Break them off into teams and give those teams autonomy. Let each team implement its piece in the simplest way possible for that particular problem.
Meanwhile, the systemwide architects don’t know the particulars of each solution. In fact, they should not know the particulars. That would allow them to base solutions on implementation details. Don’t let that happen! Raise the level of abstraction. Components must interact with abstractions, not with each other. Knowledge of inner workings of other modules is a negative.
Jeff Bezos forced this strategy at Amazon around 2002 by decree: all teams will interact via service interfaces only; they can use whatever technology they want; and all interfaces must be externalizable, ready to be exposed to the outside world. The result? The largest online bookseller became the largest vendor of cloud computing. Did anyone predict that ten years ago?
When you and I interact, our brains do not interact directly. Rather, we both interact with the physical world. I say something, you hear it. Abstractions on both sides reduce the granularity of the interaction, but maintain the independence of our individual brains. No mind-melds allowed. While this seems limiting, observation shows that amazing unpredictable structures – nations, communities, economies – emerge from these limited interactions.
Complicated systems are limited in growth. Complex systems have even greater potential for growth than the designers of the components conceived. Keep your components simple and independent, and there is no end to the problems we can solve.
[1] Complexity, A Guided Tour, by Melanie Mitchell, chapter 7
[2] Facts and Fallacies of Software Engineering, by Robert L. Glass, p. 58
VERY nicely put!Where I struggle with this is in user-interface development.
Fantastic article.
Hy jessieThat's exactly what Udi Dahan and others have been talking about for a long time. True SOA style architecture embodies your reasoning and even provides answers in how to compose user interfaces in such an architecture. The beauty with such decomposed systems is not only that you can idependently develop it or use different languages you can also apply JFHCI (just freakin hard code it). This has huge business advantage: you can invest in quality where you have good knowledge and are pretty sure you got it or hard code parts where you are uncertain but want to gain insights into the market.Nice article!
Neat comparison of \”complex\” vs \”complicated\” and how it applies to software systems. It's a great and logical expansion/application of loose coupling principles.Also worth noting, the biology analogy you cite is a great to not only show the possibilities, but the dangers of the \”simpler components\” approach. While biological systems are indeed a very complex sum of their simple parts, the system came to be through evolution, not design, and there's some serious systematic consequences to this. Vestigials, instinct-driven decisions, and uhh, male nipples? Software systems that develop through evolutionary means (especially those that are \”componentized\” by autonomous teams) can become a whole new kind of complex. It starts with: do we have a webservice that formats addresses? Oh great, err, it doesn't work exactly the way we want… ugh, that team is backed up. We'll just write our own.Years later: so…. which of the 6 address formaters should we use?Or worse, when the business rules change (\”street names now precede house numbers\”)… oy. So, I think it's good to have a balance of automony and knowledge of *some* of the inner workings to allow for planning.
Hey, male nipples are awesome!In the example given – does inner-working knowledge help with that at all?The sin here is that some team added to their piece of the system an additional responsibility. What if the address formatting module was open-source? It has a team of maintainers but anyone can contribute. That's one way to deal with the resource-allocation issue, which is really what triggered this bifurcation of responsibilities and violation of DRY.
In starting up a business, colorado llc would like to know how those software helped the business to improve and to gain such success