Before balance, separation.

We don’t program in a language, these days: we program in systems. I may write Clojure, with ring and schema and clj-http and clj-time and test.check and lein and jetty and many more inclusions. I may write Scala, with akka OR with scalaz and Shapeless, or a weird combination.

We never programmed in a language: we always programmed in systems. Dick Gabriel discusses the subtle paradigm shift when researchers separated the two, when the science and engineering of software parted ways in the 1990s. Before that, “language” and “system” were used interchangeably in some papers.

This is a valuable separation: now we can optimize each separately. In particular, languages are designed for extensibility, for building systems on. Both Scala and Clojure aim for this, in different ways. Let’s have all the existing components on the JVM available to us, and make it clean for people to build new ones. Languages are seeds of systems, and language designers seed of communities.

Take Node.js — it is not a new language, but it might as well be: it’s a new system. ┬áThe creators of Node and npm seeded a system: they left obvious necessities unimplemented, and made it easy to implement them. A community formed, and built something bigger than one small team could sustain.

When you choose a programming language, recognize that what you’re choosing is a whole ecosystem and the group of people who work on it, growing and evolving that system. Do you want that to be stewarded by a strong company? or full of individual discoveries and advances, in many open source communities? to hold you to a good style or offer free combinations of multiple styles? to be welcoming to beginners, or full of serious thinkers?

[Game: Match each of these generalizations with a community: Clojure, Scala, Haskell, .NET, JVM, Ruby.]

I’m glad that people separated languages from systems, because we can build better languages and better systems from the decoupling. Both are important, and need each other. I thought about this today while my child separated her S’Mores Goldfish into marshmallow, chocolate, and graham. She used this to produce a perfectly balanced bite.

And then she ate all the marshmallows.

The power of embedded developers

Meet my friend Sean.

Sean with a big smile

Sean is one of the most powerful developers I know. Not best, not brilliantest, but most powerful because: he provides more business value than some 50-person teams.

What is the job title of such a powerful developer? It isn’t “ninja” (although maybe it should be). It’s “Accounting Manager.” WAT.

Program: runs on the author’s machine when the author runs it.
Software: runs on any machine at any time, serving many people’s purposes.

Here’s the thing: Sean can code — he was a top developer when we worked together fifteen years ago — but he no longer writes software. He writes programs instead.

Dual wielding

When it comes to business-value power, domain knowledge adds potential; technical knowledge adds potential; and the combination is drastically better than each.

business knowledge, short bar. Technical knowledge, short bar. Both, very tall bar. Y axis, Value.

Why? In Sean’s words, “because you can actually question what you’re doing.” While breaking a process down in order to encode it as a program, his knowledge of accounting and the business needs lets him distinguish between what’s important or not, and how relevant a particular case is.

Typically, when software impacts a business process flow, it’s because the designers and developers didn’t understand. It’s a negative impact. When the designer/developer is on the business team, process changes emerge from the program design. Codifying well-understood activities reveals opportunities for synergy between businesspeople and the program. It’s a positive impact.

Sean can spot problems, observe trends, and improve processes. Software created from static requirements can only ossify processes.

How he does it

When Sean does reporting, that means: finding out what needs done and why, learning all the databases that have the information and acquiring it, joining millions of rows and performing whatever accounting business logic, putting this into a useful Excel spreadsheet, and distributing that to people – in a couple hours per month.

Those few hours of “doing reporting” each month are possible because the rest of his time is spent thinking, investigating, analyzing, troubleshooting, and programming. Given this freedom, Sean adds more business value every month.

Gardening: gradual improvement of a program. Adding error handling where errors occur, visibility where it’s needed, performance improvements where it counts. Gardening addresses the pain points after they’re felt.

When Sean does reporting, that means: automating every repetitive step, scheduling his programs, monitoring them, and gardening them. What he already does becomes trivial, and then he’s free to do more and more useful work. He and his programs are not separable; they’re both part of getting the job done. And when that job changes, he’s ready.

a person and a box inside a wall

This is an example of the code+coder ecosystem. Sean’s programs aren’t useful without him around to run them and change them, and Sean wouldn’t get much done without the stockpile of code he’s written over the years. The benefit of this system is: flexibility! When the real accountants need more or different information, Sean makes a change in 5 minutes to 3 days. ┬áCompare this to weeks when a small feature is requested of the IT department or software vendor.

There are good reasons the software vendor takes so long. Software should be bulletproof. It’s backed by fixed, negotiated, contracted requirements. It has security, authorization, failover, automated testing. Coordination, contracts, APIs. The difficulty multiplies, and software is 1000x more work to write than a program.

software system: several boxes inside many walls, and lots of people, all inside another wall

Sean doesn’t need all that, because he is the monitoring, the security, the failover. He tests with his eyeballs and his business knowledge as the results appear. As a single point of responsibility, every problem that Sean hits is a learning opportunity, and he has the ability and desire to implement the next program, or the next change, better.

The advantage of software over programs is: stability. The disadvantage is: stability. When a particular program is broadly useful and has hardly changed since Sean implemented it, that’s a good candidate to transfer over IT as a feature request.

Who can do this?

A lot of people can be this powerful. Sean is a developer who learned the business. I know other business-types who have learned to program. It’s wicked effective when it happens. It takes free time, a lot of learning, great management, and a contact network for support.

Having started as a developer with the major software vendor, Sean can talk to DBAs and sysadmins and programmers. His management had to push to get him access to some of these people, and now those communication channels are super valuable both ways. When there’s a problem, the DBAs call him to find out about the significance of various data.

the code-coder ecosystem, underneath an umbrella with a person in it to represent the managerSean’s management values his work, values him, and they work to cut through the red tape. Accounting Managers don’t usually have access to unix machines, production Oracle databases, and automated emailers. Clever people do clever things, and sometimes that surprises the corporate security departments.

The breadth of knowledge, through the business domain to finance to programs to databases to the specific software his programs interact with, doesn’t come for free. And it doesn’t stay for free. There are an infinite number of tools that might help; one person can’t know them all, but it helps to know what’s possible. When Sean took over some other person’s Access databases, it didn’t take him long to knock them down from several gigs to 250MB, because he knew what was possible. Knowing the questions to ask is more important than knowing the answers.

This knowledge doesn’t stay current for free. Most of Sean’s time is spent in meetings, keeping up on what is changing in the many systems he and his programs interact with. This is the limiting factor in growing his power.

One thing you can’t do to people like Sean: budget their time. When a person is production support and development and maintenance and design and interface to many departments, their time is unpredictable. Sometimes Sean will implement a last minute change to compensate for a bug introduced in some other software that can’t be fixed for a few weeks, and he’ll work crazy hours for a couple days. Other times, he can chip away at niggling errors, gardening his programs so that bugs like that will be caught earlier or easier to fix in the future. It isn’t predictable. Sean is flexible, and his management recognizes this. Slack gives us freedom to get better at our jobs.

What can stop us?

Motivation comes for free when people feel valued, but it’s easy to kill. One is cramming our time so full that we don’t have time to improve how we work. Another is red tape: jumping through hoops to get access to data, to get programs installed, to get questions answered. These say, “your ideas are not valued, not shut up and do the same thing every day for the rest of your life.”

There’s HR rules, which say Sean can’t get promoted or get a raise because Finance doesn’t have a job title for “Powerful Accountant-Programmer Combination.”

There’s budgets: Sean could double his power if he had a person to train. Someone to split the meetings with. And someone to carry on his legacy when he moves on to another amazing thing. Right now he’s too essential to move, and that’s another de-motivation.

The Big Problem

And when Sean does move on? In his case, the code-coder ecosystem would collapse. No one else knows his code, so they can’t re-use it. Maybe they could run it (maybe), but they couldn’t change it. They couldn’t monitor it, support it, or recognize problems when they occur sneakily. Sean is working in a dangerous silo. And he just bought a fast car.

Sean's new Honda S2000

He doesn’t want to be in a silo; a team with one or two more people would be ideal. This kind of ninja doesn’t have to work alone. There is a balance between programs and software. For many business departments, that balance may be far closer to the “program” side than software developers like me care to admit.