Progress in Learning

Back in the day, artists mixed their own paints. They bought minerals, ground them up, and mixed them with binder. Then in the 1800s, metal paint tubes became a thing, and people bought smooth, bright paint in tubes.

Do they still teach mixing your own pigments in art school? Maybe, in one class. I took one class in Assembler when studying Computer Science. That gave me some useful perspective and some vocabulary, but the baby computer designs we looked at were nothing like the computers we use.

I’m betting that art schools don’t return to pigment-grinding for months out of every year of school. There are new styles to learn, higher-level methods of expression that build on top of paint tubes.

Why do my kids spend months out of every year learning to do arithmetic by hand? They could be learning set theory, new ways of thinking that come from higher-level math.

Why are they learning cursive instead of coding? We can express ourselves at higher levels through computers.

Human culture advances through products. Paint in a tube is a product that lets artists focus on painting. It lets them work in new locations and at new speeds. Monet and the other Impressionists painted as they did because they could finally paint outside, thanks to tubes of paint!

I don’t want new computer programmers to learn the basics the way I had to learn them. They don’t need to program in C, except maybe one class for perspective, to learn the vocabulary of memory overflow. Learn queueing theory, that’s a useful way of thinking. Don’t implement a bunch of queues, except as extra credit.

Some artists still mix their own pigments.

Viewing a painting produced entirely with hand-ground mineral pigments is a completely different experience than looking at one made with modern chemical paints. The minerals scintillate and their vibrations seem to extend from the canvas.

Laura Santi

We need a few specialists to implement data structure libraries and programming languages. There are contests for mental arithmetic, if you enjoy that game. Calligraphy is a great hobby. When I sit down to learn, I want to learn new ways to think.

When younger people skip the underpinnings and learn higher-level concepts, that’s called progress.

Systems and context at THAT Conference

It’s all that

THAT Conference is not THOSE conferences. It’s about the developer as more than a single unit: this year, in multiple ways.

I talked about our team as a system — more than a system, a symmathesy. Cory House said that if you want to change your life, change your systems. As humans, our greatest power lies in changing ourselves by changing our environment. It’s more effective than willpower.

Cory and his family on the grassy stage

Many developers brought family with them; THAT conference includes sessions for kids and partners. It takes place in the Kalahari resort in Wisconsin Dells. My kids spent most of their time in the water parks. Socialization at this conference was different: even though fewer than half of attendees brought family with them, it changes the atmosphere. There’s a reminder that we are more than individuals, and the world will go on after we are gone.

my friend Emma, daughter Linda, and me at the outdoor water park

Technical sessions broaden perspective. Joe Morgan put JavaScript coding styles in perspective: yes, they’re evolving and we have more options to craft readable code. But what is “readable” depends on the people and culture of your team. There are no absolutes, and it is not all about me.

Brandon Minnick told us the nitty-gritty of async/await in C#, and how to do things right. I learned that by default, all the code in an async function (except calls with await) runs on the same thread. This is not the case in Node, which messes with the thread-local variables we use for logging. But in C# it’s easy to lose exceptions entirely; the generated code swallows them. This makes me appreciate UnhandledPromiseException.

Ryan Niemeyer gave us 35 tips and tricks for VSCode. I love this IDE because it is useful right away, and sweetly customizable when you’re ready. Since this session, I’ve got FiraCode set up, added some custom snippets for common imports, enabled GitLens for subtle in-line attributions, and changed several lines in a file simultaneously using multicursor. And now I can “add suggested import” without clicking the little light bulb: it’s cmd-. to bring up the “code actions” menu for arrow keys. Configuring my IDE is a tiny example of setting up my system to direct me toward better work.

Then there was the part where my kids and I goaded each other into the scary water slides. They start vertical. They count down “3, 2, 1, Launch” the floor drops out from under you and you fall into the whooshy tube of water. I am proud to have lived through this.

From personalizing your IDE, to knowing your programming language, to agreeing with your team on a shared style, our environment has a big effect on us.

McLuhan’s Law says: We shape our tools, and our tools shape us. This is nowhere more effective than in programming, where are tools are programs and therefore malleable.

But our tools aren’t everything: we also shape our environment in whom we hang around, whom we listen to. Conferences are a great tool for broadening this. THAT Conference is an unusually wholesome general-programming conference, and I’m very happy to have spoken there. My daughters are also ready to go back (but not to do that scary water slide again).

Understanding Promises in JavaScript (with yarn and Legos)

TL;DR: creating a Promise splits off another path of execution, and the Promise object represents the end of that path. Calling .then adds code to the end of that path.

You can think of your program’s execution as following a piece of yarn. this video illustrates the difference between a synchronous program and the same program using Promises:

Promises let you by explicit about what needs to happen after what, while giving you more flexibility than “each of these things happens one at a time in this order” (the default flow of a simple synchronous program).

The negative is that when you want to specify “do this after that Promise,” you have to package up that code and pass it to .then() . The Promise object holds the end of the yarn representing its path of execution; .then() ties more code onto the end and returns the new end.

See this in the readConfig function, which reads a file and parses its contents. The synchronous version executes on the program’s usual path of execution: readFileSync retrieves some bits, and then JSON.parse turns them into a useful object.

synchronous: one piece of yarn proceeds straight down the code.

In the version with promises, readConfig returns immediately, but what it returns is the end of a piece of string. It’s a piece of string that includes readFile, which fetches some bits; tied on by .then() is JSON.parse, which turns those bits into a useful object.

promises: the end of a string is returned, an orange piece tied on to a dark blue piece

The useful object will be available at the end of the orange string to whatever code gets tied on to it later.

Promises beat callbacks in this respect: when you start up the asynchronous task, you don’t have to provide alllll the code that needs to execute after it. You can add more later, as long as you keep hold of the end of the string.

Don’t lose the end of the string! If you don’t need it to add any more code, tie the string off neatly with .catch() — otherwise an error might come out of a stray end and mess up your program. (I could do another video on that.)

Promises don’t beat callbacks in that you still have to wrap subsequent code up into a function. It gets messy when you have .then() calls within .then() calls. But wait! Don’t get discouraged!

In TypeScript and ES2018?, we can write asynchronous code in the same simple format using async and await. While the code looks almost the same as the synchronous version, the paths of execution are more like the Promises one.

async function: just as many pieces of yarn as the Promises, with no calls to .then()

The async function returns immediately — don’t be fooled by that return statement way at the end. It splits off a path of execution, which does work (here, reading the file) until it hits theawait keyword. The rest of the code (parsing) becomes another piece of string. await ties the strings together just like .then() (except way prettier). At the end of an async function is a return statement, which supplies the value that will come out the end of the string. Anasync function always returns a Promise.

Promises give you more control, so they give you more to think about. This means they’ll always be more complicated than synchronous code. With async and await we get both control and clarity: what Avdi calls “straight line code that just thunks itself onto the work queue whenever it gets stuck.” Don’t fear Promises, do use TypeScript, and do keep hold of the ends of your strings.

The Imperial Yaks

(continued from Attack Yak; series begins with Taxonomy of Yak Shaving) Sometimes you’re coding along, writing tests as little experiments “this should fail because I haven’t implemented the parser for it yet” — and it fails in a way you didn’t expect. And then you start digging and the parsing library isn’t working how you expected. And that library (it’s our library) doesn’t have a test for this case. Now you need to switch your development environment over. The yak stack is getting deeper. You have encountered …

The Imperial Yaks!

Imperial Yaks, aka the Yak Stack; quote from yakbreeder.com

These are the yaks that stack on top of each other. They all deserve to be shaved. This is a real bug in the library. It needs more tests. Then fix the bug, then deal with the linter and make a PR and wait for its review and do a release… and in the meantime to move forward we must figure out how to get npm to use local dependencies, which is not trivial with TypeScript.

How far do you go? When do you look for a workaround — maybe we can parse this with regular expressions and not use our library. When do you ask “Is this change even worth making?”

Pairing helps. I love pairing because I can be down in the weeds of the syntax and the particular bug while my pair still has part of her brain operating at the level of “why are we doing this again?”

If your original task is urgent: look for workarounds sooner. Never abandon the yak silently; create an issue if you don’t have time to fix it. Leave enough information to reproduce the bug. (This is work.) At the very least, throw something into Slack for tracking.

If the yaks you uncovered are dangerous: they might be more important that your original task. Realize you’re doing something important. Update your team and re-orient.

In the example about that failing test, it’s Rod who hit that last Saturday. He didn’t get his particular goal accomplished over the weekend. But he fixed a bug that was going to hold Clay back. He added tests. I popped in and paired on it, and added a few more pending tests for corner cases, which Rod later fixed. Sylvain merged the changes and did a release. All of us are more concerned with Atomist as a whole than whatever assigned tasks are on our list. That’s our culture.

When I aim for generativity, I can think more broadly about yaks. I can decide whether the yak stack confronting me is worth defeating, or going around. I feel good about creating an issue, if that’s all I can do right now: a warning sign to other people. Or leaving the code a little clearer than I found it: more null checks, an extra test, type declarations (in gradually-typed languages like TypeScript or Clojure).

My yak stack is now deeper than it used to be. Every confusing error I get, I solving it until I’ve made the error more communicative, seen the new message, and then committed that. Or changed the README. When I started at Stripe, after 4 months I felt like I wasn’t getting enough work done, and when I expressed this to others — because when I’m freaking out, it’s my job to ask for help — Julia and Danielle were like, but you’ve been making so many commits! Some managers at some companies look only at assigned tasks, but you can bet all my coworkers saw those README and error message and infrastructure changes and thought “Thank you!” about each one. Stripe has a culture of generativity, so I was fine.

It isn’t just about me and my personal productivity. Teamwork isn’t a race, with each person trying to be faster than the rest of the team.

Yak Racing

In the past, I’ve worked with developers who hoarded information, excluding others from customer meetings, insisting on a code structure that makes sense to them and no one else, turning work into a competition with the rest of us.

This exclusively-personal-maximization is also a danger of the next kind of yak: the Trim Yak.

The Attack Yak

(This post describes the first yak category. You might choose to start with the Taxonomy of Yak Shaving intro.)

Black Yak, aka Attack Yak. (quotes are from yakbreeder.com)

When you feel that your task is 80% complete, these yaks form the next 80%. They are the distance between “works on my box” and “this is done.”
All the data setup. Verification. Approvals. Status checks. Linters and code review. These are the yaks we have summoned voluntarily. There are also surprises like, I’m ready to integration test, but the dev environment is broken. Or merge conflicts. Or an unrelated test failure. Or someone wants to argue about whether this change is even a good idea.

Attack yaks are very frustrating, but there’s no getting around them (without cheating). You have to shave them. They make our development flow rough, by forcing us to context switch. What can we do?

Be cautious about summoning these yaks. We choose to all write automated tests. It doesn’t make us faster today, but it makes our development smoother in the medium and long term. Except when it doesn’t; intermittent failures, especially in tests that are too wide to be diagnostic, can make development rough instead. Many fine-grained tests prevent refactoring, or enforce functionality that isn’t needed anymore but nobody knows it. Be conscious in your yak shaving. Don’t shave its eyebrows.

On some projects, we summon the yak of code reviews. What is the objective? If it’s code quality, pairing is live, yakless code review with no context switches yeah! If it’s keeping up the team’s mental model, then keep reviews focused on that; discourage nitpicks. As a reviewer, if you want to see another test, add it. Give me a commit instead of an attack yak.

:attack-yak:

Track the attacks. We have a yak emoji in Slack for these.

:attack-yak: Staging environment is down, I can't test

If we all do that, we can search Slack before the retrospective to find out which ones are interrupting us the most. Our retro can have a little more data than our gut feel, which is really based on the last few days.

Coordinate your attack. Once we know which yaks are holding us back, shave them with automation. Shave them until they’re not an interruption anymore. Autofix on the linter. Script creation of the dev environment, so that “it’s broken” means “recreate it” instead of “figure out what the problem is today and create a special custom fix.” Then automate regular re-creation.

This work can feel like a waste of time, because it isn’t getting features delivered. Yet, if we think about our work as more than personal productivity, we see the value. On my team, we care about generativity (as opposed to productivity): the difference in what your team does with you, vs what it would get done without you. It’s about the team, not about me. And it’s about the team in the future, not only right now. Not about going faster, but about going smoother: removing obstacles for everyone, present and future us.

Every time you hit a build or environment failure, ask yourself, “How could this be easier to diagnose?” It might be “put a better error message here.” Fill the potholes you encounter. Smooth the path for future-you and future-team.

Sometimes you’re coding along, writing tests as little experiments “this should fail because I haven’t implemented the parser for it yet” — and it fails in a way you didn’t expect. And then you start digging and the parsing library isn’t working how you expected. And that library (it’s our library) doesn’t have a test for this case. Now you need to switch your development environment over to work on that (have you automated that?). The yak stack is getting deeper. You have encountered …

The Imperial Yaks!

A Taxonomy of Yak Shaving

Yak Shaving is: doing seemingly-unrelated tasks to get some real task done.

The name has obscure roots and doesn’t matter; the key is that it makes you think, “Why would you do that?”

Yak shaving can be: I want to make dinner, but I’m doing stairs; moving objects around in order to clear a path to put away the blender which is on the counter in the spot I need to unload the groceries. And then I’ll search through my email to find the name of the recipe so I can google it, and there in my email there will be something urgent to respond to, it’ll make me check my calendar which will remind me that I have to be somewhere tomorrow morning which means I need to tell my husband right now that I’m going to need the car, which starts a discussion. The milk is getting warm on the counter. These are all yaks: they’re not what I want to accomplish, but they are on the way to cooking dinner.

Oh and then the recipe says, “Beat the egg whites until stiff” and I look up what that means. I learn that the best tools for beating egg whites are a copper bowl and a balloon whisk — do I go buy these now? Gathering and learning the proper tools is a yak shave, too.

In programming, yak shaving has a reputation as hoop-jumping or time-wasting. Sometimes it is, when I’m shopping for the perfect keyboard under shadow of a promised delivery date. Other yak shaves are key to a smooth development process: activities that save you production outages, that spare your team days of troubleshooting, that guide your users toward the happy path and make them love your app.

There’s a balance here, between making the task doable and forcing the task to be done. Between scrubbing the counters and unloading groceries on the floor. Between researching top balloon whisks, and using a plastic whisk that’s dirty from last week.

Let’s categorize these yaks. Then we can talk about when to fight them, when to avoid them, and when to lovingly shave them clean. Because some of them are worthy opponents! Some of them are distractions, and some of them have secret wisdom on their skin underneath all that hair.


Yakbreeder.com lists five varieties of domestic yak. I’ll use them to categorize the yak shaves that we might do while programming, so that we can decide which ones to skip, and how to tackle the others.

The goal is a team development flow that moves forward while getting smoother every day, more predictable and more fun.

There’s a lot to cover, so this is a series. Start with: the Attack Yak!