Development aesthetic: experiments

Software development is a neverending string of “What happens if I…?” Each new runtime, language, or tool is a new world with its own laws of nature. We get to explore each one with experiments.

Objective

Today I added another alias to my PowerShell $profile:

echo "Good morning!"
# ...

Function ListFilesWithMostRecentAtBottom {
    Get-ChildItem | Sort-Object -Property LastWriteTime
}
Set-Alias ll ListFilesWithMostRecentAtBottom

To use that alias in my current shell, I need to source the profile again. I googled how to do this. The first useful page said:

& $profile

So I typed that. It echoed “Good morning!” but the alias did not work.

Hmm, did it not save?

I can test that. I changed the echo to “Good morning yo!” and tried again.

It printed the new text, but still didn’t get the alias.

Hmm, is something wrong with the alias?

I opened a new shell window to test it.

The new alias works in the new window. Therefore, it’s the & $profile command that is not doing what I want.

Investigation

I could ignore the problem and continue work in the new window. My alias is working there. But dang it, I want to understand this. I want to know how to reload my $profile in the future.

Time for more googling. The next post had a new suggestion:

. $profile

I typed that, and it worked. yay!

But wait, was that the old window or the new window? What if it only worked becuase I was in the new window?

I want to be certain that running . $profile brings in any new aliases I just added. For a proper experiment, I need to see the difference.

Experiment

I add a new alias to my $profile, and also change the echo so that I’ll be sure it’s running the new version.

echo "Good morning yo tyler!"
# ...

Function ListFilesWithMostRecentAtBottom {
    Get-ChildItem | Sort-Object -Property LastWriteTime
}
Set-Alias ll ListFilesWithMostRecentAtBottom
Set-Alias tyler ListFilesWithMostRecentAtBottom

In my terminal, I run tyler as a test case, then the command I’m investigating (. $profile), then the test case tyler again.

Now I can see the before and after, and they’re different. I can tell that . $profile has the desired effect. Now I have learned something about PowerShell.

Epilogue

I remove the extra tyler stuff from $profile.

As far as I can tell, & runs the script in a subshell, and . runs the contents of the script in the current shell. The . command works like this in bash too, so it’s easy for me to remember.

Today I took a few extra minutes and several extra steps to make an experiment and figure out what PowerShell was doing. Now I know how to reload my $profile. Now you know how to run a tiny experiment to ascertain that what just happened, happened for the reason you think it did.

This verbosity makes me happy

Today I learned how to create aliases in PowerShell. I’m switching from Mac to Windows, and I want the terminal in VS Code to do what I want.

No terminal will work for me until it interprets gs as git status. I type that compulsively.

In bash, setting that up looks like this:

alias gs='git status'

But in PowerShell, aliases can only refer to single words. No parameters. Wat.

You can make a function with the whole command in it, and then set an alias to that function.

Function GitStatus { git status }
Set-Alias gs GitStatus

The first time I did this it felt kinda silly. But then the second time …

Function CommitDangit { 
    git add .
    git commit -m "temp" 
}
Set-Alias c CommitDangit

This alias c makes a crappy commit as quickly as possible. I use it when live coding, to make insta-savepoints when stuff works. (I’m a bit compulsive about committing, too. Just commit, dangit!)

The PowerShell syntax requires a long name for my command before I give it a short one. This is more expressive than the bash:

alias c='git add . && git commit -m "temp"'

My CommitDangit function is named for readability, plus a tiny alias for fast typing.

This is a win. I like it more than the bash syntax. PowerShell is a more modern scripting language, and it shows.

Bonus: in bash I put those aliases in a file like .bashrc or .bash_profile or sometimes another one, it depends. In PowerShell, I put the aliases in a file referenced by $profile. Edit it with: code $profile, no figuring out which file it is.

Next: reload the $profile in an existing window with . $profile

Progress in Learning

Back in the day, artists mixed their own paints. They bought minerals, ground them up, and mixed them with binder. Then in the 1800s, metal paint tubes became a thing, and people bought smooth, bright paint in tubes.

Do they still teach mixing your own pigments in art school? Maybe, in one class. I took one class in Assembler when studying Computer Science. That gave me some useful perspective and some vocabulary, but the baby computer designs we looked at were nothing like the computers we use.

I’m betting that art schools don’t return to pigment-grinding for months out of every year of school. There are new styles to learn, higher-level methods of expression that build on top of paint tubes.

Why do my kids spend months out of every year learning to do arithmetic by hand? They could be learning set theory, new ways of thinking that come from higher-level math.

Why are they learning cursive instead of coding? We can express ourselves at higher levels through computers.

Human culture advances through products. Paint in a tube is a product that lets artists focus on painting. It lets them work in new locations and at new speeds. Monet and the other Impressionists painted as they did because they could finally paint outside, thanks to tubes of paint!

I don’t want new computer programmers to learn the basics the way I had to learn them. They don’t need to program in C, except maybe one class for perspective, to learn the vocabulary of memory overflow. Learn queueing theory, that’s a useful way of thinking. Don’t implement a bunch of queues, except as extra credit.

Some artists still mix their own pigments.

Viewing a painting produced entirely with hand-ground mineral pigments is a completely different experience than looking at one made with modern chemical paints. The minerals scintillate and their vibrations seem to extend from the canvas.

Laura Santi

We need a few specialists to implement data structure libraries and programming languages. There are contests for mental arithmetic, if you enjoy that game. Calligraphy is a great hobby. When I sit down to learn, I want to learn new ways to think.

When younger people skip the underpinnings and learn higher-level concepts, that’s called progress.

Why is CSS a thing?

All I want is a web page. I want this one thing on the left and this other thing on the right — why is this so hard?? Can I just make a table in HTML like I used to do in the nineties? Why do I have to worry about stylesheets? and, why are they so hard?

As a backend developer, I’m used to giving the computer instructions. Like “put this on the left and this on the right.” But that is not how web development works. For good reason!

As the author of a web page, I do not have enough information to decide how that page should be laid out. I don’t know who is using it, on what device, in what program, on what screen, in what window, with what font sizes.

You know who does know that stuff? The user agent. That’s a technical term for an application that presents documents to people. The browser is a user agent. The user agent could also create printed documents, or it could speak the document to a person whose eyes are unavailable.

The user agent runs on a particular device. Computer, phone, TV, whatever. It knows the limitations of the hardware. It can be configured by the user. The user agent can conform to various CSS specifications.

CSS is not a programming language. It is a syntax for rules, rules which give the browser (that user agent) clues about how to display the document. The browser combines that information with what it knows about the world to come up with a format to display (or speak) the document.

It turns out that rule-based programming is hard. It sounds like it should be easier than imperative code, but it is not.

So no, you don’t get to decide that this thing goes on the left and that thing goes on the right. The browser gets that choice.

But here’s something I learned yesterday: put each thing in a div, and give those divs display: inline-block. then the browser has the option of putting them next to each other, if that fits with those constraints that only it knows.

Implementing all the interfaces

Humans are magic because we are components of many systems at once. We don’t just build into systems one level higher, we participate in systems many levels higher and everywhere in between.

In code, a method while is part of a class which is part of a library which is part of a service which is part of a distributed system — there is a hierarchy, and each piece fits where it does.

An atom is part of one molecule, which combines into one protein which functions in one cell in one tissue in one organ, if it’s lucky to be part of something exciting like a person.

But as a person, I am an individual and a mother and a team member and an employee and a citizen (of town, state, country) and a human animal. I am myself, and I participate in systems from relationship to family to community to culture. We function at all these levels, and often they load us with conflicting goals.

Gregory Bateson (PDF) describes native Bali culture: each full citizen participates in the village council. Outside of village council meetings, they speak for themselves. In the council, the speak in the interests of I Desa (literally, Mr. Village).

Stewart Brand lists these levels of pace and size in a civilization:

  • Fashion/art (changes fastest, most experimental)
  • Commerce
  • Infrastructure
  • Governance
  • Culture
  • Nature (changes slowest, moderates everything else)

Each of these work at different timescales. Each of us participates in each of them.

We each look out for our own interests (what is the fashionable coding platform of the day) and our family and company’s economic interest (what can we deliver and charge for this quarter) and infrastructure (what will let us keep operating and delivering long-term) and so on.

Often these are in conflict. The interests of commerce can conflict with the interests of nature. My personal finances conflict with the city building infrastructure. My nation might be in opposition to the needs of the human race. Yet, my nation can’t continue to exist without the stability of our natural world. My job won’t exist without an economic system, which depends on stable governance.

If we were Java classes, we’d implement twenty different interfaces, none of them perfectly, all of them evolving at different rates, and we’re single-threaded with very long GC pauses.

Tough stuff, being human.

These are not the only options. Wineglass edition

Today I found myself in the kitchen, near the fridge with the wine (it’s an excellent Chardonnay from Sonoma, thanks XYZ Beverages in Tybee Island, you exceed my expectations although you don’t have a website to link to). My empty glass was out on the screened porch.

Do I go outside for the glass? Or take the wine bottle to the glass, and then return it to the fridge?

These are not the only options. I snag another wineglass from the cupboard, fill it with wine, and take that out to the porch.

Now I have two dirty wineglasses, but who cares? The dishwasher washes them all at the same rate.

This is garbage collection in action. The dishwasher acts as a garbage collector for dirty dishes. It adds the capability of “do not worry about how many dishes you dirty. They will all be cleaned for the same fixed cost that you have already incurred.”

This removes one consideration that I need to think about in my actions. I’m free to optimize for my higher-level objectives (“be on the porch, with wine in a glass”) while ignoring the accumulation of resources (dirty wineglasses). It takes some adjustment to benefit from this condition.

It takes some adjustment in a brain to move from scarcity (“Dishes are a resource with a cost”) to abundance (“dirty dishes meh, not a problem anymore”). Once adjusted, the options opened to me are widened, in a way that a clearly optimal path is opened.

Now pardon me while I finish this delicious glass of wine and fetch another, from the nice cold bottle still in the fridge.

For cleaner code, write ugly code

We want to write only clean code, right?

Wrong. I want to write eventually-clean code. It starts exploring a space, and then I refine it to be cleaner and more suited to purpose. Usually, that purpose becomes clearer through writing, reading, and using the code.

That process of refining or tidying up can feel tedious, compared to implementing more features. It can be tempting to leave off error handling. I have a strategy for that: meaningful ugliness.

When I’m prototyping, I make the code so ugly that it will be satisfying to clean up. No attempt at appearing clean. I put a bunch of casts, bad casing maybe, random names instead of plausible-but-inaccurate ones. No null-checking.

for (const gerald of allKindsOfStuff.fingerprints) {
    (gerald as any).displayName =
         allKindsOfStuff.feature.convertinate(gerald.name);            }

(exaggerated to show detail)

When cleaning, I often start by making the code uglier. To move from an OO style toward functional, start by replacing all global or class-level variables with parameters. Twelve parameters on a function?! That’s hideous! Yes. Yes. Let ogres look like ogres.

This lets me feel productive when I come back to the code and clean it up. Later, I know more about what this code is for, what might be null and what won’t, what dependencies I can eliminate and which are meaningful. This is a better time to clean.

Disguising rough code in socially-acceptable clothing prevents cleaning. Appearance-of-good is the enemy of better.

When the db is the interface

There are two huge sources of inertia in software: data, and interfaces.

When two systems connect to the same database, you combine both. Ow.

When some other system is doing reporting on my database, I can’t change my table structure. That severely limits any internal reorganizations I might do. Especially when I don’t know how they use it, so I am afraid to touch it.

Then what if it’s a schemaless database? This means the schema resides in the application. When the schema resides in two applications at once, designing change is an exercise in hope.

Sometimes two systems access the same database because the database is the interface. We have an example of that at Atomist currently: one system (called squirrel, I don’t know why) populates a database, and another system (I call it org-viz) uses that data to make visualizations. How can that be okay?

Database as interface is not horrifying when:

  1. The database is not the system of record. We could repopulate all or part of it from elsewhere, so it’s somewhat disposable.
  2. The database has a schema. Interfaces must be well-defined.
  3. One system updates, the other reads.

In our case, we use PostgreSQL, so we have a schema. Some of our fields are JSON, which gives us extensibility, while the essential fields are guaranteed. The populating system (squirrel) operates from a queue, and we can replay the queue as needed, or play it into multiple databases. There are options for designing change.

Database as an interface is never going to be the cleanest decoupling, but it is not unreasonable when it is carefully designed and the teams don’t mind talking to each other. When the database is accidentally an interface, then you’re horked.

Feature interaction

Why is legacy software so much harder to work in?

Why does development velocity slow down as a system grows?

Lots of reasons, sure, but I suspect the biggest one is oft unspoken.

Every new feature comes with the invisible requirement: “and everything else still works.”

Every existing feature, and even past bugs, makes every new feature harder. Every user with expectations is a drag on change.

Feature interactions are hard! And they surprise you. My favorite examples come from The Sims release notes:

  • Fish are no longer duplicated in the fridge when moving homes.
  • Televisions no longer play video after they are burned or broken.
  • Sims will no longer walk on water to view paintings placed on swimming pool walls.
  • Pianists will no longer continue playing pianos that have been detonated.
  • Sims will no longer receive a wish to “Skinny Dip” with Mummies.
  • Sims who are on fire will no longer be forced to attend graduation before they can put themselves out.
  • Sims can no longer “Try for Baby” with the Grim Reaper.

Your feature interactions are probably not this much fun.

Software gets harder to change as it becomes part of a larger system, with more users and more uses. Development slows down because it’s useful, and we want it to stay useful. This also makes each change more valuable, because it’s helping more people in more ways.

Don’t be discouraged. Do be diligent, and be okay with being slower. This is the price of success.

Teams are like bread

Maybe when companies make you do “team-building” activities, what they’re looking for is a phase transition into a gelled team. Because it is a sudden, magical thing, right? When a group of people turns into a team.

Once you get there, to that feeling of team, it’s self-reinforcing. You trust each other, so y0u don’t take miscommunications personally; you work to restore communication, and so trust increases. You understand each other, so it’s easy to build further understanding. Working together gets smoother and smoother.

But how do you get there? pfft. How do you make sourdough bread? You grow it from sourdough starter, which you got from … someone else’s starter. Or from putting sugar water out on a windowsill and hoping some yeast lands in it. Seriously.

Will Larson suggests that when you have a gelled team, keep it. If you need to adjust how many people are helping with which pieces of software, then shift responsibilities from one team to another, not people.

When you have a gelled team, you can grow it gradually. Let the team reform with the new member incorporated before adding another.

Will suggests that when you want another team, gradually grow a gelled team up to 8-10 (max size) and then fork it into two teams of 4-5 (minimum size). It’s kind of like the sourdough starter: grow it, divide it, make the bread. Keep it alive the whole time.

If you have one team where the magic is flourishing, don’t kill it. Feed it, grow it, and let it be a source of further strong teams. No rushing.

Otherwise – if you take the group to paintball, or get them to mob program, or put them in a team room with sugar and water, maybe the yeast will blow in?

(if you want these little posts as I make them, plus a bit of extra context, sign up for my newsletter!)