Shaping our Tools to the Flow of Conversation: Issue Creation

Collaboration is hard. There are the essential difficulties of dealing with people, and then there are incidental challenges in the tools we use. Communicating with people, while clicking into to Travis to see the build status and asking: where is this deployed? Who tested it? I want to remove all those incidental challenges, even the tiny ones.

We collaborate with tools: Slack, an issue tracker, source control, many more. We want the best tool for each purpose, but switching between them is extra work for my brain. A tiny bit of extra work, but every distraction matters when I’m focused on a complex task.

For instance, we’re talking in Slack, and realize we need to track this with a GitHub issue. Sometimes two of us create an issue at the same time — oops! I want to create that issue right in the conversation where people can see it.

The Atomist bot can do this for me:

@atomist create issue

I don’t need all the functionality of GitHub Issues in Slack, just enough plus a link. This way I rarely have to switch contexts, and neither does anyone else there in the conversation. I can claim the issue and vote for it and add a label right here. If I pop over to GitHub and update the issue, the Slack message redraws when new information has come to light.

This is a small thing.

Small things matter when they let us keep more of our brain on the hard problems of coding and collaboration with people. Every tiny mental bump I can smooth out for my team makes each of us effectively smarter.

Create Issue is a tiny piece of Atomist’s built-in automation, interweaving the tools we have to suit the way we smoothly work. There’s more: commit messages are pretty and are integrated with build and deploy status, PR notifications are mergeable, even new projects are created in line with the conversation.

We don’t need a new interface for collaboration. We need to make the ones we have as perfect as possible, then customize per team.

As developers, we have the power to create the world we want: the world that gets the tiny problems out of our way so we can solve the hard stuff.

Spend some time automating your own work, with whatever tools you have. Invite the Atomist bot into your team to get Create Issue and other integrations. Join the development automation conversation in our community Slack. We’d love to hear how you shape your tools to fit your flow.

Dictionary Objects in JavaScript and TypeScript

TL;DR: when using an object as a dictionary in TypeScript/ES6, iterate through it using `Object.keys()`.

Coming from statically typed languages, I keep looking for a Map or Dict type in TypeScript (or JavaScript). People use objects for this, though. Objects have key-value pairs in them, and you can add them and delete them and declare them statically and it looks nice, especially in literals.

const dictionary = { 
   impute: “attribute to”,
   weft: “the holding-still strings on a loom”,
}

But using a JavaScript object as a dictionary has its quirks.

There are three ways to iterate through the values in the object. (This is TypeScript syntax.)

Flip through the enumerable keys defined on an object:

for (const key of Object.keys(dictionary)) {
   const valuedictionary[key]
   console.log(`${key} -> ${value}`)
}

This one makes the most sense to me; it’s what I expect from a dictionary. It flips through the values in an array of the keys defined on that object directly, no prototypal inheritance considered. This is how JSON.stringify() prints your object.

Flip through the enumerable keys defined on that object and its prototype chain:

for (const key in dictionary{
   const value = dictionary[key]
   console.log(`${key} -> ${value}`)
}

This is the easiest one to write. It flips through the keys defined on the object and its prototype chain. If you’re using an ordinary object for this, and no one has done anything bizarre like add an enumerable property to Object, it’s fine. tslint hates it though; it bugs me about “for(…in…) statements must be filtered with an if statement.” tslint is like “OMG, you do not know what is on that thing’s prototype chain it could be ANYTHING”

I find it backwards that for(…in…) flips through property names of an object, while for(…of…) flips through values in an array. This confuses me daily in TypeScript. If you accidentally use for(…of…) instead of for(…in…) on an object, then you’ll see 0 iterations of your loop. Very sneaky. If you accidentally use for(…in…) on an array, you get the indices instead of the values, also confusing. TypeScript and tslint don’t warn about either error, as far as I can tell. 😦

Flip through the enumerable and non-enumerable keys defined on that object:

for (const key of Object.getOwnPropertyNames(dictionary)) {
   const value = dictionary[key]
   console.log(`${key} -> ${value}`)
}

This one flips through only keys on the object, not its prototype chain, and also gives you the names of non-enumerable properties. You probably don’t want those.

What are non-enumerable properties?

Conceptually, they’re properties that don’t make sense to flip through, that we don’t want JSON.stringify() to look at. They’re hidden from for(…in…) iteration and from Object.keys(). You can still access them on the object. For instance, constructors of TypeScript classes are non-enumerable properties. Methods on built-in types like Array and object are non-enumerable. They stay out of the way.

When would we want to flip through them, like in Object.getOwnPropertyNames()?
I don’t know, maybe for debugging.

Why make a non-enumerable property?
I hit a use case for this today: serializing an instance of a class with recursive fields. JSON.stringify() can’t print recursive structures.

Side quest: Making recursive objects printable

In TypeScript, every field in a class instance (including any inherited field) is an enumerable property, returned from Object.keys() and printed by JSON.stringify(). See this TreeNode class that tracks its children, and its children track it:

class TreeNode {


private _parent: TreeNode;
public children: TreeNode[] = [];


public constructor(public readonly value: string) {
}
}

Printing an instance of TreeNode gives me: `TypeError: Converting circular structure to JSON` waah 😦

Here’s a tricky way to say “Hey JavaScript, don’t print that _parent field”. Explicitly override its enumerable-ness in the constructor.


class
TreeNode {

private _parent: TreeNode;
public children: TreeNode[] = [];

public constructor(public readonly value: string) {
Object.defineProperty(this, “_parent”, { enumerable: false });
}
}

We can get tricky with TypeScript class fields. After all, they get tricky with us.

Properties on a class instance

In TypeScript, class instance fields show up in Object.keys() and accessor-based properties (like with getters and setters) do not. They’re properties on the prototype, which is the class itself. So if you want to see accessor properties, use for(…in…) on class instances. That gets the enumerable properties from the prototype chain. Watch out: these include methods.

Why iterate through the properties on a class? I don’t know, maybe for debugging again. If you do it, I suggest skipping methods. This makes tslint happy because its an if statement:

for (const propertyName in classInstance) {
  if (typeof classInstance[propertyName] !== “function”) {
    console.log(`${propertyName}=${classInstance[propertyName]}`);
  }
}

Recommendations

If you have a class instance, access its properties with dot notation, like treeNode.children. That way TypeScript can help you avoid mistakes. If you have a dictionary object, access its properties with index notation, like dictionary[“impute”] (and turn off the angry tslint rule). Class instances have specific types; dictionary objects are type object. Access the contents of a dictionary using Object.keys().

the sweet spot: Local Automation

When you learn to code, you acquire a superpower: automation. We turn the computer into a machine to do our (very specific) bidding. We get paid to automate what other people want. We can and should use this superpower for ourselves too!

Levels of automation

Are we programmers or software developers?

Probably both.

I think of a program as a piece of code that runs on my box and does something useful for me. It can be a one-line shell script or a Scala program backed by a graph database that downloads all my dependencies.

Software is also made of code. Lots of code, code on servers and code in browsers and code on mobile devices. Software runs on all the computers and is useful to millions of people. It has compatibility, integration, redundancy, persistence, security, tests, user experience, multi-region back ends and multi-platform front ends. It has program managers and technical writers and designers and operations. It has stakeholders.

Scaling and safety and compatibility make software super expensive: thousands or millions of times more expensive than a program. And then there’s the risk: will the Software be useful? (If it’s internal software people are required to use, it might even make their jobs harder. Net negative benefit.) What changes will make it more useful?

Optimization

Software and programs are degrees of automation. We build this automation in order to make a system work more smoothly. Optimizing that system is easy when the beneficiary of a change is the same person who can make it — in a program, I am my stakeholder, I define “useful.” In Software, Optimizing is super hard because the people benefiting from the software and the people who can change it are distant in the communication graph. The feedback loop is long and circuitous.

On one hand, we can work on bringing developers closer to people who can measure benefits. On the other hand, can we find a place where this is already the case?

In development teams! Developers automating for their own team is low effort, high payoff. Personally, I can make programs that only benefit me, which is limited. Go one level up: how much more work is it to write code that benefits my team? If we automate what we already do, then: it only has to run on a few known computers, no one is forced to use it, everyone who uses it can change it so tests are optional. This gives me some scale of value, with little scale in cost. If it turns out really useful, other teams in the org can copy it.


I wish I had automation to set up a build on Travis. A minimum value is certain: I want it right now. The cost is uncertain, but discoverable. I can risk spending one 20-minute iteration on it. At the end of that iteration, I’ll have more information about the cost. I can give it another 20 minutes, or set the idea aside until I hit the same problem again, which gives me information about the value. My pair and I (or the mob in mob programming) become a microcosm of a product team, for whatever timebox we set.

It helps, so do it more.

At some point we automated tests, installed version control, set up continuous integration. These are automations we no longer do without, although at one point we didn’t know how useful they’d be. What will be the next automation we won’t work without?

Here are some examples of local or personal automation that I heard about or used today.

As developers we have this superpower of automation. I think we should use it more on ourselves. We will learn more about our own work by studying how to improve it. And we will improve it, and continually grow our ability to create reliable, persistent, user-experienceable, compatible Software.

Write shell scripts! and Slack bots! and Jenkins plugins! At Atomist I have a platform to more easily scale up my automations from local to team; we’re building it. Don’t wait for us, though. Local automation is a sweet spot no matter what tool you use. Accelerate yourself.

Migrating some services from AWS to Pivotal Web Services

My objective is to run some services on Pivotal Web Services (PWS; hosted instance of Pivotal Cloud Foundry), and have them respond to requests to `https://survey.atomist.com` at various paths. Currently these services run on AWS, along with services that respond at other subdomains of atomist.com.

TL;DR: this is easy enough for HTTP requests and prohibitively difficult for real HTTPS, for only one subdomain.

This posts describes some tricky bits in this process, and the bits that leave me stuck.

Prerequisites: I have PWS set up and a few apps deployed. Meanwhile all our existing infrastructure runs on AWS.

First: multiple apps responding at satellite-of-love.cfapps.io

The instructions tell me how to point my own domain at a single app in PWS, but I want multiple apps to serve paths from my domain. The caller should not know or care which service is responding to its request for a resource.

To do this, I set up a route in cloud foundry, with a hostname (which seems to be PCF’s name for the third-from-the-right segment of the domain name, anyone know why?) that doesn’t correspond to any one app.

`cf create-route jessitron cfapps.io –hostname satellite-of-love`

Here, jessitron is my space in PWS. cfapps.io is PWS’s domain, this gets requests into Cloud Foundry for routing. satellite-of-love is a domain name that I like, it matches my github org.

That path is going to 404, but I have called dibs on satellite-of-love.cfapps.io. It’ll route to my jessitron space and no one else’s.

Now I can make routes for each endpoint and tell it which app serves it. For the /vote endpoint on Kitty Survey, I have an app running called london, so I hook that up:

`cf map-route london satellite-of-love.cfapps.io –path /vote`

Now I can hit https://satellite-of-love.cfapps.io/vote and my london app receives a request at path /vote. This is good for testing.

This part totally works with HTTPS. If you don’t mind changing your clients to point to this URL, stop here.

Second: HTTP: pointing survey.atomist.com to satellite-of-love.cfapps.io

This is DNS setup. We happen to use AWS Route53 for this. I go into the AWS console to set up a CNAME record for survey.atomist.com -> satellite-of-love.cfapps.io. There was one tricky bit to this in Route53: I clicked on the existing survey.atomist.com record (if it didn’t exist I would click Create Record Set), and tried to enter my target BUT NO
It was all “The record set could not be saved because:
– Alias Target contains an invalid value.

Here’s the trick: choose Alias: No.

With a regular CNAME (the Alias ones are an internal-to-AWS thing), I can route to an external domain from Route53.

Next, over in Cloud Foundry land, I can tell it about this domain.

 `cf create-domain atomist survey.atomist.com`

Here, atomist is my PWS org. Then I tell it to send requests to my space please:

`cf create-route jessitron survey.atomist.com`

And then I create routes for each of the endpoints, but with this new domain. (I’m pretty sure this is necessary.)

`cf map-route london survey.atomist.com –path /vote`

I’ll need to make these two routes (or at least the last one) for every endpoint I add to my service. Soon I’ll add this to my “add REST endpoint” automation in Rug.

Third: security certificates and https

There are two ways to get an HTTPS:// endpoint on PWS. They recommend using CloudFlare, which can be free. There are two problems with that. 

CloudFlare -> Cloud Foundry

The first is, to route anything at atomist.com through CloudFlare, I have to route atomist.com through CloudFlare. I have to change the routing for my entire company. 😦
Even if I did reroute our whole domain through CloudFlare, the second problem appears: I can get the appearance of security but not actual end-to-end SSL. The easy option to choose is “Flexible”, meaning users get SSL from browserCloudFlare and it looks secure to them, but behind the scenes it’s HTTP between CloudFlare and my app. This seems unprofessional to me, letting everyone’s requests happen without SSL behind the scenes while telling them it’s secure.
The other option to choose is “Full SSL,” but then I need SSL on Cloud Foundry anyway, so …

SSL in Cloud Foundry 

There’s a Pivotal SSL service available in the PWS marketplace for SSL termination. For $20/month (they don’t mention that in the documentation), it’ll let you upload one certificate.

Currently, we use AWS Certificate Manager, which provides free certificates that only work on AWS.
Can I get that for survey.atomist.com separately, while leaving the rest of atomist.com alone? I’m going to try that, from some other source — but not today.

Therefore, because our security certificates are tied to AWS
and because I decline to change the routing of our entire domain in order to experiment with this subdomain,
I give up. My toy website doesn’t need HTTPS anyway.
The moral is: if you want to experiment with moving part of your infrastructure off of AWS (designated by a subdomain), be prepared to change how requests are routed to the root domain.
Thank you to Glenn Oppegard for information about SSL on PWS, and Simon Brown who is finding this just fine with CloudFlare. 

personal automation example: deleting a local maven artifact

I want to start recording the benefits I get out of personal and local (team-level) automation.

When I have a local version of a maven library installed, say 0.19.0-SNAPSHOT, but I want to go back to using the last released version, say 0.18.0, then I need to delete the library from my local maven repo. I’ve learned the hard way that it is not enough to delete the directory in the jar; I have to delete another file too. So a while back, I scripted that.

This tiny program deletes a version of the `rug` library, since that’s usually the one I’m messing with.


~/bin/delete-version
#!/bin/bash
set -eset -x
version=$1
cd ~/.m2/repositorycd com/atomist/rugls $1rm maven-metadata-local.xmlrm -r $version

This is in ~/bin/delete-version
Today I wanted to remove a version of different library, so I didn’t call the program, but I did look at it.

cat `which delete-version`
and then I said oh right. There is the directory to go to and there’s that other file I must delete. Even though I haven’t automated precisely what I want to do, my automation was useful. 
The only real documentation is code.

The Golden Yak

(This post continues from the Royal Yak, and concludes the series A Taxonomy of Yaks.)

When we improve how we work, we make tasks faster. We make progress smoother. This is magnified when we improve how all our team members work, or our whole community. Now and then, though, an improvement turns into something more: a real change. This is what I’m aiming for when I shave a Golden Yak.

Golden Yak

The Golden Yak is the yak that changes everything, the one that alters your reality. Sometimes when you follow a problem all the way down — how do I make this easier? … now how do I make that easier? … now what if this were no work at all? you find a different universe. Making something sufficiently easier can produce a sea change, which changes our behavior.

quotes from yakbreeder.com

Example: git. It changed my behavior. Git gives me logs at my fingertips, where they used to be behind a network connection so I never used them. And it gives me a capability I didn’t have before: separating saving my work from sharing my work. The combination is magic; now I save my work carefully with a story and later read that story.

Continuous Integration. This was a huge yak for the people who first did it. It’s still a lot of work, but we wouldn’t live without it, because we (as an industry) have learned that it smooths our future work. It changes our universe by eliminating the tradeoff between frequent vs safe releases. Automated tests also change our work. They reduce fear, and that’s huge. Tests turn canyons into potholes.

Golden yaks change behavior. They start by making something so much easier that we do it way more, then they give us an option we didn’t have before or eliminate a tradeoff.

Here’s a tiny one: when I find myself typing the same command a lot, there’s the question: create an alias for this? well, I have an alias set up for “open .bash_profile in an editor” and another for “reload the aliases in .bash_profile”. Creating an alias is so easy now, there’s no reason not to. It’s gone from a tradeoff (is it worth it?) to a “duh.” Like making a commit. Don’t even ask, just do it!

Here’s a team-sized one: automate spinning up a new project. This takes away a tradeoff: “this is really a separate thing, I should make a service for it… but it takes so long, I should glom it on to an existing one.” With the Atomist bot, I made it two slack commands and two in-Slack button clicks to create a repo and a channel and a build and a deployment, and bam I have an empty service in production ready for my new endpoint. It makes creating a new service “duh.” That decision is now all about architecture, not about tradeoffs. This yak is shaved once and for all. (until there’s a change in infrastructure. And then I’ll accommodate that change in one place, instead of various documentations and people’s heads.)

Here’s one that might turn out bigger: Dependency management, unsolved problem. Code reuse, not all it’s cracked up to be. I am against creating dependencies for code that happens to be the same, that isn’t business-crucial-same. So, what’s the alternative? Cut and paste, but that’s a lot of work and we often mess it up. 
This is my current golden-yak project: libbits. Maybe this will chew a corner off the nasty problem of dependency management. Atomist has automation infrastructure for modifying code, so I can make a program to do copy, paste, modify for a bit of code that’s handy in multiple projects. And then I can make a program to create that program. I’m following the yak stack, looking for the pot of gold at the end.

Then I’d like to make it easier to make the program that writes a program. Another thing we are making at Atomist is microgrammars, a new way of parsing, somewhere between Regex and a BNF grammar. (I don’t think it counts as a yak when your CEO tells you he wants your help with it.) My personal yak is to put an API on this that is super easy to use, and who knows, we might create a replacement for `sed`.

When are we justified in shaving a Golden Yak? If you don’t work for a development automation company, I don’t know. Getting an automation smooth enough that people use it like breathing is a lot of localized work for a huge but spread out benefit. Still, coding is fun! These yaks make great play.


There’s rarely a direct work-related reason to shave them well enough to find their deep secrets, the surprising wisdom on their shining skin under all that hair. The new way of working that is revealed only after we try it. Change our environment, and thereby change ourselves.

When I sit down to work on a weekend (or some Friday afternoons), I let myself dig into some Trim Yaks, and see how far I can take them. When I do this, I’m studying my own work. As automators, we study our users’ work, right? how they do their jobs. When we automate our own work we learn about our work deeply. Writing code to change code teaches me about code.

Here’s a shortcut to Golden Yak wisdom: cultivate an awareness of what we’re doing while we do it. I learned this trick from a book about how to ride a motorcycle: Devote 5% of your attention to how the work is going, and where it could be smoother. This awareness, then, feeds into our prioritization. And into understanding of our teammates’ experiences.

Conclusion

Yaks are a part of life and of progress.

Pair: We aren’t in this alone, and we can do it better together. Pairing lets us keep perspective on what we’re doing and why. We can use that space to think about doing it better.

Proven usefulness: look for it. Don’t spend a ton of time until you’ve hit a problem a ton of times. Track your yaks and timebox your attacks. Remember it’s about generativity, not all about you. Fill potholes, and shave a little more so your team can share the benefit.

Play: Stay curious. Especially learn about the system you work in, both socially and technically. Learn a little extra each time, about your tools, your system, and your companions on this journey. Interact and automate for the wisdom; for changing us, and thereby the world.

Yaks. They’re so much more than hair.

Stupid mocha stupid error

For reference only: (and generativity)

This needs to be on the internet, because I didn’t find it in a search.

I try to run mocha via npm and pass it `-grep` or `-fgrep` and I get this awful error (can you tell how annoyed I am) about “Cannot find module ‘-e'” and I think something is wrong with npm install BUT NO

$ npm run mocha — -fgrep “annotation with string content found”
> @atomist/spring-team-handlers@0.5.10 mocha /Users/jessitron/code/atomisthq/spring-team-handlers/.atomist
> mocha –require intelli-espower-loader ‘target/ts-built/mocha/**/*.js’ “-fgrep” “annotation with string content found”
Error: Cannot find module ‘-e’
    at Function.Module._resolveFilename (module.js:485:15)
    at Function.Module._load (module.js:437:25)
    at Module.require (module.js:513:17)

What that should say is

“Invalid option -fgrep. Try –fgrep”

So when future me finds this by searching the internet next time: Jess, use two dashes before the `grep` or `fgrep` options to mocha.

The Royal Yak

(continued from Trim Yaks; part of the Taxonomy of Yak Shaving series)

Royal Yak, aka Yakkity Yak (quote from yakbreeder.com)

Talking to people is yak shaving; it is an intermediate task that helps you get your official tasks done. It’s usually seen as a separate way to “waste” time. I’m here to call relationship-building out as a productivity booster.
Conway’s Law says that our system architecture resembles our communication flow (approximated by our org chart). Indeed, the human relationship graph has a close relationship to our system relationship graph — and if it doesn’t, that’s going to be a dysfunctional hunk of software.

Those interludes of chit-chat while you’re pairing? They matter. Especially on a remote team, when we don’t have chit-chat in a kitchen. Here are three crucial objectives for talking to people at work:

The more we know about what people know, the more we know whom to ask for help. This is the first thing I ask about when I meet a new person in the company. If my service interacts with another one, it helps to know who understands that one and can expound on its history. If I need to change nginx config, it helps to know that person who likes it.

Asking for help is a beautiful thing: it gets you help, and it gets you friends. Counterintuitively, people like you more if they have helped you, if they feel like you owe them one. This is great, especially with people outside the team, whom you can’t all maintain a close relationship with.

We need to know what our teammates don’t know. In particular, on a close team — and this is why a close team is limited to 7 people — we need to track what each other person knows and doesn’t, so that we can tell them the pieces of information they need. We have our own mental model of the system, our model of each other, and our model of each other’s models — all this is necessary for close coordination. This is how we avoid the potholes of communication failure. This is how we get distributed decisionmaking with trust: because we can predict each others’ actions. For smooth work, we need to know each other this well.

Let our teammates know that we want to know what they know, we want to hear their input. Psychological safety. We each need to feel safe suggesting ideas and asking questions. Google found that this is the #1 determiner of team effectiveness. So part of generativity is making your team a safe place for everyone to contribute. Do this by asking people for input and listening carefully to what they say. We don’t to agree; we have to welcome the information.

Social yaks are yaks. Get to know people, find out what they know, and make them feel safe. Keep them up to date on what you know.

One way I do this is with my own slack channel, #jessitron-stream. I use it as a notebook. I throw error messages in them when I hit them, and record what I did (especially when changing infrastructure). I track my own yak stack. People can pop in and answer questions if they want, or I can @ them and then they see the context of my inquiry. Lots of other Atomists have started their own stream channel; we have #radio-russ and #kipz-corner and #sylvain-stream and #hoffhaus and #arrrrr-day. It’s one way to work in public from the privacy of our own homes.

Don’t neglect the Royal Yaks. Hold court in Slack or in the kitchen. Interact with each other. Find out what is useful and what is in the way of other people, and you can prioritize your own yak shaving. These conversations, when they get interesting, might even lead you to the Golden Yak (ooooo).