Scaling Intelligence

You can watch the full keynote from Scala eXchange 2015 (account creation required, but free). The talk includes examples and details; this post is a summary of one thread.

Scala is a scalable language, from small abstractions to large ones. This helps with the one scaling problem every software system has: scaling the feature set while still fitting it in our heads. Scaling our own intelligence.

Scala offers complicated powerful language features built from combinations of simpler language features. The aim is a staircase of learning: gradually learn features as you need them. The staircase starts in the green grass of toy programs, moves through the blue sky of useful business software, and finally into the outer space of abstract libraries and frameworks. (That dark blob is supposed to represent outer space.)

This is not how people experience the language.

The green grass is great: Odersky’s Coursera courses, Atomic Scala. Next, we want to write something useful for work: the blue sky. It is time to use libraries and frameworks. I want a web app, so I bring in Spray. Suddenly I need to understand typeclasses and the magnet pattern. The magnet pattern? The docs link to a post on this. It’s five thousand words long. I’m shooting into outer space — I don’t want to be an astronaut yet!

The middle of the staircase is missing.

Who can repair this? Not the astronauts, the compiler and library authors. They can write posts around program language theory, defining one feature in terms of a bunch of other concepts I don’t understand yet. I need explanations by people who share my objectives, people a little bit ahead of me in the blue sky, who recently learned how to use Spray themselves. I don’t need research papers, I need StackOverflow. Blog posts, not textbooks.

This is where we need each other. As a community, we can fill this staircase. At a macro level, we scale intelligence with teaching.

Scala as a language is not enough. We don’t work in languages, especially not in the blue sky. We work in language systems, including all the libraries and tooling and all the people. The resources we create, and the personal interactions in real life and online. When we teach each other, we scale our collective intelligence, we scale our community.

Scaling the community is important, because only a large, diverse group can answer two crucial questions. To make the language and libraries great, we need to know about each feature: is this useful? and to make this staircase solid, we need to know about each source and document: is this clear?

Useful isn’t determined by the library author, but by its users. Clear isn’t determined by the writer, but by the reader. If you read the explanation of Futures on the official Scala site and you don’t get it, if you feel stupid, that is not your fault. When documentation is not clear to you, its maintainers fail. Teaching means entering the context of the learner, and starting there. It means reaching for the person a step or two down, and pulling them up to where you are.

Michael Bernstein described his three years of learning Haskell. “I tried over and over again to turn my self doubt into a pure functional program, and eventually, it clicked.”
Ouch. Not everyone has this tenacity. Not everyone has three years to spend becoming an astronaut. Teaching makes the language accessible to more people. At the same time, it makes everyone’s life easier — what might Mr Bernstein have accomplished during that year?

Scala, the language system, does not belong to Martin Odersky.  It belongs to everyone who makes Scala useful. We can each be part of this.

Ask and answer questions on StackOverflow. Blog about what you learned, especially about why it was useful.[1] Request more detail — if something is not clear to you, then it is not clear. Speak at your local user group.[2] The less type theory you understand, the more people you can help!

Publish your useful Scala code. We need examples from the blue sky. If you do, tweet about it with #blueSkyScala.

It is up to all of us to teach each other, to scale our intelligence. Then we can make use of those abstractions that Scala builds up. Then it will be a scalable language.

[1] example: Remco Beckers’s post on Option and Either and Try.
[2] example: Heather Miller’s talk compensates for bad documentation around Scala Futures.

The Emperor has no clothes: Bad actors in tech

Maybe you are interested in a language, or an open source project, but you feel like the community is unwelcoming: Some big voices are rude, they’re downright hostile to newcomers and anyone who disagrees with them. Let’s not get involved.

Or in your workplace: influential people in the organization aren’t nearly as helpful, or as smart, as your teammates. Yet their opinions, and bad behavior, are followed by everyone else, and you feel bullied into decisions, you accept their little abuses. Ultimately the people with the most freedom move away, until the workplace is a surreal island where time stood still

There’s this old Christian Andersen tale, The Emperor’s New Clothes. Some weavers sold the emperor a suit, which cannot be seen by those who are incompetent or unfit for their position. It was really not a suit: they gave him nothing, the emperor was running around nude. Nobody said a thing. Each assumed they were in the wrong. Only when a child cried out “he has no clothes!” did everyone else realize that they were not alone in pretending.

While the Emperor’s situation is humorous and extreme, the open source and workplace situations are not. They are real, and they have mathematical underpinnings! A paper, called “The Majority Illusion in Social Networks,” studies this very phenomena. The Washington Post describes how public opinion appears to change rapidly, when sometimes it’s really that public opinion suddenly became known.

Those whose opinions are the most visible control what opinions are seen as acceptable and polite. The moment enough of the social network sees their own opinion as acceptable, bam! a major change in sentiment appears. That feeling was already there, but it was masked by a few local celebrities who didn’t share the values of the majority.

When we dislike bad behavior, we feel alone, but we are not special. We all see it, we all dislike it. We all wish we were using the right tools for the job. We all wish the mailing list contained only polite helpful responses. But social norms — set by the few who are also the loudest — make us not care enough, not invest enough, to communicate our opinions with such volume. Shame and fear of rejection freeze us. Or we lack the hunger for conflict. None of us say the emperor has no clothes, bad behavior persists.

The math says the evident culture is not always the predominant culture. A few confident, inconsiderate people in key positions intimidate an entire department. A few derogatory voices fence off a language, leaving erstwhile contributors to chew on rocks outside.

If you care about your organization, work to make everyone’s voice heard.
If you care about your open source community, speak out against behavior that masks the majority attitude. Watch for a red flag in your head: “I think his opinion is wrong, but I’m not going to say anything because arguing with him is not worth it.” You’re not the only one who sees through that clothing.

This was not OK (regrets)

I have one major regret from StrangeLoop. I want to apologize and find a kinder way to be.

At dinner the last night, I said something mean about Tony Morris. I’ve never met Tony Morris. I have a feeling of certainty that he was mean to people in the Scala community, and this contributed to a splintering of the community. I am not in favor of communities including or elevating people who are mean to anyone else. See Pieter Hintjens on this.
And I was wrong to speak scornfully of him. I can see this because I saw the hurt in Philip Wadler when I said it. Later he mentioned collaborating with Tony Morris on some important work.
People aren’t all bad or all good. Some of us are horrible and fantastic. Mean in some situations and great contributors elsewhere. The good and the bad, they don’t cancel each other. Both exist. We are not a sum; there is not a one-dimensional number line between “good” and “bad.” 
If he caused a splintering in a community, then probably that community is better off without his direct participation. And if he collaborated on great work with someone else, or did great work alone, I am grateful for it. 
If I denigrate him or that work, by implication or indirectly, then I am causing splintering in the community. I ruined a chance to exchange ideas with Philip Wadler, who was extremely kind to me (he even didn’t show his hurt feelings) and who invited me to that dinner. Who is bringing excellent ideas, in code and in talks, to the whole programming community. Who is wide open to new ideas and experiments.
I don’t yet know the best way to talk about these community problems, which are very important to discuss. Now I know that it is not by denigrating or scorning anyone. I need, I feel it in my soul, to celebrate everyone in the times when they shine, to cherish the contributions they do make, even when they don’t shine in every situation. We can celebrate this without perfect inclusivity (there is no such thing). I deserve to be excluded from that dinner; it would have made everyone else more comfortable, a net win for the community. And separately, I do help in other ways. Can’t belong everywhere. 
I am sorry. I see that my words caused pain and it’s my fault. In the future I will endeavor to not speak scornfully of anyone. To criticize actions and people-in-roles only when working on improving the system, not a whole person ever. To deal with my own experience of being bullied instead of lashing out whenever my brain makes an association with it. 
I want to be a source of healing and encouragement to a community, not further splinter it. Thank you for your patience. 

Why Aren’t There More Women Programmers

When we feel like we’re good at something, that’s when we can really learn it, sink our teeth in and love it and do it until we’re experts.

It is a great motivator, this feeling of confidence. This belief that we can accomplish what we want to do is called self-efficacy. There are four sources of self-efficacy at a particular task (in order of strength):[1]

  1. doing it
  2. seeing people like me do it
  3. social persuasion
  4. your body

Why aren’t there more women programmers? Because many women don’t feel like they can do it. Therefore they don’t try it, or don’t persevere. These sources of self-efficacy explain why women (in aggregate) are less likely to program than men.

1. doing it: If you try something and have success, this is the best source of a feeling of self-efficacy. In my generation, more boys than girls started programming at an early age.

2. seeing people like me do it: If my mom can do it, so can I.
People choose careers by picturing themselves in that job, and the picture is based on people we know. If we can’t picture ourselves in a career, we don’t consider it. The gender dichotomy is strong in our culture, a big part of whom we consider “like me.” A boy can picture himself as a programmer. When a girl looks around, she doesn’t see people like her coding, attending conferences, speaking, blogging, contributing to open source. Even women developers: looking up in the hierarchy, they see themselves in management, analysis, QA, BI, maybe DBA. Not system administration. Not architecture.

3. social persuasion: Do my friends approve of it?
Here, we look not at the culture of programming, but the culture of women. When I go to Kindergarten Moms Night and someone asks what I do, “computer programming” usually ends the conversation. I don’t fit in. Growing up and being grown up, if you enjoy the company of other women, the social persuasion is against development.

4. your body: Do you get a good feeling in your stomach? I don’t know about this one. Is it different for men and women?

Women are just as capable of programming as men, but (in aggregate) they don’t feel as capable. If we can change that, then we can change the ratio.

#1 is the most direct route, and many in the community are working on introducing programming to young people, especially young women. Yay for them!

#2 is something else we can change. Increase the visibility of women who are already developers, especially those at the highest level. I want to look up, up on stage or up the decisionmaking structure, and see people like me. Conference organizers who go out of your way to elicit proposals from women, thank you. You’re helping.

#3 comes the larger social culture, not the culture of programmers. I can’t be a typical mom and a community-involved developer. This is not something the programming community can change. Personally, I see #3 as the most intractable obstacle for women.

As a community, #1 and #2 are the ones we can do something about. And hey, they’re at the top of the list! If we keep working at this, we’ll reach s critical mass. Once programming is (say) 33% women, then #3 will fix itself. Without intervention, the social pressure and lack of role models combine to attract and retain fewer and fewer women. With work, we can turn that spiral around.


[1] At PyCon Canada, Mel Chua gave a great talk on learning. This list is hers.

Welcome All the People

On our team, in the user group, and at conferences, we try to build community. In each conversation, we do this by emphasizing ways we are alike.

  •  Who doesn’t like beer?
  •  Monty Python is funny.
  •  Everyone has done Object Oriented programming.
  •  We all went to college.
  •  Each of us can grow a beard.
  •  Video games are awesome.
  •  Everyone can see and hear.
  •  We all grew up watching Saturday morning cartoons.

Each of these shared references draws together most of the developer audience and promotes a feeling of belonging. And each of them further isolates people who don’t fit these assumptions.

I’ve been guilty of this. Writing presentations or posts, I look for domains familiar to my audience, and choose role playing. This emphasizes the programmer == geek stereotype, and the people who don’t know what it means to “level up” feel stupid and out of place.

When a reference appeals to the majority then people who don’t get it feel isolated. No one asks, “What does that mean?” when 80% of the audience reacts with knowing laughter.

Let’s build community on what every one of us shares.

  • We want to learn from each other. 
  • Each of us wants to help build better software.

If you don’t want to learn or build good software then don’t read this blog. From now on, I aim to make only these assumptions about the people on my team, in my user groups, and at each conference.

Dick jokes? Funny, yes. Offensive, no. It isn’t about offensive/not offensive anymore. It’s about welcoming/isolating. When a particular joke applies to 80% of the audience differently than to the other 20%, that joke is isolating. That builds homogeneity. I want everyone to feel welcome, I want diverse ideas and contributions. I’m aiming for inclusion.

These are my action items:

  • Consider my target audience as the attendee mix I hope to see someday. Men and women and transgendered, Asian and African and European, 20s and 40s and 60s. People who like sports, and cooking, and quilting.
  • Give context to my references. The next time I mention the Prime Directive of Programming, I’ll talk about how Captain Picard in Star Trek: The Next Generation follows the Prime Directive of “do no harm.”
  • Have hallway and table conversations about a diverse range of topics, that any random person might be able to jump in on. I’ll keep the stuff that freaks some people out for parties and private.

Let’s make each other think. Development communities should be about learning, not warm fuzzies of all being alike, when “all” means “people who resemble me.” I want to be all in, and let all in.

Saint Louis, Technology City Extraordinaire

Saint Louis is a sweet spot to be a developer right now because there are tons of jobs.

Saint Louis is a sweet spot to be a passionate developer because we have an amazing user group community. There are active user groups for Java (two), Ruby, JavaScript, and .NET (two). There’s user groups for mobile, AppleHadoop, vim, and Perl. Then there’s Lambda Lounge for the really interesting stuff that you won’t use in your day job.  If you want to skip the talking and just code, you can’t beat Code Til Dawn!

All of these groups are friendly to attendees and speakers. We have many great recruiting firms that sponsor food and space, sometimes even flying speakers in from out of town.

Now here’s the real skinny – the user groups that I attend regularly ranked according to alcohol content:

1. STLJS – it’s above a bar. The bar has scotch. It closes at midnight, but we’re still drinking and talking about code past that. Food: sandwiches
2. Lambda Lounge – we bring beer, and go to the Hive (nearest bar) for more drinks afterward. Food: pizza
3. ALT.NET – some of us bring beer, and at least once the speaker brought vodka. Food: snacks at best
4. JUG – no beer. No bar. Only technical. But – Ted Neward in July! Food: pizza

Special kudos to ALT.NET, because Nick recruits some of the best speakers: Steve F-in Bohlen, Ken Sipe, me. It is a more intimate user group than the others, with a smaller venue and great discussions, and beer. The topics are technical and a good depth. Now if only I could get those guys to the bar afterward, it’d be perfect.

Expect to fly

I love being a female developer. When I go to a hacker night, I feel like Scarlet in GI Joe, Firestar with Spiderman and his friends. Sysadmins and DBAs are happy to answer questions for a smile. In interviews and in conference speaker selection, I stand out. Colleagues open doors and invite me to happy hour. Teammates respect me and enjoy my company. My gender is an asset in my programming career.
Yet – when I meet another developer, I don’t expect to be taken seriously.
It’s a subtle effect. It took me a decade to notice, but much less time to cope with it. When I start on a team, half of what I say is a joke. That way it’s okay when my words are laughed off. I demonstrate competence with good questions – these are more welcome than statements. When necessary, I can be very assertive, forceful enough that the men in the conference room can’t blow me off. These strategies get me past the initial impressions, and within a few days I have the team’s full respect. Gender is a consideration only before they know me.
The effect is so subtle I can’t call it a prejudice. It’s a shortcut our brains take: categorization. I do it myself. In my experience, women tend to contribute less in discussions. They don’t fit the image of “strong developer” in my head. This adds up to lower expectations than for a man, at least for a short time. 
The scary part is that expectations influence behavior. Subconsciously we conform to the expectations of those around us. Because females are expected to be quiet, because they are interrupted more than men, women volunteer less. Which leads our subconscious minds to expect female developers to be quiet and contribute less.
The spiral of lower expectations can be reversed. For every woman technical speaker we hear at a conference, we’ll take the next female developer we meet a little more seriously. For each woman who shows up to the user group, we’ll be more likely to invite our female colleagues. Every time we direct a question to one of the women in the meeting, or ask an interrupting colleague to let the woman finish speaking, she’s inspired to speak up more.
We turn this spiral around one tiny action at a time. Women, go to user groups. Men and women, recognize the bias and fight it – expect more from the women around you. Every woman can fly like Firestar.