Thursday, March 31, 2016

Scaling Intelligence

You can watch the full keynote from Scala eXchange 2015 (account creation required, but free). The talk includes examples and details; this post is a summary of one thread.

Scala is a scalable language, from small abstractions to large ones. This helps with the one scaling problem every software system has: scaling the feature set while still fitting it in our heads. Scaling our own intelligence.

Scala offers complicated powerful language features built from combinations of simpler language features. The aim is a staircase of learning: gradually learn features as you need them. The staircase starts in the green grass of toy programs, moves through the blue sky of useful business software, and finally into the outer space of abstract libraries and frameworks. (That dark blob is supposed to represent outer space.)

This is not how people experience the language.

The green grass is great: Odersky's Coursera courses, Atomic Scala. Next, we want to write something useful for work: the blue sky. It is time to use libraries and frameworks. I want a web app, so I bring in Spray. Suddenly I need to understand typeclasses and the magnet pattern. The magnet pattern? The docs link to a post on this. It's five thousand words long. I'm shooting into outer space -- I don't want to be an astronaut yet!

The middle of the staircase is missing.

Who can repair this? Not the astronauts, the compiler and library authors. They can write posts around program language theory, defining one feature in terms of a bunch of other concepts I don't understand yet. I need explanations by people who share my objectives, people a little bit ahead of me in the blue sky, who recently learned how to use Spray themselves. I don't need research papers, I need StackOverflow. Blog posts, not textbooks.

This is where we need each other. As a community, we can fill this staircase. At a macro level, we scale intelligence with teaching.

Scala as a language is not enough. We don't work in languages, especially not in the blue sky. We work in language systems, including all the libraries and tooling and all the people. The resources we create, and the personal interactions in real life and online. When we teach each other, we scale our collective intelligence, we scale our community.

Scaling the community is important, because only a large, diverse group can answer two crucial questions. To make the language and libraries great, we need to know about each feature: is this useful? and to make this staircase solid, we need to know about each source and document: is this clear?

Useful isn't determined by the library author, but by its users. Clear isn't determined by the writer, but by the reader. If you read the explanation of Futures on the official Scala site and you don't get it, if you feel stupid, that is not your fault. When documentation is not clear to you, its maintainers fail. Teaching means entering the context of the learner, and starting there. It means reaching for the person a step or two down, and pulling them up to where you are.

Michael Bernstein described his three years of learning Haskell. "I tried over and over again to turn my self doubt into a pure functional program, and eventually, it clicked."
Ouch. Not everyone has this tenacity. Not everyone has three years to spend becoming an astronaut. Teaching makes the language accessible to more people. At the same time, it makes everyone's life easier -- what might Mr Bernstein have accomplished during that year?

Scala, the language system, does not belong to Martin Odersky.  It belongs to everyone who makes Scala useful. We can each be part of this.

Ask and answer questions on StackOverflow. Blog about what you learned, especially about why it was useful.[1] Request more detail -- if something is not clear to you, then it is not clear. Speak at your local user group.[2] The less type theory you understand, the more people you can help!

Publish your useful Scala code. We need examples from the blue sky. If you do, tweet about it with #blueSkyScala.

It is up to all of us to teach each other, to scale our intelligence. Then we can make use of those abstractions that Scala builds up. Then it will be a scalable language.

[1] example: Remco Beckers's post on Option and Either and Try.
[2] example: Heather Miller's talk compensates for bad documentation around Scala Futures.