Sunday, November 22, 2015

Getting off the ground in Elm: project setup

If you have tried Elm and want to make a client bigger than a sample app, this post can help you get set up. Here, find what goes into each of my Elm repositories and why. This template creates a fullscreen Elm application with the potential for server calls and interactivity. This post is up-to-date as of Elm 0.16.0 on 11/21/2015, on a Mac.
TL;DR: Clone my sample repo; Change CarrotPotato to your project name everywhere it appears (elm-package.json; src/CarrotPotato.elm; index.html). Replace origin with your remote repository.

Step 0: Install Elm (once)

Run the latest installer.
To check that this worked, run `elm` at a terminal prompt. You should see a long usage message, starting with `Elm Platform 0.16.0 - a way to run all Elm tools`

Bonus: getting Elm text highlighting in your favorite text editor is a good idea. That's outside the scope of this post, because it was hard. I use Sublime 2 and this Elm plugin.

Step 1: Establish version control (every project)

Step 1A: create a directory and a repository. 

Make a directory on your computer and initialize a git repository inside it.
mkdir CarrotPotato
cd CarrotPotato
git init

 Step 1B: configure version control

In every project, I use the first commit to establish which files do not belong under version control.
I'm going to have the Elm compiler write its output to a directory called target. I want to save the source code I write, not stuff that's generated from it, so git should not save the compiler output. Git ignores any files or directories whose names are in a file called .gitignore, so I put target in there.
The Elm package manager uses a directory called elm-stuff for its work. That doesn't belong in our repository, so put it in .gitignore too. I recommend making .gitignore the first file committed in any new repository.
echo "target" >> .gitignore
echo "
elm-stuff" >> .gitignore
git add .gitignore
git commit -m "New Elm project"

Step 2: Bring in core dependencies

The Elm package manager will install everything you need, including the core language, including the configuration it needs. To bring in any dependency, use `elm package install <dependency>`, where <dependency> is specified as github-user/repo-name. Most of the packages come from github users elm-lang or evancz (Evan Czaplicki is the author of Elm). All the packages that elm-package knows about are listed on

In keeping with the Elm Architecture, I use StartApp as the basis for all my projects. Bring it in:
elm package install evancz/start-app
elm-package is very polite: it looks at your project, decides what it needs to do, and  asks nicely for permission before doing anything. It will add the dependency to elm-package.json (creating the file if it doesn't exist), then install the package you requested (along with anything that package depends on) in a directory called elm-stuff.

Here's a gotcha: the StartApp install downloads its dependencies, but you can't use them directly until they are declared as a direct dependency of your project. And you can't actually use StartApp without also using Effects and Html. So install them too:
elm package install evancz/elm-html
elm package install evancz/elm-effects
Note: This step won't work without internet access. Elm's package manager doesn't cache things locally; everything is copied into elm-stuff within each project. On the upside, you can dig around in elm-stuff to look at the code (and embedded documentation) of any of your project's dependencies.

Step 3: Improve project configuration

3A: Welcome to elm-package.json

You now have an elm-package.json file in your project directory. Open it in your text editor.
    "version": "1.0.0",
    "summary": "helpful summary of your project, less than 80 characters",
    "repository": "",
    "license": "BSD3"
    "source-directories": [
    "exposed-modules": [],
    "dependencies": {
        "elm-lang/core": "3.0.0 <= v < 4.0.0",
        "evancz/start-app": "2.0.2 <= v < 3.0.0"
    "elm-version": "0.16.0 <= v < 0.17.0"
The project version, summary, etc. become crucial when you publish a new library to the central Elm package list. Until then, you can update them if you feel like it.

Note: the project's dependencies are specified as ranges. Elm is super specific about semantic versioning. It is impossible for one of the libraries you use to introduce a compilation-breaking change without going up a major version (the first section in the version number), so Elm knows that (for instance) any version of StartApp that's at least as high as its current one "2.0.2" and less than the next major version "3.0.0" is acceptable. This matters if you publish your project as a library for other people to use. For now it's just cool.

3B: Establish a source directory

With the default configurtion, Elm looks for project sources in "." (the current directory; project root). I want to put them in their own directory, so I change the entry in "source-directories" to "src". Then I create a directory called `src` in my project root.
mkdir src
[editor] elm-package.json
and set:
"source-directories": [

Step 4: Create the main module

4A: Bring in "hello world" code

Create a file src/CarrotPotato.elm (if the name of your project is CarrotPotato), and open it in your text editor.
touch src/CarrotPotato.elm
Every StartApp application starts about the same. I cut and paste most of this out of the StartApp docs, then added everything necessary to make it compile. It had to do something, so it outputs Hello World in an HTML text element.

Copy from this file, or this gist.

To understand this code, do the Elm Architecture Tutorial. (It's a lot. But it's the place to go to understand Elm.)

4B: compile the main module

I want this compiled into a JavaScript file in my `target` directory, so this is my build command:
elm make --output target/elm.js src/CarrotPotato.elm
When this works, a target/elm.js file should exist.

Note: by default, elm-make (v0.16) creates an index.html file instead of elm.js. That's fine for playing around, but in any real project I want control over the surrounding HTML.
Note: I ask elm-make to build the top-level module of my project. Once I add more source files, elm-make will compile all the ones that my top-level module brings in.

To remind myself of how to do this correctly, I put it in a script:
echo "elm make --output target/elm.js src/CarrotPotato.elm" >> build
chmod u+x 
Then every time I want to compile:

Step 5: Run the program in a web page

Elm runs inside a web page. Let's call that page index.html because that's the default name for these things. Create that file and put something like this into it:
touch index.html
put this in:
  <meta charset="UTF-8">
  <script type="text/javascript" src="target/elm.js"></script>

<script type="text/javascript">
  var app = Elm.fullscreen(Elm.
CarrotPotato, {});

The important parts here are:
  • in the header, set the page's title
  • in the header, bring in the compiler output; this matches the file I told elm-make to write to
  • in the header, you're free to bring in CSS
  • the body is empty
  • the script tag at the end activates my Elm module.
Save this file, and open it in your default browser:
open index.html
You should see "Hello World". Quick, make a commit!

Note: opening index.html as a file doesn't always work smoothly. If the browser gives you trouble, try running an http server in that directory instead. There's a very easy one available from npm.

Step 6: Go forth and Elminate

The foundation is set for an Elm project. From here, I can start building an application. Here are some things I often do next:
  • change the view function to show something more interesting. see elm-html for what it can retusrn.
  • make a git repository, push my project to it; update this in elm-package.json, and create a
  • create a gh-pages branch to serve my project on the web (blog post on this coming soon, I hope)
  • break out my project's functionality into more modules, by creating files like src/CarrotPotato/View.elm and importing them from my main module
You can get everything up to this point without doing it yourself by cloning my elm-sample repo.
I do this:
git clone carrot-potato
< create repo at github called my-user/carrot-potato; copy its git url>
cd carrot-potato
git remote set-url origin
Comments and suggestions welcome! I'm sure this isn't the most optimal possible setup.

Tuesday, October 13, 2015

The Emperor has no clothes: Bad actors in tech

Maybe you are interested in a language, or an open source project, but you feel like the community is unwelcoming: Some big voices are rude, they’re downright hostile to newcomers and anyone who disagrees with them. Let’s not get involved.

Or in your workplace: influential people in the organization aren’t nearly as helpful, or as smart, as your teammates. Yet their opinions, and bad behavior, are followed by everyone else, and you feel bullied into decisions, you accept their little abuses. Ultimately the people with the most freedom move away, until the workplace is a surreal island where time stood still

There’s this old Christian Andersen tale, The Emperor’s New Clothes. Some weavers sold the emperor a suit, which cannot be seen by those who are incompetent or unfit for their position. It was really not a suit: they gave him nothing, the emperor was running around nude. Nobody said a thing. Each assumed they were in the wrong. Only when a child cried out “he has no clothes!” did everyone else realize that they were not alone in pretending.

While the Emperor’s situation is humorous and extreme, the open source and workplace situations are not. They are real, and they have mathematical underpinnings! A paper, called “The Majority Illusion in Social Networks,” studies this very phenomena. The Washington Post describes how public opinion appears to change rapidly, when sometimes it’s really that public opinion suddenly became known.

Those whose opinions are the most visible control what opinions are seen as acceptable and polite. The moment enough of the social network sees their own opinion as acceptable, bam! a major change in sentiment appears. That feeling was already there, but it was masked by a few local celebrities who didn’t share the values of the majority.

When we dislike bad behavior, we feel alone, but we are not special. We all see it, we all dislike it. We all wish we were using the right tools for the job. We all wish the mailing list contained only polite helpful responses. But social norms -- set by the few who are also the loudest -- make us not care enough, not invest enough, to communicate our opinions with such volume. Shame and fear of rejection freeze us. Or we lack the hunger for conflict. None of us say the emperor has no clothes, bad behavior persists.

The math says the evident culture is not always the predominant culture. A few confident, inconsiderate people in key positions intimidate an entire department. A few derogatory voices fence off a language, leaving erstwhile contributors to chew on rocks outside.

If you care about your organization, work to make everyone’s voice heard.
If you care about your open source community, speak out against behavior that masks the majority attitude. Watch for a red flag in your head: “I think his opinion is wrong, but I’m not going to say anything because arguing with him is not worth it.” You’re not the only one who sees through that clothing.

Friday, October 2, 2015

ElixirConf keynote: Elixir for the world


Slides with notes

Big PDF with notes (15M)

Slides only (on speakerdeck)


Camille Fournier on distributed systems: video
Caitie McCaffrey on stateful services: video 
Denise Jacobs, creativity: video
Marty Cagan on what's better than agile: video
How to Measure Anything: book
BrenĂ© Brown on vulnerability, Grounded Theory: video book
Property testing: video
Property testing: QuickCheck CI 
Elm: Richard Feldman's talk: video
My talk about React and Elm: video
Structure of Scientific Revolutions: about the book

Tuesday, September 29, 2015

This was not OK (regrets)

I have one major regret from StrangeLoop. I want to apologize and find a kinder way to be.

At dinner the last night, I said something mean about Tony Morris. I've never met Tony Morris. I have a feeling of certainty that he was mean to people in the Scala community, and this contributed to a splintering of the community. I am not in favor of communities including or elevating people who are mean to anyone else. See Pieter Hintjens on this.

And I was wrong to speak scornfully of him. I can see this because I saw the hurt in Philip Wadler when I said it. Later he mentioned collaborating with Tony Morris on some important work.

People aren't all bad or all good. Some of us are horrible and fantastic. Mean in some situations and great contributors elsewhere. The good and the bad, they don't cancel each other. Both exist. We are not a sum; there is not a one-dimensional number line between "good" and "bad." 

If he caused a splintering in a community, then probably that community is better off without his direct participation. And if he collaborated on great work with someone else, or did great work alone, I am grateful for it. 

If I denigrate him or that work, by implication or indirectly, then I am causing splintering in the community. I ruined a chance to exchange ideas with Philip Wadler, who was extremely kind to me (he even didn't show his hurt feelings) and who invited me to that dinner. Who is bringing excellent ideas, in code and in talks, to the whole programming community. Who is wide open to new ideas and experiments.

I don't yet know the best way to talk about these community problems, which are very important to discuss. Now I know that it is not by denigrating or scorning anyone. I need, I feel it in my soul, to celebrate everyone in the times when they shine, to cherish the contributions they do make, even when they don't shine in every situation. We can celebrate this without perfect inclusivity (there is no such thing). I deserve to be excluded from that dinner; it would have made everyone else more comfortable, a net win for the community. And separately, I do help in other ways. Can't belong everywhere. 

I am sorry. I see that my words caused pain and it's my fault. In the future I will endeavor to not speak scornfully of anyone. To criticize actions and people-in-roles only when working on improving the system, not a whole person ever. To deal with my own experience of being bullied instead of lashing out whenever my brain makes an association with it. 

I want to be a source of healing and encouragement to a community, not further splinter it. Thank you for your patience. 

Tuesday, August 25, 2015

Functional principles come together in GraphQL at React Rally

Sometimes multiple pieces of the industry converge on an idea from
different directions. This is a sign the idea is important.

Yesterday I spoke at React Rally (video coming later) about the
confluence of React and Flux in the front end with functional
programming in the back end. Both embody principles of composition, declarative
style, isolation, and unidirectional flow of data.

In particular, multiple separate solutions focused on:

  •   components declare what data they need
  •   these small queries compose into one large query, one GET request
  •   the backend gathers everything and responds with data in the requested format

This process is followed by Falcor from Netflix (talk by Brian Hunt) and GraphQL from Facebook (talks by Lee Byron and Nick Schrock, videos later). Falcor adds caching on the client, with cache
invalidation defined by the server (smart; since the server owns
the data, it should own the caching policy). GraphQL adds an IDE for
queries, called GraphiQL (sounds like "graphical"), released as
open source for the occasion! The GraphQL server provides introspection
into the types supported by its query language. GraphiQL uses this to let the developer
work with live, dynamically fetched queries. This lets us explore the available
data. It kicks butt.

Here's an example of GraphQL in action. One React component in a GitHub client might specify that it needs
certain information about each event (syntax is approximate):
  event {
    actor {
and another component might ask for different information:
{  event {    actor {      image_uri    }  }}

The parent component assembles these and adds context, including
selection criteria:
{  repository(owner:"org", name:"gameotron") {    event(first: 30) {       type,       datetime,       actor {         name,         image_url      }    }  }}
Behind the scenes, the server might make one call to retrieve the repository,
another to retrieve the events, and another to retrieve each actor's
data. Both GraphQL and Falcor see the query server as an abstraction
layer over existing code. GraphQL can stand in front of a REST
interface, for instance. Each piece of data can be
fetched with a separate call to a separate microservice, executed in
parallel and assembled into the structure the client wants. One GraphQL
server can support many version of many applications, since the
structure of returned data is controlled by the client.
The GraphQL server assembles all the
results into a response that parallels the structure of the client's
{  "repository" : {    "events" : [{      "type" : "PushEvent",      "datetime" : "2015-08-25Z23:24:15",      "actor" : {        "name" : "jessitron",        "image_url" : "https://some_cute_pic"      }    }    ...]  }}
It's like this:
The query is built as a composition of the queries from all the components. It goes to the server. The query server spreads out into as many other calls as needed to retrieve exactly the data requested.
The query is composed like a fan-in of all the components'
desires. On the server this fans out to as many back-end calls as
needed. The response is isomorphic to the query. The client then spreads
the response back out to the components. This architecture supports
composition in the client and modularity on the server.
The server takes responses from whatever other services it had to call, assembles that into the data structure specified in the query, and returns that to the client. The client disseminates the data through the component tree.
This happens to minimize network traffic between the client and server.
That's nice, but what excites me are these lovely declarative queries that
composes, the data flowing from the parent component into all the
children, and the isolation of data requests to one place. The exchange
of data is clear. I also love the query server as an abstraction over
existing services; store the data bits in the way that's most convenient
for each part. Assembly sold separately.

Seeing similar architecture in Falcor and GraphQL, as well as in
ClojureScript and Om[1] earlier in the year, demonstrates that this is
important in a general case. And it's totally compatible with
microservices! After React Rally, I'm excited about where front ends are

[1] David Nolen spoke about this process in ClojureScript at Craft Conf
earlier this year. [LINK]

Sunday, August 16, 2015

An Opening Example of Elm: building HTML by parsing parameters

I never enjoyed front-end development, until I found Elm. JavaScript with its `undefined`, its untyped functions, its widely scoped mutable variables. It's like Play-Doh, it's so malleable. And when I try to make a sculpture, the arms fall off. It takes a lot of skill to make Play-Doh look good.

Then Richard talked me into trying Elm. Elm is more like Lego Technics. Fifteen years ago, I bought and built a Lego Technics space shuttle, and twelve years ago I gave up on getting that thing apart. It's still in my attic. Getting those pieces to fit together takes some work, but once you get there, they're solid. You'll never get "method not found on `undefined`" from your Elm code.

Elm is a front-end, typed functional language; it to JavaScript for use in the browser. It's a young language (as of 2015), full of opportunity and surprises. My biggest surprise so far: I do like front-end programming!

To guarantee that you never get `undefined` and never call a method that doesn't exist, all Elm functions are Data in, Data out. All data is immutable. All calls to the outside world are isolated. Want to hit the server? Want to call a JavaScript library? That happens through a port. Ports are declared in the program's main module, so they can never hide deep in the bowels of components. Logic is in one place (Elm), interactions in another.
one section (Elm) has business logic and is data-in, data-out. It has little ports to another section( JavaScript) that can read input, write files, draw UI. That section blurs into the whole world, including the user.

This post describes a static Elm program with one tiny port to the outside world. It illustrates the structure of a static page in Elm. Code is here, and you can see the page in action here. The program parses the parameters in the URL's query string and displays them in an HTML table.[1]

All web pages start with the HTML source:
  <title>URL Parameters in Elm</title>
  <script src="elm.js" type="text/javascript"></script>
  <link href="" rel="stylesheet"></link>
<script type="text/javascript">
  var app = Elm.fullscreen(Elm.UrlParams,
                           { windowLocationSearch:

This brings in my compiled Elm program and some CSS. Then it calls Elm's function to start the app, giving it the name of my module which contains main, and extra parameters, using JavaScript's access to the URL search string.

Elm looks for the main function in my module. The output of this function can be a few different types, and this program uses the simplest one: Html. This type is Elm's representation of HTML output, its virtual DOM.

module UrlParams where

import ParameterTable exposing (view, init)
import Html exposing (Html)

main : Html
main = view (init windowLocationSearch)

port windowLocationSearch : String
The extra parameters passed from JavaScript arrive in the windowLocationSearch port. This is the simplest kind of port: input received once at startup. Its type is simply String. This program uses one custom Elm component, ParameterTable. The main function uses the component's view function to render, and passes it a model constructed by the component's init method.

Somewhere inside the JavaScript call to Elm.fullscreen, Elm calls the main function in UrlParams, converts the Html output into real DOM elements, and renders that in the browser. Since this is a static application, this happens once. More interesting Elm apps have different return types from main, but that's another post.

From here, the data flow of this Elm program looks like this:
The three layers are: a main module, a component, and a library of functions.
The main module has one input port for the params.  That String is transformed by init into a Model, which is transformed by View into Html. The Html is returned by main and rendered in the browser. This is the smallest useful form of the Elm Architecture that I came up with.

Here's a piece of the ParameterTable module:
module ParameterTable(view, init) where

import Html exposing (Html)
import UrlParameterParser exposing (ParseResult(..), parseSearchString)

type alias Model = { tableData: ParseResult }

init: String -> Model
init windowLocationSearch =
  { tableData = parseSearchString windowLocationSearch }

--- VIEW
viewModel -> Html
view model =
  Html.div ...
The rest of the code has supporting functions and details of the view. These pieces (Model, init, and view) occur over and over in Elm. Often the Model of one component is composed from the Models of subcomponents, and the same with init and view functions.[2]

All the Elm files are transformed by elm-make into elm.js. Then index.html imports elm.js and calls its Elm.fullscreen function, passing UrlParams as the main module and in the extra parameter. And so, a static (but not always the same) web page is created from data-in, data-out Elm functions. And I am a happy programmer.

[1] Apparently there's not a built-in thing in JavaScript for parsing these. Which is shocking. I refused to write such a thing in JavaScript (where by "write" I mean "copy from StackOverflow"), so I wrote it in Elm.

[2] Ditto with update and Action, but that's out of scope. This post is about a static page.

Monday, August 3, 2015

Data-in, Data-out

In functional programming, we try to keep our functions data-in, data-out: they take some data as parameters, return some data as output, and that's it. Nothing else. No dialog boxes pop, no environment variables are read, no database rows are written, no files are accessed. No global state is read or written. The output of the function is entirely determined by the values of its input. The function is isolated from the world around it.

A data-in, data-out function is highly testable, without complicated mocking. The test provides input, looks at the output, and that's all that it needs for a complete test.[1]

A data-in, data-out function is pretty well documented by its declaration; its input types specify everything necessary for the function to work, its output type specifies the entire result of calling it. Give the function a good name that describes its purpose, and you're probably good for docs.

It's faster to comprehend a data-in, data-out function because you know a lot of things it won't do. It won't go rooting around in a database. It won't interrupt the user's flow. It won't need any other program to be running on your computer. It won't write to a file[2]. All these are things I don't have to think about when calling a data-in, data-out function. That leaves more of my brain for what I care about.

If all of our code was data-in, data-out, then our programs would be useless. They wouldn't do anything observable. However, if 85% of our code is data-in, data-out, with some input-gathering and some output-writing and a bit of UI-updating -- then our program can be super useful, and most of it still maximally comprehensible. Restricting our code in this way when we're writing it provides more clarity when we're reading it and freedom when we're refactoring it.
Think about data-in, data-out while you're coding; make any dependencies on the environment and effects on the outside world explicit; and write most of your functions as transformations of data. This gets you many of the benefits of functional programming, no matter what language you write your code in.

[1] Because the output is fixed for a given input, it would be legit to substitute the return value for the function-call-with-that-input at any point. Like, one could cache the return values if that helped with performance, because it's impossible for them to be different next time, and it's impossible to notice that the function wasn't called because calling it has no externally-observable effect. Historically, this property is called referential transparency.

[2] We often make an exception for logging, especially logging that gets turned off in production.