Wednesday, June 6, 2012

Why Functional Matters: Your white board will never be the same

Why learn functional programming? For better design skills!

The other day we designed a process to match cleared deposits with dispensations. This is how I would have white-boarded it a few years ago:

Since then I've played around with functional programming. It encourages one to think about processing in terms of data streams: data comes in, gets transformed and filtered and manipulated, and finally results are output. Not only do programs work this way, but each piece of the program -- each function -- is also modeled as data in, data out. No state changes in the middle.
Thinking about the problem in this way leads to this kind of diagram:
Thinking about the program as a data pipeline keeps me thinking about what needs to happen, instead of how each step should be done. It's a higher level of thinking that keeps us from bogging down in implementation details at the design phase.

Whatever language we use to write the solution, thinking about it this way has the following advantages:
* It breaks down into pieces. (The orange boxes represent JIRA tasks.) The requirements are already specified on the diagram: what goes in, what comes out. Each task can be developed independently.
* Each bit is testable. The database-impacting pieces are called out; other than that, each box is defined entirely by input and output. That's the easiest kind of component to test.

In this way, thinking functionally helps with agile (task breakdown), TDD, and maintainability. The code is more composable. Thinking about the problem is easier because we can see what the data looks like at each step.
Independent, testable components: that's functional.

For how functional thinking helps at a lower level too, check this post.

New coding tricks are fun, but new white board techniques are serious business.

17 comments:

  1. Masterly, and exactly my experience: not loops and mutations, but sets and mappings.

    ReplyDelete
  2. you're pretty cute. wanna date?

    ReplyDelete
  3. @hans - your stupidity lowers the level of discourse.

    ReplyDelete
  4. oh, look! an internet white knight

    ReplyDelete
  5. So does the presumption that acting with mappings is somehow functional, and not a topic suited for college freshmen.

    Functional, TDD, Agile, Maintainability? The presumption that modularity is functionality? The complete failure to notice that her new method is transactionally in error?

    Yeah, this isn't Law Of The Tool bingo at all. Maybe if you line the coconuts up this way, the metal birds will come back with their cargo and golden hammers?

    ReplyDelete
    Replies
    1. Why shouldn't college freshmen learn functional techniques like mappings?

      In the real code, transaction boundaries were carefully considered and applied. Breaking the problem up into pieces like this still helped, and it helped to break up the work and define the ins-and-outs of each piece.

      No tool helps everywhere. If you don't know where its limits are, where it is not appropriate, then you don't know the tool.

      hans's comment was better.

      Delete
    2. That person does hit on some points and it doesn't have to be functional to meet the needs of independent and testable components. However if your not stuck in a functional language you'd probably never come to that type of conclusion. You'd not come to that conclusion if you used java or c++ to program the first diagram. However If you tried to program that in a language like Haskell, you'd rip your hair out, which is why the second diagram makes so much more sense. The side effect of functional programming is that you learn to think different. Which is what seems to be the goal of this article.

      However this is really what an architect is for, and since many programmers are taking the role of not just a programmer but and an architect too its becomes very hard to reach the "right solution".

      We learn to program imperatively, so the notion of functional programming will take time to learn. Languages like Scala and Rust encompass both paradigms making the transition easier. In the end, just like you said, no tools helps everywhere and that is what truly is important.

      Delete
    3. This comment has been removed by the author.

      Delete
  6. You've rediscovered a 70s concept http://en.wikipedia.org/wiki/Data_flow_diagram

    ReplyDelete
  7. You just hid the interesting part in the box marked 'pair'. Looks like you think that when these two collections are ordered by date neither will ever have two items on the same date. If you order them by date and time, what makes the times match up. If you order them by date, what makes the dates match up? If you have multiple on the same date, how do you pair them? If you have a date mismatch, will you still pair items that happen to collate according to your sorts? If you do, that kind of suggests that the sort criteria is only approximate, since addition of another item could cause the pair to be missed. A functional programmer would likely deal with this by recursively putting the two sets of unmatched items back through the same pairing process.

    Take a minute to make your best guess at stating the problem accurately before drawing solutions.

    ReplyDelete
    Replies
    1. > You just hid the interesting part in the box marked 'pair'.

      The matching algorithm was not part of the first diagram, either. It just says "try to match with a deposit". That's not the point of this part of the design problem.

      But I'm willing to bet that if a non-trivial matching algorithm is indeed needed, its design and implementation will go much more smoothly if they are using the kind of thinking in the second diagram than in the first.

      Delete
    2. @yitzgale - you're right.

      Conveniently, that method is described in detail in this other post: http://blog.jessitron.com/2012/05/f-made-java-easier-today.html

      Delete
    3. >> The matching algorithm was not part of the first diagram, either. It just says "try to match with a deposit". <<

      But you leaked quite a few details of the lower level module up into the high-level design, where it really does not belong. Sorting by age means that you are not sorting by amount, check number, voucher number, payee, ... It also means that you are using a sequential collection, not a set, not a dict, not a database, etc.

      The other low-level detail that is in the high-level design is the decision to store before marking something as reconciled. How come you store after you reconcile but mark as reconciled after you store? Anyway, that's a low level detail, too.

      Get rid of those details, and you've got three boxes:

      input -> process -> output

      That can be a fine functional design, but there is nothing peculiarly functional about it. It's the recommended morphology of structured systems from Demarco and Yourdon in 1977.

      Delete
  8. Welcome. Ignore the naysayers and whiners. It's par for the course.

    ReplyDelete
  9. lol what's this article........

    ReplyDelete
  10. @JessiTRON, listen to Tony and ignore the mean people (and watch his videos). Keep learning and exploring and influencing other people!

    ReplyDelete
  11. using white boards, writing flow charts are first step towards programming and design.

    ReplyDelete