Tuesday, November 29, 2011

Humility in programming languages

John Backus (pdf) distinguishes two parts of a programming language: "First, its framework which gives the overall rules of the system, and second, its changeable parts, whose existence is anticipated by the framework but whose particular behaviour is not specified by it."

The framework part is all the built-in language constructs.

The changeable parts are everything we create in the language. Classes, methods, functions, modules, libraries.

Backus's point is that strong languages are a platform for developers to build on. Adding a bunch of framework -- for loops, case statements, new syntax -- might increase productivity, but it's what we can build on top of the language that makes it take off.

When Scala tries to be a SCAlable LAnguage, this in part of what scalable means. The language lets developers add to it.

For example -

Java and C# give us case statements, very restricted matching mechanisms that only work on constants. It was a Big Deal when Java added Enum and supported in case statements, in addition to int.

Scala has more pattern syntaxes, including matching strings and lists and arrays and tuples etc. Scala gives the program access to the matched parts.

F# goes one better and lets the programmer define patterns. Active patterns let the programmer create whatever matcher is needed, exponentially increasing the power of pattern matching.

For details and examples on active patterns, read my article at developerfusion.

In general, functional languages please Backus because they let the programmer create and use changeable parts in all sizes with minimal ceremony. F# active patterns exemplify this goal.

Let the language remain humble, and let its framework get out of the way of the changeable parts.

Saturday, November 26, 2011

Stupid Gradle Tricks #1: what can I do in this block?

In the previous post, we saw that each labeled block in a Gradle build file is really a closure with DELEGATE_FIRST scope. Each of these blocks has access to all the methods on the delegate object. The class of that delegate object varies by block.

Even looking at the source code to identify the class of the delegate won't tell you every method that's available on it. This is because Gradle plugins dynamically add methods to these things.

Here is a sneaky trick to find out what's available to you in a particular block. If you give the closure one argument, Gradle will populate it with the delegate. Then you can have the build print out information about it.

The following build script prints out the class of the repositories delegate and every method that's on it, formatting the output to remove bulky package information.

apply plugin: 'java'

def printDelegateInfo (String blockName, Object d) {
  println "---- ${blockName} block ----"
  println "Delegate is of class " + d.class
  d.class.methods.each {
     println (("Method: " + it + " from " + it.declaringClass =~ /[a-zA-Z\.]*\./).replaceAll("") )
  }
  println "----------------------------"
}

repositories { d ->
  printDelegateInfo("repositories", d)
}


The output is something like:
---- repositories block ----
Delegate is of class class org.gradle.api.internal.artifacts.dsl.DefaultRepositoryHandler_Decorated
Method: public void DefaultRepositoryHandler_setProperty(String,Object) from class DefaultRepositoryHandler_Decorated
Method: public Object DefaultRepositoryHandler_getProperty(String) from class DefaultRepositoryHandler_Decorated
Method: public Class DefaultRepositoryHandler_getType() from class DefaultRepositoryHandler_Decorated
Method: public String DefaultRepositoryHandler_getDisplayName() from class DefaultRepositoryHandler_Decorated
Method: public FileResolver DefaultRepositoryHandler_getFileResolver() from class DefaultRepositoryHandler_Decorated
Method: public void DefaultRepositoryHandler_setConvention(Convention) from class DefaultRepositoryHandler_Decorated
...
Method: public void whenObjectAdded(Closure) from class AbstractDomainObjectCollection
Method: public Action whenObjectRemoved(Action) from class AbstractDomainObjectCollection
Method: public Set findAll(Spec) from class AbstractDomainObjectCollection
Method: public DomainObjectCollection withType(Class,Action) from class AbstractDomainObjectCollection
Method: public DomainObjectCollection withType(Class,Closure) from class AbstractDomainObjectCollection
...
Method: public final native void notify() from class Object
Method: public final native void notifyAll() from class Object
----------------------------


You could call printDelegateInfo in any similar block in Gradle, including a nested one. Just add the "d ->" at the beginning of your block to capture the delegate in a variable.

This trick is not as cool as autocomplete, but it can give you some clues. Note that a delegate's method with a Closure argument means that you can probably create a block with that label and do more detailed configuration.

Friday, November 25, 2011

Lifting the blankets of Gradle

Gradle, the build system, is a DSL on top of Groovy. The build scripts (which are really configuration scripts) are Groovy code. So why do they look so weird?

Today I configured a particular obscure property of the eclipse plugin in milestone 6 of Gradle. The answer looks like:


apply plugin: 'eclipse-wtp'

eclipse {
wtp {
facet {
file {
whenMerged { config ->
config.facets.each {
if (it.name == 'jst.web') {
it.version = 3.0
}
}
config.facets.unique()
}
}
}
}
}


What is it doing? What is that "eclipse" word anyway? Is it a type? a variable? No -- it is a method. The eclipse method (defined on the Project, but I'm not sure how) accepts a closure. Similarly, "wtp" is a method on EclipseModel, which accepts a closure. "facet" is a method on EclipseWtp.

But wait, how within the closure passed to the "wtp" method do we have access to a method on EclipseWtp? What determines that? The answer is: before calling our closure, Gradle sets some properties on the groovy.lang.Closure. The following snippet is modified for clarity from org.gradle.util.ConfigureUtil:


Closure copy = (Closure) configureClosure.clone();
copy.setResolveStrategy(Closure.DELEGATE_FIRST);
copy.setDelegate(delegate);
copy.call()


... where the delegate was passed in like so, in org.gradle.plugins.ide.eclipse.model.EclipseModel:


EclipseWtp wtp = new EclipseWtp()

void wtp(Closure closure) {
ConfigureUtil.configure(closure, wtp)
}


What happens is:
- wtp is a method that accepts our closure
- the wtp method sets an object of type EclipseWtp as the delegate on the closure
- the wtp method tells the closure to resolve methods on the delegate first.
- finally, the closure is called.

So, the strange scoping blocks in gradle turn out to be method calls. The way to find out what delegate you're getting access to within each of these is to look at the DSL Reference or (for the real nitty-gritty details) the source code. For instance, the DSL Reference documentation for EclipseModel says that within the wtp {} block, you should see the documentation for EclipseWtp. (Note that the javadocs are useless. This is groovy trickery, not Java.)

This concept of a resolve strategy within a closure is interesting.

Wednesday, November 23, 2011

What to look for in a BPM system

This week I presented a lunch'n'learn about Oracle BPM 10g. The theme of the presentation was "This is why you should never use this product if you have a choice."

Business Process Modeling (BPM) can have a lot of value to the business. It's effectively a graphical Domain-Specific Language (DSL). Even when the implemented logic is complex enough that it takes programmers to create the BPM process, the analysts and business people can read it. The serious value of BPM in a very complicated application is the universal language: the processing steps each have names. The analyst or the support person can point to the chart and talk about what should happen. Since the process design is executable, it is never out of date. This communication tool is priceless.

When implementing BPM, there's a choice of systems out there. My recent experience is with Oracle BPM 10g. These are the things we long for after two years of working with this product:
  • Traceability. We need to see the history of an instance, what route it took through the process and what its variables contain.
  • Flexibility. How hard is it to change a process? What restrictions are there? In Oracle BPM, there's no merge and no locking mechanism, so we often discard changes and start over. Once a process is in production, activities can never be deleted or moved to another role, instance variables must remain forever, and various other restrictions that we can't even define clearly.
  • Stability. Oracle BPM just plain lies, returning nothing or an error on some queries.
  • Testability. Testing in Oracle BPM requires lots of custom code and is entirely manual. There's no automating it, and there's no dropping an instance in where you need it; it has to start from the beginning.

So what's a BPM-hungry company to do? What I've heard about Oracle 11g (the newer version, more closely integrated with SOA) is that it doesn't fix the problems in 10g and is not production-ready. That's hearsay. So far people are more impressed with Lombardi, which is now (unfortunately, for the price-sensitive) owned by IBM. For open-source, Mule has a workflow engine built in, which can serve as a form of BPM. What else is out there? Who works with a BPM system that they actually like?

Thursday, November 17, 2011

Dependency Injection vs Functional

At work, we use dependency injection to build up small, decoupled classes each with a single responsibility. DI lets us assemble these in multiple different ways, so we can use the same class for similar processes. This strategy of combining small pieces of functionality in different ways is also a goal of functional programming. Let's contrast the two methods.

There's a chunk of code whose job it is to string together the contents of a list of segments. The contents can be in one of two different fields in the segments. The segments might have connectors, which might need to go between the contents. We might need to skip the first and last segment if they're not interesting.

To avoid duplication of any part of this, our current Spring-y solution at work uses dependency injection to construct different SegmentContentsConcatenators with different components. The result is 9 Java classes and 6 Interfaces.

  • Builder
    • has a SegmentRemover (or a not-remover)
    • has a Concatenator
      • has a ConnectorRetriever (or a not-retriever)
      • has a first-or-lastness determiner
      • has a ContentExtractor (two implementations, one for each possible field)


The Spring configuration defines 3 beans that are useful, but in order to hydrate these in all possible combinations, 17 bean definitions are required. Fortunately for us we care about three of the eight possible behaviour combinations, so we only need 12 bean definitions. Only!

That's a at least 120 lines of Java in 15 files, plus about 30 lines of XML. Not counting tests.

Just for fun, I implemented the same functionality in F# using tryfsharp.org. The result is 12 little function (including two that are generic and could be placed in a list-processing library). It took 30 lines of F# code. The functions build on each other, and the three desired functionality combinations are pieced together from other small functions. Best of all, there's no XML at all!

Both methods have the same goal: take small, testable pieces of functionality and piece them together in multiple ways. This is what functional languages DO. Trying to achieve the same goal in Java with Spring was not so pretty.

For reference, here's the F# code:

// defining the types
type Contents = string

type ConnectorMethod =
| Easy
| Complicated of Contents

type Segment = { ExpectedContents : Contents; ObservedContents : Contents; Connector : ConnectorMethod; IsExtraneous : bool;}

// Functions that build up all the pieces
let findConnector c : Contents =
match c with
| Easy -> ""
| Complicated x -> x

let expected s = s.ExpectedContents
let observed s = s.ObservedContents

let expectedInternalJunction s = expected s + findConnector s.Connector

let rec concatSegmentsInternal f fIJ acc list =
match list with
| [] -> acc
| [x] -> acc + f x
| head :: tail -> concatSegmentsInternal f fIJ ( acc + fIJ head ) tail

let concatSegments f fIJ l : Contents =
match l with
| [] -> ""
| x -> concatSegmentsInternal f fIJ "" l

let isExtraneous s = s.IsExtraneous

// these last two are generic, could apply to any predicate and list
let rec removeLastIf s l : 'a list =
match l with
| [] -> []
| [x] when s x -> []
| [x] -> [x]
| head :: tail -> head :: (removeLastIf s tail)

let removeEndsIf s l =
match l with
| [] -> []
| head :: tail when s head -> removeLastIf s tail
| head :: tail -> head :: removeLastIf s tail

// Publically useful methods - these put the pieces together.
let concatExpectedSegments : Segment list -> Contents = concatSegments expected expectedInternalJunction
let concatObservedSegments : Segment list -> Contents = concatSegments observed observed
let concatInterestingObservedSegments ss : Contents = concatSegments observed observed (removeEndsIf isExtraneous ss)

// Tests
let s1 = {ExpectedContents = "ABC"; ObservedContents = "ABCD"; Connector = Complicated "-"; IsExtraneous = true;}
let s2 = {ExpectedContents = "EFG"; ObservedContents = "EFGH"; Connector = Complicated "a"; IsExtraneous = false }
let s3 = {s1 with IsExtraneous = false}

let ss = [s1;s2;s3]

assert (concatExpectedSegments ss == "ABC-EFGaABC")
assert (concatObservedSegments ss == "ABCDEFGHABCD" )
assert (concatInterestingObservedSegments ss == "EFGHABCD")