Friday, May 18, 2012

Return of the Stateful Client

We used to write thick clients, fat happy Swing apps that executed application logic and business logic on the user's computer, and connected to the server mostly for data access. Then we moved toward service-oriented architecture and the thin, browser-based client. This solved all our deployment problems, and it moved the business logic back to the server.

Now that browsers are more powerful, more of the application logic is moving back into the front end. Instead of calling the client "thick," we now call it "rich." Where is this going? Toward stateful, browser-based apps. Forget maintaining a session on the server -- store all the state in JavaScript.

Think about apps that run in a browser, but aren't dependent on constant network access.

Now think about developing these apps. Think about reloading to bring in a JavaScript or CSS change... and starting over. With all the state in the JavaScript, reloading a page means restarting the app. More than that: reinstalling the app. That's harder than reloading one thin page to see changes, or running that Swing app from your IDE. So what's to be done?

Bring the IDE into the browser. With even more JavaScript, dynamically reload pieces of the application.

Last night at STLJS, Bill Edley from Technical Pursuits Inc. demonstrated his TIBET library, which makes the browser into the IDE. Change your application while it's running. Type commands in the little in-app REPL. The library also handles asynchronously getting data back to the server, abstracting temporary disconnections (motto: "There is no server, there is no server, there is no server"). It provides a richer OO framework for JavaScript as well.
What's the negative? 3 megabytes. yikes! and it's not out yet, so you can't play with it.

But hey, the thick rich client is back, still functional when the server is unavailable, and it is integrated into the IDE (Smalltalk lives!). Just think, no session data on the server -- that smells like freedom. We're bringing the code to the data again, where this time the data is user input. This bodes well for responsiveness.

Wednesday, May 16, 2012

F# made Java easier today

Today I had a sticky problem to solve, the kind business app programmers drool over because it's nontrivial and contained. Halfway through the TDD process, my logic got all jumbled and I couldn't see the solution. Stepping back, I took ten minutes to write pseudocode in an F# style, then translated that into Java. Bam! green all over. Turns out this problem is more suited to recursion and pattern matching than loops and variables. The same style works just fine in Java, but thinking in F# brought it to light.

As an illustrative example, here's the code in both languages.

The problem:
Money comes in through deposits. Money goes out through dispensations. Take an ordered list of each and pair them up into Utilizations, which identify which deposits were used to satisfy which dispensations. If any dispensation cannot be satisfied with the provided deposits, do not satisfy the dispensation at all.

The F#:

type Deposit = { amount : int }
type Dispensation { amount : int }

type Utilization = 
  | DepUtil of Deposit * Dispensation * int

let rec utilizeInternal outputSoFar currentDispOutput (deposits:Deposit list) (dispensations:Dispensation list) depUsed dispUsed : Utilization list = 
 match (deposits, dispensations) with
  | [] , _ -> outputSoFar
  | _ , [] -> outputSoFar
  | dep :: depTail, disp:: dispTail ->
     match(dep.amount - depUsed, disp.amount - dispUsed) with 
       | (depAmt, dispAmt) when depAmt = dispAmt ->
           utilizeInternal (DepUtil(dep, disp, depAmt) :: currentDispOutput @ outputSoFar) [] depTail dispTail 0 0
       | (depAmt, dispAmt) when depAmt < dispAmt -> 
           utilizeInternal outputSoFar (DepUtil(dep, disp, depAmt) ::  currentDispOutput) depTail (disp :: dispTail) 0 (dispUsed + depAmt)
       | (depAmt, dispAmt) when dispAmt < depAmt -> 
           utilizeInternal( DepUtil(dep, disp, dispAmt) :: currentDispOutput @ outputSoFar) [] (dep :: depTail) dispTail (depUsed + dispAmt) 0

let utilize deposits dispensations = utilizeInternal [] [] deposits dispensations 0 0

The key portion, the recursive function definition, works out to nine long lines. It divides the problem up neatly into: the deposit and dispensation match exactly; the deposit is more than the dispensation; the dispensation is more than the deposit. Those are the three non-trivial cases.

Translating this into Java simply puts these cases into if-then statements.

The Java:

public class DepositUtilizer {

    public static class Utilization {
        public final Deposit deposit;
        public final Dispensation dispensation;
        public final long amountInPennies;

        public Utilization(Deposit deposit, Dispensation dispensation, long amountInPennies) {
            this.deposit = deposit;
            this.dispensation = dispensation;
            this.amountInPennies = amountInPennies;

    public Iterable<Utilization> utilize(Iterable<Deposit> deposits, Iterable<Dispensation> dispensations) {
        return utilizeInternal(new ArrayList<Utilization>(), new ArrayList<Utilization>(),
                peekingIterator(deposits.iterator()), peekingIterator(dispensations.iterator()), 0l, 0l);

    private Iterable<Utilization> utilizeInternal(Iterable<Utilization> outputSoFar,
            List<Utilization> currentDispensationOutput, PeekingIterator<Deposit> deposits,
            PeekingIterator<Dispensation> dispensations, long amountUsedSoFarInCurrentDeposit,
            long amountUsedSoFarInCurrentDispensation) {
        if (!deposits.hasNext() || !dispensations.hasNext()) {
            return outputSoFar;

        long dispensationAmount = dispensations.peek().getAmountForJPA() - amountUsedSoFarInCurrentDispensation;
        long depositAmount = deposits.peek().getUnutilizedAmount().getPennies() - amountUsedSoFarInCurrentDeposit;

        if (depositAmount == dispensationAmount) {
            // great, use them both up
            Deposit usedDeposit =;
            Dispensation usedDispensation =;
            currentDispensationOutput.add(new Utilization(usedDeposit, usedDispensation, depositAmount));
            return utilizeInternal(concat(outputSoFar, currentDispensationOutput), new ArrayList<Utilization>(), deposits,
                    dispensations, 0l, 0l);
        if (depositAmount < dispensationAmount) {
            // use all of the deposit
            Deposit usedDeposit =;
            currentDispensationOutput.add(new Utilization(usedDeposit, dispensations.peek(), depositAmount));
            return utilizeInternal(outputSoFar, currentDispensationOutput, deposits, dispensations, 0l,
                    amountUsedSoFarInCurrentDispensation + depositAmount);
        if (dispensationAmount < depositAmount) {
            // use all of the dispensation
            Dispensation usedDispensation =;
            currentDispensationOutput.add(new Utilization(deposits.peek(), usedDispensation, dispensationAmount));
            return utilizeInternal(concat(outputSoFar, currentDispensationOutput), new ArrayList<Utilization>(), deposits,
                    dispensations, amountUsedSoFarInCurrentDeposit + dispensationAmount, 0l);
        throw new IllegalStateException("The sky is falling!");

The Java is twice as long as the F#, sure, but it works beautifully. What matters is that knowing a different language, and thinking about the problem in terms of the better-suited language, led me to a good solution in the language of my day job.

Functional programming! It's not just for math geeks!

Monday, May 14, 2012

Oh, IntelliJ IDEA, how I miss you

I'm switching from IDEA to Eclipse for the sake of conformity, with a great deal of mourning. No, it's not ideal, but it's a startup and not a good time to shave a yak.

Here are some features I miss. If you know how to do this in Eclipse, please, post a comment!

* Save the file whenever it loses focus!
* JavaScript highlighting even within .vm velocity templates
* JavaScript autocompletion. It's a dynamic language so the IDE can never be sure what properties an object will have, but IDEA comes remarkably close. It highlights known properties in one color, and underlines ones it can't verify. Great jQuery integration.
* Automatically importing classes that are only found in one place, or classes from specific libraries. Like, I want to type List<..> and the IDE should import java.util.List without asking me.
* Populating the filter of packages to exclude from the import suggestion list, from the import suggestion list. Not requiring a separate trip to the preferences dialog and typing in the useless ones.
* Ctrl-W. In IDEA, this widens the selection. Widen it to the word, then the expression, then the containing expression, etc.
* At the end of a line that contains an expression, command-shift-V to introduce a variable that stores the result of that expression. -- solution from Moandji: Command-2 (or ctrl-2 on Windows), then L
* Static imports: from an explicit static method call, alt-space for suggestions including "static import"-- solution from Moandji: Command-shift-M
* Recognition and auto-import of static methods from certain files (such as JUnit assertions)
* add a private final field; have the IDE generate the constructor or add the parameter to the existing constructor.
* switch back and forth between a source file and its test file.
* generate a test class and have it automatically go into test sources rather than defaulting to main
* When I copy a class, open the new copy. Don't make me go looking for it.
* In the debugger, increase the font size on variables. This is possible through some deep menu options, but then it increases the font without increasing the height of the row with the text in it. Not helpful.
* Reload projects when they change on the filesystem. Eclipse sometimes opens files with an error "This resource is out of sync with the filesystem" and no option to refresh it. Sometimes F5 on the project doesn't cut it and I have to restart the IDE. IntelliJ handles reloads smoothly, with a prompt and a quick reload.

and, to be fair, things that are handy about Eclipse:

* Opening multiple projects in one window. Our project structure was invented by Eclipse users, so instead of one project with several modules, we have several projects with tribal-knowledge-based dependencies between them. Having three IDEA windows open and switching between them was a pain.
* Make "missing @Override annotation" an error that occurs at compile-time, without a separate code-inspection step. Same with "empty method."
* Run tests when there are compile errors elsewhere in your project, in files that won't be hit by the test.

I'll add to this, and update it when people point out solutions. Suggestions in comments welcome.

Monday, May 7, 2012

Testing Google Analytics locally

When you're first setting up Google Analytics for your website, you'll want to do some testing in your local or test environment. The good news is: testing of the transmission is fast. The bad news is: testing the processing is not. This post describes the process.

Here's what you need:
  1. A google analytics account. It's free, no big deal.
  2. A profile. The default one will do fine. In profile creation, when it asks for your Website's URL, go ahead and enter the real, production domain name. It doesn't matter.
  3. Tracking Code: the javascript to include in your pages. It looks a lot like this, but I've changed it in three places:
<script type="text/javascript">

  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-000000000-0']);
  _gaq.push(['_setDomainName', 'none']);


  (function() {
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);


UA-000000000-0 is your Google Analytics account identifier. When you copy and paste the code from Google that'll be populated with real numbers, just for you.

_gaq.push(['_setDomainName', 'none']); is the magic make-it-work flag for local testing. This says, ignore the domain name in the URL and just post the data.

u/ga_debug.js is where the real work happens; in real life (production) this will say ga.js. For testing, use the debug version of the script. Notice the extra /u in front of the script name; the debug script is in a slightly different spot.
  1. Trigger some page loads in your local environment with that javascript included in the page.
The debug script will print useful information to the console every time information is transmitted to Google. Look for this in the debug console (command-alt-I in Chrome or Safari on a Mac):

Track Pageview
ga_debug.js:24Tracking beacon sent! utmwv=5.3.0d&utms=1&utmn=1124665081&utmhn=localhost&utmcs=UTF-8&utmsr=1920x1080&utmvp=1469x795&utmsc=24-bit&utmul=en-us&utmje=1&utmfl=11.2%20r202&utmdt=&utmhid=548350195&utmr=0&utmp=%2F&utmac=UA-31368436-1&utmcc=__utma%3D1.558561847.1335972658.1336071437.1336081900.9%3B%2B__utmz%3D1.1335972658.1.1.utmcsr%3D(direct)%7Cutmccn%3D(direct)%7Cutmcmd%3D(none)%3B&utmu=qB~
ga_debug.js:24Account ID : UA-000000000-1 Page Title : Your Title Host Name : localhost Page : /pathFromContextRoot/pageOfInterest?anyQueryParameters=stuff Referring URL : 0 Hit ID : 548350195 Visitor ID : 558561847 Session Count : 9 Session Time - First : Wed May 02 2012 10:30:58 GMT-0500 (CDT) Session Time - Last : Thu May 03 2012 13:57:17 GMT-0500 (CDT) Session Time - Current : Thu May 03 2012 16:51:40 GMT-0500 (CDT) Campaign Time : Wed May 02 2012 10:30:58 GMT-0500 (CDT) Campaign Session : 1 Campaign Count : 1 Campaign Source : (direct) Campaign Medium : (none); Campaign Name : (direct) Language : en-us Encoding : UTF-8 Flash Version : 11.2 r202 Java Enabled : true Screen Resolution : 1920x1080 Color Depth : 24-bit Ga.js Version : 5.3.0d Cachebuster : 1124665081

  1. Check Google Analytics Real-Time to see whether your visit has been recognized. (Click on Home in the orange header, then Real-Time on the left, then Overview.) This tells you that Google Analytics is receiving data.
  2. Now comes the hard part: waiting. Seriously, you have to wait a whole day to see the page views you generated. Then you can play around with the reporting, filtering, site search config, etc etc. While the data transmitted is received and recorded immediately, it is not processed for three hours or so, and more processing occurs overnight.
Keep in mind that when you change Google Analytics configuration (like site search query parameters), that affects only records that have not been processed. Once data is processed and stored, it doesn't get re-processed with updated rules. Testing configuration means make a change; generate the relevant traffic; come back tomorrow.