Franklin Webber[1] went on a quest to find out whether
utility code was better or worse than core code in open source
projects. Utility code is defined as anything necessary but not directly related to the application’s
mission. Franklin found that overall quality of utility code was higher, perhaps because this code changes less. He also found a lot of repetition among projects, many
people doing the same thing. A function to find the home directory
in several possible env vars, for instance. HashWithIndifferentAccess,
which doesn’t care whether you pass the key as string or symbol (keyword
in Clojure). He asks, why not break these out into tiny little gems?
Dependency management is an unsolved problem in our industry. We want
single-responsibility, and we want re-use. Reuse means separately
released units. But then when we compile those into one unit, transitive
dependencies conflict. Only one version of a particular library can make it into the final executable. In the example below, the Pasta library might not be compatible with the v2.1.0 of Stir, but that’s what it’ll be calling after it’s compiled into Dinner.
Programs like WebLogic went to crazy ends to provide different
classloaders, so that they could run apps in the same process without
this dependency conflict. That’s a mess!
We could say all libraries should all be backwards compatible. Compatibility is great thing, but
it’s also limiting. It prevents API improvements, and makes code ugly.
There’s more than API concerns as well: if the home-directory-finding
function adds a new place to look, its behavior can change, surprising
existing systems. We want stability and progress, in different places.
With these itty-bitty libraries that Franklin proposes we break out,
compatibility is a problem. So instead of dependency hell, we have chosen
duplication, copying them into various projects.
What if, for little ones, we could inline the dependency? Like
#include in C: copy in the version we want, and link whatever calls
it to that specific implementation. Then other code could depend on
other versions, inlining those separately. These transitive dependencies
would then not be available at the top level.
Seems like this could help with dependency hell. We would then be free
to incorporate tiny utility libraries, individually, without concern
about conflicting versions later.
I wonder if it’s something that could be built into bundler, or leinengen?
This isn’t the right solution for all our dependency problems: not large libraries, and not external-connection clients. Yet if we had this choice, if a library could flag itself as suitable for inlining, then microlibraries become practical, and that could be good practice.
——-
[1] Talk from Steel City Ruby 2014
[2] Angry face:
You're describing static linking with your C example. It's a function of how the thing is built, not the use of includes. You can dynamically link as well.The JVM really only supports dynamic linking, which means that if you want duplicate symbols you either need to runtime partition (e.g. app servers, OSGi, Classworlds) or remap symbols at build time (e.g. JarJar). Runtime partitioning is notoriously difficult and very much out of fashion. Many larger libraries use JarJar (and other build time remapping) techniques to achieve what you're describing).Build time remapping generally works pretty well, but it can break all kinds of stuff. Consider any app that references class objects in the config files. When you rename the class in the bytecode you also need to rename it in the config. JarJar has some support for this. Where it usually hurts more though is when debugging tools come into the picture. Unless your toolchain supports something like Javascript/CSS sourcemaps (and I don't know any that do) you can have a hard time mapping runtime diagnostics back to source. For stable utilities, this usually isn't so much of a problem.Gradle, Maven and Ant have pretty good support for JarJar. I'd be surprised if Leiningen doesn't.