Forking upstream software

December 03, 2009

Tom wrote a rant about the way we've incorporated other free software, and while I know it's easy to get carried away when you're mid-rant and talk in generalities, I thought it'd be interesting to look at the details.

First, some background. On Windows, your software is basically on its own: if the code you want isn't provided by Microsoft, it's your responsibility to package, ship, and update it. So for any third-party libraries we use (take sqlite as an example) we already must dedicate resources to integrating it into our build system, tracking security vulnerabilities upstream, and updating the code we use. This cost is fundamental to the endeavor, and any approach to software management beyond that (notably the Linux way) is by definition extra work to support. (That doesn't mean I don't want to change things, but just that doing so doesn't come for free.)

An interesting side effect of having all this code locally is that there is no longer a difference between your bugs and someone else's bugs: in tracking a bug down, you trace the code to the source of the problem and then you fix the bug, regardless of whose it is. Contrast this with development of software on any other platform, including Linux, where if there's a bug in Karmic's version of GTK the best we can say is "darn, let's file a bug upstream then attempt to find a workaround". (Which is what we've done; for a particular example I'll note my coworker Dan has written patches for two or three window managers, not that that helps any of our existing users.)

I believe this is generally the approach we have taken: do whatever it takes to make the best product we can, contribute bits upstream where we can. So let's take a stroll through the READMEs in the third_party directories and see what we can find. As I type this post I don't actually know the conclusion I'll reach, so this is for my education as much as yours. (Edit: Now that I've gone through more of this, I'll use the short phrase "build changes" to mean "changes made to let this to build as part of our cross-platform build system, ending up as a static lib that gets pulled into our final binary.")

There are a few other directories other than the top level for third-party code. Let's now go into them alphabetically:

...*whew*, that was exhausting! And I am actually rather surprised — I expected to find more projects hacked up like sqlite, when in fact that's one of the few projects we've actually forked in a meaningful way.