Turning Dross into Gold: Alchemy and Offline Browser Access

Share via Twitter Share via Facebook Share via Linkedin Share via Reddit

As Stephenson dutifully chronicles in the excellent Baroque Cycle, no less a personage than Sir Isaac Newton was captivated by the process of alchemy. As defined by Wikipedia, alchemy is a practice that combined “elements of chemistry, physics, astrology, art, semiotics, metallurgy, medicine, and mysticism.” Among its better known ambitions was to turn dross or base metal into gold. While modern science has – a couple of hundred years later – succeeded in that particular transmutation via a nuclear reaction, to the best of anyone’s knowledge none of the pre-scientific method alchemists ever succeeded in their endeavours – most likely because they lacked the necessary tools.

Give the context of the term’s history, I find the selection of Alchemy as the project name for Adam Bosworth’s BEA project an intriguing one. As James reminds us, Jon Udell’s covered this ground before, saying:

Adam Bosworth probably regrets having shown me the Alchemy prototype in the summer of 2004. I’m like a dog with that bone, constantly gnawing. Alchemy was about pushing data intelligence into the browser. Its core components were a lightweight XML datastore and…wait for it…a simple synchronization engine.

In other words, by combining seemingly disparate elements, Bosworth was seeking to turn a base application – the browser – into gold by giving injecting it with elements of intelligence and persistence.

With Bosworth now at Google rather than BEA, I’m unaware of the status of that work, except to say that to the best of my knowledge it hasn’t seen the light of day (seems like an excellent potential donation, BEA). The problem, however, has only grown more acute. See, for example, my post here, Scott’s here, Stewart here, and the man himself, Adam, here. But even without such testimonials, the problem should be obvious to just about anyone who’s used a browser: once you lose a network connection, the browser almost immediately becomes the least useful piece of software on your machine. But does it have to be that way? Maybe not.

Earlier this week, as Simon was kind enough to alert me via IM, Sun not only announced support for the Derby codebase from Apache (nee Cloudscape at IBM/Informix) but demonstrated their version – called Java DB – functioning as the offline persistence mechanism for a browser based application. Go read Simon and Tim for the Sun perspective, and Ted for the view from a non-Sun guy. While I do think that Sun supporting multiple relational DBs – different as they may be – opens the door for FUD and potentially confuse customers, I’m going to be ecstatic if someone solves the browser persistence problem for me.

The really interesting thing, however, is that Derby is but one amongst several potential candidates that could “fix” the browser. If JavaScript is the new app server, what might be its back end? Well, as Ted has, I’ve heard rumblings that the Mozilla folks have at least kicked the tires with respect to the embeddable DB SQLite, which given its role as the back end towards several of the applications I run on my Linux desktop would seem to be a viable candidate. And if SQLite’s being actively considered, it might be interesting to look at Sleepycat’s Berkeley DB – as that’s also heavily used in behind the scenes roles in products like Evolution. But what about forgoing the database entirely? As Udell covered above, the Alchemy work was at least in part about giving the browser access to a lightweight XML datastore, and as James and Tim have linked to recently, there are folks out there proving that at least in some situations, XML text files might serve just fine. Particularly if they had a next gen filesystem behind them…but that’s a topic for another day.

Now is the solution to the problem of offline browser access as simple as grafting a persistence mechanism onto the existing browser? Hardly. If that were the case, Bosworth or some of the talented Mozilla folks would have solved this problem eons ago. The difficulties in determining what content to cache, when to cache it, how to refresh it, and how to expire it are not trivial. They’re solvable – and I believe will be addressed – but not overnight, regardless of how plentiful the back end options might be.

One particular concern I don’t share, however, is that expressed by one of the commenters on James’ piece, Alex Goncalves, who says:

AJAX, local persistance with Derby, seems to me like we’re making the browser fatter and fatter. I mean, why not use WebStart? with rich client apps that have permissions to use local resources? Shouldn’t we just leave the browser what its good at, i.e thin-client? instead of transforming it into an abstract OS environment?

I don’t agree with this for three reasons:

  • First, the line between thin and thick clients will be blurred regardless of what happens within the browser simply by virtue of the introduction of the technologies formerly known as Avalon.
  • Second, I don’t see any reason why the offline persistence mechanism can’t be loosely coupled to the browser, thus preserving at least the option of a pure, lightweight browser that lacks non-network capabilities.
  • Last, I think that the direction of Software-as-a-Service, browser based applications will inevitably result in the introduction of offline features and functions. Just think what Salesforce.com or Zimbra could do with a browser that handled network degradation gracefully.

Sooner or later someone’s going to turn the metaphorical dross of an offline browser into seamless offline gold. I don’t know who, I don’t know when, and I don’t know (precisely) how. The one thing that I do know, however, is that it’s probably not going to take as long as it did for the real thing. The modern day Alchemy-ists, after all, aren’t lacking the proper tools.


  1. The other nice thing about Alchemy is that it made coding rich dynamic data driven web apps really easy.

    We had designs on not just replacing the way that offline-required apps were made, but also the way online apps worked, as people later saw with ajax there is a lot of potential in applications that don’t have to reload the entire page just to do something trivial.

    I wouldn’t read too much into the name of Alchemy, it’s my Dad’s pet name for any project, he had it on his license plate while he worked at MSFT.

  2. Every time I think about working on offline access for one of my web apps, I just can’t do it…

    Connectivity is increasing and will continue to be available in more and more places. Why spend lots of time working on hard problems that could be irrelevant within a matter of years (especially when there are so many great features to build!)?

    Maybe I’m too optimistic about the spread of connectivity – maybe not.

  3. Alex,

    The problem is that the toolset is not there today for you to do it. You braved the AJAX world long before it had a name, or even XML support for that matter.

    Once you get an interface, like XMLHttpRequest, that lives in every browser, just like document and window do today, your mind might change πŸ™‚

    I want to take Tasks with me, but usually can’t, and that would help a HUGE amount to have those base APIs available.

    Once I’ve made a few mil, maybe I’ll run off and tackle this problem πŸ™‚

  4. Offline browser persistance = personal web server?


  5. Scott–

    Maybe, but I’d rather have connectivity everywhere and not have to worry about online/offline. I’m getting closer to that. πŸ™‚

  6. Alex B: i hadn’t even considered it in that dimension, but it’s a great point. persistence is as important to the developer as it is to the user, perhaps even more so.

    Alex: i simply can’t rely on wireless, b/c there are too many occasions when i don’t have it. at my parents house in NJ, for example, which is quite near the junction of two major interstates, the cell coverage is quite poor – nearly 10 years after i got my first cell phone. is it getting better? surely. but will there be many situations where i need persistence and either don’t have it or am not willing to pay a fee to a wifi provider just so that i can check an email for 30 seconds? i think so.

    Scott: let’s hope those FL checks start rolling in, so you can sit back and tackle it πŸ˜‰

    Anon: good link, and definitely related.

  7. FYI, a prototype for an AJAX-based online/offline wiki: http://blog.monstuff.com/archives/000272.html

    This approach still takes a lot of work and needs to be improved, but it shows some promise.

Leave a Reply

Your email address will not be published.