Alpha.gov.uk: What's in Your Stack?

For the third installment of What’s in Your Stack? I spoke with James Stewart, of Ket Lai, who’s been working on a team hoping to make the UK’s government’s public face on the web a better, more productive experience. This one is extensive, but it touches very nicely on both technologies used and the people management side of the project, including how it may fit into the governments “mainstream” process when the experimental phase is done:

First, describe company, the business that the stack is supporting

Last year the UK government commissioned Martha Lane Fox (founder of lastminute.com) to produce a report on government online. That report recommended a “digital by default” strategy where the first stop for citizens to access government services was through a single government website. Following on from that Tom Loosemoore (ex-BBC, Channel 4, etc) was asked to pull together a team and produce a public alpha of what that single government site could look like. Three months later, it’s live and we’re very actively soliciting feedback that will help shape the future of the government’s digital strategy.

How would you describe your development process?

Historically government has outsourced most of its development, but for this project it was seen as very important to have an in-house on-site team, following agile methodologies. Around 12 of us worked from an civil service office in Lambeth, completing weekly sprints. We worked from referrer and search logs to identify the 100 “needs” that most frequently brought people to government sites and we identified a few custom tools and some ways of packaging content to meet those needs. Getting the composition of the team right was obviously crucial for a project with a tight timeframe and high profile, but thankfully it really gelled. We were helped enormously in that by a fantastic project manager called Jamie who kept us all focussed and provided a huge amount of support.

In terms of the back-end development we tried to stay as close to the user needs as we could. This wasn’t the time to be building out platforms or making choices that we’d be bound to should the prototype be taken forward to be the government site. Our team was comfortable with Rails, Sinatra and Django, and so we used a combination of them along with some custom middleware that would support some of the geographic utilities we wanted, and that would ‘skin’ the content to provide a consistent look and feel for the site. Everything went through at least a couple of iterations, but there are a lot of rough edges.

Despite the whole team being committed to automated testing, the way we got started and a couple of architectural mis-steps early on meant we didn’t automate our testing as much as we should. That bit us a few times and it’s something I’ll be very keen to address in the future. Where we do have tests we are doing Continuous Integration, reporting into an IRC channel.

I was very pleased we were able to make use of ScraperWiki to gather a lot of the raw data we needed.

We’re deploying everything onto EC2 and amazon were very helpful with scaling advice and support. We’re managing configuration with Puppet, and it’s been great to have the freedom to experiment with server configurations as we’ve built the rest of our stack. And with the potential level of attention being a constant concern, Varnish has been a god-send, even if I am a bit disappointed that we couldn’t make version 2 work well with simultaneous mod_deflate and edge side includes.

What development (IDE, build tool, etc.) and project management tools did you guys use, if any? Given the dynamic languages, I’m curious what the team uses.

The front-end devs both worked in Textmate. We set up a VMWare VM for them with a full check out of all the code so they had a convenient environment to develop locally. We were really lucky that they’re both pretty comfortable with git so were able to fit right into the way the backend was developed.

The backend team was evenly split between Textmate and Vim. When working in dynamic languages we’ve not really found a need for much more.

All our code was hosted on github, using one of their “organisation” accounts. We started out using github issues for ticketing, but switched to lighthouse as we needed a bit more flexibility, and better ways to see issues across the project rather than just per-repository. Our project manager then used google docs to track progress, and we used a wiki to keep notes on work in progress, research done, and other documentation. He’s written a bit about the project management under the banner “agile does work in government.”

Over time, I’m curious how you guys will integrate all the cool, whiz-bang stuff into the staid, normal stack that I’d expect government to be more comfy with. To start, will the project transition to civil servants rather than the current mixed team?

Most of the team aren’t civil servants. A couple of people were seconded from other parts of government, but a deliberate effort was made to bring in “outsiders” to avoid being caught up in existing politics and long-running debates. Our Editorial Lead wrote a bit about that on the blog. That’s a pretty radical step for government which has long outsourced development work (though most departments have in-house teams for communications, editorial, etc). I can’t speak for any formal decisions that might have been made, but my understanding is that there’s a shift going on where at least some of the development work will be brought in-house and if this project is taken forward an internal team will eventually be picking it up. That’s partly about integrating it into the “normal way” but also about shifting that normal a bit.

In terms of the actual tech used, I get the impression there’s quite a bit of scope for continuing to use whichever tools are most practical for the job at hand. We’ve seen the US government adopting drupal quite widely over the past few months, WordPress has been used pretty extensively in UK government over the past few years, and there are some rails apps in production for government. So there’s definitely some momentum behind open source options and building on top of frameworks that can be built on and/or deployed rapidly and there are quite a few signals that we’re entering a period where there’ll be quite a bit of flexibility in technology choices and a lot more developer involvement in those decisions.

(If you’re willing to answer a few questions for a What’s in Your Stack? piece, feel free to simply fill out the new questionare, I’d really appreciate it!)

Disclosure: GitHub is a client.