In this edition of What’s In Your Stack? we find out how a cloud-bound application is developed and architected. The notes on how Redis is used in the architecture are especially interesting:
Who are you?
I built and run Listening Room, a website for listening to music for other people. It was one of the first websites that allowed multiple people to collaboratively build a playlist and listen to it together in real time.
My name is Abe Fettig. I’ve been building applications for about 12 years now. One of my first college web projects was a Javascript-driven webmail app, and I’ve spent most of my career since then writing programs that run in web browsers.
How would you describe the development process you follow?
Up until about a year ago I was working at Google, which has a very comprehensive set of practices they follow for development. I learned a lot from the way people work at Google, but coming away form that process-heavy environment, and working by myself instead of on a team, has given me a lot of freedom to be fast and flexible instead of process-driven.
When I’m thinking about a new feature, I usually start by writing my ideas out with a pencil in my notebook. If I feel like I’m onto something good, I’ll sketch out some UI screens. When I have a UI that I think will work, I start coding. If it’s a larger piece of work I’ll try to think about the smaller tasks involved and write a TODO list on an index card or something, which helps keep me focused.
I try to think about the expected life cycle for a chunk of code before I invest too much time in it. For a lot of more experimental things I don’t write any unit tests or documentation, because I’m expecting that my whole approach will probably change based on my experience with the first version of the code.
What tools are you using for development and delivering your software?
I use Emacs for coding, terminal windows to run my code, and a whole bunch of web browsers for testing things. I code on a Mac and deploy to Debian/Ubuntu on my cloud servers. There’s so many developers working on Macs, and using similar stacks, these days, that it’s usually not hard to get all the same tools and frameworks I’ll be using in production installed (in particular homebrew is awesome here).
The main technologies I use for Listening Room are node.js, MongoDB for permanent data storage, Redis for message queues and an event bus, and FFmpeg for transcoding audio. As a long time client-side JS programmer it’s been fun using node on the server. My main concern when I started using node was the lack of libraries, but in practice that hasn’t been a problem, for two reasons. First, there’s an amazing community of programmers on GitHub churning out open source node libraries all the time and publishing them to npm. And second, node makes it really easy to call out to other processes. For example, there’s no node.js bindings for ffmpeg (or at least there weren’t when I started working on Listening Room), so I just call the ffmpeg binary from node, pass the music data through stdin/stdout, and read metadata from stderr.
Tell us about a recent tool, framework/SDK, or practice that you started using that worked out really well, much better than you’d thought. And/or, what’s one that didn’t work out well?
For a specific technology, I’ve been really happy with the choice of MongoDB. Coming from an SQL background it took me a while to acclimate, but now I really love it.
As far as overall system design, Listening Room uses lots of independent processes communicating through Redis instead of monolithic servers. Breaking up the server into lots of small services lets me take advantage of multiple cores, easily scale up parts of the app (by running multiple copies of the same service in parallel), and split tasks that used to run on the same machine across multiple machines without changing code. I was concerned about the performance of using Redis this way at first, but that hasn’t been a problem at all – Redis is super fast.
Anything else you want to tell us?
My Twitter handle is @SleeperCo (after my contract programming business, Sleeper Industries). My blog is http://fettig.net/weblog. I’m in Portland, Maine, an awesome little east coast mini-city. If you’re ever coming though feel free to ping me and I’ll be happy to buy you a coffee.
Recent Comments