Scalability: Damned if You Do, Damned if You Don’t

Share via Twitter Share via Facebook Share via Linkedin Share via Reddit

One year and three days ago, I wrote an entry entitled “What Do These Services Have in Common?” The services in question were Audioscrobbler, del.icio.us, Flickr, Technorati, Paypal, and Weblogs.com. On a cursory glance, there seems to be little in the way of a common thread. Certainly financing was not the common denominator: some were bootstrapped, some were VC funded, and some were the province of very large commercial entities. Nor is there a great deal of overlap functionally speaking. The linking factor, instead, was scalability – all of the mentioned services had been experiencing significant latency and/or downtime in the weeks prior to December of last year.

Scalability, I concluded then, was of paramount importance in a service driven world.

But what this does indicate, I think, is that in a services world, scalability is at a premium. It’s ironic from where I sit, because I sat through far too many WebLogic/WebSphere bakeoffs (with their respective tens of thousand per CPU licensing deals) in the late 90’s that were intended for deployments of a few thousand seats, while the services above are seeing traffic at least in the millions and are often delivered through quote unquote non-commercial server software.

This line of thinking was hardly original or prescient, of course, given that scalability’s not exactly a new concern. But I do think it’s increasingly driving the success or failure of the next generation of web applications.

Google Blogsearch, I think, might be the best example of this phenomenon at work. After having very little nice to say about it at its release, I’ll confess that it’s the first blog search engine I turn to these days. Why? Because its results are the most relevant? Negative. Because its results are the most timely? Nope. I use it because it’s just so damn fast. Google’s absolutely mastered the fine art of providing absurdly low latency in response times (with a few notable exceptions), and while I might not consciously be aware of it that affects my usage in interesting ways.

When one of the folks on my blogroll that I respect (who should feel welcome to name themself if they so choose) and I got into a debate about the virtues – or lack thereof – of tagging some time ago, the individual mentioned that his web logs saw me travelling into his site via Google rather than del.icio.us, which I’d been defending. I did not then and do not concede now that that invalidates the value of tagging and del.icio.us, but it was an interesting observation nonetheless. I went in via Google not because of search efficiency or any obvious product features, but because of performance. On the one hand it seems slightly absurd; as fast as Google is, we’re literally talking about a difference of seconds. But I’ve accepted as a fact of life for a while – as have a great many web application designers – that such miniscule slices of time can be the difference between a successful and unsucessful application.

So what does this all mean? Simply that scalability is important now, and is likely to become more so. In a critique of FeedLounge’s (disclosure: FL are not RedMonk customers, but I know Alex and Scott personally) approach, Jeremy Wright says the following:

I’m not sure when building a scaleable web app became optional. But Feedster, Technorati, Delicious, Google Analytics (and numerous other Google apps of late), BlogPulse and many of the other “big apps” have “suddenly” been hit by scaleability issues.

And so has FeedLounge…

Erm? Hello? Should the recoding have happened after step 1? I mean, if you draw a graph of “okay if we use 10% of a CPU with 10 users, with 100,000 users we’ll need 10K CPU’s”. Something’s wrong.

Maybe I’m just spoiled, having worked in high performance, high availability apps before, but it constantly astounds me what some folk consider “scaleable” and “available” applications. I’ve spent about 10 hours this month working with really, really high profile Web 2.0 ish companies nearly yelling at them about their lack of true infrastructure.

From the FeedLounge side, Scott responds in a post entitled “Scalability is not the only concern,” saying:

Nothing is wrong. It is all choices that you make trying to start a company. Alex and I chose to focus on the user experience instead of scalability, and now we know that we have to scale. I knew going in to this venture that it would be a huge amount of data to move. I do have a great number of years experience making fast things faster in the software world. You mention that is “astounds you” as to what people define as scalable and available. No one has ever used those two words to define FeedLounge. Nor will they, until we have proven that we can. Ask any of our alpha users. They don’t stick around for the availability, they stick around for the features that they have been given. And yes, scalability is a feature, but not one that should be the major focus in incubation.

To me, the question that Jeremy and Scott are debating boils down to two choices:

  1. Build scalability in from the start
  2. Focus on features first, scalability second

In a perfect world, I’m with Jeremy. While I’m at none of these guys level technically (particularly these days), I’ve got enough hands on experience to know that it’s very difficult to build in scalability after the fact – just as it is with security. More to the point, in this perfect world, I’m fairly sure Alex and Scott would agree. But it not being a perfect world, with resources being limited, choices have to be made. It’s the old “pick two” quandry. A choice in favor of scalability would likely mean a glacial pace of development for the application, and a far less sophisticated user interface. And further, is it likely that Alex and Scott would ever be able to differentiate on the basis of scalability – can they out scale Google, for example? I think not. Instead, they chose to focus on features at the expense – in certain cases – of scalability. The result is a truly differentiated interface that now has a need to scale better.

In a way, the FeedLounge guys and every other developer wishing to cater to an internet sized audience is damned if they do and damned if they don’t. They can design boring applications that scale, or cool applications that don’t. Doing both simultaneously is exceedingly difficult, even for an organization the size of Google (see Analytics, Google). But as much as I do believe that scalability is important, and potentially a compelling feature in and of itself, if I’m in Alex and Scott’s shoes I probably make the same choice. I’d much rather have too much demand for my application than too little, which is what I think you risk if the primary focus from the outset is on scalability as opposed to compelling features. Longer term, I’d invest heavily to scale, but while in alpha I’m not convinced that’s the way to go.

One interesting question that I’m beginning to ask from all of this: is it even possible for smaller players to scale competitively? As good as the del.icio.us, FeedLounge, et al guys are codewise – and many of them are very, very good – there’s the non-coding aspects of scalability to consider. Networking, hardware, etc require capital investments that can be onerous when bootstrapping, and it’s a simple fact of life that the smaller guys don’t benefit from the economies of scale that say a Google or a Yahoo will. They buy hardware piecemeal, and get run-of-the-mill bandwidth deals, all of which adds up to high entry costs.

But might there be an opportunity there? Might there be an opportunity to aggregate lots of these Web 2.0 (for lack of a better term) services in one place, and offer them an affordable, scalable infrastructure? Maybe. There are lots and lots of hosts out there, but are there any that provide specific expertise in scalability? None that I’m aware of.

Either way, watching the different approaches taken towards better scalability should be interesting. As I write this, Gmail is non-responsive and timing out. Anyone need more proof that scalability is hard?


  1. FeedLounge Scalability

    There is a great conversation going on about web applications and scalability, with FeedLounge as an example.
    A few months back, there was a meme going around talking about how cheap it was to build a web based application/service/company compared to…

  2. Perhaps Sun Grid can open for business to Web 2.0 services?

    You can convince them, can’t you πŸ˜‰

  3. Bang on. This isn’t cheap. And there isn’t any reason services that are all bootstrapping can’t go in together to help each other out or that a service couldn’t start out with scalability in mind (basically you’d pay for your traffic or something).

    Great points πŸ™‚

  4. terrific post. it would be terrific if someone could provide scalability on some sort of basis other than cash up front, for small Web 2.0 companies.

    I a certain sense, that is what Akamai did for Web 1.0 companies–at the Dean campaign we would pay extra for bandwidth at peaks, to Akamai, and though it was expensive it was better than the alternative. To send out the “Judy Dean interview” video online we spent $70k in a day or so.

    Another better solution, certainly for a Web 1.0 site, would be some sort of bittorrent-based cooperative arraingment. Had bittorent been fartr along during the campaign, we could have gotten 30k or so users or more to host torrents on their computers, and driven our cost for video dowloads to zero. That would have let us use a lot more video, which would have been fun and perhaps effective.

    It strikes me that in the Web 2.0 world scalability is not as simple as in Web 1.0. The server and processing issues are more central, because at heart the services are more computationally intensive. Bandwidth alone is not a solution.

    Technorati is a perfect example, with complex combinatorial searches as its core offering, but very light bandwidth requirements. No way is it going to be able to keep up with Google’s server and processing and computational capacity, methinks.

    It would be interesting if Microsoft or another big company found a way to do the core backend work for Web 2.0 companies, in trade for some sort of revenue shares or traffic building. This could lead to a federated organizational structure competing with Google’s and Yahoo’s more centralized organization and technology infrastructure.

    That might be an interesting development in the ecosystem..

  5. Jim – my company (Digipede) has been working with Microsoft to create a software solution that makes it very easy for developers to write scalable applications built on their platforms (that is, Windows and .NET). We, in turn, have been approached by at least one service provider who is interested in building facilities that provide a hosting service that would provide true scalable functionality to Web 2.0 companies.

  6. many apps won’t “scale” regardless of the hosting infrastucture. a lot depends on the application design. the specifics of “scale-out-ness”. i am not commenting generically on the ability of these web 2.0 services to scale, but rather considering application design.

    same in the Java world. a “J2EE” app that doesnt index data but constantly hits a database is not going to scale, regardless of how many servers you throw at it.

    in terms of your resource question, lets just look at google analytics- which, it did *not* design, but acquired. google doesnt lack the resources to “host” these services it acquired. so what *is* happening. why can’t google do it? i would argue that the initial google architecture design was scalable in a way some of these new services might not be.

    the whole notion of scale is problematical too. there is a lot of expectation management in this space. when Danny Sabbah asks questions of PHP lesscoders call him an asshat. When Stephen O’Grady asks about web 2.0 scale he is applauded.

    now i fully understand that web 2.0 and PHP are not interchangeable. but the point i am trying to make is about expectations.

    there is also an interesting similarity with Microsoft’s growing up. in that didnt microsoft focus initially on user experience, with no real scalability of design. the question in my mind is can web 2.0 players really deliver scalability as a service (rather than the lumpy improvements and forklifts we saw with windows). ie happening at the back end without the user noticing. we’ve seen some moves in that regard-typepad for example (we noticed the degradation but not the upgrade). which i guess brings us back to your question.

    i think a definition of scale would be useful. subsecond response times across many users? or what? that is one reason i like your google example.

    that is there is no latency in a google search. compare and contrast with my supposed rich client, where i spend many minutes a day wondering what in the hell the machine is doing.

    that is another advantage of services 2.0 – that is, it makes a lot of sense to just focus on scaling one service, rather than trying to scale *any* service. back to google…

    just a few thoughts.

  7. Scott: you can be sure i’ll be trying to do just that πŸ˜‰

    Jeremy: totally agreed. the way that i was talking about it w/ Alex last night, it should be some sort of union type situation. pool together for collective purchasing power, etc.

    Jim: you actually scooped me there – well done. i was thinking about a scenario in which some of the bigger providers – folks such as Google, Microsoft and Yahoo – that presumably know how to scale, would become providers of a sort as well. more on that later.

    Dan: interesting. would love to talk to you more about that. one question: do you support Mono apps as well?

    James: not sure if that’s exactly the pushback the lesscode folks had against sabbah. danny didn’t ask questions, unfortunately, he said it couldn’t scale – and the lesscoder’s acted as they probably should.


  8. saying something doesnt scale is asking a question of the something. no need to tie ourselves in knots here, but i would have thought there was more to my response that that. hey ho.

  9. Stephen – we’ve had one user running a couple of agents on Mono. We’re not officially supporting it yet–it would be biting off more than we can chew at this point–but we’re .NET based, and it seems to work well. There are some security issues (WSE 2.0 and 3.0 don’t exist on the Mono side, if I’m not mistaken), but I believe that those can be overcome with policy. Check out my blog or our website (www.digipede.net) if you’re interested.

    One other point to make: scalability is very expensive, it’s true. But planning for scalability isn’t necessarily expensive–you have to make smart decisions when designing your solution. On the other hand, if you don’t make smart decisions at design time, then you can end up having to re-write your software–and that can be a very expensive process.

    Not planning to scale is like planning not to succeed. You have to plan to succeed.

  10. Dan: thanks for the feedback, and understand about keeping the expectations reasonable relative to resources. have added your blog, and would love to get a briefing some time to hear more about what you’re doing.

  11. Stephen –
    Thanks for getting back to me (and thanks for adding my blog–I hope you find it of interest!).

    As far as a briefing, I’m giving a 30-minute webinar on Tuesday. It has some high level description, but then I actually get in and write some code (grid-enabling a .NET app as you watch). If you’d like to take 30 minutes, this is a good opportunity.

    Or, if you’d like a more personal, directed briefing, e-mail me at dan (at) digipede (dot) net and we can set something up.

  12. Ah. And, of course, if you are interested in the webinar, you probably want this link:

    You can sign up for the webinar there.

    (note to self: must remember to preview before posting!)

  13. Bloglines is back with a Christmas present: the Service Scales

    Unless I am very much mistaken bloglines has finally nailed the performance issues plaguing the service. Performance was bad enough that Stephen, for example, has been trialing an alternative (Bye For Now Bloglines). Others are too, if the co…

Leave a Reply

Your email address will not be published. Required fields are marked *