“If Amazon/eBay/Google/Yahoo offer platforms – and they will, I believe.” – me
That was Saturday. By Monday, Google App Engine had launched with much fanfare. When I said cloudy days were ahead, I obviously believed that to be the case. I just didn’t know that meant Monday.
Some forecaster I am.
Anyway, you’re hearing it here last (seriously) as usual, but as the highest profile entrant into the cloud computing market since Salesforce.com, Google has earned itself a Q&A. Better late than never and all that.
Q: For those few that haven’t seen the news, can you summarize the announcement for us?
A: Certainly. Monday evening, at their now annual Campfire One developer gathering, Google announced the release and immediate, limited availability of a cloud computing platform called Google App Engine. In essence, it’s a cloud based platform similar and yet distinct from competitive offerings from Amazon, Salesforce.com, and so on. Like those mentioned, it’s what Marc Andreessen might term a Level 3 platform.
Q: How does Google App Engine compare, specifically, to alternative cloud options available to developers?
A: If we think of cloud computing offerings on a spectrum of closed to open, with Salesforce.com’s Force.com on the closed end and Amazon’s EC2/S3 at the open (though the Joyent guys can credibly argue to be more open than Amazon), Google’s App Engine is somewhere near the midpoint. It does impose a proprietary database (Bigtable) on would-be developers, as does Force.com, but it allows for the use of a standard programming language (Python) rather than the proprietary one required of Force.com developers. It’s more constrained than Amazon, of course, which essentially offers developers the freedom to pick and choose from the bare metal of a virtualized platform on up, but Google’s approach does offer some advantages to with its disadvantages.
Q: What kinds of advantages?
A: Transparent scalability, primarily. Relative ease of implementation, secondarily. As Google describes it, App Engine includes “automatic scaling and load balancing,” and this feature along will be worth it to a certain class of application developers. Says my office mate Alex King:
At Crowd Favorite we are web developers, not system admins. We understand how to build something so it can scale, but actually doing the server config and managing them on a day to day basis is not our core competency. This service is designed for us.
To be sure, self hosted offerings or competing services like Amazon’s can be scaled, but they – short of an abstraction layer provided by a third party – require both know how and time. Both of which may be in short supply at many startups.
Google’s service, by contrast, aims to deliver this as part of the offering.
For developers like those at Crowd Favorite, who’d prefer to focus on the actual construction of applications, this is, as Alex put it, a potential game changer.
But it does not come without a cost, of course.
Q: Let’s talk about the costs: what are the downsides of App Engine?
A: Alex Bosworth lists several that are worth considering, but four seem most important to me. Two are relatively minor, two are more substantial.
Q: What are the minor objections?
In ascending order of importance, they are:
- No CRON:
According to what I’ve read, App Engine includes no CRON-like scheduling component. Meaning that, as Alex suggests, developers will – where such scheduled tasks are required by the application – be compelled to maintain external application components to “wake up” App Engine via HTTP requests. - Python Only:
In its initial iteration, App Engine supports Python only, so developers in Java, PHP, Ruby, and so on are left with two choices: port their applications to Python, or host elsewhere. Frankly, I find this constraint as understandable as it is logical, given both Google’s internal Python focus (remember, they employ Guido) and the difficulties of launching a new service. But for some developers, it will mean App Engine is a non-starter.
Aside from the fact that there are obvious work arounds for the above, both could – and likely will be – addressed in some future version of the App Engine platform. So I don’t see either as worth focusing on, at least at the present time.
Q: So what is worth focusing on? What are the major objections>?
A: They’re actually somewhat intertwined, but again in order of ascending importance the big issues from my vantage point are:
- No Export:
At the current time, according to the answers Alex got, there is no simple export – no mysqldump, as it were – available for App Engine. Meaning that anything your application generates and persists on the App Engine platform will have to be extracted programmatically. This is suboptimal, even if we can expect such tools to emerge relatively quickly post-launch. - The Lockin Issue:
The most obvious concern, both for myself and several other commenters. In a nutshell, as Ian writes, it’s wise to be somewhat cautious “[basing] my startup on a infrastructure which can only be provided by a single company.” ARS Technica, in an excellent piece, is more blunt, saying:Although Google App Engine offers some clear advantages and lowers the barriers to entry for startups and independent developers, the potential for lock-in creates risks that could prove more costly in the long-term.
To be sure, App Engine is less closed than Force.com, as the construction language in Python is at least portable. And from the initial reports, it would appear that GQL – the customized version of SQL used to access Bigtable – is not a radical departure from its ancestor, and thus the persistence layer shouldn’t be hugely challenging to port. But a port it would still be. Not to mention the fact that if you’ve been relying on Google to transparently handle scaling for you, on exiting that environment that challenge would loom alongside the task of porting.
In short, while it’s technically possible to leave Google’s platform from what we know at this point, there are a variety of factors in the design that actively disincent this choice. Which may be wholly necessary in order to horizontally scale the service over a substantial user base.
But developers and startups selecting App Engine should do so understanding that any future migration will be a significant challenge. Which it always is, of course, but in my view there are unique considerations for App Engine based offerings.
Q: What could Google do to reduce concerns about the potential for lock-in?
A: Open source as much of the infrastructure as is possible. One person I’ve spoken with, in fact, is of the opinion that this will happen, but I haven’t seen any public confirmation of this one way or another. If any of the Googlers are at liberty to comment on the subject, I’d love to hear what if any plans there are for opening the doors.
If Google is unable or unwilling to open source these assets, it might instead assist in the creation of API compatible back ends, either with resources, documentation, or both. More than a few developers believe that this will happen anyway, with Ivan Krsti saying:
And now, as people begin to implement their own Google datastore API-compatible backends on top of everything from Apache CouchDB to (awkwardly-fitting) relational databases like MySQL and Postgres — which, make no mistake, will happen very soon — we might yet see the long-unfulfilled WinFS dream of rich metadata and tremendously powerful search come to life.
If the initial assessments of the APIs prove correct, it seems likely that this is correct. And while some might remain skeptical of the possibility of reverse engineering the APIs because of the lack of incentive – even if you’re successful, are you likely to more or less efficient than Google – there are other potential use cases for external implementations of the underlying infrastructure.
Think test, staging, and so on.
Q: What about the libraries and applications I rely on? If my application takes advantage of Java libraries and is built on top of MySQL, what are my options?
A: For the libraries, seek Python equivalents. As for MySQL, Google’s persistence mechanism is Bigtable so MySQL is unsupported; your application will have to leverage that instead.
Q: What about Django? Is Python’s answer to Rails supported on App Engine?
A: My understanding is that it is, with some exceptions, which should speed application generation up considerably.
Q: Besides the language and persistence support, are there other relevant APIs available?
A: There are. Besides the standard mail interface, Google apparently is providing an API to its own identity system. This is important from a developer perspective because, if leveraged, it could obviate the need for the creation of an authentication/authorization/signon infrastructure. At the cost of mandating Google account usage or creation, and increasing the platform lock-in, of course. As Tim Bray put it, the API has the potential to turn adopters into sharecroppers.
It’s no surprise that Google has made this available, as it offers them potentially rich new pipelines for account creation, which can both be mined and cross-sold into other Google application usage.
Q: What are the service level restrictions?
A: Applications are currently limited to “500MBs of storage, 200M megacycles of CPU per day, and 10GB bandwidth per day” which Google expects to translate to the ability to serve “around 5 million pageviews per month.” Beyond that, commercial rates will begin to apply, though those rates are not currently available. Though I’ve seen several question these caps, they seem relatively reasonable to me, particularly at launch.
Q: And are there any levels of service promised?
A: If there are, I have not seen them. Nor would I have expected them; Amazon, remember, launched without them, and I think it’s unrealistic to expect a newly launched offering of this nature to promise anything in the way of uptime. Once they can profile the demand from the initial 10,000 users, it will be easier to tackle the capacity planning – and the attendant service level expectations – for a wider scale rollout.
Q: Are SLAs even worth anything in this day and age?
A: Strictly speaking, no. Particularly in instances such as this, where the service is offered at no cost. That said, SLAs do speak to the expectations for uptime that the offering vendor believes are reasonable, and as such are of interest, at least to me. Even if I can’t hold a cloud computing vendor to the terms of an SLA, it is useful to know what metrics they believe they can meet.
Q: Does this make sense for commercial organizations as well as individual developers?
A: It depends, of course. Assuming that the basic restrictions – Python, Bigtable, etc – are not obs tacles, you need to assess the long term probability of a departure from the platform, and decide whether the risks are under threshold for your organization. And I would also want to understand more fully the over-limit costs that can be expected, and whether or not those were predictable over at least a two to three year period.
Q: What about the costs of the service, both now and in future?
A: At the present time, with the restrictions already noted, the service is free. More, the platform at those same levels will remain so. Meaning that, unlike any of the other cloud computing offerings, users can begin building at no cost, assuming only the risk of porting should that become necessary. As many have said before, it’s difficult to compete with free, so in spite of the some of the aforementioned concerns, Google has essentially guaranteed itself a market by eliminating cost as a barrier to entry. Few if any competitors or potential competitors, with the possible exception of Microsoft, would seem willing to try this model.
The question of costs above and beyond the basic levels remains an open one, and until we have numbers it’s difficult to speculate on its positioning relative to offerings from Amazon et al.
Q: Speaking of Amazon, what impact do you expect this to have on them?
A: It’s difficult to say precisely, because the answer is entirely dependent on a question I don’t know that anyone will be able to answer: are there more developers that will put a premium on seamless scalability, or are they outnumbered by the set that values openness and fears lock-in? I certainly cannot claim to know the answer to that question at the present time, and neither, in my view, can anyone else. It will be fun to watch.
In the short term, however, I expect that Amazon will remain a viable and often preferred choice for a very sizable portion of the addressable market. It was inevitable that they would face competition in this market, and like Dave Winer I believe that it will affect Amazon’s direction in a positive way for users. It’s reasonable to predict, for example, that Amazon will redirect AWS resources into the tooling and management of the platform, both which are likely to suffer in comparison to Google’s front ends.
Q: Who are the other winners and losers here?
A: Well, the clear winners are the Python language and Google. App Engine, from the discussions I’ve had over the past few days, has galvanized interest in both, and besides the obvious short term wins observable, there are more interesting long term implications to the offering, including the potential for dramatically streamlining acquisitions.
Q: Wait, what does that mean?
A: Several people have made the point, but ARS puts it most subtly:
This sounds great to small developers with small sites, but what happens when your cool idea takes off and you’ve got thousands or millions of users? You’ll be paying a lot of money to Google each month—with no easy way out. No matter how much your user base and technology is worth, almost no company will be willing to purchase your idea because of the high cost of migrating that code out of Google.
Everyone except Google, of course.
Google’s unimpressive track record with respect to the handling of its acquisitions is well known; Dodgeball, Grand Central, Jaiku, Jotspot and others have all shown signs of stalling under Google’s care, if not outright neglect, and one potential explanation for this is the difficulty of rearchitecting the acquired technologies for Google’s unique environment. But what if the acquired technologies were already running on Google’s infrastructure?
IBM, unlike Google, has a history of integrating its acquisitions into its portfolio efficiently. Part of that ability is the smaller size of the majority of its acquisitions, but the bigger factor, in my view, is the fact that IBM tends to acquire partners. More specifically, partners that are preintegrated with IBM technologies.
It’s possible that App Engine represents, in some fashion, Google’s digestion of that particular lesson.
Q: Back to the winners and losers: who stands to lose from this announcement?
A: All of the larger systems vendors, in my view, including HP, IBM, Microsoft, Sun and so on. For every week or month they delay their own cloud computing strategies, their losing developers that – particularly in the case of Google – they may never get back.
Besides the vendors, languages other than Python may take a hit. Not permanent, particularly as Google has promised to support languages other than Python on App Engine in the future, but I’ve already spoken with a few developers that are either porting or considering porting various assets to Python. For example, here’s Alex on the subject:
While I’m not going to run out and rewrite all of my PHP products in Python, I’ve already mentally re-architected a component of MyFreeBusy that we probably will rewrite in Python so that it can leverage GAE to scale.
Another idea that I’ve had for a service that has been backburnered for a while is probably going to be written in Python instead of PHP so that we can roll it out on GAE.
None of this spells wholesale departures, of course, but it unquestionably means that Python is a far more formidable competitor than it was on Sunday.
Q: What was the controversy around one of the initial sample applications, Huddle Chat?
A: As John Gruber noted, Huddle Chat effectively cloned the 37Signals application Campfire:
HuddleChat is just a feature-for-feature clone of 37signals’s Campfire. The layout is the same, the tabs at the top of the screen are the same, the right-side sidebar listing participants and file uploads is the same. It even copies Campfire’s trick of formatting a message as “code” if it contains literal newline characters.
As he went on to conclude, it’s one thing to borrow ideas, it’s another to precisely replicate an entire interface, as I’ve written before.
Fortunately, the potential for distraction was ended when Google offlined Huddle Chat, leaving the following message in its place:
Hi, a couple of our colleagues wrote Huddle Chat in their spare time as a sample application for other developers to demonstrate the power and flexibility of Google App Engine. We’ve heard some complaints from the developer community about it and because of that we’ve decided to take it down. If you’d like to see more sample applications written on Google App Engine please check out our documentation and our App Gallery.
Apparently not everyone agrees with the decision, but personally I think it’s appropriate.
Q: Considering the above, if you were in a developer’s shoes, would you or would you not consider hosting on App Engine?
A: Consider it? Of course. If your application is written in Python or not yet written, I think you’ be foolish not to consider it. Even I downloaded Mark Pilgrim’s Dive Into Python this afternoon.
But as for the decision process on whether or not to host there, I’d consider my own skillset and language preference, the resources available to me, the risks of lock-in, the need and difficulty of scaling. Depending on how you answer weight the above, App Engine may or may not be a good option for you.
Consider Adrian Holovaty’s EveryBlock. Adrian, one of the creators of the Django framework and a noted advocate of Python, would on a first pass seem to be an ideal candidate for App Engine. While it’s mere speculation, based on the infrastructure of his earlier and similar Chicagocrime.org project, my guess is that Adrian is building EveryBlock off of PostgreSQL but more specifically PostGIS. If that’s true, that portion of the application at least would be ineligible for Google’s cloud platform.
Just a few paragraphs prior, of course, we heard Alex talk about rearchitecting an existing application in just such a fashion.
Whether or Google’s offering is right for you, then, is a complicated question with no easy answers.
Q: With two or maybe three major and several smaller cloud based initiatives now in place, what are the remaining opportunities for late-market entrants?
A: Being first to market in cloud computing is a significant advantage, because inertia is as much a friend to Amazon, Google et al in the cloud as it is to Windows on the desktop.
That said, all of the current offerings have limitations that throttle their usage. Many of which are related to the lack of open standards. Apart from the mostly standard Python implementation, App Engine is decidedly non-standard. Even Amazon Machine Images are uniquely packaged to that platform.
My general recommendation for potential cloud computing players is to seek opportunities to conflate cloud computing with virtualization. It should, but is not, be straightforward for potential cloud computing customers to package running instances into snapshots that can be seamlessly deployed on competing cloud computing environments. The technical capabilities are widely available and utilized, but not – typically – within the clouds.
If I were a vendor looking for an opportunity to differentiate, then, that’s where I would start.
Q: What’s the most interesting aspect of the App Engine announcement that you haven’t seen discussed?
A: That no one has so much as mentioned operating systems when discussing it.
Disclosure: IBM, Microsoft and Sun are RedMonk customers, while Amazon, Google, and HP are not. Joyent is not a RedMonk customer, but hosts a personal site of mine gratis.
