The original idea behind J2EE was as simple as it was compelling. “Write once, run anywhere” promised customers a layer of abstraction which would provide independence from underlying operating system and hardware layers, and thus, in theory, vendor substitutability. In practice, “write once, run anywhere” was as aspirational as not, with providers of J2EE middleware constantly in search of features that might provide the customer lock-in the market had trained them to crave. Aspirational or no, however, customers bought into the vision, as companies like BEA rode the middleware market to $8.5B valuations.

Oddly enough, however, multiple runtime PaaS platform providers like Red Hat (OpenShift) and VMware (Cloud Foundry) appear to be discinclined to borrow from the middleware playbook. While “write once, run anywhere” mantra may be a tired cliche, the similar functional goals would seem to offer the opportunity to more easily position PaaS platforms within enterprises. Overall, the idea behind PaaS is the same as it once was for J2EE: to containerize applications, thereby making them portable across different environments, regardless of the underlying infrastructure. Neither Cloud Foundry nor OpenShift seems eager for the comparison, however.

OpenShift.com, for example, says that users can “enjoy support for a broad choice of languages, frameworks, middleware, datasources, and developer tools” – the obvious implication being that OpenShift is not, itself, middleware. CloudFoundry.com even more aggressively disavows any relationship between itself and the term. From its FAQ page:

Cloud Foundry allows developers to focus on applications, not machines or middleware. Traditional application deployments require developers to configure and patch systems, maintain middleware and worry about network topologies. Cloud Foundry allows you to focus on your application, not infrastructure, and deploy and scale applications in seconds.

In spite of the functional similarities, then, neither Cloud Foundry nor OpenShift is positioning itself as middleware. To some extent, this may be explained by product portfolios as both Red Hat (JBoss) and VMware (Spring) have existing middleware lines of business, and terming their respective PaaS offerings as middleware would complicate marketing efforts. It is not clear, however, that an attempt to persuade customers to see PaaS as something other than middleware will be successful. Nor that this is the correct approach. Our old colleague Michael Cote’s “burger model” – which characterizes PaaS as middleware – has proven quite popular in various talks and webinars, because it’s easily understood. And technologies that are well understood are easier to sell.

Ultimately, if it looks like middleware and acts like middleware, it probably is middleware – whatever the vendors may assert. PaaS, in other words, is the new “write once, run anywhere.”

Most of the commentary regarding the donation of the CloudStack assets to the Apache Software Foundation by Citrix yesterday has centered around the landscape implications. This is understandable, because CloudStack’s break from OpenStack has an impact on multiple communities.

Given the stakes involved with the cloud market, the growing number of market entrants is no surprise. Incumbents like Microsoft and VMware cannot afford to be locked out of the cloud; what’s more, each vendor’s leaership understands the financial opportunities associated with owning the platform. Amazon, which is for all practical purposes the Microsoft or VMware of the cloud, can and will leverage their position to simultaneously undermine competitors and preemptively defend lines of attack. If this means creating ancillary private cloud markets and leaving money on the table in the process – as they may with Eucalyptus and now CloudStack – so be it. Everyone who’s not Amazon or aligned with them, meanwhile, from Joyent to Rackspace, is intent on ensuring that Amazon’s tenure as the Microsoft of the cloud is as short as possible. If that means open sourcing what would have been considered core software in the process – as with Rackspace and OpenStack – so be it.

That landscape, as convoluted as it appears, is fairly well understood within the industry. While everyone wants to predict outcomes on project and API futures, the fact is that it’s too early in most cases to project accurately. The CloudStack transaction, however, does make obvious certain facts regarding the licensing mechanisms and governance models employed by the various projects.

Permissive Licensing Continues to Make Gains

Since 2009 we’ve been predicting (and observing) gains for permissive licensing at the expense of copyleft or reciprocal alternatives. Reciprocal licenses, which require that any changes to a licensed asset that are distributed be made available under the exact same terms, have in the past been employed to disincent forks and to create artificial purchase triggers (e.g. dual licensing). As forks have become something to encourage rather than fight, however, one justification for the usage of copyleft licenses has faded.

Faded to the point that permissive licenses are increasingly seen as a license of choice for maximizing participation and community size. It’s not true that copyleft licenses are unable to form large communities; Linux and MySQL are two of the largest open source communities in existence, and both assets are reciprocally licensed. But the case can be made that this will in future be perceived as anachronistic behavior.

Commerical entities in particular favor permissive licenses like the Apache Software Foundation’s , because they impose very little overhead. As Cloudera CEO Mike Olson – whose first business was built around reciprocal licensing, but whose Hadoop business is built around Apache – says of permissive licenses, “There’s very little legal complexity for people to deal with.” Permissively licensed assets can be repackaged and sold as closed source software, for example, per the terms of the license itself. For projects, then, that wish to encourage the participation and engagement of commerical vendors, the permissive license can be a good, differentiating, choice.

Citrix is being less than explicit about this, but it is probable that as they continue to build an ecosystem around CloudStack, one of the “features” they’ll cite is the permissive license which allows would be players to leverage the code however they see fit. Contrast this, for example, with Eucalyptus, which is at present governed by the reciprocal GPLv3 license.

The same license, in fact, that CloudStack left behind in favor of the ASF’s more permissive license.

The Asymmetry of Permissive vs Reciprocal

Permissive licensing isn’t an unequivocal win for the CloudStack project, however. Underreported is the licensing asymmetry created by Citrix’s relicensing of the CloudStack assets. Because of the licensing mechanisms involved, the Eucalyptus project will now be able to incorporate code from CloudStack – or OpenStack, for that matter – at will. Technical innovations within Eucalyptus, meanwhile, are protected from usage by permissively licensed cloud stacks. Code sharing, in this case, is unidirectional towards Eucalyptus. CloudStack also forfeits similar protections from the OpenStack project; anything that OpenStack wishes to consume from CloudStack, they will now be able to, per the terms of the new licensing scheme.

Historically, this has not been a particularly important dynamic. MySQL, for example, has no major history of incorporating code from the more permissively licensed PostgreSQL. But with the cloud stack projects’ broader scope and mandate, it is not out of the realm of possibility to believe that Eucalyptus may opportunistically incorporate features or components from CloudStack, OpenStack or both.

You Won’t Get Fired for Using Apache

In the wake of GitHub’s meteoric ascension, there have been many questions about the role of open source foundations like Apache or Eclipse today. There are those who argue, in fact, that they have outlived their original purpose; see, for example, Mikeal Rogers’ “Apache Considered Harmful.” But while it is certain that foundations will need to adapt to changing needs, there are many reasons yet to justify their existence [coverage]. One of which is branding.

Citrix was very careful to put the Apache Software Foundation front and center in their announcement. This does two things. First, it allows them to benefit from the halo effect of the Apache brand and the goodwill of becoming a sponsor. Second, it differentiates them from two of the more visible alternative open source cloud stacks. Eucalyptus is primarily a single vendor initiative, much as MySQL once was. OpenStack is developed by a broader community of participants, and is being transitioned to a foundation. But that process has not been without its growing pains, with one of the co-founders questioning the governance structure.

Contrast that with CloudStack and the ASF. The latter is a known quantity, and vendors like IBM have historically advantaged Apache at the expense of independent foundations (e.g. their OpenOffice.org participation). What the ultimate impact of the Apache brand will be on CloudStack’s visibility and traction remains to be seen, but it’s undeniable that the Apache brand is being positioned as a feature for the project.

Disclosure: The ASF, Citrix, Eclipse Foundation, Eucalyptus, GitHub, Joyent, Microsoft, and VMware are clients. Rackspace has been a client in the past. Amazon is not a RedMonk client.

Born as a research project in the computer science department at UCSB, Eucalyptus the company was founded in January of 2009. Originally intended to replicate a subset of the Amazon cloud’s featureset in software that could be run locally, one of the project’s primary differentiators was its compatibility with the Amazon API. Importantly, however, this support was unofficial: Amazon neither supported nor legally blessed this feature. Which meant that its appeal was throttled by the uncertainty of Eucalyptus’ legal footing. More than one large vendor has privately characterized the Amazon API as a “non-starter” because their legal departments could not be assured of Amazon’s intent with respect to the intellectual property issues involved.

Meanwhile, a year and a half after Eucalyptus was commercialized, NASA and Rackspace jointly launched their own cloud project, OpenStack. While the initial versions were closer to a set of tools than the stack the name implies, OpenStack had an impressive partner roster at launch. And industry skepticism of the level of commitment of those partners has been offset with sustained momentum. Due in part to its more permissive licensing – OpenStack is Apache licensed, while Eucalyptus is reciprocally licensed under version 3 of the GPL – the NASA/Rackspace effort has enjoyed wide corporate interest and support. From AT&T to Dell to HP, OpenStack’s traction has been such that even skeptics like IBM and Red Hat have lately appeared to be moving towards acceptance of the project.

For all of OpenStack’s momentum, however, Amazon remains the dominant player in public cloud, with one researcher estimating its datacenter size at just shy of a half a million servers. Amazon itself has validated external estimates of its growth trajectory, saying (PDF) “Every day Amazon Web Services adds enough new capacity to support all of Amazon.com’s global infrastructure through the company’s first 5 years, when it was a $2.76 billion annual revenue enterprise.”

The question facing Amazon was if, or perhaps when, interest in on premise functionality would become sufficient to incent its participation – whether through build or a partnership – in private cloud solutions. The answer to that question appears to be this year. In January, the retailer turned technology giant announced the availability of the Amazon Storage Gateway, a virtual machine run locally that bridges data to Amazon’s storage service, S3. And then this morning, Amazon announced a partnership with Eucalyptus. As Om Malik put it, “On paper it looks like one of those strategic agreements that large companies sign-up with small startups,” but the announcement belies the larger significance. Amazon is, for the first time, playing the intellectual property trump card it has been holding in reserve.

By strategically and selectively removing the uncertainty regarding its APIs, Amazon gains literally overnight a credible private cloud offering, minimizing that as an angle of attack for competitors who might otherwise attempt to sell against Amazon by emphasizing its public cloud-only technology story. Nor does it have to deviate from its public cloud orientation by creating a more traditional software organization. This deal instead effectively outsources that to Eucalyptus.

Eucalyptus, for its part, may now realize the full potential of its differentiated access to Amazon public clouds. As Amazon’s only approved platform, it can expect its attach rate within organizations consuming Amazon cloud resources to improve substantially. While the deal is apparently not exclusive, OpenStack is not likely to either ask for or receive the same blessing from Amazon that Eucalyptus received. Assuming that the API licensing would survive a transaction, Eucalyptus has with this announcement substantially increased its valuation. The success of one organization is the success of the other; wider Eucalyptus adoption poses no risk to Amazon’s growth, while its success would push Amazon’s APIs closer to becoming the de facto standards of the public cloud.

From a landscape perspective, this cements the perception that it’s Amazon and Eucalyptus versus OpenStack and everyone who’s not Amazon, with the notable exceptions of Joyent, Microsoft and VMware, each of whom owns and sells their own cloud stack. In general, betters would be best advised to take the field over any single vendor, but the cloud market is an exception: Amazon is that dominant. By virtue of being first to market as well as executing consistently for six years, Amazon made itself into the proverbial 600 pound gorilla in one of the most strategic markets in existence. If you’re going to pick sides, as Eucalyptus did more emphatically this morning, that’s not a bad choice to make.

Disclosure: Dell, Eucalyptus, Joyent, HP, IBM, Microsoft, Red Hat and VMware are RedMonk customers. Amazon is not a RedMonk customer.

“In San Francisco cafes and bars, even on the street, I overhear people talking about their startup ideas, business plans, and goals. And there are tons of incubators, Angels, wannabe Angels, VC firms, making investments in startups.”
- “Silicon Valley’s dirty little secret: The ‘Startup Boom’ is a disguised jobs fair for big corporations,” Tom Foremski: February, 2012

The conventional wisdom in this industry is that we are amongst a startup boom. Besides the piece above, there’s Bloomberg: “New York Is Having ‘Incredible’ Startup Boom, Hippeau Says“; Forbes: “How Cloud Computing is Fueling the Next Startup Boom“; GigaOm: “Why the big data startup boom will likely be short-lived“; the Wall Street Journal: “Veteran Investor Defends Start-Up Boom,” and on.

One of the things missing from these articles, however, is an attempt to examine the question quantitatively. Is there, in fact, a boom, and if so, what does it look like? Ever since running across the Kauffman Foundation’s report “Starting Smaller; Staying Smaller: America’s Slow Leak in Job Creation” via Andrew McAfee and Erik Brynjolfsson’s “Race Against the Machine,” I’ve been curious about how the technology industry’s job creation compared to the rate overall. Working backwards from the same source of data that the Kauffman Foundation used – the US Census – it was simple enough to collect basic job creation data for the US. What was missing was insight into technology company starts. Fortunately, AppFog’s Chris Tacy had sourced relevant data from the National Venture Capital Association and the UNH Center for Venture Research. Plotting overall industry business creation as measured by the census against the volume of seed and venture deals, normalizing the data in the process, this is what we find (Census data is only available for 2009 and earlier).

The trendlines are more gradual than “startup boom” rhetoric might lead one to believe, but they do support the assertion that business creation in the technology sector exceeds that of the wider job market. There are two notable spikes in venture activity; the first, from 2002-2004, roughly coincides with the dot com boom. The second, beginning in 2006, is likely attributable to something else entirely.

Over on the AppFog blog, Tacy notes an odd phenomenon: while the normalized volume and deal size metrics initially track closely, beginning in early 2006 they are effectively decoupled as the volume of deals spikes while the average deal size falls dramatically. A replication of his dataset is below. The question is what caused this phenomenon.

Tacy attributes this to the rise of Amazon Web Services and the subsequent rise of the cloud market. And it’s difficult to argue that S3, launched March 2006, and EC2, launched in August of the same year – not to mention the market they helped to create – haven’t had a profound impact on the costs associated with business formation, and thus, the attendant risk. As Flip Kromer put it: “EC2 means anyone with a $10 bill can rent a 10-machine cluster with 1TB of distributed storage for 8 hours.” The cost model is inarguably more disruptive than the underlying technologies that power the cloud.

It is also likely, however, that the rise of seed-stage startup funding companies like Y Combinator (founded March 2005) were a catalyst in the above equation. It’s difficult to envision the comparatively small funding levels having had the same impact in the absence of cloud platforms that minimize upfront capital expenses. But with 316 companies funded from 2005 through 2011, Y Combinator would account for approximately 13% of the total venture deals in that span according to the data above. Factor in the funding firms such as Tech Stars that have followed, and the impact of Y Combinator is undeniable.

However the trends are accounted for, the takeaways are clear. Business creation in the technology sector is outperforming the wider market, while the capital required for formation – and thus the risk attached – is in clear decline. Both of which bode well for an economy desperate for an increase in employment opportunities.

“This paper described Dynamo, a highly available and scalable data store, used for storing state of a number of core services of Amazon.com’s e-commerce platform. Dynamo has provided the desired levels of availability and performance and has been successful in handling server failures, data center failures and network partitions. Dynamo is incrementally scalable and allows service owners to scale up and down based on their current request load.

Dynamo allows service owners to customize their storage system to meet their desired performance, durability and consistency SLAs by allowing them to tune the parameters N, R, and W.”
- “Dynamo: Amazon’s Highly Available Key-value Store [PDF],” Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall and Werner Vogels

In October 2007, Amazon published a paper describing an internal data store called Dynamo. Incorporating ideas from both the database and key-value store worlds, the paper served as the inspiration for a number of open source projects, Cassandra and Riak being perhaps the most visible of the implementations. Until yesterday, these and other derivative projects were the only available Dynamo implementations available to the public, because Amazon did not expose the internally developed database as an external service. With Wednesday’s launch of Amazon DynamoDB, however, that is no longer true. Customers now are able to add Amazon to their potential list of NoSQL suppliers, although to be fair they’ve technically been in market with SimpleDB previously.

The following are some points of consideration regarding the release, its impact on the market and likely customer questions.

AWS versus Hosted

The most obvious advantage of DynamoDB versus its current market competition is the fact that it’s already in the cloud, managed and offering consolidated billing for AWS customers. Requiring minimal setup and configuration versus native tooling, a subset of the addressable market is likely to be of a similar mindset to this commenter on the DataStax blog:

“Cassandra’s tech is superior, as far as I can tell. But we’ll probably be using DynamoDB until there is an equivalent managed host service for Cassandra. Moving to Cassandra is simply too expensive right now.

…

All those are clearly better served by a service like DynamoDB than trying to run their own Cassandra clusters unless they happen to be very proficient in Cassandra administration and want to dedicate precious human resources to administration. That takes a lot of the benefits of “cloud” away from small and mid-sized companies where cost and management are the limiting factors.”

For many, outsourcing the installation, configuration and ongoing management of a data infrastructure is a major attraction, one that easily offsets a reduced featureset. Like Platform-as-a-Service (PaaS) offerings, DynamoDB offers time to market and theoretical cost advantages when required capital expense and resource loading are factored in.

Like the initial wave of PaaS platforms, however, DynamoDB is available only through a single provider. Unlike Amazon’s RDS, which is essentially compatible with MySQL, DynamoDB users will be unable to migrate off of the service seamlessly. The featureset can be replicated using externally available code – via those projects that were originally inspired by DynamoDB, for example – but you cannot at this time download, install and run DynamoDB locally.

It’s true that the practical implications of this lack of availability are uncertain. NetFlix’ Adrian Cockroft, for example, asserts that migration between NoSQL stores is less problematic than between equivalent relational alternatives, because of the lower complexity of the storage, saying “it doesn’t take a year to move between NoSQL, takes a week or so.” It remains true, however, that there are customers that postpone upgrades to newer versions of the same database because of the complexity involved. And that’s without considering the skills question. Given the uncertainty involved, then, it seems fair to conclude that the proprietary nature of DynamoDB and the potential switching costs will be – at least in some contexts – a barrier to entry.

The question for users is then similar to that facing would be adopters of first generation PaaS solutions: is the featureset sufficient to compel the jeopardizing of later substitutability? Amazon clearly believes that it is, its competitors less so. EMC’s Mark Chmarny, additionally, notes that Amazon may be advantaging adoption at the expense of migration in its pricing model.

Competition

DynamoDB clearly has the attention of competitive projects. Basho – the primary authors of Riak – welcomed DynamoDB in this post while pointing out the primary limitation, and DataStax wasted little time spinning up a favorable comparison table. One interesting aside: the Hacker News discussion of the launch mentioned Riak 23 times to Cassandra’s three.

Basho and Datastax are right to be concerned, because the combination of Amazon’s increasingly powerful branding and the managed nature of the product make it formidable competition indeed. The question facing both Amazon and competitors is to what extent substitutability matters within the database space. Proprietary databases have had a role in throttling the adoption of PaaS services like Force.com and Google App Engine in the past, but we have very few market examples of standalone, proprietary Database-as-a-Service (DaaS) offerings from which to forecast. Will DaaS or more properly NoSQL-as-a-Service be amenable to single vendor products or will they advantage, as they have in the PaaS space, standardized platforms that permit vendor choice?

The answer to that is unclear at present, but in the meantime expect Amazon to highlight the ease of adoption and vendors like Basho and DataStax to emphasize the potential difficulties in exiting, while aggressively exploring deeper cloud partnerships.

NoSQL Significance

It’s being argued in some quarters that DynamoDB is the final, necessary validation of the NoSQL market. I do not subscribe to this viewpoint. By our metrics, the relevance of distinctly non-relational datastores has been apparent for some years now. Hadoop’s recent commercial surge alone should have been sufficient to convince even the most skeptical relational orthodoxies that traditional databases will be complemented or in limited circumstances replaced by non-relational alternatives in a growing number of enterprises.

Throughput Reservation

Perhaps the most compelling new feature of Amazon’s new offering isn’t, technically speaking, a feature. Functionally, the product is (yet) another implementation of the ideas in the Dynamo paper; Alex Popescu has comprehensive notes on the feature list. Receiving the most attention aren’t technical capabilities like range queries but rather the concept of provisioned throughput, levels which can be dynamically adjusted up or down.

This type of atomic service level provisioning is both differentiating and compelling for certain customer types. Promising single digit latency at a selected throughput level with zero customer effort required is likely to be attractive for customers that require – or think they require – a particular service level. And by requiring customers to manually determine their required provisioning level, Amazon stands to benefit from customer overprovisioning; customers will feel pain if they’re under-provisioned and react, but conversely may fail to observe that they’re over. Much like mobile carriers, Amazon wins in both scenarios.

Timing

With DynamoDB having been extant in some form since at least 2007, one logical question is: why now? Amazon did not detail their intent with respect to timing when they prebriefed us last week, but their track record demonstrates a willingness to be first to market balanced with an understanding of timing.

In 2006, Amazon launched EC2 and S3, effectively creating the cloud market. This entrance, however, was built in part from the success of the Software-as-a-Service (SaaS) market that preceded it; Salesforce, remember, went public in 2004. With enterprises now acclimated to renting software via the network, the market could be considered primed for similar consumption models oriented around hardware and storage.

Three years later after the debut of EC2 and S3, and one year after MySQL had achieved ubiquity sufficient to realize a billion dollar valuation from Sun [coverage], Amazon launched the first cloud based MySQL-as-a-Service offering [coverage]. That same year, the first year that Hadoop was mainstream enough to justify its own HadoopWorld conference, Amazon launched Elastic MapReduce.

The pattern is clear: Amazon is unafraid to create a market, but attempts to temper the introductions with market readiness. Logic suggests that the same tactic is at work here.

NoSQL has, as a category, crossed the chasm from interesting science project to alternative data persistence mechanism. But while NoSQL tools like Cassandra and Riak are available in managed form via providers like Joyent and Heroku, DynamoDB is, in Popescu’s words: “the first managed NoSQL databases that auto-shards.”

It is also possible that SSD pricing contributed directly to the launch timing, with pricing for the drive type down to levels where the economics of a low cost shared service finally make sense.

SSDs

One underdiscussed aspect to the Dynamo launch is the underlying physical infrastructure, which consists solely of SSDs. This is likely one of the major contributing factors to the performance of the system, and in some cases will be another incentive to use Amazon’s platform as many traditional datacenters will not have equivalent SSD hardware available to them.

The Net

While discussion of the DynamoDB offering will necessarily focus on functional differentiation between it and competitive projects, it is likely that initial adoption and uptake will be primarily a function of attitudes regarding lock-in. For customers that want to run the same NoSQL store on premise and in the cloud, DynamoDB will be a poor fit. Those who are optimizing for convenience and cost predictability, however, may well prefer Amazon’s offering.

Amazon would clearly prefer the latter outcome, but both are likely acceptable. Amazon’s history is built on releasing products early and often, adjusting both offerings and pricing based on adoption and usage.

In any event, this is a notable launch and one that will continue to drive competition on and off the cloud in the months ahead.

Disclosure: Basho is a RedMonk client, while Amazon and DataStax are not.

The cost of delaying my 2012 predictions is that one has already come to pass. Nginx – the web server now powering all of the redmonk.com properties – passed IIS according a January 4 Netcraft release. Because the quantitative data available to us has indidicated surging interest in the alternative web server – the logical result of which was a commercial response – we’ve been expecting something like this. But of course we can’t count this as a prediction any longer because it’s January 13th.

Here instead are a few things that have not yet come to pass, but will, I believe, in the year ahead. These predictions are informed by historical context and built off my research, quantitative data that’s available to me externally or via RedMonk Analytics, and the conversations I’ve had over the past twelve months, both digital and otherwise. They cover a wide range of subjects because we at RedMonk do.

For context, my 2010 predictions graded out as 66% accurate while 2011′s were 82% correct.

With that, the 2012 predictions.

Data & The Last Mile

It is not technically correct to assert that large scale data infrastructure is a solved problem. Decades of innovation remain, as the Cambrian explosion of projects demonstrate. It is nevertheless true that relative to the user interface, data storage and manipulation is a solved problem. Since the original creation of Hadoop in 2006, for example, we have seen multiple user interfaces applied: connectors (e.g. R), standard MapReduce, scripting (e.g. Jaql/Pig), SQL (e.g. Hive), spreadsheets (e.g. BigSheets), client tooling (e.g. Karmasphere). Each has its strengths, none bridges the last mile: putting the power of Big Data in the hands of ordinary users.

Which is perhaps unsurprising; even the mature relational database world uses abstractions of varying levels of complexity to interface with business users. But with data driven decision making on the rise, premiums are being placed on tooling which can expose in sensible fashion data to those without degrees in computer science. Hence, the elevated visibility of startups such as Metamarkets, who excite data scientists with tools like Druid but whose valuation may ultimately depend on its last mile expertise.

At this point in time, whatever my preferred model for data storage and whatever the type, there will be greater than one credible option for a data engine. The same cannot be said for presentation. Which would be less problematic if the market for Big Data talent were not so desperate; outsourcing to shops like Mu Sigma will be an option in some quarters, but comes with its own inefficiences and risks, not to mention per inquiry premiums.

This, then, will be an area of focus in 2012, for both innovation (look for assisted anomaly and correlation identification, a la Google Correlate) and M&A.

Desktop Importance Declines

The most interesting characteristic of the forthcoming Windows 8 release isn’t the technology, which is curious because it’s revolutionary from a Microsoft standpoint. From the support for ARM to the addition of the Windows Store to the ability to author in JavaScript and HTML5, there is much to digest. Instead, the single most defining characteristic of the pending launch is apathy.

Overall inquiries and discussion of the platform demonstrate curiosity but limited interest; the visibility of the once dominant Windows platform is secondary to mobile platforms like Android and iOS.

While this is not a function of any specific or general design failures on the part of Microsoft – indeed, the platform is incorporating important changes while making itself more developer accessible – it is symptomatic of a broader and more difficult to attack problem: the declining role of the desktop.

The desktop is simply not as important as it once was. Mobile usage is eroding the central role PC’s once played; while they are still the dominant form of computing, the trendline is declining and there is no reason to expect it to invert. It’s been suggested that mobile computing in general is additive; that it’s being used to extend the usage of computing to areas where PCs were not employed, and is thus non-competitive. But our data as well as Asymco‘s indicates that, at least in part, mobile usage is coming at the expense of traditional platforms. General search volume data, as we’ve seen, validates this assertion.

There are two implications here. Most obviously, Microsoft’s ability to generate interest in and thus leverage for its flagship operating system is jeopardized. Worldwide developer populations are not necessarily zero sum as skills overlap, but they tend to be rivalrous; an Android or iOS developer is often a lost potential Windows developer – experiments like BlueStacks aside. We can therefore expect Microsoft to have to expend more effort to attract fewer developers to their platform, a negative cycle which becomes cyclical. Second, as the desktop’s primacy abates, we can expect to see greater competition in the marketplace. As enterprises become by necessity more heterogeneous, incorporating Android and iOS devices, the costs of supporting second operating systems drifts towards marginal, which means that forecasts of greater Apple penetration become more probable.

Developer Shortages

It’s become axiomatic that industry hiring is all demand and short supply, and none of our clients expect any relief in the year ahead. Nor will they receive it. Shortages for in demand skillsets will continue over the next twelve months, advantaging entities that are either geographically positioned to leverage markets less competitive than the Valley or with the logistical ability to incorporate remote hires.

That said, we will in 2012 see the first steps towards a more rational market, through a combination of cultural shift and educational model innovation that will increase supply. Regarding the former, it’s no secret that technology has had a profound impact on the erosion of middle class jobs. In Race Against the Machine, MIT Professors Andrew McAfee and Erik Brynjolfsson document the role that rapid innovation has had on jobs:

Digital technologies change rapidly, but organizations and skills aren’t keeping pace. As a result, millions of people are being left behind. Their income and jobs are being destroyed, leaving them worse off in absolute purchasing power than before the digital revolution.

Even skill industries are not immune. From John Markoff’s New York Times piece, “Armies of Expensive Lawyers, Replaced by Cheaper Software“:

“From a legal staffing viewpoint, it means that a lot of people who used to be allocated to conduct document review are no longer able to be billed out,” said Bill Herr, who as a lawyer at a major chemical company used to muster auditoriums of lawyers to read documents for weeks on end. “People get bored, people get headaches. Computers don’t.”

While Brynjolfsson and McAfee are ultimately optimistic about the prospects of technical progress as they relate to employment, the outcome is far from certain.

What is becoming clear, however, is that unemployment rates that have been north of 8% in the US since February of 2009 are driving people into industries that are desperate for help. For some, this means oil & gas employment in traditionally underpopulated environments like North Dakota. For others, however, technology – long an enemy – is becoming a refuge.

We’re seeing a spike in inquiries about transitioning to technology careers. Lawyers, management consultants, teachers and others are seeking – and often finding – homes for themselves within the technology sector. Some are self-taught or trained on the job, others merely apply existing skills in new contexts, but both represent a potential cultural shift. Which begs the question: could technology be the next major middle class employment sector?

For that to happen, the education system needs to improve, because even an industry which has been one of the few economic bright spots of the last decade can only absorb so many unskilled workers without slowing. This is the real significance of applications like Code Academy or programs like Harvard’s free CSCI E-52, MITx or Stanford Engineering Everywhere: they are one potential solution to the perpetual shortage of talent. For all of the limitations of distance learning, the scale means that some subset of motivated students will become productive developers, and by extension, contributors to the larger economy.

This is a long term process, so obvious progress within 2012 will be minimal, and talent shortages will continue. But we will in the next twelve months begin to see distance trained students hired at scale, and this will be one of the first steps towards lower talent costs as well as, possibly, the restoration of middle class employment opportunities.

Monitoring as a Service

We are not oriented around category definitions at RedMonk; we prefer market driven names to those conceived and marketed by the analyst industry. That said, it seems clear that the time of Monitoring-as-a-Service (MaaS) is at hand. New Relic’s growth led to a $15M round in November, Boundary took $4M a year ago this month, Monktoberfest speaker Theo Schlossnagle’s Circonus has been in market for over a year, and virtually every vendor that we speak with today is adding monitoring and management facilities, from 10gen’s MMS to Cloudera’s Cloudera Manager.

The proliferation of these services is a direct response to the increasingly heterogeneous nature of application architecture and the reality that the substrate is frequently network based, rather than local. Given accelerating rather than declining consumption of network resources, we predict a strong increase in interest and adoption of MaaS tools. Much as I don’t care for the term itself.

Intelligent usage of generated telemetry – which we’ll come back to – will further cement adoption, delivering previously unseen value.

Open Source and the Paradox of Choice

Gartner in March of last year asserted that open source had hit a tipping point, saying:

“Mainstream adopters of IT solutions across a widening array of market segments are rapidly gaining confidence in the use of open source software.”

We concur, although we would argue that the tipping point actually occured ten years or more prior. The Apache web server and MySQL were originally written in 1995. In 1999, we saw the public offering of Red Hat and the creation by IBM – as mainstream a technology brand as there is in the enterprise – of the Linux Technology Center. Firefox was first released in 2003. None of these reached their relative levels of popularity in the past twelve months; they have instead been the de facto infrastructure for the better part of the last decade.

Regardless of when one asserts that open source crossed the chasm, however, it remains that it is a model whose popularity is increasing over time. As understanding of the benefits increases and concerns about the risks abate, more organizations are not only consuming open source but contributing to it. Evidence suggests, in fact, that perceptions of the value of software are in decline – we’ll come back to that too, and that the end result of this is that more proprietary code is being released as open source software.

Widely perceived as a net benefit, however, the influx of new projects does present problems for would be adopters. Specifically, the paradox of choice implies that developers will increasingly be forced to select from a growing sea of projects which may or may not be suitable for their needs. And while the nature of open source guarantees developers the ability to apply this code to their projects without restriction or commercial engagement, this is a process with a limited ability to scale. Consider the NoSQL space, as an example. Presuming for the sake of argument that the developers in question understand the different categories of database – key value stores, document databases, columnar databases, MapReduce engines, graph databases and so on – well enough to understand their high level needs, there are at least two and sometimes as many as half a dozen credible options to consider.

This paradox of choice, or too much of a good thing, will become more problematic over time rather than less as contributions will continue to rise. The net impact is likely to be increased commercial opportunities around selection, and therefore attention to vendors like Black Duck, Open Logic, Palamida and Sonatype.

PaaS: The New Standard

It has been evident for some time that runtime fragmentation – an aggressive diversification of programming languages and frameworks, specifically – will change the development landscape. The market failure of the first generation PaaS providers, in fact, was primarily a function of their over-prescriptive natures. The benefits to outsourcing management and scale were obsoleted by the constraints; Java shops were never likely to rewrite their application stack in Python or Ruby strictly to benefit from a platform. Which is why virtually every relevant PaaS provider today offers a choice of runtimes, so as to maximize their addressable market.

But in a fragmented world, what might emerge as a standard? From a developers’ perspective, the standard is most often the framework they’re deploying to, whether that’s Django, Node.js, Lift, Play, Rails, Spring, the Zend Framework or another. From a vendor perspective, however, the new standard is likely to be one level of abstraction up from individual language frameworks: the platform itself. Certainly this is VMware’s opinion, as they are in Maritz’ words trying to construct “the 21st-century equivalent of Linux” – i.e. the substrate that everything else is built on top of.

In 2012, this will become more apparent. PaaS platforms will emerge as the new standard from a runtime and deployment perspective, the middleware target for a new generation of application architectures.

Service Proliferation

With the inevitable adoption of multiple third party services – varying cloud resources, multiple, possibly overlapping, management and monitoring services and so on – will come challenges in making sense of the whole. Overall, instrumentation and visibility on a per service level is improved, but aggregating these views into a cohesive picture of overall architectural health and performance is likely to be highly problematic. Not least because the services themselves may present conflicting information and data. Google Analytics and New Relic, for example, are frequently at odds over load times and other delivery related performance metrics. Introduce in to that mix services like Boundary or CloudWatch and the picture becomes that much more complex. Connecting their data back to underlying log management and monitoring solutions such as 10gen’s MMS or Splunk is more complicated still.

The challenges of service intregration will create commercial opportunities for aggregating services which consume individual performance streams, normalize it and present customers with a consolidated single picture of their network performance. Commercial solutions will not fully deliver on this vision in 2012, but we will see progress and announcements in this direction.

Telemetry Usage

Five years ago, we began publicly discussing revenue models based around what we termed telemetry, or product generated datastreams. The context was providing open source commercial vendors with a viable economic model that better aligned customer and vendor needs, but the approach is by no means limited to that category: Software-as-a-Service vendors, as an example, are well positioned to leverage the data because they maintain the infrastructure. In 2011, we finally began seeing vendors besides Spiceworks take the first steps towards incorporating data based revenue models. For products like Sonatype Insight [coverage], data is not a byproduct, but the product.

In 2012, this trend will accelerate as necessary monitoring capabilities are added to product portfolios and industry understanding and acceptance of the model overcomes conservative privacy concerns. Many more vendors will begin to realize that like New Relic, which observed a decline in commercial application server usage, their accumulated data is full of insights on customer behaviors and wider market trends both.

Value of Software Will Continue to Decline

Capital markets have not, traditionally, been overly fond of software firms, perhaps because comparatively few of them eclipse annual revenue marks of a billion dollars – less than twenty, by Forbes‘ count. Microsoft’s share price has languished for over a decade in spite of having not one but two licenses to print money. The mean age of the PwC’s Top 20 software firms by revenue is 47 years; a fact which cannot be encouraging to startups.

Higher valuations instead are being awarded to entities that employ software to some end, rather than attempting to realize revenue from it directly. Startups today realize this, and the value of software in their models has commensurately been adjusted downward. Tom Preston-Werner, for example, describes the GitHub philosophy as “open source (almost) everything.” Facebook, LinkedIn, Rackspace, Twitter and others exhibit a similar lack of protectiveness regarding their software assets, all having open sourced core components of their software infrastructure that would have been even five years ago fiercely guarded.

This is becoming the expectation rather than the exception because it is nothing more or less than an intelligent business strategy. Businesses can and will keep private assets they believe represent competitive differentiation, but it will be increasingly apparent that less and less software is actually differentiating. As a result, 2012 will see even less emphasis on the value of software and more on what the software can be used to achieve.

Bonus: Facebook’s Most Important Feature

In 2012 will be Timeline. Mark it down.

Disclosure: Black Duck, Cloudera, GitHub, IBM, Microsoft, Sonatype and VMware are RedMonk customers, while 10gen, Boundary, Circonus, Facebook, Open Logic, Palamida, and New Relic are not.

Predicting is an easier business than it once was. True, technology is hysterically accelerating rates of change and disruption, but that’s only relevant if the substance of your predictions matters. Which all too often, these days, it doesn’t. Analysts and pundits are able to prognosticate with relative impunity; who has the time to go back and check their accuracy? Pageview driven models, in fact, reward wilder predictions because the error cost is, generally, approaching zero. Unless you predicted, say, that Linux would be killed off by Windows NT, nobody will remember later.

I find value in reviewing my annual predictions, however. If they prove correct, that’s useful. If they were not, understanding the reasons why is important to adjusting our models moving forward.

Because I made the mistake of making better than a dozen predictions last year, this year’s review will be delivered in two parts. Part 1, below, will cover my predictions for browsers, the cloud, data, developers and programming language frameworks. Part 2, covering predictions within hardware, mobile, NoSQL, open source and programming languages, will hit tomorrow.

If you’d prefer to read last year’s first, they can be found here.

Browsers

Firefox Will Cede First Place to Chrome, But Not Without a Fight

According to RedMonk Analytics, whose data reflects our developer-heavy audience, Firefox was able to hold off Chrome for two quarters. On the first of June, Firefox held a 32.58 share of our audience to Chrome’s 32.08. By the second, Firefox was in second place and would remain there for the balance of the year, widening the gap in the process. At present, our browser metrics peg Chrome at 36.38 with Firefox a distant second at 25.48.

I feel safe counting this one as a hit.

Cloud

PaaS Adoption Will Begin to Show Traction, With Little Impact on IaaS Traction

The first Platform-as-a-Service providers essentially asked developers to trade choice for development speed. Like Ruby on Rails – itself the basis for multiple first generation PaaS platforms – PaaS was built for those that would embrace constraints. But PaaS platforms never saw the type of growth that Rails experienced, in part because of the further loss of control that the cloud represents. It’s one thing to have a web framework like Rails dictate the way that you build web applications; having PaaS platforms also choose the operating system, database, version control systems and more was too much.

Which is why second and third generation PaaS providers have furiously removed barriers to entry, adding additional runtimes, open sourcing the underlying platform and allowing you to pick your provider. Which, in turn, is why adoption of PaaS is accelerating. VMware CEO Paul Maritz calls PaaS “the 21st-century equivalent of Linux,” which explains not only why they feel compelled to compete in the space, but also why Red Hat might.

Virtually every vendor in this space is reporting growth similar to the Hacker News trajectories for Cloud Foundry and Openshift (below).

In spite of the growth of PaaS, however, none of the metrics we track reflect any decline in usage of general infrastructure platforms. Quite the contrary, in fact.

I count this as a hit.

Data

Firms Will Increasingly Seek to Leverage the Data They Generate

Turning data into revenue has been one of the core themes of the past year, as well as the focus of my talk at the Open Source Business Conference in May. We’ve long held SpiceWorks up as a model of monetizing data, and as customers adjust to the reality that they’re already sharing data and vendors cease to regard it as a third rail issue, we’re seeing more businesses embrace data based revenue streams, as with Sonatype Insight. From 10gen to Black Duck, vendors are increasingly positioning themselves to be purveyors of data as much as software. Data is no longer the byproduct, but a product itself.

I count this as a hit.

Hadoop Will Become the MySQL of Big Data

EMC, HP, IBM, NetApp and even Oracle all have Hadoop – or in EMC’s case, MapReduce – plays in market. Microsoft actually deprecated its own Dryad initiative in favor of the Apache project. Players from AsterData to CouchBase to EnterpriseDB to MarkLogic to Tableau to Vertica have purpose built Hadoop connectors. The commerical distribution space, once essentially owned solely by Cloudera, has expanded to multiple third parties with varying points of differentiation.

Hadoop interest elsewhere, meanwhile, has not slowed.

Need I say more about the growing ubiquity of Hadoop? I count this as a hit.

Developers

Talent Shortages Will Continue

Granted, predicting a shortage of qualified development talent will be seen in some quarters as controversial as predicting that the sun will rise in the east. But part of this is context: certainly in January, the economic direction was less than certain. And in spite of an unemployment rate that has hovered just south of 10% for the better part of the last calendar year, hiring continues to be an issue for the majority of our clients. To the extent that several are spinning up offices solely for purposes of recruitment. This is not surprising, given the historical growth in employee headcounts (see above) that has, to date, been relatively resistant to the global economic crises.

Demand varies by skillset, as might be predicted, but 2011 remained – by our metrics – a tight market. Other market watchers support this assertion.

“Hiring talent in Silicon Valley is the toughest since the last bubble and investors are starting to openly wonder how this one will end.”
Steve Blank

“There is a war for talent, particularly developer talent, going on. Not just in Silicon Valley but also in NYC and many other places around the country.

Companies, small and large, are resorting to all sorts of creative ideas to recruit. Free lunches, free yoga, pushing code day one, cool schwag, options, RSUs, pretty much whatever it takes.”
Fred Wilson

While we don’t have good data then on market specific hiring (Bureau of Labor data is not fine grained enough), the evidence available to us seems to support the contention that shortages of tech talent remain.

I count this as a hit.

Frameworks

Node.js Will Continue its Growth Trajectory

October was a rough month for Node.js, with posts like Node.js is Cancer and node.js Is VB6 – Does node.js Suck? following the tradition of March reddit discussions like Is NodeJS Wrong? The Trough of Disillusionment, it seemed, had arrived well ahead of schedule.

Except that interest metrics showed no commensurate decline. Node took – again – three of the Top 5 spots in inbound search queries within RedMonk Analytics. Which is unsurprising against the backdrop of Google’s Insights for Search numbers.

Over on GitHub, meanwhile, which itself has achieved dramatic growth, Node.js is the second most popular watched repository, ahead of Rails, jQuery, HTML5-Boilerplate, and Homebrew. Microsoft clearly perceives this growth, because it has worked with Joyent to create a stable build of Node for Windows which in turn led to an SDK for Azure.

All of which means nothing except that Node’s growth trajectory continues.

I count this as a hit.

Part 2, tomorrow.

“From the beginning of time two forces have vied for influence over us. One is bottoms-up, decentralized, and emergent. The other is top-down, centralized, and directed.

The first force catalyzes change and divergence, while the second tends toward order and convergence. The first gives birth to new ideas, and the second enshrines them.”
- Adam Ludwin, From LOLcats To Occupy Wall Street, Everything Is Happening From The Bottom Up Now

Traditionally, industry analyst firms have been oriented around top down adoption patterns. CIOs and other IT decision makers comprise both the research subjects and purchasing audience for the majority of firms in this industry, large and small. Which was logical given traditional procurement patterns. When hardware, software and services are available only at high prices, command and control is an appropriate management structure. Attempting to scale the decision making process for big ticket items across a large body of middle managers is not likely to yield acceptable outcomes.

An approach that makes sense in one context, however, may be misapplied in another.

The technology purchasing landscape today looks very different than it did even five years ago. Where once CIOs might reasonably expect to have the clearest understanding of what technologies are leveraged within their own organizations, today they are, as Billy Marshall put it, “the last to know.” This pattern manifests itself every day within the majority of businesses. Not because CIOs are failing, but because of trends that have fundamentally and likely permanently disrupted their ability to centralize the technology adoption process.

The four trends we see as most important in driving this are arranged here in rough chronological order.

Open Source

In the late nineties, startups and enterprises alike were effectively beholden to commercial suppliers for the majority of their software needs. Because each piece of the requisite software infrastructure had to be licensed, the capital expenses associated with new initiatives was high. This represented a barrier to entry, and thus a brake on innovation.

With the popularization of open source software, developers from enterprises and startups alike were able to operate independently. For the first time, the actual software practitioners were free to choose their own software rather than having it selected for them and subsequently imposed upon them by upper levels of management. Even in situations where the ultimate production infrastructure targets remained commercially licensed software, open source software like Linux and MySQL allowed for prototyping and rapid development without the attendant costs, both financial and in procurement latency.

This was the first major shift affecting procurement, and perhaps the most profound. None of the infrastructure we take for granted today – Linux, Apache, MySQL, PHP, etc – were originally adopted from the top down. Their adoption was, instead, a fait accompli. CIOs – the last to know – gradually became aware that increasingly significant portions of their infrastructure, unbeknownst to them, were running on free and open source software. The inevitable demand for production support options for this software is what fueled, in time, the valuations of MySQL, Red Hat and others.

Bring Your Own Device

In October, Apple CEO Tim Cook asserted that 92% of the Fortune 500 were “testing or deploying iPad in the course of less than 18 months,” which may help explain why the iPad revenue stream by itself would place within the top third of that group. The interesting thing about this is that the majority of businesses appear uncertain about precisely why they’re deploying tablets: “Most participants, 51 percent, indicated that they did not have a clearly articulated strategy.”

The answer, in most cases, is that there isn’t one. iPad adoption, much like the penetration of iPhones and Android handsets is being driven by users who simply want the device. Faced with a choice between users – chief executive officers among them – who will employ their own devices for work purposes with or without the permission of IT, many businesses are compelled to support the platforms even without concrete business justifications for them.

The consumerization of the enterprise is decentralizing the process of technology selection, but its importance may lie rather in design. Like all products, technology is designed and built to be sold to a specific buyer. For enterprise products, historically, the actual user has been a secondary concern; the buyer – typically centralized IT – was the priority. Consumer technology companies like Apple, however, are designed for a user. What they give up in IT friendly features they more than make up for in usability and the ability to delight.

The Bring Your Own Device trend, therefore, may well improve user productivity by driving devices designed to be used rather than managed into organizations, from the bottom up.

Software as a Service

Software as a Service is a classic case study in timing with respect to market acceptance. Not many remember today that the model actually failed the first time around, when its practitioners were known as Application Service Providers. Pyschologically, few enterprises were prepared for either the idea of renting software or externalizing critical data like that stored in customer relationship management systems. By the midpoint in the last decade, however, these concepts were sufficiently commonplace to see Salesforce.com a publicly traded company with a valuation north of a billion dollars.

Consumer markets, meanwhile, had adapted much more quickly. Hotmail debuted in 1996, Yahoo Mail the year after and Gmail dropped in 2004.

Some of those same consumer services were pressed into service by enterprise workers, in fact; it was once common for Exchange users to forward all of their email to Gmail due to the disparity in storage limits between typical Exchange implementations and Google’s webmail product.

This pattern has played out repeatedly over the years, from webmail to CRM to project management software to website hosting to online helpdesks. All were adopted from the bottom up. By making applications available to anyone with a browser, often at low or no cost, SaaS has surged up through the ranks of enterprises. The inexorable nature of the model is reflected by the growth of providers large (Salesforce.com) and small (37signals).

Cloud

The single most important feature of the cloud has nothing, or at least very little, to do with technology. It is, rather, the pay as you go economic model. As Flip Kromer puts it, “EC2 means anyone with a $10 bill can rent a 10-machine cluster with 1TB of distributed storage for 8 hours.”

What this means in practical terms is that for the first time, hardware procurement is democratized. From an accessibility and availability standpoint, cloud is the hardware equivalent of open source software. Where open source allowed developers to bypass traditional procurement channels by making quality infrastructure and development software freely available, so does the cloud allow the growing class of devops technologists to leave the world of high latency hardware procurement – where same day server provisioning is a feature – behind. Armed with nothing more than a credit card, instances can be spun up and ready for use in ninety seconds.

Cloud is the final piece of the bottom up puzzle. Open source software and to a lesser extent SaaS allowed for the decentralization of enterprise technology development, but at some point hardware would become necessary which was the insertion point for IT. With public clouds, it is possible for the first time to entirely bypass the traditional gatekeepers.

The Net

It should be evident that traditional procurement and purchasing is not dead, just increasingly bypassed by a more efficient process. Also, that a great many enterprises continue to function largely as they always have: top down. More important than the question of whether this model is sustainable in the face of the trends above is whether it should be.

Before lamenting the fact that the above forces are disrupting and destabilizing your enterprise IT, consider that that may be a net gain. If the primary drivers of BYOD, Cloud, Open Source, and SaaS include ease of use, lower costs, frictionless availability, and speed of provisioning, are these trends worth opposing? Particularly since efforts to do so will, in all probability, fail?

Or are they instead assets to be strategically leveraged? There is little debate that businesses that move the most quickly have a competitive advantage. It’s not clear how businesses that prohibit the same tools that enable this will benefit.

Either way, bottom up adoption is here to stay: use it or lose.

In 1999, Napster was unleashed upon the world. A year later they were sued by Metallica and Dr. Dre. A year after that the service peaked with 26.4 million users. A year after that the company filed for Chapter 7 to liquidate its assets.

While the record industry would have you believe that Napster’s meteoric rise was driven exclusively by thieves, the reality was that it was a desire for a product that record companies would not provide. Napster offered music available for dowload without draconian restrictions as well as the ability to download individual tracks rather than an entire album. Services that the record industry reluctantly agreed to years later. Years that the record industry also spent fighting a multi-front scorched earth legal battle against users of file sharing services that sprang up in the wake of the fall of Napster. When they finally did break down and sell music online, they were compelled to work with a much stronger player than Napster ever was.

But what if the music industry had been a rational actor and made the decision not to fight the tide, seeking agreements with Napster similar to the ones they employ today with Amazon, Apple and eMusic? What if they had recognized in those 26 million Napster users not thieves but potential customers and given them what they implicitly asking for: a more convenient way to obtain music?

The question is important because it’s essentially the same question facing enterprise IT today.

Napster made music available to anyone with an internet connection. For decades, enterprises have endured provisioning delays measured in months. Today, as Flip Kromer says, “EC2 means anyone with a $10 bill can rent a 10-machine cluster with 1TB of distributed storage for 8 hours.” It’s the end of procurement as we’ve known it.

Enterprise IT faces the same decision that the record industry once did: fight the tide or get out in front of it. Even setting the public relations damage aside, the returns of the former strategy for the record industry have been unimpressive. Given that developers have an increasing portfolio of accessible open source software and cloud services available to them, it’s unlikely that an enterprise crackdown on so-called shadow IT will be materially more effective. And then there’s question of whether throttling the constituency within your business that wants to move fastest is generally a good idea.

Why not enable them, then? Instead of firewalling the services Shadow IT wants, provide them centrally. Turn the tools that you are wasting your time fighting into an enticement to come out of the shadows. You’ll have better, if still imperfect, visibility into consumption and usage patterns as well as shorter development cycles. What’s not to like?

The RIAA missed their Napster opportunity. You don’t make the same mistake.

Bonus: My slides from the RightScale conference address this subject in more detail if you’re interested.

There are two interesting takeaways from this week’s – and last, if we include EnterpriseDB’s news – flurry of PostgreSQL news. Most obviously, it’s a win for the Postgres community. Whether you believe that Salesforce or VMware are in this for the long haul as strategic database suppliers, each of the new guys is a large, publicly traded enterprise software provider visibly committing to Postgres. Which is big.

The tortoise to MySQL’s hare, Postgres has built its reputation slowly, incrementally over the years. Well regarded by those in the market for relational databases, the project’s biggest limitation historically has been its more limited visibility.

With Heroku and VMware bringing Postgres to the cloud the same week – no coincidence there – the problem of visibility should presumably be lessened. Confidence in the future of the platform coupled with improved accessibility should have a positive material effect on overall traction for the project.

The second impact of this news, however, is perhaps less obvious. With two more market entrants, the importance of the wider cloud database market, and therefore the need to have a credible cloud story, increases.

Cloud database offerings are generally greenfield; it’s often difficult to justify the costs of even internal database migration, let alone transitioning to an offsite platform. But as enterprises consider their big picture database investments and strategy, the ability to consume their platform of choice from the cloud as necessary will become more strategic and less tactical.

Which means that existing database vendors lacking a cloud story for their database strategy – i.e. most of them – should accelerate their efforts in this regard, whether that’s through development or partnership.

Update: Missed the EnterpriseDB news while I was out on vacation, so that’s been updated.

Disclosure: Both Salesforce.com and VMware are RedMonk customers.