Skip to content

The 2016 Monktoberfest


The week of the 2016 Monktoberfest didn’t start out particularly well. First our daughter, and then my wife, came down with fevers. Next, my brother got in a car accident (he’s fine). Last, there were the questions around whether Hurricane Matthew, the first category five Atlantic hurricane since 2005, would lead to widespread flooding in Portland as we had last year.

It didn’t, and the rest of the week was enjoyably drama free. Thanks in part to the blameless post-mortem policy that we borrowed from Etsy – from a past Monktoberfest talk by Rafe Colburn, in fact – our process improves perceptibly each year. Organizing a conference remains an enormously time consuming and stressful challenge, but it takes far less of a toll than it did the first few times we put the show together.

As I tell new attendees every year, using our twin broad themes of social and technology, the Monktoberfest has two primary goals. First, to encourage attendees to engage with other attendees. From the initial Monktoberfest through the sixth iteration, we design the conference with the explicit goal of giving people time, space and the opportunity to meet each other. To share stories, to make connections or just to talk about their favorite craft beer.

The other hope is that attendees of the Monktoberfest are challenged, that they walk away from the show with different perspectives than what they arrived with. Sometimes these insights may arrive in the form of new information, at other times through the sharing of personal experiences. Some of these experiences are intensely personal and difficult, so much so that speakers have told me that the Monktoberfest is the only conference they can imagine talking about these subjects. Which means that they can be hard, in some cases, for attendees to hear. But every talk selected for the Monktoberfest is selected for a purpose, whether that’s discussing how oral traditions are passed down in the technology industry or how technology is impacting children today in ways that are surprising and, at times, frightening.

This year’s slate of speakers met that goal and then some. They were, as always, incredible: researched, prepared and, in a few cases, very brave. To answer a common question: yes, the talks were filmed, and we’re working to get them online as quickly as possible.

The reactions to the event were, as they always are, incredibly humbling.

Based on that feedback, it should come as no surprise that the best part of the event for me are the people. Every year we hear the same thing from our catering staff, our organizers, the venues we select, from Ryan and Leigh: it is an incredibly kind, smart and welcoming group. As I tell each new class of Monktoberfest attendees, it’s not often that catering staff fights to work on your event because the attendees are such a good group of people. But somehow, that’s the kind of group that we’re able to attract to the event every year.

If you want more of a flavor of the event, Mike Maney documented his experience here.

The Credit

I said this at the show, but it’s worth repeating: the majority of the credit for the Monktoberfest belongs elsewhere. My sincere thanks and appreciation to the following parties.

  • Our Sponsors: Because we do not like conferences where talks are simply paid advertisements, we do not sell speaking slots at the show. This means that we have to find sponsors that understand that the benefit to their sponsorship won’t be standing up and pitching but letting attendees know that they helped to make possible an event that they enjoyed. We’re fortunate that we find such sponsors every year, and we can’t thank them enough. Without their support, there is no Monktoberfest.
    • Nuxeo: For a content-driven event, it’s appropriate that our lead sponsor was a content management vendor. Nuxeo stepped up in a major way, which went a long way towards ensuring that the Monktoberfest happened. If you enjoy conferences without pitches, make sure to thank Nuxeo for making this one possible. We couldn’t have done this show without them.

    • GE: Another major reason that we were able to hold the Monktoberfest was GE, who was both an Abbot sponsor and will be bringing you the videos as soon as they’re released.

    • Red Hat: As the world’s largest pure play open source company, there are few who appreciate the power of the developer better than Red Hat. Their support as an Abbot Sponsor – the only sponsor to have been with us all six years, if I’m not mistaken – helps us make the show possible.
    • Pivotal: It should come as no surprise that a company like Pivotal, which is doing so much to rethink the way software is developed in a modern way, should sponsor an event that is similarly trying to rethink the way conferences are put together. Thanks to Pivotal for making the Monktoberfest happen.
    • Apprenda: Hopefully your brilliant new footed goblets made it home safely. When you get a chance, thank the good folks at Apprenda for them.

    • GitHub: Every conference needs food to get you through the day. To help keep our attendees powered up, GitHub bought everyone breakfast and snacks:

    • CircleCI: Our coffee, supplied to us by Arabica, got excellent reviews this year. Part of the reason it was there? CircleCI.

    • Funcatron: More conference snacks were brought to you by the Funcatron framework.

    • O’Reilly: Lastly, we’d like to thank the good folks from O’Reilly for being our media partner yet again and bringing you free books.
  • Our Speakers: Every year I have run the Monktoberfest I have been blown away by the quality of our speakers, a reflection of their abilities and the effort they put into crafting their talks. At some point you’d think I’d learn to expect it, but in the meantime I cannot thank them enough. Next to the people, the talks are the single most defining characteristic of the conference, and the quality of the people who are willing to travel to this show and speak for us is humbling.
  • Ryan and Leigh: Ever wonder why I call Ryan and Leigh the best craft beer team not just in this country, but the world? Take a look at this:

    The 2016 Monktoberfest Tap List from Mike on Vimeo.
    As I told them, we could not do this event without them; before I even start planning the Monktoberfest, in fact, I check to make sure they’re available. It is an honor to have them at the event, and we appreciate that they take time off from running the fantastic Of Love & Regret to be with us.
  • Lurie Palino: Lurie and her catering crew did an amazing job for us, and as she does every year, delivered on an amazing event yet again. With no small assist from her husband, who caught the lobsters, and her incredibly hard working crew at Seacoast Catering.
  • Kate: Though as many of you noticed Kate wasn’t around the conference during the day because she had an event several times as large to coordinate the day after the Monktoberfest ended, she did all of her usual hard work putting the fundamentals of the conference in place: booking and working with venues, coming up with menus in collaboration with the caterers, making sure we have t-shirts, designing our glassware and so on. As I like to say, the good ideas you enjoy every year come from here. I can never thank her enough.
  • Rachel: Her first Monktoberfest as a full time employee, Rachel had, if anything, even more heaped on her plate than last year. Just as with 2015, however, everything was delivered and executed smoothly. Which should come as no surprise, because Rachel is one of the most organized people I have ever met. At one point on Thursday, one of our staffers asked “When are things going to get weird?” because they always have in the past. They didn’t get weird and out of control this year because of Rachel. We couldn’t have done this without her.
  • The Staff: Juliane did her usual excellent job of working with sponsors ahead of the conference, and with James secured and managed our sponsors. She also had to handle all of the incoming traffic while we were all occupied with the conference. Marcia handled all of the back end logistics as she does so well. Celeste, Elizabeth, Emma, Izzi, Jamieson, Katie and Kim handled the chaos that is the event itself with ease. We’ve got an incredible team that worked exceptionally hard.

  • The Alchemist: Our favorite Vermont brewery was fantastic to us as always, helping us change things up this year from Heady Topper to Focal Banger, an incredible and incredibly hard to get IPA. They’re fantastic people, and if you’re in or around Stowe, VT, you really need to go see them.

With that, we’ll hope to see all of you next year. Don’t miss it, or risk potentially lethal levels of FOMO:

Categories: Conferences & Shows.

DevOps: All Development, No Database

Since the last time I touched working code in a production environment, it’s no exaggeration to say that no part of the development process remains untouched. Over the last decade plus, effectively every aspect of the application development process has been scrutinized, rethought and in many cases reinvented. From version control to build systems to configuration and deployment to monitoring, modern development’s toolchain is multi-part and sophisticated.

As it must be. Processes that work for code released in cycles measured in months cannot be expected to handle workflows measured in days or minutes.

For all that the process of developing software has evolved, however, the database remains curiously overlooked. Consider the example of Cloud Native. Describing a modern, typically legacy-free approach to building applications appropriate for cloud environments, the term Cloud Native has gone from informal descriptor to accepted industry shorthand in short order – to the extent that it has its own technical foundation.

If we look at the membership of that foundation, the CNCF, it would appear that the roster includes no database vendors at the Platinum or Gold membership levels, at least if you assume Google’s involvement is around Kubernetes and not tools such as BigQuery. Of the 41 silver members, meanwhile, two can be considered database vendors: Crunchy and Treasure Data.

For its part the Cloud Native architecture diagram from Pivotal, the company that first got behind the term, does not explicitly call out databases or database administrators.

Nor does Red Hat’s Open Hybrid Cloud architecture diagram.

If you query the website for the term “database,” meanwhile, you get about 20 results back, none of which are database-centric and many of which are translations.

From a market standpoint, it’s clear that databases are anything but an afterthought. The market valuations are substantial, and customers choices are expanding as are commercial investments in the space. But if you were judging simply by visibility of databases within the developmental toolchain and associated discussion, it is clear that databases do not occupy a position of prominence.

The question is why.

Some of the vendors in the space make an implicit argument that it’s a technology problem. But while it certainly cannot be argued that database tooling has undergone as rapid or thorough of an evolution as their application development counterparts, there are many operationally focused database vendors with technology that could certainly be useful in a DevOps context. Some market to this – see, for example, DBmaestro’s tagline of “DevOps for Database.” Others with applicable functional coverage such as Datos, Delphix or Redgate do not, with their messaging clearly aimed at traditional database administration buyers. Even developer-accessible offerings such as Citus Data or Heroku Postgres don’t explicitly try and make the connection from the operational data layer back to DevOps, or on a detailed level the task of application lifecycle management.

To some degree, this is to be expected as an artifact of a divide that continues to persist within most enterprises. While barriers between developers and operators have been actively targeted for elimination, the boundaries between developers and database administrators have remained well off the radar. Operators were once a distant, disconnected constituency, but are increasingly being integrated into development teams and vice versa. A similar thawing between the developer and DBA has yet to occur.

Which is why we still have application development models that don’t mention the database, and development-oriented database tooling that doesn’t mention DevOps. Neither of which, it should be noted, makes business sense.

Or is likely sustainable. Sooner or later, someone is going to realize that the continuing divide between the people developing the applications and the people that manage the data behind those applications is an obvious inefficiency, and like all market inefficiencies it will be targeted.

The question is not if, but when and who.

Disclosure: Citus, Pivotal, Red Hat, Salesforce (Heroku) and Treasure Data are RedMonk customers. Crunchy, Datos, DBmaestro, Delphix and Redgate are not currently customers.

Categories: Cloud, Databases.

On Digital Transformation

Lokomobile 3

The importance of technology, from an organizational perspective, is cyclical. Misunderstood and feared in its infancy, in a manner that will be familiar to those who’ve studied the history of the industrial revolution, technology was subsequently embraced first in specialized roles but went on steadily annex every unoccupied area of an organization. Having conquered the office, it then spilled over into the home with the introduction and rise of the Personal Computer in the early 1980’s.

But something happened along the way to everyone having a computer in their pocket and thirty in their car: people stopped caring. When everyone had technology, technology no longer mattered, at least according to the core thesis from the influential title by Nicholas Carr. A book that was indirectly responsible for unintended side effects from booms in outsourcing to downward price pressures on technology vendors. If technology was no more differentiating for an organization than its plumbing, the thinking went, why pay premiums for superior technology, or indeed own the capability at all? Many organizations, in an effort to focus on a core set of business concerns which did not include technology, chose to effectively divest themselves of responsibility via large outsourcing contracts.

This trend has begun to reverse itself in recent years, however. Seven years after the publication of “Does IT Matter,” many CIOs read Marc Andreessen’s Wall Street Journal Op-Ed describing the evisceration of traditional markets by digital players such as Amazon, Netflix, Pandora, Pixar and Spotify (for the record, no, Andreessen mentioned neither Airbnb nor Uber). When they asked themselves whether IT mattered in 2011, a number of them now answered the question differently.

Which is why and how the term “Digital Transformation” has come into fashion. It is no longer enough for technology providers to be able to solve complex technical problems. Customers today take the infrastructure for granted, and want instead to know how they can end up being Amazon or Netflix rather than Borders or Blockbuster.

While understandable, there are three basic problems with this request.

  1. Most Technology Providers Aren’t Equipped to Have That Conversation
    Independent of industry, most companies do a limited number of things well. The technology industry is no exception. Apple, for example, is rightly regarded as being the best in the world at user experience. In the context of delivering online services, however, Apple’s UX prowess has not translated particularly well because services are not a core competency for Apple – however much users wish otherwise. When enterprises turn to technology vendors for their digital transformation needs, in turn, what they are effectively asking for is a combination of deep technical experience and product design skills with very high end management consultancy or venture capital skills on top of that. It is difficult to envision a single company providing both at a high level of quality. One, certainly, but not both.

  2. Even if the Technology Providers Could, The Organization’s DNA Will Fight It
    There is a famous and possibly apocryphal story told in the research community about a call that an analyst had with one of the major US telcos in the early days of the cloud. According to the account, the analyst got on a call with sixty or so telco team members. The goal, the analyst was told, was to compete head to head with Amazon Web Services. The analyst then asks which fifty people on this call the telco is planning on firing, saying that you don’t tend build the next generation infrastructure with the people who built what you’re replacing. True or not, the implication is clear: organizational change, and in particular the type of organizational change that is required to respond to true disruption, is exceptionally difficult. There’s a reason most companies don’t survive the experience. Even if technology providers are equipped to provide not just the required technical expertise but the business innovation consulting that is more typically the province of a Bain or a McKinsey, many and perhaps most organizations will find it difficult to execute the necessary changes.

  3. The People Whose Job It Is to Predict the Next Airbnb, Uber, et al Can’t
    In June of 2008, a friend of the Airbnb CEO and co-founder introduced the company to seven prominent Silicon Valley investors. The company was attempting to raise $150,000 on a $1.5 million valuation. Five investors sent notes back declining the opportunity to purchase 10% of the company for $150,000, two didn’t bother to reply at all. In June of 2016, eight years later, the company was raising money at a $30 billion valuation. The point here isn’t to laugh at the unnamed investors, the point here is that predicting outcomes generally, and digital transformation specifically, is hard. Hard enough that very highly paid and educated professionals who do nothing but evaluate businesses that intend to transform industries digitally fail more often than they don’t. Expecting that your technology supplier, then, can be responsible for shepherding you through this process is probably not reasonable.

None of which is to say that technology providers cannot and should not be good partners as businesses in industries roiled by change seek to retake control of their technology stack and transform themselves amidst highly dynamic markets. Indeed, technology providers that can provide more than technology, in the form of education, guidance and, yes, good tech – are seeing tangible rewards in deal size and customer goodwill. More to the point, as one vendor put it this week, having the conversation is not optional for vendors any longer.

But enterprises must have realistic expectations, both of themselves and their technology providers. If they expect to be Uber tomorrow based on purchasing technology from a vendor with a deck that could have come from McKinsey, they’re likely to be very disappointed.

It is nice, however, to see people coming back around to the idea that yes, IT matters.

Categories: Business Models.

Lighting Out For the Territories

Apart from the fact that working at RedMonk affords me a legitimate opportunity to make a difference in the lives of developers, perhaps the best part of this job is the people that I work with – both at RedMonk and the clients and contacts we talk with daily. For those wondering, I am fully aware of how lucky I am to like both what I do and the people I work with.

Still, given the travel schedule analysts sustain for months on end, with long hours and a lot of time spent on planes and in hotels away from home, it’s nice to have some downtime – not in a metal tube – to take a step back. Every summer, as things in our industry slow down, I try and take a few consecutive weeks away from the job. While there are some in the industry who would make a virtue of never taking vacation, from my perspective, the time away is essential, not just to recharge the batteries but also to gain some necessary perspective. If you spend all of your time within the industry, it’s easy to lose that.

Fortunately, here in Maine, as much as Portland is attracting more and more industry talent, it’s not hard to get away from tech. As of tomorrow morning, I will be dividing my time between building furniture and swimming under waterfalls, painting the house and meeting friends for an afternoon beer at Allagash, ripping up shrubs from our front yard and, well, heading back to Allagash. My last week of vacation, with any luck, will be spent waking up to the sound of AC/DC and Led Zeppelin cranking from lobster boats a hundred yards away on the water.

Hopefully, it’ll be a few quiet weeks while I’m out. More likely, all hell will break loose, markets will roil and major acquisitions will come fast and furious. If you’re interested to hear about how and how badly I injure myself working on home improvement projects, Twitter, as always, is your friend. If you’re a RedMonk client and need help while I’m out, Juliane (jleary @ will be able to direct you appropriately.

Otherwise, I’ll see you on the other side, and until then, enjoy your time.

Categories: Personal.

Where the Database Market Goes From Here

At the crossroads

It’s hard to remember now, but a decade ago the idea of non-relational databases was a foreign one. Outside of successful and widely adopted alternatives such as Berkeley DB, generally the word database could reasonably be assumed to mean relational database. When we wrote about the possibility of non-relational alternatives then eleven years ago last March, the general reaction was a shrug, consternation or both.

As developers increasingly took control of the decision making processes around technology selection, however, they looked outside the enterprise to the likes of Google for architectural inspiration, and non-relational databases first emerged and then exploded. From a consolidated handful of enterprise-oriented relational databases which are still the backbone for millions of existing applications, the database market added a wide variety of new specialized database types: columnar, distributed storage and process, document, graph, in-memory, key-value and more.

Each of these categories began with the creation of specialized engines that excelled at a particular task, but that also involved tradeoffs traditional database buyers were unfamiliar with. Hadoop’s Map Reduce, for example, was less accessible to traditional DBAs (at least until companies such as Facebook wrote SQL-like interfaces such as Hive), but it could attack larger scale datasets than was practical with traditional relational databases, and it could do so far more efficiently.

The database market today, then, looks very different than the database market of a decade ago. The traditional relational databases are all still around, but they are increasingly one of many databases employed in a given business rather than the database employed.

Just as it was clear a decade ago that the market would be expanded, however, it is equally apparent today that the database market is poised for change. Functionally, we will continue to see a steady, even accelerating evolution of new approaches – fueled in large part by the release or replication of technologies developed at companies occupying the bleeding edge of web scale. Strategically, however, the available evidence suggests we should look for two major shifts in market.

A Return to General Purpose Datastores

By necessity, most of the major emergent non-relational database platforms of the last decade – projects such as Cassandra, Hadoop, MongoDB or Redis – were specialized in their design. In order to compete with the incumbent general purpose relational database platforms, their focus was asymmetric. Of all of the technology categories, database buyers have perhaps the least tolerance for risk. Which means that to justify using something other than the tried and true relational database technologies that had evolved and been improved over decades, alternatives couldn’t just be a little bit faster or a little bit more accessible: they had to be an order of magnitude improvement or more.

This, plus their built-from-scratch nature, inevitably produced a host of new database software that was highly differentiated from the traditional relational databases in approach, scale and function. Which is how we ended up with a database market composed of half a dozen or more relatively distinct categories.

Inevitably, however, these specialized platforms will seek to become less specialized over time. Much as lightweight, developer–friendly MySQL steadily added features such as stored procedures and triggers due to enterprise demand, many of today’s vertical non-relational stores will trend back towards their general purpose, relational ancestors.

In many ways, this transition has been underway for some time.

  • The category, for example, that once defined itself by its universal lack of a structured query language has been, in many cases, working to add that functionality. Software once born out of frustration with databases shackled to SQL, in other words, have been compelled over time to add query languages that look very like SQL.
  • Or look back to MongoDB’s acquisition of WiredTiger, a storage engine built by some of the same architects of Berkeley DB once upon a time. In many respects, the WiredTiger acquisition was a crucial step for MongoDB, because it allowed the database once prized for its ease of use and accessibility to add features such as better write performance, compression and document-level locking. The kinds of features, again, which enterprises look for.
  • Cloudera, for its part, began as a company with a single mission: applying Map Reduce and HDFS to data problems of unusual size and complexity. Today, its message is much broader, encompassing the traditional batch workloads, but search, streaming and SQL on top of those. Much more general purpose.

This trend will continue. As enterprises have acclimated to more complex functional markets, their willingness to purchase commercial solutions or support in specialized categories has ticked up. This is incentive both for players in market, but also players in nearby or adjacent markets. Importantly, open source has also lowered barriers to entry between these markets, as some of the core developmental costs can be offset and because in some cases necessary integration work between projects has already been performed.

The key question facing the market around this development, however, concerns developers. There is little question, historically, that enterprise buyers have preferred to consolidate purchases amongst the fewest possible number of suppliers. Which means that all-in-one, general purpose offerings will be welcome to CIOs and other purchasing agents. Developers, however, have historically advantaged more specialized, single purpose offerings over general purpose alternatives.

Which means that while the trend towards general purpose commercial datastores is seemingly inevitable, its outcome is not. It will be important for commercial vendors making such a transition to ensure that their developer engagement and narrative is at least as strong as the one-throat-to-choke message they can present buyers. Because otherwise, as the market performance of general purpose relational databases suggests, even perfect buyer messaging cannot make up for a lack of developer interest and adoption.

The Rise of as-a-Service Offerings

If the developer appetite for general purpose non-relational on premise solutions is uncertain, however, the interest in Database-as-a-Service offerings is not. This has been evident for some time, as the two fastest growing services in the history of Amazon’s fast growing Web Services business are Redshift (datacenter-as-a-service) and Aurora (MySQL-compatible proprietary database). Merger and acquisition activity in the space has likewise been steady: IBM, after previously acquiring Cloudant, bought Compose. Elastic purchased Found. Both of which follow acquisitions such as CenturyLink/ and Rackspace/ObjectRocket. And of course there is organic development. MongoDB’s Atlas service [coverage], for example, is very likely the shape of things to come. As are updates such as Cloudera’s 5.8 drop which added Impala support for AWS’s S3.

As-a-service offerings have limitations relative to on premises alternatives: they have less of a track record, latency between the database and compute tiers can be an issue, and it can be difficult to migrate large scale datastores to the cloud if only because of network limitations. But the advantages of instant-on, pay-as-you-go services that allow developers to make the database and everything that comes with it someone else’s problem have proven to be more than attractive enough to offset those and other concerns. Convenience, as ever, will trump just about everything more often than it won’t. Faced with the prospect of a fraught selection of the appropriate database, scaling it as required, protecting it and keeping it backed up, many developers are opting out.

Further, while as-a-service databases are compelling enough as stand-alone options, they also stand to benefit from general market adoption of cloud services. If your workloads are on premises, DB-as-a-service offerings are significantly disadvantaged, but with so much growth coming from cloud at the expense of on premises alternatives, the growth opportunities for hosted databases are substantial. This is true for base IaaS, but even more true for cloud services operating at higher levels of abstraction such as PaaS or serverless. If you’re content to outsource the infrastructure for your application, you’re more likely to do so for your database as well.

The Net

Given that the database market is subject to the same market forces as other enterprise categories, on premises software both specialized and general purpose is likely to be a tightening market over time. There is a great deal of revenue to be had in the category, without question, but it will be more difficult to obtain as there is more competition generally, more competition from open source specifically and on premises alternatives increasingly compete directly with service based alternatives. These price pressures are one reason vendors are increasingly moving back towards general purpose datastores from specialized roots: the broader the functional capabilities, the wider the addressable market, at least in theory.

Even long time database incumbents, however, are scrambling to develop or acquire their way into service-based businesses because that is where much of the growth will occur. Whether adopted as more convenient stand-alone alternatives to on premises databases or deployed in conjunction with other cloud infrastructure, DBaaS offerings are attractive to both developers and their employers, if for very different reasons. Importantly, this is the case in spite of the fact that the DBaaS market is in its infancy; many popular databases are not yet available as services, and those that are don’t yet have the provider choice that they will ultimately. Which implies that the DBaaS market has been successful, to a degree, in spite of itself.

From a provider perspective, then, a choice is implied. The existing spend on on premises relational solutions is measured in tens of billions of dollars, which is why many database providers today still regard their primary market competition as Oracle, even if they’re selling non-relational solutions. Vendors focused on trajectory, however, tend to see Amazon as the more important target, given that the most common report when talking to purveyors of on premises software is that a significant percentage of their existing customers are already in the cloud and most of those are on Amazon.

Implied choice or not, however, a legitimate market approach is also not to choose. On premises providers in most cases will need to follow the lead of players like MongoDB, because competing with DBaaS players without an as-a-service option is a non-starter. Worse, competitors that operate as-a-service businesses have an enormous intelligence advantage over purely on premises competitors, thanks to the difference in available operational telemetry. But neither should on premises vendors deprecate their existing business; instead, they can differentiate from pure play as-a-service options by offering customers their choice of running in an existing datacenter or in the cloud.

What is clear, however, is that a status quo approach in this market is one that will lead to diminishing returns over time. Choose your path carefully.

Disclosure: Amazon, CenturyLink, Cloudera, MongoDB and Oracle are RedMonk customers. Rackspace is not.

Categories: Cloud, Databases.

What Amazon Learned From Microsoft

As the company most clearly associated with the rise of the commercial software industry, it is no surprise that Microsoft’s business model historically reflected a profound, even emotional, attachment to software as an asset. Microsoft didn’t make money with software, it made money from software. At rates that were unprecedented in the technology industry at the time.

In practical terms, Microsoft was a platform business – and arguably even still is as it moves aggressively into services. Microsoft owned two of the most important platforms in the business and productivity software and operating system markets, and it studiously built on top of and alongside these platforms to deliver not just base platforms to their customers, but complete end-to-end stacks. Need a database? Microsoft had one. Ditto for web servers, app servers and integrated development environments. The value of these individual pieces was intended to be more than the sum of the parts, thanks to a practice known as integrated innovation. This describes an approach in which individual components are improved when used in conjunction with one another.

The byproduct of both the integrated innovation approach and Microsoft’s fixation on software as an asset was that company felt compelled to own the stack. If software was an asset, and higher switching costs meant higher profits, then logically Microsoft needed to sell more and more software that would raise the switching costs which would result in higher profits. While the company pointed to integrated innovation as an opportunity for partners as well as Microsoft, the Redmond software giant left little available oxygen amidst the core infrastructure – and later, segments of the packaged applications market – for partners. The end to end stack that Microsoft took to market was, by and large, built and sold by Microsoft.

Which in turn meant tradeoffs for customers. On the one hand, the stack offered the promise of lower integration costs, the proverbial one throat to choke and Microsoft’s characteristically strong developer experience. On the other hand, the stack was in many respects an all or nothing proposition. At the time, developing in Microsoft languages or leveraging Microsoft infrastructure meant that you used Windows. And while there was certainly a wealth of non-Microsoft server software that ran very capably and could be mixed and matched on Windows, it was difficult to do this with Microsoft server software – SQL Server being a notable exception.

Microsoft’s vision for customers, in other words, was complete, end-to-end and full service. Which, again, was not a surprise given that the company had internalized – was probably the best example of, in fact – Shapiro and Varian’s maxim that “the profits you can earn from a customer – on a going forward, present-value basis – exactly equal the total switching costs.” Customers that used less Microsoft software could switch platforms more easily. If customers that used one Microsoft product could be incented to use another Microsoft product, however, they would be less likely to make a jump from the platform. Each additional Microsoft product consumed further raised the switching costs, which according to Shapiro and Varian meant that Microsoft’s profits would rise in direct proportion.

This strategy worked very well, and for a very long time. While it’s impossible to know for sure, it’s not impossible that it could have survived the transition to mobile had the company executed more effectively in that market.

In the cloud, however, the player that appears to have learned the most from Microsoft’s platform success ironically has not been Microsoft, but Amazon. What the companies have in common – besides their geographical location, respective market dominance and some shared history amongst its executives – is that they are both platform businesses. Amazon’s Web Services business, it can be argued, is the new Microsoft, in the cloud. Which would imply that the AWS Management Console, as mentioned above, is the new Visual Studio.

(click to embiggen)

Like Microsoft, AWS is intent on delivering a complete, end-to-end experience for its customers. Whether the market need is a database, a CDN, directory service, streaming data support, logging, storage, messaging, data warehousing capabilities or something else, AWS has an offering. Or if they don’t today, they are likely to eventually. Like Microsoft with Office or SQL Server, AWS is comfortable augmenting its internal development with inorganic acquisition or OEMing where need be. AWS’ relentless drive to deliver the most complete platform in the cloud, at a rate that is impressive bordering on intimidating, has garnered the company a great deal of attention over the past few years. But it is not unprecedented. Those who followed Microsoft before it was temporarily stalled by the emergence of cloud and mobile will recognize a great deal of Amazon’s behavior and ambition.

Amazon has broken from the Microsoft playbook in one very important way, however: they have provided more and more accessible on ramps.

Microsoft woke to this strategy late, as evidenced by decisions such as the one to explicitly court the PHP community. As popular and ubiquitous as Microsoft technologies were, the theory went, there were open source alternatives that were far more popular that might be growth opportunities for IIS, and by default, the only operating system it ran on.

Amazon, by contrast, has had this essentially baked in from day one. The Elastic Compute Cloud, for example, may be accessible via proprietary APIs, but what runs on an EC2 LAMP stack will run equally well on premises or in a competitor’s cloud. Likewise, when Amazon’s Relational Database Service launched three years after EC2 and S3 debuted, it was an implementation of an open source project – MySQL. Unlike SQL Server, which could only be obtained from Microsoft, Amazon could plausibly tell its users that they if they were unsatisfied, they could get MySQL support from any number of alternative providers. Buying into Amazon, in other words, doesn’t mean a commitment to only buying from Amazon.

Which is not to say, of course, that Amazon has pursued a purely open strategy where Microsoft’s was proprietary. Quite the contrary; the majority of Amazon’s services are proprietary to Amazon, and in cases such as Aurora even open source-based offerings have a proprietary flavor to them. Amazon, and its sizable body of employees from Microsoft, have not forgotten the lessons of the software giant. But by including open or open-ish entrypoints such as Elasticsearch, MariaDB, MySQL, PostgreSQL – not to mention the core support for distributions such as RHEL or Ubuntu, or the Docker support within ECS – Amazon is not requiring customers to make the same tradeoffs that Microsoft did once upon a time. If customers choose to leverage standard services such as compute or open source databases, Amazon can monetize them. If they expand from there into proprietary services such as Kinesis or Redshift, AWS profit potential goes up, but the company will happily pursue volume and margin markets simultaneously. Amazon is like Microsoft, in other words, if Microsoft had been able to offer more, better and wider support for open source platforms years earlier.

Whether this is because Amazon understands that its cloud services’ instant availability and the inertia that comes from its usage is a powerful staying factor, the fact that they can monetize even non-AWS software services by selling the hardware to run it on, or some combination of the two is ultimately irrelevant. The net result is a provider that appears to have understood the most important lessons learned by one of the most dominant technology companies on the planet, and has even managed to improve upon them. It also is suggestive of the best way to compete with Amazon, but that’s a post for another day.

Disclosure: Amazon and Microsoft are both RedMonk customers.

Categories: Economics, Platforms.

The RedMonk Programming Language Rankings: June 2016

With the spring and summer travel schedule drawing to a close, we finally have had time to sit down and run the numbers collected back in June. As always, aside from the fact that we run our own GitHub rankings now, the process used for our bi-annual programming language rankings remains the same as when Drew Conway and John Myles White first looked at the question late in 2010. We have continued this analysis, comparing the performance of programming languages relative to one another on GitHub and Stack Overflow twice a year. The idea is not to offer a statistically valid representation of current usage, but rather to correlate language discussion (Stack Overflow) and usage (GitHub) in an effort to extract insights into potential future adoption trends.

With the exception of GitHub’s decision to no longer provide language rankings on its Explore page – they are now calculated from the GitHub archive – the rankings are performed in the same manner, meaning that we can compare rankings from run to run, and year to year, with confidence.

Historically, the correlation between how a language ranks on GitHub versus its ranking on Stack Overflow has been strong, but this had been weakening in recent years. From its highs of .78, the correlation was down to .73 this time last year – the lowest recorded. For this run, however, the correlation between the properties is once again robust. As with last quarter’s ranking, the correlation between the properties was .77, just shy of its all time mark. This is arguably noise, but we believe the correlation is worth noting at a minimum.

Before we continue, please keep in mind the usual caveats.

  • To be included in this analysis, a language must be observable within both GitHub and Stack Overflow.
  • No claims are made here that these rankings are representative of general usage more broadly. They are nothing more or less than an examination of the correlation between two populations we believe to be predictive of future use, hence their value.
  • There are many potential communities that could be surveyed for this analysis. GitHub and Stack Overflow are used here first because of their size and second because of their public exposure of the data necessary for the analysis. We encourage, however, interested parties to perform their own analyses using other sources.
  • All numerical rankings should be taken with a grain of salt. We rank by numbers here strictly for the sake of interest. In general, the numerical ranking is substantially less relevant than the language’s tier or grouping. In many cases, one spot on the list is not distinguishable from the next. The separation between language tiers on the plot, however, is generally representative of substantial differences in relative popularity.
  • GitHub language rankings are based on raw lines of code, which means that repositories written in a given language that include a greater amount of code in a second language (e.g. JavaScript) will be read as the latter rather than the former.
  • In addition, the further down the rankings one goes, the less data available to rank languages by. Beyond the top tiers of languages, depending on the snapshot, the amount of data to assess is minute, and the actual placement of languages becomes less reliable the further down the list one proceeds.

(click to embiggen the chart)

Besides the above plot, which can be difficult to parse even at full size, we offer the following numerical rankings. As will be observed, this run produced several ties which are reflected below (they are listed out here alphabetically rather than consolidated as ties because the latter approach led to misunderstandings). Note that this is actually a list of the Top 21 languages, not Top 20, because of said ties.

1 JavaScript
2 Java
4 Python
5 C#
5 C++
5 Ruby
9 C
10 Objective-C
11 Shell
12 R
13 Perl
14 Scala
15 Go
16 Haskell
17 Swift
18 Matlab
19 Visual Basic
20 Clojure
20 Groovy

JavaScript retains its position atop the rankings for yet another quarter, as do Java and PHP in their second and third positions respectively. There is no movement, in fact, among languages ranked within our Top 10. The positions have solidified, and it’s becoming apparent that it will take a serious push – or crisis – to significantly alter the dynamics of the top tier absent minor and statistically irrelevant drifts from quarter to quarter. It may or may not suggest that fragmentation is beginning to slow, but that’s an analysis outside the scope of these rankings.

We do have movement outside of the Top 10, however. Here they are in no alphabetical order.

  • Elixir: Elixir jumped again this quarter, but to a smaller degree (2 spots) than last (6) run. Its trajectory and functional appeal make it a language to watch, but whether or not Elixir can sustain this momentum is the important question. As even very popular languages like Swift have proven, the difficulty of growth is proportional to the rankings themselves – as one rises, so does the other. It’s also worth noting that Erlang has not seen a bounce from Elixir; it was static this period, holding at 26.

  • Julia: Julia’s growth has always been slow, but this is the first period in a number of quarters where Julia actually slid. Having moved up to #51 last quarter, it slid back to #52 for this run. This is not particularly surprising, as the language is not currently demonstrating the traction, visibility and enthusiasm characteristic of faster adoption rates. We’ll watch over the next quarter or two to see whether Julia can resume its climb, or whether it has stalled in a manner similar to CoffeeScript.

  • R: Out of all the back half of the Top 20 languages, R has shown the most consistent upwards movement over time. From its position of 17 back in 2012, it has made steady gains over time, but had seemed to stall at 13 having stuck there for three consecutive quarters. This time around, however, R took over #12 from Perl which in turn dropped to #13. There’s still an enormous amount of Perl in circulation, but the fact that the more specialized R has unseated the language once considered the glue of the web says as much about Perl as it does about R. Which is irrelevant to R advocates, of course. Whatever the cause, R’s relatively unique Top 20 path is one for fans of the language to cheer.

  • Rust: Interestingly, given that the past two quarters have anecdotally seen an uptick in Rust discussion, the language actually followed Julia’s lead and gave up one spot in the rankings this quarter. From a big picture standpoint, this is not particularly problematic, given that individual ranks should be taken with a grain of salt always, particularly so the further down the rankings a given language sits. That said, upward trajectories are preferable to the opposite, even if the actual rankings themselves are not to be obsessed over. Like Julia, it will be interesting to see whether or not Rust will gain next quarter or if it has instead plateaued.

  • Swift: Swift at this point has become the canonical example for the inertia of incumbent languages. Have followed an unprecendented growth trajectory since its introduction, this run is the first in which Swift has not gained but merely held its position of #17. In Swift’s defense, it at least performed better than the language directly ahead of it, Haskell, which fell out of a tie with Go for 15th place into #16. But it’s clear that further gains for Swift will not come easily, and will instead be the product of widespread usage across an array of communities. As discussed in the last iteration of these rankings, Swift has opened up new avenues for growth beyond iOS development via its release as open source software and the embrace of third parties like IBM or Perfect, but these have yet to yield gains in new discussion or code sufficient to propel it forward in these rankings. We’ll be watching for signs of this type of new growth closely.

  • TypeScript: Outside of Go or Swift, the fastest growing language we’ve observed in recent years is TypeScript. The Microsoft-backed JavaScript superset and Angular 2 foundation has made significant gains for the second consecutive quarter, jumping from #31 to #26. That was the biggest single change in any Top 30 language, and the second largest jump overall (Standard ML, 7 spots). At #26, in fact, TypeScript is now tied with Erlang, one spot behind Powershell and four behind CoffeeScript, which is just outside the Top 20. The question facing the language isn’t whether it can grow, but whether it has the momentum to crack the Top 20 in the next two to three quarters, leapfrogging the likes of CoffeeScript and Lua in the process.

The Historical Rankings

As we did last quarter, this visualization will allow you to dynamically select or deselect languages at will, tracing their individual rankings back to the first runs of this exercise.

A few notes:

  • This is not a complete ranking of all the languages we survey. It includes only languages that are currently or have been at one time in the Top 20.
  • This graphic is interactive, and allows you to select as many or as few languages as you prefer. Just click on the language in the legend to toggle them on or off. This is helpful because Swift fundamentally breaks any visual depiction of growth: de-select it and the chart becomes much more readable.
  • The visualization here, courtesy of Ramnath Vaidyanathan’s rCharts package, is brand new and hasn’t been extensively tested. Mobile may or may not work, and given the hoops we had to jump through to host a D3-based visualization on a self-hosted WordPress instance, it’s likely that some browsers won’t support the visualization, HTTPS will break it, etc. We’ll work on all of that, and do let us know if you have problems, but we wanted to share at least the preliminary working copy as soon as we were able.

With that, we hope you enjoy this visual depiction of the Historical Programming Language Rankings.

The Net

In general, these rankings are experiencing less volatility over time. The Top 10 in particular is fairly static, and even within the Top 20 movement is becoming more limited. It may be, as mentioned above, that we’re at or near peak fragmentation, and that the Cambrian explosion of programming languages is leading to a predictable period of consolidation. We’ll look at this question in more detail shortly.

Even if that is the case, however, it is not true that there are no interesting trends to watch in the programming language landscape. R’s continued ascent is interesting, particularly following the acquisition of Revolution Analytics by Microsoft. Another language with ties to Microsoft, TypeScript’s ascent is as notable as it is surprising, and whether Swift can pass Haskell and potentially even Go will be fascinating to watch. In the last few spots in the Top 20, meanwhile, there are suggestions that change is coming: Clojure, Groovy and Haskell all fell back this quarter.

We’ll be back with you in two quarters to assess these and other questions.

Categories: Programming Languages.

Hark Episode 3, “Getting Medieval on You”: Guest, KellyAnn Fitzpatrick

As newsletter subscribers are aware, Episode 3 of my podcast Hark, “Getting Medieval on You,” dropped the first of July. In this month’s episode, KellyAnn Fitzpatrick dropped by to get medieval on everyone. We discussed everything from Dungeons and Dragons to Zelda, Game of Thrones to Tolkien, the Arts and Craft Movement to Shrek…wait, what? Yes, that Arts and Crafts movement. Victorian England. Stephenson’s the Diamond Age. Ready Player One. The History of Rome podcast. And more. If you like references, this episode’s for you.

For those of you who don’t do podcasts, however, we offer a transcription of the episode below. Enjoy.

Stephen: Maybe it’s the location. Maybe it’s the great people who attend every year or maybe it’s just the beer. But we get a lot of proposals to talk at the Monktoberfest every year. Depending on sort of which year we’re talking about, there’ll be somewhere between 10 and 20 proposals per open slot. Competition to speak, in other words, is fierce.

Every so often, however, there is a talk that is so unique that it’s accepted even before I get around to reading the abstract. KellyAnn Fitzpatrick’s talk was one of these. I got a note one day from one of her coworkers that read, “I’ve got a proposal for you from our technical writer who’s writing her English doctoral thesis on MMORPGs.” To which my reply was a polite but non-committal “Interesting, have her send it along to the link for the CFP.”

Then the talk proposal came in and it was titled “Dungeons and Towers: Medievalism, Gaming, and the Academy.” Now let me ask you. If you were running a conference for developers and geeks, would or could you turn down a talk with that title? The answer is of course you can’t. And that’s how KellyAnn came to join the ranks of Monktoberfest speakers. She was kind enough to stop by to revisit the subject with us and to talk more broadly about medievalism, its role and its importance in media, gaming, and society today. Welcome to Hark Episode 3, “Getting Medieval on You.”

Stephen: So excellent. So, welcome to the show, KellyAnn. As we like to do to start the show, can you tell me who you are and what you do?

KellyAnn: My name is KellyAnn Fitzpatrick and I am a writer and probably, more importantly, a reader. And at present, I kind of have this like split path going on in terms of what I do. I try to be an academic and in that, I teach at the University at Albany. I teach classes and I work with the writing center. And I present at conferences for kind of academic English and medieval-related stuff and I attempt to publish. But I’m also a technical writer at a company called Apprenda and we do enterprise platform as a service.

Stephen: Indeed. Yeah, so this conversation, at least from my end, it’s, you know, originally a calling back from your Monktoberfest presentation, which touched on medievalism and gaming and so on. Which is I think for probably a not insignificant portion of our listeners, a couple of topics I think that are potentially of interest. So, you know, I guess my first question is, how did you get into what you do? Like what career path lad you down the “Hey, I teach at the University of Albany. I’m an expert on medievalism,” you know? So, how did you end up where you are?

KellyAnn: I, like many people who were at Monktoberfest, had as a child like a strong interest in the works of J.R.R. Tolkien and the videogames that kind of come out of his work. So I’m from the generation of the first Nintendo so I played “Zelda.” Before that, I played “Gauntlet” not in the arcade but on my cousin’s Tandy.

Stephen: Oh, nice.

KellyAnn: And if anyone actually remembers that, you can kind of date me there. So, between that type of kind of games that I was exposed to as a child and the things I would read, I became interested in the Middle Ages. You know I read up on it. I had some type of history class in high school that world history and we kind of covered the Middle Ages. This was kind of before the internet had this wealth of information that it does today. So I would spend a lot of time at a library looking at actual books. And, when I ended up going to college, I thought I was gonna go and be a doctor. But I got to college and I did my…

Stephen: Didn’t we all? Yeah.

KellyAnn: Yes, we all thought we were going to be a doctor or something else. I went to… I did my undergraduate at the University of Notre Dame and they have an excellent Medieval Institute there. The entire seventh floor of their library is kind of dedicated to the Medieval Institute. And they have it not only as a space but also as this kind of like cross-disciplinary program in the university.

So I thought I would be…go. You know, I would come out and go to medical school. But by the time I got through college, I decided I wanted to actually pursue Medieval Studies and kind of English jointly. So that’s kind of what set me on the track of learning more about the Middle Ages.

At the time, I didn’t entirely understand that the actual study of the Middle Ages, like looking at the history, you know, culture, literature necessarily tied in to the games that I played as a kid. Until I stumbled upon a couple… there were a couple of different books, a couple of different scholars that were very important for me for this. One is Tom Shippey who wrote a book called “J.R.R. Tolkien: Author of the Century” in which he talks about J.R.R. Tolkien as an author, as a product of the 20th century not necessarily the author of the 20th century. But that really, you know, kind of turned my interest to Tolkien in a more academic way. And then Tolkien himself, of course, was an Anglo-Saxon scholar. And so there are these ties to the types of work I was doing with like “Beowulf” and actual medieval literature.

And the second scholar that really kind of turned me on this path is… her name is Jane Chance and she also did a lot of work with kind of like Tolkien’s work and the images you come out of Tolkien. But she also was an Anglo-Saxonist so she’s done kind of a lot of “Beowulf.”

So you know somewhere between the end of my undergraduate and the beginning of even my graduate studies, I kind of had this in my head that this is what I wanted to do. And I’m glad I ended up this way because it ties into a lot of things that I work with now. I mean I’m surrounded by kind of technical people all the time. So the fact that part of what I study is the Middle Ages in videogames, I actually have a lot more kind of understanding of where those videogames come from and how.

Stephen: No, I can certainly see that. So that actually sort of begs the question. As you mentioned, just dating back to “Zelda” and “Gauntlet” and obviously going through to board games, you know the “Dungeons & Dragons” up to a lot of today’s present day games, “World of Warcraft” and so on. There’s obviously a lot of ties between medievalism and gaming which was again a big part of the presentation you gave at Monktoberfest. Where does that date to do, do you know? In other words where did that all start? How did these ties originally form?

KellyAnn: And I think I covered this in my Monktoberfest presentation. But even before, I think it was necessarily a videogame tie, it was like a game tie.

Stephen: Right.

KellyAnn: So “Dungeons & Dragons” is for me the big space where you have medievalism and gaming coming together in a way that becomes just very, very popular. There are like manifestations of, you know, medievalism that you can kind of see maybe hopping up in like smaller spaces. But that becomes the basis for taking this idea, not just of the Middle Ages, but of like medievalism itself. So the type of like high fantasy world of Tolkien that Tolkien kind of puts together and translating that into this game system to the point where almost anytime you see a game that has medieval components, especially the earlier ones like something like “Zelda,” there’s a dungeon in it.

Stephen: Indeed. Yeah, I remember the screenshot from the Mocktoberfest presentation. So you know that’s sort of some of the roots. Where do you think the appeal comes from? Because to be perfectly honest, and sort of I guess by way of full disclosure, when I was a kid I was very into medievalism as well. You know, loved Tolkien, read tons and tons of medieval history, took clearly not as much medieval history at undergrad as you did. But certainly, it was an area of interest.

And, I couldn’t honestly tell you sort of where that came from or what the appeal was. Right? I mean, do you have any sort of better sense of that having studied it and obviously embraced it on the game side as well? In terms of you know culturally, we look at the phenomenon like “Game of Thrones,” do you have any idea sort of where that universal appeal for the time period comes from?

KellyAnn: There are a number of theories about this. So any information I kind of you know give you on this has been talked about and debated and probably comes from people who are much smarter than myself. But one of the bigger ideas is that the Middle Ages, and even especially in the versions of the Middle Ages, in the medieval that we get kind of passed down and modified and appropriated in these types of games. It’s a break from what we’re dealing with today. It’s a break from what we consider the modern or post-modern world. So that it’s so different from what we do. That most of us get up and go to work every day.

Stephen: Right.

KellyAnn: You know, some of us get to get up and work in our pajamas which is always kind of fun.

Stephen: Guilty.

KellyAnn: But yeah. But the idea that this a break from that and that it’s a space where you get to control the position that you imagine yourself in. So, if you think about something like medieval Times, the restaurant where you go and you watch jousting. Right? Every customer who walks into medieval Times gets to wear a crown. And everyone is kind of the king, even though there is a king and a princess and things like that. But everyone is treated as if they are of this kind of like noble class within that. And they get to cheer for their knight and they get to rip apart a chicken with their hands because they won’t give you a fork.

So it’s this very fun way of imagining the Middle Ages. There’s really no space where you go and you’re like, “I’m gonna be the peasant who’s cleaning up after the horse” at medieval Times. Right? Thank goodness!

Stephen: In spite of the fact that, you know, according to the statistical probability if you look at the actual distributions of populations of the time period, if you’re gonna be anyone, you’re probably gonna be a serf. But we’ll sort of let that go.

Actually, on a facts bent, as long as we’re sort of talking about that. As you’ve brought up, and I think you used the example of dungeons being equated with medievalism in spite of the fact that they pre-date and post-date the period. As somebody who is academically an expert in the field, you know has studied it a great deal, does the sort of factual departures, and I’m not even talking about things like dragons which don’t exist. But, in other words the liberties, I guess, that some of the games or sort of other media take with the period. I mean does that bug you at all? Is that something that you’re fine with as a fictional liberty? Like from an academic standpoint, how does that strike you?

KellyAnn: It strikes me as good that we can take something like the Middle Ages and change it and alter it. There are a number of different ways to kind of approach that. For instance, if I were teaching a course on medieval history, I would probably be horrified by it in some ways. In that, if I walked in the first day of classes, say it was like an undergrad class on medieval history, some of the kind of preconceptions that I would expect from my students about what the Middle Ages are could be difficult to deal with. Right?

So the idea that they’re always associating the Middle Ages with like princesses and dragons that don’t exist and dungeons.

Stephen: Right, right.

KellyAnn: And there is like a school of thought that looks at that and is just kind of banging their heads against the wall. Like “No, this is not what the Middle Ages are. This is exactly what it is.” But there’s also the argument that the term medieval itself is a construct. It’s something that even within the realms of academia that we’ve kind of put together. Not that we can’t point to kind of what we would think of as a period in history and maybe point to like this happened on this date. But, there’s no event where it was announced to the world that the Middle Ages have started.

Stephen: Right.

KellyAnn: Or the Middle Ages have ended so, you know, therefore everyone go out and be modern.

Stephen: Yeah, yeah, yeah.

KellyAnn: So from that point of view, it suddenly becomes less disturbing. Even if you’re teaching something like history to think of the Middle Ages and the medieval as something that is kind of taken off and changed by people. And to me, the interesting thing is what we do with it.

Stephen: Right.

KellyAnn: For…

Stephen: Go ahead.

KellyAnn: For instance, in the 19th century, the idea of Medieval Studies itself like as a contract, construct and as a discipline. I mean that’s very much in a response to like this 19th century need to kind of categorize things and understand them and give this kind of empirical view of the world. And videogames are very much a 20th and 21st century way of taking the Middle Ages and using it as a response, I think.

Stephen: Yeah, so that actually so let’s… We’ll come back to the modern incorporation or modern, I guess, iteration of medievalism. But let’s go back to Tolkien for just a second. So, from a history standpoint, you know how did he sort of arrive on that? Obviously, he was a sort of linguist by trade. But, you know, was he one of the first to pick up on a lot of the history and incorporate it? Or is this… You know, was he drawing on traditions of other storytellers? In other words, how did that first get going that medievalism became sort of a “thing” so to speak, from a fictional narrative standpoint?

KellyAnn: Yeah. So Tolkien is… He’s born at the end of the 19th century and he’s very much a product of the 19th century. Both in that he becomes a linguist, he becomes a philologist. And philology has its own kind of like history. And it’s something that was still going on and have been a profession that he could enter into when he came of age.

But it also means that, you know, when young Tolkien decided to start reading, he could read things like William Morris especially. So he inherits this kind of like 19th-century medievalism and there is a very big, what we call the “Medieval Revival” in Britain in the 19th century. And William Morris is one of the author’s that he like cites directly as someone he had read. That he actually tries to… when he’s maybe like in his late teens, he tries to write a story after the style of William Morris. And for those of you who… Like I don’t know if you’re familiar with William Morris?

Stephen: I am not.

KellyAnn: He was this 19th-century kind of like polymath. He did everything. Like he wrote, he painted. He started a publishing company, the Kelmscott Press. Morris & Company, the kind of if you’ve ever heard of that, the type of design firm, Morris & Company, that’s what he started as well.

Stephen: Interesting. Okay.

KellyAnn: He’s part of what is known as the Arts and Crafts Movement.

Stephen: Okay.

KellyAnn: Which…

Stephen: I was gonna say Kate’s a big fan of that so.

KellyAnn: Well it’s great. And in the context of kind of like 19th century Britain, if you can imagine what we often think of when we think of like Victorian décor. And you can think of maybe of a lot of like frills and lace. And things kind of being produced at a time when the Industrial Revolution is in full swing. So a lot of the decor that people would have in their homes like furniture would be like mass produced, the result of factories, like coming out of factories. And the decor itself often being very cluttered.

Stephen: Mm-hmm.

KellyAnn: Morris and other people kind of involved in this Arts and Crafts Movement look at that and they’re like, you know, “This is terrible. We’re getting all this horrible, cheap furniture and everything is cluttered.”

And for Morris, he actually turned to the idea of the Middle Ages in terms of design. So he saw the Middle Ages as this space where you could kind of pull out these more like clean designs. And the way things were produced were not on this mass scale. That they were done by hand through these like kind of artisanal craftsmen. And that the labor involved…and this was very important for him because he was also a socialist. The labor involved in a craftsman creating a chair is very different than the number of people in a factory who are maybe putting different pieces of the chair together.

Stephen: Yeah, yeah.

KellyAnn: So for him, he has this idea, very, very polymathic understanding of the world and what he did with it. But one of the things he also did was he wrote. And he wrote travel narratives of his own travels. He did translations. And then towards the end of his life, he started writing what he called these kind of romances that were these part fantasy, part semi-historical fiction narratives, and these influenced J.R.R. Tolkien a lot.

Stephen: That’s actually, that’s… Particularly the angle that resonates with me, well, there are several, but the idea of the Arts and Crafts movement. And essentially almost a rejection of industrial production actually reminds me of I wanna say it’s “The Diamond Age” by Neal Stephenson. You know, it’s essentially a world where they have, sort of I guess the term we would use today would be 3D printing. So you can 3D print basically anything. And yet, you still have craftsmen which they call “Vickies,” obviously harkening back to Victorian England who manufacture things by hand. And there is a value attached to that. You know, just as being distinct from things that are sort of mass produced by 3D printing.

But that’s obviously an aside and not necessarily on the topic of medievalism. But going back to sort of the Tolkien topic. So Tolkien fed into, I assume and maybe this is a direct link or maybe it’s indirect, Gary Gygax and the foundation of “Dungeons & Dragons.” Do you know how to trace that? What the direct influence was or lack thereof?

KellyAnn: There… I mean and I’m not an expert on kind of the exact places where “Dungeons & Dragons” is pulling from Tolkien. But to look at the different types of what we would consider like races involved like elves and dwarves. And then the one that is probably the biggest giveaway would be what in “Dungeons & Dragons” would be a halfling.

Stephen: Mm-hmm. Yeah, yeah, yeah.

KellyAnn: Because using the word hobbit would be probably something actionable by law.

Stephen: Yeah. So similar but legally distinct, shall we say?

KellyAnn: Yes, very legally, legally distinct.

Stephen: Yeah. So, yeah. It’s always interesting to me because it’s one of those things that are obviously trends and fads that come up from time to time. And, you know, each generation has its time period that they may sort of focus on. But sort of the interesting part of medievalism for me is that it really is pretty enduring. Right?

It seems like… you know, I can remember reading, oh, god. They had to be published I guess in the late 1800s. It was books that were in my grandmother’s house growing up, crusader tales and things like that. All the way through “Dungeons & Dragons” and obviously we have the sort of mass-produced media today like “Game of Thrones.” And it’s all sort of very, very medieval. So, it’s had a longevity that I don’t know that I would have expected sort of given the time period.

KellyAnn: It does and there’s the argument that medievalism existed before the middle ages ended.

Stephen: Gosh, that’s excellent. How so?

KellyAnn: Yeah. Well, a great example of this would be the Tudors and that’s another topic that has kind of been taken up and re-imagined in popular culture like with the series “The Tudors” about Henry VIII of England. But Henry VIII’s father, Henry II, and this is probably like late 15th century, early 16th century we’re talking about then. He actually ends the War of the Roses in England, which you’ve had different factions fighting over who is gonna be king. The Lancaster and York are the two names that kind of get thrown out. And interestingly enough, another aside, that is one of the basis for “Game of Thrones,” right?

Stephen: Yeah. Yeah. Right. Yeah.

KellyAnn: The War of the Roses in England. And you can see the name was translated. Like “Lannister” for Lancaster and “Stark” for York and things like that. That’s an entire other study, right?

Stephen: Of course, yeah.

KellyAnn: But when Henry VII, he claims the crown by conquest. Basically, he kind of has a claim on the throne through blood. There are a whole bunch of people that have a claim to the crown ahead of him but he wins. His armies are triumphant. So, he gets to be, you know, King of England. And he does a very smart thing in that he marries the York princess, Elizabeth of York. So he kind of like ties the two houses together in some ways.

But the other thing he does is he really plays up on Arthurian legend. In that he, he names his first son Arthur. And you know as much as possible goes… you know, relies on the legends of like King Arthur and claiming descendency from King Arthur which is always the thing to do in England, if you can.

And his son, Henry VIII takes this up in that he… There is this giant round table in Winchester and you can kind of go see. It’s hanging up on the wall. I think it dates from one of the Edwards, like Edward II or III or whatnot in terms of when it was made. But he has it repainted in Tudor colors with a giant Tudor Rose in the middle of it. So that the Tudors themselves in terms of establishing their own legitimacy to the English throne use this older idea of King Arthur which is like going back to even a perceived earlier time in what we would consider to be the Middle Ages.

So, even then you have this idea of medievalism, this looking to the past. This idea that you can find this knightly or courtly or chivalric tradition. And use it as a basis for anything is happening even then.

Stephen: So would you say that’s one of the reasons I guess that would keep medievalism and sort of medievalism-themed media, be that games or TV or movies you know, competitive an environment? Like looking at games is just one example. Games like “World of Warcraft” more than hold their own, at least from what I can tell as a casual observer, against the “Calls of Duty” things like “Grand Theft Auto” which are certainly big titles in and of themselves. But medievalism is again basically very competitive in that environment. So do you think it’s basically that sort of look to the past, that escapism again that keeps it on…well, yeah, that keeps it sort of a first-tier option as far as media goes?

KellyAnn: I think so. And, the other element that kind of works into that, especially in other…even outside of gaming and definitely within gaming is the possibility of magic.

Stephen: Okay.

KellyAnn: Right? So, one thing that you can do in “World of Warcraft” and other types of games like “Dark Souls,” there’s always some type of sorcerer…

Stephen: Right, right, right, right.

KellyAnn: …or mage. You know, there are these elements of magic that you don’t necessarily find something like in a first person modern day shooter.

Stephen: Yeah, yeah, actually it’s funny you bring that up because I just read “Ready Player One.” In “Ready Player One,” it’s this sort of large virtual universe is one of the settings. And they have to sort of explicitly define in any given region, “Does magic work here or not? Does technology work here or not?”

So you can have Tolkien-like worlds where no technology is allowed but magic works. You can have “Blade Runner” worlds where technology works and magic doesn’t. You can have other areas where they both work at the same time. So yeah, it’s… I don’t know, the interplay is, again not being a gamer. It’s always one of those things that’s sort of more academically interesting to me. But yeah, like I said, I think it’s just very curious to see. Like I said the longevity of the material because it’s… You know, sort of on paper, you’d think, “Hey, kids are typically obsessed with things that are new.” Right? In other words, well for a lot of good reasons, political correctness among them, you don’t see things like cowboys and Indians anymore. There’s not too many kids who are interested in being cowboys. But, they are still interested in being knights and going on quests, which, yeah, I find that really interesting.

KellyAnn: Yeah and it’s something especially for kids that gets tied into the whole idea of fairy tales as well. So, like Disney culture. There’s an excellent article that I read while I was completing my dissertation. And it talks about how in the kind of Disney realm, in just, you know, talking about the films and specifically the princess movies, which there are many of. That if you ask the typical person on the street, how to describe all these different things, the term medieval comes up to describe almost all the fairy tales and all of the princesses. Even if it’s one that’s not even necessarily in a medieval context or a medieval setting.

Stephen: Interesting. Okay.

KellyAnn: Yes. So if you think back just kind of like the new classic and new Disney movies, something like “Sleeping Beauty”is it’s set very firmly something in the 14th century. Right?

Stephen: Yeah. I was gonna say you don’t think of it that way but that’s absolutely true.

KellyAnn: Yeah. In that, you can make the argument that this is a medieval fairytale. But if you look at something like “Snow White” or “Cinderella,” which is “Cinderella” is very much more 19th century in context. Even that gets branded as medieval or like associated with the Middle Ages for a lot of people. To the point where something like the princess or the idea of the princess being rescued and all the other kind of like tropes that kind of go with it are… they become these medieval tropes. So you can’t really go through much of the fairytale culture without getting some idea or glimmer or redirection to the medieval.

Stephen: So what do you think… You know, sort of when we think about those medieval tropes, right? In other words obviously, one of the things about tropes is they are to some degree narratively helpful in that they are constructs that a reader or a listener or a viewer will understand, sort of instinctively. They have some frame of reference to deal with. But on the flip side, they’re also at risk of course of becoming clichés. Right? “Okay, yes, it’s been done a thousand times. I’ve seen this. Like give me something new.” Are there aspects to medievalism that you think are sort of undermined, not undermined but sort of under, I don’t know, underappreciated, not yet discovered that may yet emerge?

In other words, when you’ve been sort of doing… well when you were doing your undergrad research, were there aspects to it that have yet to be embraced? I’m thinking of things like “Game of Thrones.” Right? “Game of Thrones” obviously incorporates a lot of the standard tropes, you know, dragons, sort of magic and things of that nature. But also is for example, exploring currency issues in sort of one of the plot lines and so on. So are there things like that you’ve studied that you think have yet to emerge but will begin to pop up in medieval media moving forward?

KellyAnn: I don’t and I don’t know if I can think of something that I have studied that I haven’t seen pop up somewhere. But one kind of image of the Middle Ages that I think I would like to see more would be more of what we get through “Monty Python and the Holy Grail” which is this very almost like disjointed version of the medieval narrative. Right?

And if you can kind of look back at it, it’s like it kind of has a story. Arthur is going to find the Holy Grail. But all these other things just kind of like pop up and thrown in and very clearly are being used to talk about other things. For instance, there’s the… Arthur shows up at this kind of farm. Right?

Stephen: Mm-hmm.

KellyAnn: This is the one where he’s being questioned as, “Well, what do you mean you’re the king?” So you’re very much talking about these kind of leader politics being re-represented in this. What is allegedly this movie about King Arthur and the Middle Ages. So, that where it’s just very clearly pointing out this is how we’re using the Middle Ages kind of forward our own agendas or our own ideas. And doing it in this very just disjointed way. I think I’d like to see more that or I think we’re hopefully going to see a little bit more of that where it’s not necessarily the simplistic, relying on of tropes but actually turning these tropes on their heads.

Stephen: Yeah, which actually is a perfect segue to the next question, which is, why should we today continue to study medievalism, right? And obviously, I have my own sort of answer to this as somebody who is certainly not an expert in medieval history but somebody who is a fan of history generally. But I’m curious, for somebody who is very well-versed in this particular time period, what are the things you think it offers obviously beyond the entertainment value that make it a subject that is worth us studying and really understanding in a fundamental way today?

KellyAnn: I’m very curious to hear what your answer is.

Stephen: Well, we can… I can go first if you like.

KellyAnn: Yeah, please.

Stephen: Yeah. I mean I think from my perspective, always having some distance from a historical standpoint is useful. One of the really interesting things, I think… I don’t know if you’ve listened to the “History of Rome” podcast but one of the things I thought was fascinating about that, aside from the fact that I thought I was somewhat versed in Roman history before I started listening to that. And as it turned out, I was not at all. I’d forgotten – I’d either forgotten or not known most of it. So, for anybody listening to this if you haven’t listened to that podcast, I would highly recommend it.

But anyhow, one of the interesting things that essentially Mike Duncan sets up in that podcast is how essentially the decline of Rome leads to essentially the formation of feudal states. Which obviously later give rise to what we would consider the medieval power structure. Right? You know, feudal lords and serfs and so on. And that transition from basically the large, centralized Roman Republic to feudal states is one that, as I said, I frankly had not understood very well.

But also, I think it offers some very interesting and important lessons for us today. Because you know it’s… Comparisons between the decline of the Roman Empire and sot of the United States are in my opinion and I would… Actually, I know this for a fact. He wrote about it. Mike Duncan’s as well I think are overblown. But that doesn’t mean there aren’t parallels and there aren’t a lot of lessons to be learned in terms of okay, this is what happened to a republic that made certain decisions. This is where we ended up.

And studying the history of the medieval period, these are some of the long-term ramifications. Right? In terms of disruptions to commerce, disruptions to learning. Frankly higher rates of death really across the board from infant mortality all the way to adult mortality. So I think it is, from my perspective, just again certainly not being an expert but just knowing what I know of the medieval period. I think that there are a lot of important, illustrative lessons in terms of where things end up, you know, if you make certain decisions along the way. So, that’s certainly one reason, I think. You know, I’m sure there are hundreds but that’s certainly one reason I would imagine the medieval period is still very much worth studying. But yeah, like I said I’m curious to hear what you have to say.

KellyAnn: Yeah and that’s well said. I’m glad I made you go first. I think you said it better than I would have. And then just to kind of add to that train of thinking. Not only like the fall of Rome and you can kind of see the rise of these feudal states. But within the medieval period, we actually see the decline of feudalism and this nascent capitalism, the beginning of like our economic system as we have it.

Stephen: Right.

KellyAnn: So like that in terms of going back to history and seeing what we have in terms of record, there’s definitely some stuff worth there. But also, in terms of just kind of like history of nations and how nations rise and fall, the Middle Ages has that as well. Towards the end of the Middle Ages and early Renaissance, of course, we have in Europe especially religious wars. Right?

Stephen: Mm-hmm. Yup.

KellyAnn: So you have Protestants and Catholics. Not that many people that even I know who aren’t necessarily academics go back to that and understand how much fighting within Christianity there actually was.

Stephen: Oh, yeah, absolutely.

KellyAnn: They do know there is a drink called “Bloody Mary” but not necessarily that it’s named after a Queen of England who was a Catholic and put a number of Protestants to death. So there’s a lot in history for us to kind of go back and learn and we can kind of draw parallels to as well.

Stephen: Yeah. Yeah, I think it’s, it’s… Unfortunately, it’s always a sad commentary. But the Santayana quote that “Those who can’t learn from history are doomed to repeat it” really does seem to be true. You know, I don’t know. I guess the older I get, or whatever it is maybe I’m just getting old and curmudgeonly, but it really is.

You see these patterns sort of play out. And when you have a period of transition between one time period and another, right? So in other words, we go from sort of history of Rome and that goes back, that transitions out to the medieval period where we have a much more fragmented, much more sort of nation state. You know in many cases, sort of warlord type of distribution of power, distribution of economic systems and the consequences of that. Like I said, I think there’s a lot to be learned there. But we have to pay attention to those lessons which is always the trick.

KellyAnn: Yeah. And then we go back to your original question of, “Why should we study medievalism,” right?

Stephen: Yup.

KellyAnn: So not just the medieval period but the manifestations of the medieval over and over again. I think it’s vitally important to understand the way that we are using these tropes in our own kind of rhetoric and arguments today.

And just one of the easier examples is the trope that we have is that men go out and fight and women should be at home.

Stephen: Right.

KellyAnn: I mean that’s something that we locate in the Middle Ages often with… “Rescuing the princess from the dragon” is probably one of my favorite fairy tale medieval tropes. That’s something that gets held up and used to perpetuate different types of gender roles. Even in something like Tolkien, and I love Tolkien and I grew up loving Tolkien, but I’m reading about him. I’m like this is a story about boys. I don’t wanna to be the elf princess who sits around or the one woman who gets to go and pretend, you know dress up as a man and fight. I want to be Gandalf.

Stephen: Right, right, right.

KellyAnn: Right? So that’s definitely something where medievalism is not going away. And if we just kind of let it happen without being critical of it in any way, shape, or form, there’s a lot of danger there.

Stephen: Yeah and I hadn’t really thought about that. But I think it’s been interesting I guess to see… I think you said this earlier, to essentially turn it on its head in some degrees. Like in other words I’m thinking when you were talking princess and rescued by the prince, it made me think of “Shrek.” Right?

KellyAnn: Mm-hmm.

Stephen: Where you have obviously a comical retelling but a comical, like, “All right, well let’s turn this trope and let’s turn it around.” Actually, the princess is quite capable and doesn’t need to be rescued theoretically and so on. So I think, yeah, it is interesting to think about how medievalism has obviously influenced many of the movies that a lot of us saw as kids in Disney movies. And what that passes on to us in the form of things that we don’t even think about. We just take for granted.

KellyAnn: Yeah.

Stephen: Like, “Oh, a princess in a castle, must be rescued.” Well, why? You know, is that essentially a gender stereotype that we want to persist or is that something that is not helpful that we need to retire?

KellyAnn: Yeah and “Shrek” is a great example, I’m glad you brought it up. Because that’s… In my field, we kind of look at that as a movement in medievalism that gets categorized under the name of neo-medievalism, which is a term that is still in development. But the idea of to take these tropes and turn them on their head and see the idea of the medieval being constantly reinvented and “Shrek” is a good example for it. As you point out, it takes the “rescue the princess” element and turns it on its head.

But probably one of the most pointed ideas in that movie is that Fiona, the princess, she’s patiently waiting in that castle. Because it’s kind of like she’s been told that this is how this needs to be even though she is this very capable princess. She almost goes through it as a ritual and then when she… It’s not gonna work out that way, she actually kind of gets to come into her own.

Stephen: Yeah. Yeah. No. Like I said, I think history is always very useful I think for… you know, as a way of removing ourselves from a situation and allowing to view it critically. Right? Because sometimes obviously it can be difficult to evaluate current culture sort of in a self-critical way. Right?

Because you get caught up in “Oh, I know somebody like that” or “that’s like my parent.” You’re basically too close to it. So by removing sort of… or introducing distance via different time periods, like I said, I think you have the opportunity to use it as a mechanism to shine a light on things that are present day. You know, that maybe, well at least according to narrative hundreds or thousands of years removed from the actual events.

But anyway, okay. So we’re sort of getting close. One question I want to get in before we wrap up here. As somebody who obviously is a gamer today, I wanted to make sure to sort of touch on that at least. Since we’re talking about medievalism and gaming, what do you think the role of gaming is? So we’ve been talking about medievalism as a way of learning about ourselves and learning about different aspect to our society. What role do you see gaming playing either through medievalism or on just its own?

KellyAnn: That’s a big question.

Stephen: It is.

KellyAnn: I think it probably has a big answer. The super short answer is that gaming right now it’s a commodity. Right?

Stephen: Mm-hmm.

KellyAnn: It’s something that is commercialized and for the most part kind of sold and bought. And the industry that drives it is one that is mostly driving for profit. That’s the simple answer. So that the reason we keep seeing medieval style games is because people will pay for them. Right?

That said, I almost suspect that if we stopped making them, people would make their own.

Stephen: Yeah.

KellyAnn: That there is a phenomenon in and of itself separate from this exchange of goods that people would use as a way to kind of create them on their own. And, I think people already do that.

The other answer is the idea of gamifying things is becoming very prolific outside of just the industry of game production. To the point that where any time we’re talking about teaching, we’re talking about how we can work some type of gamification into that or training or anything like that. There are entire fields of study around game theory. So it’s a very big field. And I’m glad about that because it’s, to me, a very interesting one.

Stephen: Yeah. So last question from me then. How do… when you know, people making movies, people making TV shows, people making games that are medieval in nature, how do they… How do you think or how do you know they educate themselves? Like how do they get up to speed? Because a lot of them are just writers. They don’t know anything about the time period. What is the interplay there? In other words, how do experts like yourselves connect with people in those disciplines? And make sure that, all right, you’re not gonna adhere necessarily to the , you’re not gonna replicate the setting exactly. But don’t make obvious mistakes and hey, here are some things you haven’t taken advantage of?

KellyAnn: That’s a good question too. The idea is, is the job of a film to be historically accurate? Especially if it’s based on something that’s like fictional. And the best example I can think of that is like “The Lord of the Rings” and “The Hobbit” movies where it’s not necessarily historical accuracy but attempting to somehow be authentic to Tolkien. But at the same way, translate it into this other type of field. I know that that when those films were made, there were a number of Tolkien experts brought in, probably experts that I don’t… that I can’t even think about. Like how do we make this armor?

Stephen: Yeah, yeah.

KellyAnn: How do we make these weapons? How do we put together a fighting style?

Stephen: Yep.

KellyAnn: So it’s not necessarily just historical accuracy but all these other types of things we tend look at and be like “Is this authentic or is it not?” So I guess it really depends on the purpose of the project and what we consider to be accurate. If we’re thinking of history or if we’re thinking of being close and true to somebody else’s vision or work.

That said, I do know in terms of game development often there’s a role given a name something like the “Lore Master” whose job is to at least make sure that in…

Stephen: Lore Master. That’s awesome!

KellyAnn: I know. I love that. I think that’s what I might want to be when I grow up. But their job is to make sure there is some type of integrity at least within the world they are creating.

Stephen: Sure.

KellyAnn: To have that type of like internal logic be held whether or not it is tied to something on the outside.

Stephen: Yeah. Sure, sure.

KellyAnn: I think that’s what we tend to see. Like we would look at a film or game and be like “This makes no sense” or “This at least has a coherent vision.”

Stephen: Well, that has to be difficult too because I’m sure in many of the fictional worlds there are people who know them a lot better than the writers do. Because they obsess over it in the way that writers… You know, writers have a job of advancing narrative and worrying about characters. But fans can be obsessive in a totally, totally different way.

KellyAnn: Yes. The cliché example of that is at Star Trek Convention like…

Stephen: Yeah.

KellyAnn: Trekkies know more about it than like the actors.

Stephen: That’s right. That’s right. I hadn’t even thought of that. All right. So this has been great. I have one last question for you and that question is what animal are you most frightened of?

KellyAnn: Aside from humans…

Stephen: Aside from humans.

KellyAnn: Because I have to qualify it that way. I think mosquitoes.

Stephen: Mosquitoes? Disease bearers you mean?

KellyAnn: Yeah, disease bearers.

Stephen: Yeah.

KellyAnn: I mean we’ve got malaria pretty much dealt with. We have quinine. We have like tonic and our gin and tonic and everything like that but…

Stephen: True.

KellyAnn: All these other things that mosquitos can spread are kind of frightening.

Stephen: Yeah, the Zika virus is terrifying.

KellyAnn: Yeah.

Stephen: Yeah. Okay. That’s pretty fair. With that, KellyAnn, thank you so much for being on Hark.

KellyAnn: Thank you.

Stephen: Thanks again for listening to Hark. As a reminder, you can find us in Google Play, iTunes, Pocket Cast, and Stitcher. You can also listen directly or find links to all the above by heading over to If you have questions, feedback, or suggestions, you can hit us up on Twitter @harkpodcast or via email at [email protected] We’ll be back next month with episode four and until then, enjoy your time.

Categories: Podcasts.

There and Back Again: The MongoDB Cloud Story


Before it was a database company, MongoDB was a cloud company. Founded in 2007 and originally known as 10gen, the company originally intended to build a Java cloud platform. After building a database it called MongoDB, the company realized that the infrastructure software it had built to support its product was more popular than the product itself, and the PaaS company pivoted to become a database company – eventually taking the obvious step of renaming itself to reflect its new purpose.

For the better part of the last decade, MongoDB has more or less played the part of the traditional on premise software vendor. Its model necessarily differed in important ways from that of its closed source, relational predecessors, but in general the software was bought, sold and used much as Oracle or SQL Server might have been. Like every other open source vendor, it may never have matched its closed source competitors in margin or size, but MongoDB generally prospered, emerging from a very crowded open source, non-relational market as a significant and growing commercial vendor.

The landscape around MongoDB, however, was changing.

Two years after the company formerly known as 10gen was founded, Amazon released the first version of its Relational Database Service (RDS) which supported MySQL databases, as a service. RDS support for Oracle followed two years after that, SQL Server a year later, then Postgres and continuing with MariaDB and the MySQL-compatible but AWS proprietary Aurora engine.

As has become typical, others followed in Amazon’s footsteps. Google, Microsoft and other cloud providers offered similar traditional relational database systems online. Closer to home for MongoDB, several companies began offering commercial support for the Mongo database, but as a service. San Francisco-based mLab is an independent provider of MongoDB-as-a-Service; offered MongoDB-as-a-Service among other offerings, and a year ago this month was acquired by the increasingly services-minded IBM.

All of which fit the model outlined within The Software Paradox, a model in which on premise software would give way at an accelerating rate to cloud based implementations. Databases, in other words, would decline in popularity as database-as-a-service grew. Just this morning, an on premise database vendor who considered Amazon, not Oracle, as its greatest threat, admitted that “40% of his customers were in the cloud, and 80% of those were on AWS.”

On Tuesday, MongoDB announced a new product, Atlas, at its annual user summit. Atlas represents an important new step in the company’s history, as it enters the Database-as-a-Service (DBaaS) market. For the first time since its earliest days, MongoDB is once again a cloud company.

Against the wider market backdrop, it is not only not a surprise that MongoDB has decided to directly enter the DBaaS market, this move has been expected. It could be argued, in fact, that Atlas is overdue.

Of all of the various services offered by cloud providers, databases are among the most sticky. This is in part due to the inherent inertia that comes with large datasets that pose logistical challenges simply to move from one place to another, but also to the risks of changing production database infrastructure.

This stickiness can be a positive for vendors, as it decreases the likelihood of customers exiting the platform. But it also means that unlike other workloads such as compute, customers that are lost today may be lost permanently. While a competitive market exists for on premise MySQL databases, for example, with multiple avenues available for commercial support, a far smaller percentage of workloads that enter RDS or other MySQL-as-a-Service products will ever leave those platforms.

Which means that Atlas couldn’t get here soon enough for MongoDB, because workloads lost to Compose or mLab would be difficult to win back. It also implies that other on premise software companies, and on premise database software companies in particular, would do well to add as-a-service capabilities as quickly as possible.

Services are more convenient than on premise alternatives, and as one well known senior engineer said privately yesterday, “Everyone will pay for compute. It’s less clear that people are willing to pay purely for software value up the stack.” Hence MongoDB following in the footsteps of other on premise providers that have made the journey to the cloud and thus have conflated their product with compute in Atlas.

If you’re an on premise software company, you might think about doing the same. You might not believe in The Software Paradox, but the market certainly appears to.

Disclosure: Amazon, Compose, IBM, Microsoft, MongoDB, and Oracle are all RedMonk customers. Google and mLab are not a RedMonk customers.

Categories: Cloud, Databases.

Why LinkedIn and Microsoft Isn’t Crazy

View post on

As far as $26 billion acquisitions paying nearly a 50% premium go, Microsoft’s intent to buy LinkedIn is proving to be one of the most amusing. While most transactions of this magnitude have analysts and reporters racing to assess fit and infer intent, this transaction is most notable for its comedic game. Instead of being besieged on Twitter with financial metrics, the announcement has instead unleashed a torrent of pointed commentary on “Joining my professional network,” Clippy or both. Our local Portland Slack even managed to find a Coldplay/U2 connection.

All of which is fair enough, some of which are legitimately funny, but none of which tells us anything about the transaction and whether it is likely to succeed.

For my part, that is a less interesting question than why the transaction occurred. Forecasting how this acquisition will play out is, to some degree, boring. It is true, for example, that the probability of success for major acquisitions in general is low because acquisitions are hard, and as many observers have pointed out, Microsoft’s recent track record here is not particularly encouraging. It’s also true that many of its highest profile failures, such as Nokia, were transacted before Nadella’s tenure and, it’s said, against his wishes.

The short answer to whether or not Microsoft will eventually be writing down LinkedIn a few years from now the way it has Nokia then is: no one knows.

What we do know, however, is that this acquisition is interesting both in terms of what is being acquired and the valuation Microsoft attached to it, and that the market didn’t seem to appreciate either.

Let’s start with what Microsoft acquired. For all of the focus on LinkedIn the product, it is unfortunately being lost that Microsoft stands to acquire a capable set of technologists. Whatever one might think of the LinkedIn UI – and Paul Ford is not the only one who believes it’s a tire fire – LinkedIn’s infrastructure engineering and data science teams are both smart and capable. Many of the engineers currently lampooning LinkedIn that are happily relying on Kafka, as but one example, typically don’t realize that the latter was built and released as open source by the former. According to its engineering page, LinkedIn is responsible for 75 open source projects across a wide range of categories. Ten years ago at Microsoft, this would not have been an asset. Today, with Microsoft emerging as a surprising champion for open source projects in the cloud and releasing its own software as open source, LinkedIn’s engineering team and their experience with open source are useful.

Not $26 billion useful, of course. The bulk of the valuation depends on LinkedIn the product. Which is another way of saying that Microsoft spent took on debt equivalent to a quarter of its available cash and short term investments on data. Which only makes sense if you value data, obviously, but more importantly have a clear idea of how to use it in meaningful ways to generate revenue.

Queue the Clippy jokes.

But the truth is that Microsoft has acquired a unique asset in LinkedIn, the company’s financials notwithstanding. There’s no other comparable dataset that contains a reasonably up-to-date employment history and professional network for a large portion of the working world. That information has value to any third party hungry for personal information, from Amazon to Facebook to Google to IBM, but it’s potentially more leverageable for Microsoft, which has spent the better part of the last three decades building out an ever more comprehensive and ubiquitous suite of tools for office workers via the creatively named “Microsoft Office.”

As an exercise, what would you do if you first were in Nadella’s shoes and second believed the Software Paradox to be a real phenomenon. One answer would be to begin acquiring unique, exclusive datasets.

First, because they can serve as the foundation for a variety of AI and machine learning products and services. Second, because data – unlike software – is ground that cannot be made up easily. LinkedIn’s software could be replicated, and perhaps improved upon, within a reasonable timeframe by at least half a dozen different large scale internet vendors. The corpus of data it accumulated from its users could not be replicated in twice as much time and at a large multiple of the cost. Hence, presumably, Microsoft’s valuation. Lastly, Microsoft presumably has attached value to denying the LinkedIn dataset to one or more competitors. There’s no guarantee that Google, as one example, would have been able to leverage the LinkedIn dataset into more robust and information rich business services – but now Microsoft doesn’t have to wonder.

The opportunity costs to an acquisition of this size and type are considerable, of course. Acquisitions also being inherently problematic, there is also a reasonable risk of failure. But those focusing on existing LinkedIn or Microsoft services and attempting to evaluate this transaction through that lens are missing the wider market context.

If this were a decade ago, or half that even, the Redmond giant’s core answer to strategic questions – any of them – would be software. Nadella appears to have internalized the reality, however, which is that software is not the commercial engine it once was. Microsoft’s past and present with Office and Windows was and is selling standalone assets, typical of traditional shrinkwrapped software. Increasingly, it’s harder to sell software for the returns that once were common – even for Microsoft.

But if standalone software isn’t where commercial value is to be found, what’s the alternative?

Consider the current state of the market. Broadly speaking, there is widespread agreement that the focus for both buyer and seller moving forward is in two major areas: cloud and mobile. Enterprises are moving to the cloud and cloud based services at incredible rates, and the story of how mobile annihilated the traditional PC market is well understood at this point. The good news for Microsoft is that it’s performing reasonably well in the cloud market at present. The bad news is that it was too little and too late in mobile, so the company has largely punted on the market for the time being.

The better news is that Microsoft has an opportunity to transcend both markets by combining hardware and software. For the enterprise buyer, could Microsoft use LinkedIn’s dataset to add predictive analytics around retention to its CRM offering, for example, or make this available as a third party service? This seems plausible. Could the company use LinkedIn data about your employer and teammates to improve mobile sharing, say, for individual users? Certainly. Is Microsoft one of the companies with the software and hardware resources required to build and perhaps even sell a bot that would answer, on demand, questions about a given employee’s work history and professional skills?

“Cortana: Where has Jane Smith worked?”
Jane Smith has spent ten years as a Java programmer for companies in the insurance and healthcare industries.”
“Cortana: What are Jane’s professional certifications or memberships?”
Jane is an Oracle Certified Master, Java Enterprise Architect, a certification that less than 5% of the applications for this position claim.”
“Cortana: What languages does Jane Smith speak?”
Jane is fluent in English, French and Spanish.”
“Thank you, Cortana. Can you schedule an interview with Jane next week?”
Of course.”

As long as they keep Tay away from LinkedIn related offerings, this does not seem like too much of a stretch.

Which makes the most important questions around this acquisition ones of execution, it would seem, more than valuation. Given the price tag and visibility of this acquisition, LinkedIn will be an interesting test of the new, Nadella-led Microsoft. But whatever the outcome, the process that led to the acquisition is nowhere near as crazy as the Clippy asking you to join his professional network-GIFs would suggest.

Disclosure: Amazon, IBM and Microsoft are RedMonk customers. Facebook, Google and LinkedIn are not currently RedMonk customers.

Categories: AI, M&A Announcements.