OpenStack – an open source cloud platform

Rackspace announced the OpenStack project today, open sourcing much of the software it uses to run its own cloud. I spoke with Rackspace’s Jonathan Bryce on the topic to get an in-depth overview, discuss Rackspace’s intentions, and explore the operational future of OpenStack.

This is a big announcement in the cloud world, further widening the technologies that are available to start crafting public and private clouds. The nature of Rackspace as not a software company is also interesting to watch here, as well as what partners do with the project.

Transcript

Michael Coté: Well, hello everybody, I’m here in the Austin Rackspace offices in the Austin City Limits conference room as you can see. It’s, perhaps, one of the more three dimensional conference rooms I’ve ever been in. It’s very exciting and this is always Michael Coté from RedMonk and I’m joined by a guest. Would you like to introduce yourself?

Jonathan Bryce: Sure. I am Jonathan Bryce. I am the Co-Founder of the Rackspace Cloud and I’m currently leading up the technology-end of a new venture that we are just getting started.

Rackspace has been doing hosting, managed hosting, dedicated hosting for about a decade and a few years ago we started a cloud initiative to do virtual servers, cloud storage, Platform as a Service and the big news that we’ve just released is that we’re actually going to be open sourcing most of the software that runs our cloud system. Specifically, our cloud servers’ product line, our cloud files product line.

We’re going to be giving all of the source code away and opening it up under a new organization that’s called OpenStack.

Michael Coté: What does that mean, for “cloud software” to go Open Source? Do you think — I mean to the — you were, kind of, alluding to this a little bit, but a lot of what a cloud is, is the hardware and everything that is running. So, what’s the part that’s the software you can Open Source?

Jonathan Bryce: Well, most of the clouds’ out there – public, private, enterprise, you know, whatever – under the hood it’s built with a lot of Open Source components like Linux, KVM, or Xen or different Open Source hypervisors. But what the missing piece for a lot of that is an orchestrator, a controller that can make tens of thousands of physical devices, the network, all of the different components work together so that you can provision and manage virtual servers in multiple locations across all of this hardware.

So we’ve built a system that does that. Our cloud is very large. It has hundreds of thousands of cores. Our storage system has billions of files, petabytes of data, and there’s a lot of software that glues it altogether and makes all of those other technologies work.

Up to this point, there have been a few Open Source projects that do similar things, but all of the ones that have really been at scale have been proprietary systems that are running in other people’s datacenters like ours and Amazon’s and Google’s and so this, I think, the big announcement and the big change for the industry is that we’re going to have what’s kind of a carrier grade controller for clouds that is, now, going to be available and it’s going to be open and it’s going to be contributed to from a lot of different players, not just us.

Michael Coté: To that point of it being carrier grade, which is, it’s kind of fun, it’s sort of like in the — outside of the enterprise datacenter, instead of using the world “enterprise” you always use “carrier grade” to mean, like, hardcore real stuff, right? But to that point, how long have you guys been using this software internally? What’s the maturity of the Open Source cloud stack?

Jonathan Bryce: The software that we are running has been in development internally for about four years. It’s been running in production for almost that whole time. We have tens of thousands of companies from all over the world that are using it in our environment. So it is something that — it’s been battle tested and it has scaled and I’ve seen a lot of demand and a lot of success in the deployments that we’ve already done with it.

So this is not, kind of, just a think layer of virtualization control that’s meant for a test lab of ten servers or 20 servers. It really is meant for that, kind of, large provider type scale.

Michael Coté: It’s not for like your Beowulf cluster in the closet?

Jonathan Bryce: Right.

Michael Coté: I mean you guys are not really a software company necessarily. So what’s like the motivation for — I mean this is a very software company, sort of, thing to do, to Open Source something. So, why are you guys doing this?

Jonathan Bryce: Well, a big a reason of why we are doing this is because we’re not a software company. We believe that the best technologies out there in the last decade have been driven forward by Open Source, whether they were Open Source systems themselves or whether Open Source provided a real competitor to an existing entrenched closed source player. Open Source has really propelled innovation and what we see, right now, in the cloud is that it’s a huge opportunity, it’s a huge market shift, but it’s being held back a little bit by the fact that a lot of the cloud technology out there is proprietary.

It’s either closed source, commercial, very expensive, or it is only run in a provider’s datacenter. We looked around and we didn’t see anything that met our needs and thought here’s an opportunity for us to really help push this forward, to get a lot of people involved, to use our scale and success to accelerate the adoption of the cloud technology.

Some companies who — you know, you mentioned software companies that Open Source some of their products, sometimes that happens when those products are maybe on the decline or if they need a marketing boost to generate interest and there’s always a little bit of a conflict of interest there and as a software company how do I give away my software and then I also make money off of it? Lots of people have done it and made a lot of money.

For us, though, it’s actually a much simpler decision, because, for us, the software is a piece of the overall service that we deliver. But really what Rackspace is about is operating software at scale, doing it really well, really efficiently, really reliably and then offering great support and great — just an overall awesome experience on top of it.

So, the software, to us, is not a real advantage competitively. We built our company using Linux, using MySQL, and Postgres, and JBoss, and Apache and all of these freely available systems. What we did that set us apart from the competition is we delivered it in a way that it was just a superior experience. When we look at the cloud, what we see is a lot of competing proprietary systems and there isn’t really a true competitive market.

We want to help move the cloud space to that. Where you can compete on support and a premium experience or you can compete on cost or you can compete on operational efficiency or you can integrate it in to your enterprise application, your ERP/CRM system, or whatever needs to be, but really move it to a true competitive market.

Michael Coté: When you open source something there’s lots — hopefully there’s other people who are working with you and collaborating. Like who are some of these other companies that are interested in working with you or partnering?

Jonathan Bryce: In our announcements, we’ve talked about a number of them. Companies like Dell and Citrix. A bigger partner who has really kind of been a surprise to us, but has been an amazing piece of this is actually NASA.

When we started to think about this, we went and we looked at some of the Open Source — actually all of the Open Source options out there to see if any of them would meet our needs and none of them would meet our specific needs. We have a specific set of needs, because of the scale and the type of offering that we run on top of it and so we thought okay, well this is it, we’re going to Open Source our code and really take this on ourselves.

A few weeks before we were getting ready to do all of this, the software that powers NASA’s Nebula Cloud was actually Open Sourced and we saw it, and we looked at it, and we said, “Wow, this is really awesome.” We always said that if you could find something better then we’d love to partner up and work together with that and so we’ve been able to do that and it puts us — actually it gives us a head start, it pushes us forward and helps advance the whole thing and I think it really shows the power of Open Source.

Because immediately NASA has access to more technology that we have, our storage system for instance that they were missing before. We have access to technology that they’ve been running inside of the federal government and so this is really the value and the promise of Open Source.

There are a lot of other players who have been our partners for a long time and who are well known in the cloud space, companies like RightScale, and Cloudkick, JungleDisk. Other companies who have built on top of clouds that are these proprietary public clouds, this is great for them and they’re all excited about it because now it opens up a whole new market for enterprises who are going to be running this in their own datacenters, for other private cloud installations, and I think it really is just going to be an awesome opportunity for the industry as a whole.

Michael Coté: Do you anticipate that people will try to use the software to compete directly with you guys running public clouds?

Jonathan Bryce: Absolutely! Some of the other people who have already been involved in this are our direct competitors and we all see that cloud is a shift that’s going to happen and instead of beating each other up in the first few years as the technology gets ironed out, let’s work out this technology together and compete the way that we always have, which is with standard x86 hardware stacks and Open Source software stacks so that we can all have a much bigger pie to eat from, so to speak.

Michael Coté: That’s right.

Jonathan Bryce: The pie analogy is always a funny one to me, but a much bigger pie out there of customers who want this type of technology.

Michael Coté: You’ve mentioned several times that the differentiation you guys will have is basically the service you provide and, kind of, the whole package of things. So, whenever a company Open Sources there’s always the cynical people looking for the part that’s closed sourced or whatever. And nowadays this is an “open core” model essentially where you’ve got an Open Source stack of software and then there is a set of software that’s usually plug-ins or extension that are kept closed source and then that’s the part that’s monetized, if you will. Are you guys using an open core model or what does it look like?

Jonathan Bryce: No, we want a system that is truly open. And one of the reasons is cloud is all about scale and a lot of the open core software components or products, projects, whatever you want to call them, they follow this open core model where you can run a basic version of the system and then when you want to do something that’s really heavy duty then you pay for the extra features that let you do that.

Well, from my point of view the reason that you do cloud in the first place is because you have some heavy duty needs. Cloud is all about scale, it’s all about giving people access to computing scale that they just didn’t have access to before.

So an open core model for a cloud system doesn’t make a whole lot of sense to me to have a product that is trimmed down and works on a small set of servers or on a small number of clients, whatever it may be and then you pay to get to the full scale, it just doesn’t seem to apply to cloud. So that’s not the model that we’re going after.

We will monetize this by running it in a hosted model, so, by operating it and running it reliably and efficiently for our customers. And I think also we will be providing commercial support for this. This is something that fits right in with fanatical support and the way that we’ve supported other Open Source products for years and years and years and now that we are developing it, it makes even more sense for us to do support on top of it.

But the core software that you need to run this and to run it at scale, it’s going to be what we are running for our hosted versions and it’s all going to be out there. With the exception of integration points, I shouldn’t say it’s all going to be out there, because we have to tie into our billing system, our authentication system, those hooks are really the only things that we’re not open sourcing from the product.

Michael Coté: So, then the other question, I mean, especially when you are open sourcing something is, I guess it’s the whole area of where is it going to live. Like where are you going to host it and what site is it going to be at, where would people go to get it and everything?

Jonathan Bryce: Both of these initial projects are going to be under an umbrella organization that is called OpenStack and so OpenStack will live on the internet at OpenStack.org.

You can go there and get some basic information like now as the project mature and move along. It’s going to continue to fill out that website. But the source code and the project management is going to take place on Launchpad.

Launchpad is something that we chose, because it gives us some extra capabilities over something like maybe GitHub that people are probably really familiar with. But it gives us the ability to do very complicated planning and long-term product management kind of things.

So lifecycles and in addition to just basic bug requests and it also is built around managing a really large community.

Michael Coté: Another thing that you mentioned briefly was private cloud users of OpenStack, and what do you anticipate the opportunity there is going to be?

Jonathan Bryce: When we talk to larger companies about using our cloud, they are kind of like, “Whoa! We’d like to really see it, really set it up, and test it. We are a little concerned about security, sometimes a lot concerned about security, we’d like to run it inside of our datacenter, we’d like to run in your datacenter, but behind our Firewalls.”

All of these kinds of concerns around lock-in and security come up when we talk to these big enterprises. To this point we’ve had an offering which is a private cloud based on VMware at Rackspace. But we have companies who want something that is more of a commodity cloud. They’re doing things where they don’t need all of the enterprise benefits of VMware and they don’t want to pay the enterprise premium for that and so, again, this is another one of those things where the market is going to be deciding how they want their cloud.

So, this gives this, now, an option where when those customers say that we go great, we have been leading this OpenStack project and you go access the compute and the storage system, you can run them. You can get support from us on them and they can set it so that they can have really truly commodity elastic clouds internally in their datacenters or in our datacenters or in a competitor’s datacenter or wherever they need to set it up for their business requirements.

Michael Coté: So, we’re here at the Rackspace Austin office and you guys have been here for a while as you were reminding me when you were giving me your tour and it was funny, we were going on a tour and [you said] like “oh, this used to be your workout space there.” I mean you guys have expanded so much that it’s just cubicles and everything, but you guys are based in San Antonio and you guys — was it a couple of years ago now you IPO’ed or I mean some —

Jonathan Bryce: It was August of 2008.

Michael Coté: Right and so — I mean since then you guys have been expanding quite a lot.

Jonathan Bryce: Yeah, well we have. Obviously, our headquarters is in San Antonio. We’ve had this Austin satellite office for a while that we’ve continued to grow and it’s many times larger than it was now when we started just a few years ago.

We also have staff in datacenter locations and various places – Dallas, Chicago, Virginia, Hong Kong. And then we have another office that’s staffed pretty well over in the U.K. that has kind of all the functions of the business in it.

One of the things that we really want to do is find good talent and so we’ve become a little bit more flexible in where we look for that good talent. But also I think that all of this OpenStack stuff it opens up new opportunities because we’re hiring. We’re hiring people to come work full-time on what we think is going to be the ubiquitous cloud computing system and it’s an exciting chance to get to work on code like this.

So, we’ve got openings to work on it in OpenStack and our hosted cloud offerings within Rackspace and we need developers and we need marketers and we need just good people everywhere, we want good Rackers. So, yeah, I mean I think that we have a lot of positions open and we’re continuing to hire and expand in different places.

Michael Coté: Over the years mostly bumping into people at South by Southwest as your shirt attest to – it’s been interesting to see what was otherwise a very, you know a normal traditional hosting company like really get — for lack of a less corny word get hip to Open Source and start to drink the Kool-Aid with it.

Anyway, it’d be fun to see how the OpenStack stuff plays out and I appreciate you talking all this time to go over with us.

Jonathan Bryce: Yeah! Thanks for talking with me.

Disclosure: Rackspace is a client and sponsored this video.