We get all dev/ops crazy when Andrew Shafer joins John and I for this episode of the IT Management & Cloud podcast, #81.
Download the episode directly right here, subscribe to the feed in iTunes or other podcatcher to have episodes downloaded automatically, or just click play below to listen to it right here:
Show Notes
- Andrew at CloudScaling.
- We talk about telcos and service providers wanting cloud stuff. As a quick example, see CloudScaling’s work with KT.
- What are telcos looking for from cloud?
- Andrew: “we need to automate the bottom.” What do the “commodity boxes” look like, though?
- James Hamilton talk at Velocity (video or audio only) goes over how he handles this stuff.
- Optimizing to the meat-level – “it’s all about the math.”
- Andrew: if you want to do cloud, you need commodity hardware, open source [cloud] software, and a WebOps culture. What’s the software used here.
- Why is open source so important here? Flexibility, adapting to change – having to re-write your stack as things scale.
- John’s been going to a lot of dev/ops days – how’ve those been going? Check out the round-up of videos from devops days US from Damon Edwards..
- Web Operations book, mentioned by John several times.
Transcript
As usual with these un-sponsored episodes, I haven’t spent time to clean up the transcript. If you see us saying something crazy, check the original audio first. There are time-codes where there were transcription problems.
Michael Coté: Well, hello everybody it’s the 4th of October 2010 and this is IT Management & Cloud Podcast, number 81. While John was away I slipped in a number 80 with a special guest. So, now we don’t have to worry about the awesomeness of 20 X 4 as it were.
So with that, silliness aside, this is one of your co-host as always Michael Coté available at peopleoverprocess.com.
Andrew Shafer: I think the proper nomenclature is for Four Score Cote.
Michael Coté: Oh, Four Score that’s right. And as you can tell, we’re joined by a guest, old long time listeners may recognize his voice, but do you want to introduce yourself before the other co-host?
Andrew Shafer: I suppose. This is Andrew Shafer and I have been on this show. I missed the transition, when did you add & Cloud to the podcast?
Michael Coté: Well, it just seemed appropriate, because we spend a lot of time talking about that.
Andrew Shafer: Because IT Management & Cloud, those are way different topics.
Michael Coté: Totally. Well, you know what we’re going to get into some of that, but first, let’s introduce the co-host.
John Willis: The unannounced co-host guest disagrees. My name is John Willis at [email protected] and can be found also and at [email protected].
Andrew Shafer: Can we make the IT Management & Cloud Comedy Hour?
Michael Coté: That’s a lot of pressure to be funny.
Andrew Shafer: Yeah, it’s better not to say you are funny people, think you are funny, but —
Michael Coté: It’s all, you know, that there is —
Andrew Shafer: We will go understated.
Michael Coté: You know, while we’re on the meta conversation here, there is a lot of lessons I’ve learned in doing podcasts over the years. And the number one lesson is, never put anything that implies regularity in the title of your podcast, like weekly or daily or something like that.
And I think, another one — it’s a little further down the list, but you definitely don’t want to have a conceit that you are funny, because then you have to try to be funny. I think that’s a good one John.
John Willis: Well, are we funny? I didn’t even think we were funny.
Michael Coté: No, no. This is a very serious podcast dealing with things like —
John Willis: From now on, we’re very serious, sorry.
Michael Coté: You know, it’s dealing with things like ITIL and the process and basically what we’re dealing with here John and Andrew is aligning IT with business and there is nothing funny about that.
John Willis: Well Cloud is pretty serious stuff, and particularly when you marry it with IT Management.
Michael Coté: Exactly I mean —
John Willis: It’s not a laughing matter, it’s not a laughing matter.
Michael Coté: So speaking of things that aren’t a laughing matter. Andrew, so you know, I think since last time we — I don’t know talked on some recordings and you’ve been working at a new place Cloudscaling, you want to just give us a quick overview of what’s going on there?
Andrew Shafer: Cloudscaling is helping service providers and Telco’s transition to the cloud service model for their businesses.
Michael Coté: And so you know, how my predilection is to always get like the most basic things to find. And so, like I think — well I don’t know like, tell us what exactly like, what are Telco’s and service providers? What does that mean in this context exactly?
Andrew Shafer: Well Telco’s, we can pull up the Wikipedia article, but you basically have a bunch of companies that have businesses built around providing landlines, mobile, these networks for the 3G, 4G, all these services that have been provided for the last — I don’t call that space as much as I probably should now, but they’ve been packaging, hosting, you know, every little Telco has a web hosting package, I think. I mean when I usually see them, I don’t consider them for my personal business, but you see lots of these all over the place.
John Willis: Well Andrew, could you explain for our listeners just since we’re doing this what 3G and 4G means as well?
Andrew Shafer: When you look at your iPhone and it says edge you’re doing it wrong. Okay, there you go. Sorry.
Michael Coté: I think to pull out the sometimes less than obvious point of Telco’s is that, there is a kind of, there is all the consumer based Telco stuff. I mean everyone knows about AT&T. So I think, the somewhat less than obvious point of Telco’s, especially in the context of cloud stuff. Like you were saying, they all have like websites and they’re all like doing a lot of — providing services to business and stuff.
So, most people of course encounter Telco’s on the consumer end when they’re using their Internet and their voice connection and all that crap, but there’s a whole bunch of — they make plenty of money just providing Internet and data and voice and all sorts of other stuff to businesses and then more importantly running web stuff for them.
So, there’s a lot of stuff going on, the service provider or a managed service provider or a Telco or a carrier, however you want to say it, where they’re basically trying to kind of scrambling to get the software that allows them and basically get their own clouds, so that they can continue providing these services rather than that stuff going off to Amazon or Rackspace or other people.
Andrew Shafer: So, if we back up and just sort of think about this space and the evolution and you guys have been watching this real time for the last few years. There is a class of businesses and services that are being for lack of a better word, is the word I would use is, disrupted. And people that are in those businesses are highly motivated to find some other way to provide value, to create value.
Michael Coté: Yeah, definitely. And anecdotally there are several cloud people that I’ve been talking with recently and they’re seeing a lot of — they’re getting a lot of sign ups through Telco’s if you will.
So there’s a lot of interesting stuff going on there and one of the funny things that I think is, kind of, I guess cloud counterintuitive about it is, originally there was this idea there is one, maybe two or three clouds or whatever, but if you think about what’s going on here, all of the existing Telco’s are trying to make sure that there is hundreds if not thousands of different clouds out there, which is kind of a subtle, but big shift of kind original cloud thing, but we’ll, I don’t know we’ll see how this stuff sorts out, it will be — for people who get excited about the words like Telco’s and service providers it’ll be interesting to see how that disruption plays out over the next few years so.
John Willis: Well you guys from an age that you might have a perspective on this, but imagine if there was only one telephone company.
Michael Coté: Yeah. There used to be a long time ago. I mean you could get like getting a pink phone was considered a feature.
John Willis: Those were good days.
Michael Coté: That’s right, yeah. No, I mean there is a — that is like the idea of monopolistic effects on technology innovation is something that is kind of a bird in the UniCloud theory.
Andrew Shafer: Well, the other thing I think is, what I’ve been seeing from the — and I don’t have as much experience in a Telco, but is that their whole playground is some much huger, right, because of all the web apps that are now mobile apps, right. So like I remember talking to one of the big Telco’s about like, they were just building their clouds so that partners could develop their apps and test them and get them to production, kind of like a pre-built app store, for their particular brand. So I think that even drives more so why they need some type of kind of private cloud infrastructure.
Michael Coté: Yeah and along those lines so like in —
Andrew Shafer: They’re public, I mean really what you’d like as a Telco I think is to be able to package these things that’s the — the Telco space right now, you’re not competing, minutes or minutes, you’re competing on the way you could package things. So if you could put together a package where you could give someone their business, their phones, their Internet connection and also oh, yeah here’s this thing it is sort of like Google App Engine or Heroku or whatever and you can power your mobile apps with that, like that’s a pretty good package or compelling service maybe.
Michael Coté: And like are there other — I mean is that — what are kind of like the requirements you’re seeing Telco’s and service providers have for their cloud software. Like what is it they are looking for? Like obviously, like you just said, there is that ability to package, to bundle things up right. So the technology has to be vulnerable.
Andrew Shafer: So there’s the actual — there’s the story and the narrative and the packages and the pricing and then there is the technology. And I think that there is a lot of people in this space, because cloud is so undefined and people can define it however they like. For example, the Oracle World Keynote, you get up and you say everything, nothing is new, there is no such thing as cloud, oh yeah everything we do is cloud and that’s put people in a position where it’s very difficult for them to be able to make good decision about things.
Michael Coté: Right.
John Willis: And I think right now for the Telco space, the service provider space, there is a little bit of a danger, because — and this also applies to some degree to the private cloud space, but if you think you’re just going to install cloud and then run it in your data center like you do your switches, then you’re setting yourself up for a little bit of a hurt.
Michael Coté: Yeah. No I mean this is like completely like off the top of my head theoretics or whatever right, but I always — I get the sense that it’s almost like running your own private cloud is just as much hassle as running what you already had. And so, like if you’re really going to like run your own — if you’re going to do this big transformation where you’re running your own private cloud, you have to really be certain about what it is you’re getting after all of that trouble.
Andrew Shafer: In some ways it my actually be worse, because you still have all of the overhead of managing the separate services, plus you managed to create yourself a nice little distribute single point of failure if you don’t do it right.
Michael Coté: Yeah. Well do you guys — have you been seeing any exciting Telco stuff John?
John Willis: No, not really. No I mean that’s the high end of our —
Michael Coté: Yeah, I mean and you guys are a little higher up the stack of needs if you will over at Opscode. So probably you spend your time doing things more in the application and like —
John Willis: Yeah, I mean we don’t see more of the media companies or bioinformatics, those types of things. It can be leading edge service oriented service, e-tailer type companies.
Andrew Shafer: So, I think some of this has to do with culture and we can certainly talk more about the impact of that in both directions, but I think it’s a ways off before the Telco’s and we’ll see how it goes and John and those guys can you know, try to prove this wrong, but I think it’s a long ways off before they’re going to be willing to let their infrastructure be managed and configured from something else, either a firewall.
John Willis: Yeah, I mean that’s another lengthy, lengthy discussion. I think one of the maturity things is to happen is, like you know if you think about private cloud, right. So private cloud pretty much — I mean I’m sorry, public cloud people buy into the — very easily buy into the you know, the kind of managing your infrastructure or configuration management from the cloud right, like which is what Opscode does, right.
And because you’re in the cloud already and I think and it’s more then that, the public cloud is more mature right, in terms of its delivery and all that. So, I think what’s happening now is the private cloud is still on that infant stage, right you know in other words, well they’re just trying to get the meat and potato stuff right, you know get instances doled out properly and allocated and managed with the infrastructure hardware.
They are not even ready to discuss how you might mange you know, end-to-end configuration and system integration. I think there is still another year or two of maturity in the private cloud space when they can lift their head up and say yeah, we do the whole provisioning thing really well now, all right. Now let’s look at what we have to do next or to make the higher layers of the stack, wouldn’t you agree Andrew?
Andrew Shafer: Well, I would agree. I think that everything you said is true, but for the stuff we’re building now, we need to automate the lower layers. We need to automate the bottom and cloud, you know, the promise of cloud and the magic of cloud notwithstanding there’s got to be metal at the bottom.
John Willis: Right, no. I think we’re saying the same thing, right. I’m saying there is still a maturity level of just getting the basics of what we would consider a pure cloud delivery.
Andrew Shafer: The majority of the problems that we solve with configuration management now are not on instances learning —
John Willis: Right, that’s what I said.
Andrew Shafer: It’s actual metal — configuring metal machines
John Willis: Yeah.
Michael Coté: And how is that bottom metal layer like working out, like is it — are you able to just like use like commodity boxes or do you need to get specialized boxes or like, you know, and this is another interesting kind of like counterintuitive cloud thing, because you know way back when it was supposed to be like you just use really cheap boxes and they are commodity boxes and you replace them and everything and then you learn that Google has their boxes custom made and people are having shipping containers of boxes.
And it seems like, nowadays like the public cloud storage you hear about, maybe their hardware is cheap, but it’s definitely not like I just go down to Best Buy and buy a bunch of servers, like there’s kind of like specialized boxes that people are getting to operate like this. And so like, what is the metal looking like when you’re building these clouds?
Andrew Shafer: That’s a very interesting question and one we’ve been giving a lot of thought to. Our strategy is to build on commodity, commodity hardware and the question you were just asking; we talk about the way Google and Amazon are building servers or Facebook or whatever, it depends on where you’re thinking the commodity lies.
So they’re putting together specialized for their needs, you know, configurations, opinionated configurations of hardware, at the same time, the building blocks that they’re using, they are not that different from what you would buy and consider commodity, it’s just of question what relationships do you have with your suppliers, with the manufactures and what leverage do you have.
If you go look at some of the stuff, James Hamilton’s talk from Velocity Conference for example. He at this point is analyzing and optimizing for every voltage drop from when power comes into the data center on the macro scale down to that the chips and the boards.
Michael Coté: Right. Yeah, I know. As an analyst, like, nowadays it’s very popular when you go to an Analyst Summit or Conference or whatever, to have the person come, it’s always been a guy I guess. So they always have the guy come out who is like the — who just like bombards you with data center optimization awesomeness. And there’s always lots of things like heat flow and like the environmental conditions and energy things like that.
And so, you know, I’ve seen this talk from Microsoft folks and IBM folks and pretty much all of the big industry people trot out the — we have that the data center guy sort of talk, which is kind of interesting to — I don’t know I mean it’s get back to this point of like, a lot of — to use one of my favorite old quotes like a lot of effort to went into making this effortless. And it doesn’t seem like it’s always like such a snap of an easy thing to do.
Andrew Shafer: So that brings up an interesting point. Some of the things you mentioned, the difference between Microsoft trotting out the data center guy versus Amazon, is Amazon runs a service and Microsoft runs some services too, but they’re not necessarily optimizing the same way James Hamilton is. Because he is looking at his own bottom line in optimizing for his service, where Microsoft has it’s — I can’t remember the quote, I’m sure I’m going to butcher it, but James said something to the effect of if you don’t know something exists and you’re not measuring it, then there is no motive for the manufactures to optimize that anymore than kind of like the crappiest level that they can to get by. So, he’s talking about in that comment I think he was making, talking about the power supplies sort of at the board and server level.
Michael Coté: Right. And is that, I mean that’s another question for you guys. I mean do you see a lot of people obsessing about power usage and things like that, because I certainly like — I certainly hear vendors claiming that everyone is obsessed with like power consumption and things like that, but I don’t always hear a tremendous amount and it’s probably just because I don’t listen to the right places but, on this topic, but I don’t know always hear a tremendous amount of people like you know saying that’s top of their mind or things they’re worried about.
Andrew Shafer: Well, we’ll just keep talking and I don’t pretend to have the perspective on this that someone like James Hamilton does, but at the end of the day he pretty convincingly proved that the big cost is actually servers, it’s actually the hardware and the power, once you get all this stuff setup on a day-to-day operating cost, that’s a smaller percentage. The servers are more like 50% to 60% of your cost and power is more like 20.
Michael Coté: Right.
Andrew Shafer: So while you — go ahead.
John Willis: No, I recently saw that I think it was probably was another survey not from him, but it was the same thing where the — it’s almost exactly what you just said, less than 20% release power and more like in the 30’s or 40’s as servers.
Andrew Shafer: I think the reason that people have the perception of power as the biggest cost is, because their server is already some cost and they’re looking at their monthly bill to pay the power, but if you take the server purchases and amortize them properly and do the actual accounting, then the server costs are significant.
Michael Coté: You know, maybe it’s also like virtualization and consolidation where — and you were kind of getting at this right, like if you don’t measure something vendors aren’t going to optimize it or no one will optimize it, because you know if you’re wasting power in the forest and no one knows then you’re not really, not wasting power.
Andrew Shafer: Well, the only other thing is — the only thing if you look at like an IBM’s perspective, right where they try to sell products to monitor power usage, you know they might be tool schools of implementation right. The James Hamilton school of implementation is, we run overall 80% of — you know we run 80% utility on the power and 20% SAP, whereas I think that a company that hasn’t address their power concerns could be just the opposite. So I don’t know if that’s in the play too, right.
Michael Coté: Yeah I mean and I think that’s what I was getting to, it’s like it seems like, like I was in Raleigh in one of the IBM cloud locations or whatever. They’ve got you know lots of crazy gold lead certified data centers and all this stuff you know, this was another instance where they trotted out, it was the data center guys in this case, you know they’ve got the heat maps and there’s all of these words I could spin out that I really know what they mean, like there are crack units for cooling air and all sorts of stuff and it seemed like basically what you do, is there is a bunch of wasteful dumb stuff that you’re doing with energy and then you kind of fix it.
And then to Andrew’s point, it’s kind of like that cost is controlled and there’s a just lot of waste that you can take care of. And so, to the point that once you solve that waste problem just like once you consolidate your servers with virtualization then, then there is other costs that become expensive, but it does seem like there’s just a bunch of waste happening to say that again.
Adam Shafer: So that’s a nice segue into another interesting point that I already — I pointed out in previous conversations maybe not with you guys, but certainly on Cloudscaling’s blog. When you look at James Hamilton’s analysis of cost, he has no mention whatsoever of the human cost, the headcount and I think that for the scale that he is familiar with and what he is dealing with then that cost becomes fairly close to negligible, but for most of these organizations they don’t have the same kind of WebOps culture, they don’t have the same kind of approach to automation that Amazon does and the headcount costs are actually significant.
And I think if you look at the cloud story and you really want to put this in perspective, that the advantage that Amazon has in this game, is not that they’ve figured out how to put a web service in front of a hypervisor, that’s trivial. Amazon’s advantage is that they have over a decade of experience managing a massive web infrastructure.
John Willis: Absolutely, totally, yeah.
Michael Coté: Yeah, it would also be, along those lines it would be interesting to compare the like, the total overhead and cost for an average employee between like Amazon and various Telco’s, right, because I mean not to get all like this guy, but like it would seem like with a lot of Telco’s they also have like pensions and benefits and all sorts of like historic things to deal with their employees where I would wager at Amazon.
They probably love that 401(k) business where you kind of like are cutting out a bunch of stuff that you have to pay for people and it seems like, I think coupled with the fact that they’re not automating stuff, so that older companies probably have a lot more employees. It seems like your cost do become — the human side becomes a pretty big cost and as a human myself, I always bemoan you know cutting out humans, but it does seem like that’s the kind of competitive environment you are in.
Adam Shafer: Well, I’m not — I think it a lot it has to do with the time when those companies were born, right. You try to optimize given the conditions that you have and certainly the notion, and I don’t want to get too financial or clerical here, but there has certainly been a transformation in the way that things like pensions and the way that employee corporation relationships have evolved in the last 20 years.
Michael Coté: Yeah. No I mean that kind of stuff is like the ultimate leaky abstraction that drives how much you have to charge for things and therefore like the availability of the technology that you can provide and it’s kind of interesting to track down to that level of things.
Andrew Shafer: Well, what’s interesting too, I mean last week I was in Seattle and I went over to Amazon had an open house at their new office or whatever and 00:23:18 gave us kind of history of Amazon, the business not just the cloud, but included it, it’s interesting because — you know I’ve got that Cambrian explosion presentation I gave where I talk about how Google kind of came to realization that they were, you know that where they were like when they were doing 15 million Euros a day and they saw the growth pattern and they knew that they had to drastically change the game, right. It just — the math didn’t work out the way they — you know there was no mainframe or super computer big enough to handle where they were going, so they basically kind of punted and went with, distributed CAP Theorem all the things we’ve seen from Google File System and Big Data or Big Table and I think Amazon —
John Willis: It’s all about the math.
Andrew Shafer: Yeah it is. Well I think that was the same things that, when you talk about Amazon doing this for ten years, you talked about kind of four phases of Amazon and first phase was a very kind of coupled stack, web database and then second phase was a service oriented architecture and then their third phase was WebOps, Webscale, what did call the Webscale infrastructure and now they are Cloudscale infrastructure, right.
So, I think that like at certain points when he was giving that representation, he didn’t say they used the math, but it was, you know they looked and they said. “gosh where we’re going, we’re going to have to do something completely different”. And they are the ones that kind of took a lot of the Google ideas with the Dynamo White Paper and all that and you kind of further push that eventual consistency of all the CAP not to get too nutty technical, not that I am even that nutty technical, but what they did is, you know if you look at Google had to change the game to deal with the math, Amazon changed the game to deal the math, you know what I mean, or drove that one step further and I think that —
John Willis: You choice is either to do that or to get crushed?
Andrew Shafer: Yeah and I think that’s the — I mean that’s the reality like again I think that’s every company now and like you’re saying the Telco’s right, in their own way they are doing the math, right and we’ve all had this conservation offline a hundred times. I say simply that there is ships sailing to the left and there are ships sailing to the right, which ship is your company on, right. And you better deal with the reality — you know you better do the math, how are you going to be able to do the kind of growth that your competitors may or may not be able to do?
John Willis: So to that ,point if you want be building a public cloud infrastructure and be competitive as a service provider, if you’re not going to build on commodity hardware, open source software and have a deep commitment to WebOps in your culture, then you might as well just fold your hand.
Andrew Shafer: I agree. I couldn’t agree with you more sir.
Michael Coté: And on that point like we talked about the commodity boxes like a little bit, but like what is the open source software that you’re looking at and seeing used, like what fills that slot, that bucket?
Andrew Shafer: Well, if you start at the bottom and you think about the evolution of the cloud again, going back to the story of the cloud, which I think we all sort of agree starts with Amazon —
John Willis: The story of the cloud.
Andrew Shafer: Yeah, there would be no cloud, there would be no cloud computing if it wasn’t for Linux. If it wasn’t for Linux, if it wasn’t for open source, hypervisors all this kind of stuff they would not be easy too. And then you just have complexity moving up the stack, right you look at all the — what is the — I mean outside of Microsoft and if you went to Velocity two years ago. There was a guy giving a conversation, I think I actually mentioned this on a podcast with you guys before, there was a guy giving a lecture on doing profiling on the .Net stack and he asked everyone in the room if they are running .Net and only one —
Michael Coté: Oh, yeah that’s right.
Andrew Shafer: And only one guy raised his hand. So up and down the stack, if you are looking at PHP, Java, Ruby, Python, all these things are open source. All these things, everything you need to build whatever you want at the bottom is open source software and then the complexity of managing, doing system management there is Puppet, there is Shaft, there is CF Engine all those, that’s open source, that’s basically an open source lineage of configuration management frameworks. There is monitoring, monitoring is the kind of bastard stepchild of web operations, but who is running 00:28:03 Open Source software.
John Willis: Is it monitoring management, Andrew?
Andrew Shafer: We’ve been through this, monitoring is not management.
John Willis: Couldn’t resist. Well it’s interesting this might be a little be segue, I’ve been kind of fighting my way through the – either fighting my way, I just don’t see that a whole lot of time, but I’ve been reading the WebOps kind of chapter by chapter and I just read the Eric Ries Continuous Deployment or Continuous Delivery chapter, that’s pretty freaking fascinating that whole process, so —
Michael Coté: Well, let me ask you —
Andrew Shafer: Actually let me — I want to continue the line of thought I had a just minute ago about open source.
Michael Coté: Yeah, well let me ask you along that line, like is it the fact that it’s open source that makes that stuff work or it is just the software happens to be open source and the software is what you want. Like, I mean you see what I’m saying, like does it matter that it’s open source or like what or is it feature side of the software that happens to be open source?
Andrew Shafer: I think it really, really that it’s open source.
Michael Coté: And so why is that?
Andrew Shafer: Well, that’s the line I want to pursue, I think not only does it have to be open source, there’s a spectrum of openness with respect to how the different projects interact with their potential community, with respect to how people can get patches accepted upstream, with respect, if you look at all the different Linux distributions, they all have slightly different motives and they have slightly different mechanisms for how they handle this.
And if you look at the evolution right now there is just — I mean without being too bias, there is a big difference between a open source project that you know if you submit a reasonable patch it will get accepted and that you know that you can go to the mailing list and you know you can go to the IRC and get answers and there is people there that want to see that project succeed and open source where you know, it’s VC funded out the gate and they have the open source, but you can’t really — it’s not really setup to build or accept patches or there is not a community so to speak, it’s what I’ve been — I said this a few times in other contexts, but I think of it as ornamental source where you can sort of see the source and it’s sort of there, but there is not not really the true commitment to that spirit of open source, it’s open in name alone.
Michael Coté: So, if I can pull out what I think you’re saying there, like one of the reasons open source is so important for a cloud stack if you will is that the sort of needs that any given cloud might have are different, not all clouds need the same thing or more importantly, the software you are using is going to have to change per sort of like use of it. So you sort of need a community you can go to that helps support and evolve the software you’re using to fit into what you’re doing and then even —
Andrew Shafer: It goes in both directions, you need a community but you also need to have your own ability to understand what’s going on. If you kind of go back to the story John just told about the transition. If you look at all these web services and cloud is let’s not kid ourselves, cloud computing is basically a big web application.
Every single one of them went through transitions where in order to scale to meet the demand, we basically had to rewrite their whole stack. I can’t think of one example where that’s not true. Amazon, Twitter, Facebook, everyone, Google or whatever, when you get to a certain point that the scale is the limitation and you have to rethink how you approach the problem or you’re not going to be able to provide that service at that scale.
John Willis: I’ll go one step further and I think you know I agree open source, I think it’s in general open right and you could argue the difference between just being open or open sources that I think the one thing we’ve learned over the last three years is if you’re going to be a company that relies on technology which is just about every company that exists now, you need to be nimble and you can’t be restricted in terms of the changes and your ability to change, right.
And so, the technologies that you choose have to be flexible, so you can’t like wait on a vendor, you know what you need to do is you need to adopt a base of technology that doesn’t block you, you know and then there is no guarantees but you need to put yourself, because you don’t know — I mean if you look how much has changed in three years and imagine you know what could happen in the next three to five years.
So I think that part of what you are explaining about the best possible scenario for me if I was going to build a private cloud I’d want to work on commodity hardware, I’d want to make sure the operating system is as open as I can get. I’d want to make sure the stack as I build on top of it, is as open as possible. So it gives me the most flexibility to adapt to change and different things that are you know it’s the unforeseen and the potential growth you know to control my own destiny and if I have to —
Andrew Shafer: I think it’s a question of what you are going to take responsibility for and I think what’s happened in the enterprise culture is they’ve sort of accepted a level of mediocrity and advocated their own responsibility for their infrastructure and for their applications to their vendors. And if you’re going to do that, if you’re willing to do that then you know, by all means go ahead, but if you’re actually going to try to deliver value as a web service, it’s one thing if your business is selling tractors and you’re going to build a private cloud or whatever and you know God bless you, but if you’re going to deliver a cloud service and you think you’re going to go build it with kind of enterprise mindset, then you first of all, do the math, you are not going to make any money. And second of all, you’re in for a lot of heartache.
Michael Coté: A nice tractor ride.
Andrew Shafer: Yeah, and the tractors are awesome by the way.
Michael Coté: Yeah, I think this has been exciting times recently, because there is all this cloud hype that’s been going around, people are actually like trying to compile it as it were and that’s when the you know like I was chainsawing this weekend and every now and then your chainsaw hit something and sparks fly out of it, and that’s fun stuff. That’s when the good stuff happens and if you don’t end up sawing your leg off then you know you get that tree cut and it’s good stuff.
But I really like all the — you know as the — I think you were using this metaphor before we were recording Andrew about the whole when the rubber hits the road or whatever and that’s what I’ve been seeing a lot recently so it’s nice.
Andrew Shafer: Yeah, I think you’re going to go or there has been like you know the DevOps stuff, DevOps days in the U.S. There’s one coming up in Hamburg, coming what is it a week, two weeks from now, yeah I lost track of that calendar.
John Willis: It’s two weeks from this week, it’s a week from this weekend, so it’s the following Saturday. I’m actually going to be there.
Andrew Shafer: So I think it would be great to be there, but I’m actually going to be in Africa. So I’ll have to live vicariously through all my DevOps buddies over there.
Michael Coté: Well so you’ve been going to a lot of DevOps stuff recently right John, you want to tell us what’s going on with those, with the DevOps days and things like that.
John Willis: Yeah. It’s you know, I mean as Andrew knows, Andrew is kind of, you know him and Patrick, I think you know got pretty much a lot of this stuff started and then you know, we had the DevOps days after Velocity — right there was DevOps in Belgium or in Brussels last year and I was fortunate enough to attend that and then we ran the DevOps in — after Velocity, which was amazing and I know we’ve talked about that and then they’ve got the other one coming up here.
Michael Coté: There is like videos for all those sessions right, I’ve watched a couple of those.
Andrew Shafer: There’s also one in between that Lindsey did in Australia.
John Willis: Yeah, that’s right there was one in Australia. Now what’s happening is through a lot of what you know Andrew and myself here in the States and what happened was like a lot of guys left that one last year in Brussels, Belgium sorry, and everybody basically came back at it just — you know their heads exploding with my God this is great and like you know Lindsey went to Australia, the guys in England started going nuts, Andrew and I were doing a lot of stuff in the US to get stuff going and now what we’re starting to see is all these regional ones I think pop up, so I was in Chicago two weeks ago, I went to Orbitz and gave a presentation there and then we did the first Chicago DevOps at dotWorks and then there was Morningstar had that little thing too and then there’s one coming up in New York in a few weeks.
The guys out in the Valley have had a couple of ones already. I went to — I was fortunate enough also to go to the SoCal, Los Angeles they’re pulling 100 to 125 people on a Thursday night, Tuesday or Thursday night under the label of DevOps Meetup. So, it’s pretty interesting. I mean the emotion and the passion around and you know — Andrew said this and my CTO Adam Jacob says it’s a cultural movement, it’s a cultural and professional movement and that’s what we’re seeing, we’re just seeing a lot of passion in this, kind of, cultural movement or trying to figure out, you know, some people have already got it, some people don’t have a clue, right, not the people that are coming to these meetings, but enterprises and whatnot and I think when people see the ideas of just the DevOps, it’s one of those things, you don’t have to see it twice, it doesn’t have to be explained to you more than once, you know, you pretty much get it right from the get-go, if you’ve worked more than a week in IT, you know so.
Michael Coté: And so, I mean is there — what’s the explosive device that blows peoples heads up? Like is it kind of the stuff that we and other people have been talking about for a while or is it sort of like some tweaking that’s been going on that like blows heads up better?
John Willis: Well there’s a secret and we can’t let you in unless you join the cloud.
Andrew Shafer: You got to get the secret handshake.
Michael Coté: There you go.
John Willis: Well let me — go ahead Andrew.
Andrew Shafer: So I guess there’s a couple of things there that I want to at least touch on. On this notion of having to explain to you and you understand it, I went to the AGEL 00:39:04 conference that was in —
Michael Coté: Nashville. No that wasn’t right.
Andrew Shafer: Moved to Orlando. And at the — there was an analyst panel and they mentioned DevOps about five times on that panel and there was you know the Gartner guy and the Forrester guy and what have you and then afterwards because I think I might have a little bit of knowledge and a little bit of an opinion about DevOps, I went up and I tried to explain to those guys where I thought their DevOps coverage was a little bit off what I thought was actually happening.
And then I follow up by writing some email to them and encouraging them to reach out to me and have a conversation and I didn’t get responses and I don’t know about you, but I never forget when somebody doesn’t respond to my email. So if you’re listening to this Gartner & Forrester, someone should send me an email.
John Willis: So what was he missing, what was like you —
Andrew Shafer: Well I just think they have — everyone has a perspective, everyone has an opinion. And at this point there is, you know there is DevOps and there is me and there is Patrick and there’s a bunch of people building stuff and you know I got to consider all the puppet people and the chef people and sort of this community practice and the amazing infrastructures and the things they are building and the things they are doing on both a technical and a cultural level.
And there is vendors, that want to sort of attach themselves to the movement, to sell the product or whatever their strategy is and those things aren’t always 100% aligned. And just like cloud is an ambiguous undefined term that creates confusion and opportunity, I think you have the same sort of thing with DevOps and I think that it actually — it tends to weaken the potential for what could be accomplished in a lot of organizations when that message gets muddied and diluted.
John Willis: Well I, you know when I present on a DevOps thing, the first thing when I say is that you know I use the line that Adam used that Velocity, which is DevOps is a culture and professional movement. Right and then I say, you know if a vendor is coming to you and saying they’re DevOps compliant or they have a DevOps product or you know they’re implying that DevOps is anything other than a cultural and professional movement as a general discussion run. You know, they are evil, you know because it’s exactly what you said, is that there is the cultural aspect to it and which is you know and as I say that if me and David we’ve kind of come up with thing called CAMS, so you know cultural automation, measurement, and sharing. You know unless somebody else comes up with a better way to describe it I’m sticking with it. And I would say if you don’t have — if you’re not dealing with the cultural, you don’t even understand the culture.
Andrew Shafer: So let me put it in perspective, all right. As an engineer I have opinions and perspectives about technology and how some of this stuff works and how the culture works and I express them and sometimes I have ideas that other people don’t have or other people have ideas that I don’t have and that’s is fine. But when I read something that makes me roll my eyes, when it exceeds the eye-roll threshold, then that’s sort of — you know that’s my BS detector. And all I really wanted Gartner & Forrester to do is write articles that don’t make me roll my eyes. Like I don’t think that’s too much to ask. It’ll help you, it’ll them and then —
John Willis: Well and I think there is technicians that make the same mistake too, right, I mean that really don’t have an agenda and they pierce in that DevOps is you know, continuous integration. You know the whole article is what is DevOps and then the whole thing is just describing some really, you know fascinating continuing integration model, you know what I mean and I think that’s just as probably not — I mean I agree with you that I think then —
Andrew Shafer: That’s the trunk of the elephant, you know, it’s the lineman on the offense.
John Willis: Oh I agree. I think the vendors that are trying to write — but you know that’s business and that’s what happens. We saw that happen with the cloud. I think the people in the movement are pretty darn smart. You know, the truth of matter is the Forrester’s and the Gartner’s, they are never the guys that get it on the first run anyway right. I mean they were pretty much convinced the cloud didn’t exist until a good year into it, you know before they even started mentioning the cloud right.
So I think DevOps is the same thing. I think that right now their view of it is from their big clients and their big clients are driven by their thought process is mostly driven by large vendors. So they’re going to get whatever they are kind of minor, oh don’t worry about that or that’s something we can solve in Version 2 of our product.
Andrew Shafer: Clearly.
Michael Coté: You know it’s also an opportunity.
Andrew Shafer: I’m here to help that all I’m trying to say, I’m here to help.
Michael Coté: It’s also an opportune time to plug your other podcast John, the DevOps Cafe, where you guys spend a lot of time talking about the finer points of what DevOps is. And if I remember in the first episode, you have a nice little what DevOps means to me moment, which is fun. But like — I think that’s part of the issue people have is they definitely — you know and it’s kind of getting back to the point of why it’s sort of easy to explode people’s heads, is there is not — you know there is not a lot of like easy things to point to, like you can’t buy the O’Reilly DevOps book if you will and so —
Andrew Shafer: So that was the other point I wanted to make when you guys touched on it is, this exploring people’s heads the outcomes, because so many people their idea of IT best practices is such a disaster, it’s a minefield of Jenga setups where everything that they’ve ever experienced is broken, fundamentally broken where you had you’re under resourced, extreme time pressure, extreme up time pressure and it’s just — there are a bunch of people with post-traumatic stress syndrome.
John Willis: Well maybe that’s the thing I think now taking it back to where it was earlier is I think the web operations book, which I know you’re an author of one of the chapters Andrew. A very good chapter actually and — but the — you know I think that to me is hand this book out not that it like solves DevOps, but it’s clearly — you know like Adam says right, Adams says that I know you said this and others said this too, that web operations is now becoming or is the dominate culture for IT right.
So the dominate model for IT processing, this is — you know what we’ve learned over the last what five, maybe ten years about web operations is the right way, unless somebody else comes up with a better way, is the way that we need to start assuming our best practices for building IT and which includes, you know this is very much where the DevOps mindset came out of, you know, because it was less of a barrier in WebOps blog and I think that book is a great — I mean everybody should read that if they are thinking about what IT is going to look like for the next ten years, that book, you know I think you would agree is a great starting place.
Michael Coté: And I think that’s a nice concise way of putting it this 00:46:39 instant. We’ve, sort of, ironically been moaning the fact that people talk about DevOps without knowing what they’re talking about and yet we haven’t really said what it is, but which is exciting other than some top line stuff, but like, in the same way that it can be said that consumer technology drives what happens in enterprise technology now.
Like basically the point you’re making there kind of is — it kind of dips your toe into more specifics here is that rather than thinking about enterprise IT management where a lot of the innovation is happening and coming from is how high scale public websites run their operations and that’s like what the big switches you know, from Andrew’s Jenga towers or whatever it is, like there is a whole different way of running or not a whole different way, there is a much different way of running IT that comes from a different source than like what you’re used to being taught in your enterprise IT stuff.
John Willis: So back to this point, I think that the present is not evenly distributed. In every sense of the word and while I do think that there is — there are some very colorful and powerful personalities that are involved in doing WebOps I don’t think it’s the dominant paradigm and I think that you know like I love Adam, don’t get me wrong, but I think he might have a little sampling bias, because he has done a lot of WebOps and if you go — I mean I don’t know what your experience is at Opscode, but I can speak to my experience at Reductive Labs and looking at the way a lot of these enterprise IT shops run, they could learn a lot from all of the lessons in that WebOps book. They could learn a lot from the lessons and the things people do at Velocity conference or Surge conference. But at the end of the day, when they go back they’re still — it’s still a bunch of Jenga and they got a lot —
Andrew Shafer: That’s the point. I think they’re the ship sailing to the left, you know what I mean. I mean I think that in all honesty that enterprise — there is a lot that WebOps can learn from enterprise and I think I’ve had discussion with you. I mean I remember sitting and watching John Allspaw talk about incident and change. And I remember you know J.P. Morgan or Chase Bank doing that 20 years ago, you know what I mean and having it nutted on the mainframe, right.
So and that’s kind of not that it came from the banking industry in the US, but 00:48:59 came out of people that were figuring out how to get incident and problem and change in a process that everybody could share.
John Willis: I could probably articulate this a little bit more effectively but I feel like —
Andrew Shafer: Are you saying I’m dumb?
John Willis: No, no, to reinforce your point. I feel like cloud computing is really just client server coming back full circle and relearning all the lessons that people learned about how to build reliable systems in mainframe and implementing it in distributed architectures.
Michael Coté: And that answers how you’re going to get rid of all that legacy IT, you’re not —
Andrew Shafer: Yeah you’re not. And okay so that’s a great point. There are still people building mainframes or —
John Willis: There are all right. But all right, so again I’ve got a bad sampling rate to sample set as well because, I’m so immersed in world right now. But I think that when you look, it goes back to what you were saying early, I think today all things being equal, and that’s a pretty gigantic statement, but if your decision is to pay $15 million for an IBM mainframe or build a commodity infrastructure based on commodity hardware, Linux, Xen Hypervisor, possibly one of the open clouds as it matures.
Andrew Shafer: You could buy a lot of CPU and disk for 15 million.
John Willis: That’s what I’m saying — well you know yeah, I mean my point is that yeah there are still people buying mainframes, but I’m not sure they’re on the right ship, you know what I mean? I’ve never been one to bank against IBM or bet against IBM, but the point is I think that I don’t — I think if you are not picking up some major lessons from web operations, if you’re not looking at DevOps as something that should be game changing your enterprise.
Now the chances are like you said can we, will we, would we; yeah, well they will eventually. I mean eventually they’ll get a CIO that’s basically out of college that you know that’s coming out of college right now that will actually be their CIO and he’ll have no other way except WebOps, DevOps and cloud right. And — but I think that the people that are going to hold on to dear life to their Jenga infrastructure, as you would say it or I mangle it, I think they’re headed for — you know they are sailing in the wrong direction.
Andrew Shafer: Well, it’s just they’re always going to have the cost center, because they always treated it like a cost center. I don’t think —
John Willis: Let’s take it out of business right. I mean the question is, is there somebody out there going to start a business. I mean the one thing we should have learned over last 15 years and certainly within the last five, is that when the Internet boom came out companies like Barnes & Noble and Borders thought they were safe and then all of a sudden this Amazon basically shocked their world right. But the banks and the insurance companies they were like that’s never going to happen to us, right.
Well guess what’s happened in the last three to five years, right, cloud is in a struggle. So I mean your competitor, I mean there’s a bank called Simple Bank, I mean these guys are ex, I think B of A guys, right that are building commodity infrastructure, one of the interviews we did was this company called Ka-Ching on our DevOps Cafe.
Andrew Shafer: Ishe Smith, he was at DevOps days.
John Willis: Okay, I mean they blow your mind if you look what they are doing and people say well you can’t use cloud if you’re in the financial credibility. Well these guys are kind of like a mini e-trade and there are 15 guys, I called them a garage band of startups, they actually —
Andrew Shafer: They are like garage band mutual fund.
John Willis: Yeah, and they’re doing it all on commodity infrastructure, they’re doing continuous, to hold continuous deployment Eric Ries continuous delivery or whatever, Continuous Deployment. It will blow your mind what they’re doing, and I would bet, this is what I say like what they had built, I could picture going to B of A and say he tried to replicate this.
So they are probably about a $2 million investment, they’re up to like 15 guys, they’re working out of what looks like a garage and if you went to B of A and said replicate this, it probably cost them $5 million to $10 million, it will probably take them a-year-and-a-half to develop it and it wouldn’t be as half as efficient. And so, you could stick with your mainframe and AIX and DV2 and Oracle and keep going down that path and somebody is going to put you out of business.
Andrew Shafer: Disruption.
John Willis: There, I’ve said it.
Michael Coté: Oh, you closed the circle back to the beginning of Telco’s being disrupted off of one word.
John Willis: But they get it obviously, they have no choice but to get it, right, because the mobile is killing — is totally ripping them apart.
Andrew Shafer: I mean you’re obviously in a better position if you know your business model is being disrupted and if you —
John Willis: Or you are somebody like Amazon and Google who have really smart guys that do the math, right. And they figure it out ahead of time.
Andrew Shafer: Right. I mean everyone has always done some math, it’s just a matter of as you get more insight into where your costs are and how do you’re going to be able to spend things, until people see things then they’re not obvious and once they do then it’s de facto standard.
Michael Coté: Well, and also to use your metaphor again right, if you’re dealing with bunch of Jenga towers. Really the thing that usually happens is they fall down in a big catastrophe, it’s a little difficult to carefully unassemble them.
Andrew Shafer: Or you are hiring a bunch of guys to hold the little pieces up and they run around and they keep them. If you just had a bunch of Jenga all by itself and no one ever touched it, then it shouldn’t really have a problem, but you have users, you have new features, whatever page you can deliver them and then that’s why they fall, not because they’re sitting there and they’re unstable, but because that you changed them.
John Willis: Well, the other thing, are you a type of company that are willing to take the Amazon like or Google like risk to make that kind of change that’s so disrupted to yourself, right and to do that right. And that’s — and unfortunately a lot of these companies, for all sorts of historical reasons and reasons that I’m not even qualified to comprehend, but what it takes is some company to have a risk taker to basically say you know what we got to change the game.
I always think about like when I worked at the Federal Reserve doing consulting, the food always suck and what they do is every few years when the contract come up, the lowest bidder got the contract, whoever was the lowest bidder on the food services for the cafeteria got the contract. So the food always suck, right I mean it was never going to work, right.
Andrew Shafer: So, that was the result of someone doing the math and I think so let’s talk about and I think this goes back to culture. We’re making lots of little loops here, but Zappos was purchased not because they were beating Amazon on price, Amazon’s core competency is beating or winning the price battle.
Zappos was winning on culture. They had a great culture of people who loved their stuff and if you think that your differentiator is all about running the numbers and at the bottom you find the lowest one, that’s not the math I want to do. There’s the bottom line and there’s the top line and if you can build relationships with people, in 2010 you have amazing, amazing infrastructure for reaching out and interacting with your customer on any level business-to-business, consumer-to-consumer and that is another thing that’s changing the game right now.
Michael Coté: Yeah, and I think maybe the point, the thing that I encounter a lot is, it gets back to the cultural thing of CAMS or whatever just like you alluded to, is that businesses aren’t used to taking risks with IT or using IT to take risk or taking risk in the first place of course, but I think the wider thing is people don’t expect IT to be helpful or to actually help them do anything, as you were saying.
Andrew Shafer: So, yes —
Michael Coté: I think there’s little expectation that like —
Andrew Shafer: Exactly.
Michael Coté: That like you could do something interesting with your customers, because you have some new piece of technology, they just assume that any technology stuff you get involved in is going to be a swamp.
Andrew Shafer: Exactly. So to make my point more explicitly, it’s about treating people like people and it could be your outward facing customers and the culture this app was built to leverage that or it could be your colleagues and your co-workers. So if you think about the IT mindset where the guys in the data center we think they’re all a bunch of chimpanzees and they can’t do anything right, then don’t expect to get great IT out of them, right.
If you want people to solve problems, if you want real DevOps you can’t just slap a little DevOps label and say we got some automation now, basically you need to treat people like people, you need to give people interesting hard problems to solve and let them solve them.
John Willis: Well, what I’ve been seeing and kind of now circling, circling 00:58:31 that is you asked me what I’m seeing from DevOps, what kind of — there’s nothing genius about this, but I’m finding that when I interview or talk to people, the most effective DevOps culture it plays is where somebody at the top, the CIO, somebody up there gets it completely and then it just kind of, you either want to work there, because you get it or you don’t want to work there, because you don’t get it.
I mean and it’s like anything else, we say this all the time, unless you have a kind of top level commitment or something, but when you’re talking about like we always talk about top level commitment to maybe a road process or a technology, but when you start thinking in terms of like top level commitment to understanding your culture and how it affects your business, if you’ve got that, then everything falls downhill in a positive way, you know what I mean.
I think that when people ask me what’s the most effective thing about DevOps, I was like, like again if you don’t have cult, if you don’t understand like, I believe like the visible ops guy, the Gene Kim and all, you can’t change culture, you could change behavior, but you got to understand your behavior and you got to think about, well how do I change behavior to affect a positive flow of anything we do, because if we don’t understand our behavior or our culture, then throwing a whole bunch of automation and process and trying to formfit everybody into something is just going to be a waste of time.
Michael Coté: Yeah that’s why usually like it helps to have executives, giving permission to do all this stuff, because it seems like the way most businesses run, if not all of those. Basically you figure out one work flow that makes you money and if your employees deviate from that, you punish them and you basically want to do the one work flow that generates profit. So you sort of have the —
Andrew Shafer: You want a positive feedback, not a negative feedback.
Michael Coté: Exactly. And so, first you have to give — you have the employees that can innovate and then you have to kind of give them the context in which to innovate. And then finally of course, you have to give them permission to do it, and then hook it up to actually transforming things, but I mean I would wager for all the innovation talk and all that kind of whiz bangery that most companies are not really, the business side is not really interested in doing new things, they’re interested in squeezing as much cash out of their current 01:01:00 as they can.
And so, that’s why, and then the same sort of thing would happen in Agile world where you sort of acculturize all these programmers to develop software in one way and you can’t really expect them to be able to take it fully bottom up all the way until you get executive leadership that says, all right you’re allowed to do this. It’s sort of — part of business culture that you don’t do what the boss tells you not to do or wait, you only do what the boss tells you to do.
Andrew Shafer: And what has to happen is again, I think that business leaders have to come to the conclusion that DevOps ultimately is trying to mature, the principles of DevOps if there are any, will actually make you more money, right. The same conclusion that I guess the Opscode guys came to that hey there was a reason why this is going to work, we are not just doing it because it sounds cool, there was grassroots efforts where people did it and said, I’ll build it and they were calm and then there’s executive level people that got it and said, “Oh My God” or they read about their competitor doing it and I think that’s what DevOps is going to have it’s traction is that —
John Willis: It’s not about a name, it’s not about DevOps, it’s not about —
Andrew Shafer: It’s about creating value, it’s about creative capitalism, it’s about creating more value than you capture, I think that’s the bottom line.
John Willis: But yeah.
Andrew Shafer: DevOps is this idea focused on one particular slice of how you’re going to run your business. How you’re going to trade your —
John Willis: But I think that’s just like the cloud and just like Agile names are affective, right. I mean I agree that DevOps is just a place, a stakeholder in the ground right, but it’s an affective stakeholder, because it allows you to rally around it and have the kind of discussions that could affect business, positive business changes.
Andrew Shafer: I agree. And I’ve said all those words; DevOps, cloud, Agile yes, but at the end of the day it’s not about the word it’s about the action.
John Willis: Totally. No we always kind of say the same things but —
Michael Coté: Well, I think on that note it’s good time to wrap up. We got the Web Operations Book, that’s by O’Reilly right John?
John Willis: O’Reilly, yeah.
Michael Coté: You know a willingness to change, to do something with them Jenga towers and also yeah you know if the audience, you guys — do your listeners really like this kind of DevOps talk, like I said John and Damon Edwards have a great podcast series where it’s basically all they talk about all the time, which is good stuff. So, hey thanks for being on this week Andrew, it was a pleasure as always.
Andrew Shafer: I’m started to think my core competency is meandering editorialization, but thanks for having me.
Michael Coté: Well, it’s true meandering editorialization works for podcast.
John Willis: Well, he is Little I Dare on Twitter, but he’s got some Big I Dare in the clouds.
Michael Coté: Send that guy to the marketing department. And with that–
Andrew Shafer: We’ll see how the 01:03:55 or we’ll measure them over the next few years to see how it turns out.
Michael Coté: There you go and with that we’ll see everyone next time.
Andrew Shafer: Cheers.
Disclosure: CloudScaling is (new) client of RedMonk, as is OpsCode where John works. Many other companies of interest are clients as well, see the RedMonk client list if you’re interested.
http://www.truereligions.in/