John has been playing with Cloud Foundry a lot of late, so we go over that and PaaSes in general.

Download the episode directly right here, subscribe to the feed in iTunes or other podcatcher to have episodes downloaded automatically, or just click play below to listen to it right here:

Show Notes

• Cloud Foundry – a container of containers. “You need a ruby guy, for sure.”
• What is a PaaS exactly?
• A Lou Gerstner interlude
• Coté’s favorite Asymco.com chart, again.
• John uses Spot Cloud – and lives to tell!
• How many levels of wonder does Amazon have?
• DevOps workshops – what people really want is to go right to the cool stuff, e.g., “let’s go install something! Let’s go install Jenkins!”
• The Viking Ethos of Business: Party tonight, for tomorrow we may die!

Transcript

As usual with these un-sponsored episodes, I haven’t spent time to clean up the transcript. If you see us saying something crazy, check the original audio first. There are time-codes where there were transcription problems.

Michael Coté: Well, hello everybody. It’s the IT Management & Cloud Podcast episode, I’m not really sure, we’re in ‘80s or something like that. And as always this is one of your co-hosts, Michael Coté, available at redmonk.com/cote and I’m joined by the other co-host.

John Willis: John Willis at johnmwillis.com, or more recently at dtosolutions.com.

Michael Coté: And you’ve been keeping busy over there I understand, like doing stuff.

John Willis: Yeah, that stuff-stuff is hard man.

Michael Coté: So, last week we were supposed to record and then as always happens on a Friday afternoon I got busy and I had to reschedule. But you were telling me that you were actually like getting your hands dirty with like Cloud Foundry or something like that, a Platform as a Service.

John Willis: Yeah, so it was kind of not on the top of my – one of those things that like we all have is lists of things that we want to do and things we have to do.

So, the Cloud Foundry was just kind of in the middle of somewhere. I thought like cool idea to play with, write a little bit of that when it came out. But we were working with a client two weeks ago who wants to actually build it kind of strategically into their cloud which is an interesting approach, but so I had to dig in on it and it’s pretty interesting.

I actually — I wasn’t sure about it at first. I think on this podcast we’ve had a lot of discussions about PaaSs and I think I at least on a few occasions expressed my opinion that I’m not a big believer in PaaSs in general. Primarily the — clearly the public PaaS I think are — I say they are the primrose PaaS if you will, but yuk, yuk, yuk, but the — I thought that was funny.

Michael Coté: It was quite hilarious and if anything it had alliteration in it.

John Willis: Yeah, there you go. So but the thing is, is that I will say that it’s not a good idea today to outsource your core competency and I’m not sure at this stage of the game running infrastructure for most companies whether that isn’t the core competency, I believe it is and so I think when you go to a pure behind the black curtain or the black wall of a PaaS, a public PaaS now then I think you give up a lot of opportunities to be strategic.

And I’ve said this before too, I’m also — I’ve gotten to the age where I’m like 50 — I think I’m 51, I can’t remember, but 51 or 52, I have to check that, put in the show notes. But the thing is I’m not apologetic any more about like everything looks like are now, you know what I mean?

Michael Coté: Yeah.

John Willis: At some point you just say, “You know, hey” and I truly believe that — I’ve always believed that, I’ve taken the long way to get here but I believe that why I’m such a big believer in Chef. I mean I worked there and now I don’t work there, but I still — and Puppet to a certain extent. But that the right combination to get the farthest along to a PaaS is in my opinion infrastructure as a service with infrastructure as code with something like Chef and Puppet and there you have more control. Now I’m sure this could create a great debate for a lot of people.

But the thing I like first right out of the gate about Cloud Foundry is it’s an open source PaaS. So that’s the first, as far as I know the first open source PaaS and they nearly did a good job on not just making it like, like they do have a public PaaS for Cloud Foundry, cloudfoundry.com you can run it at basically somewhat similar to say what Heroku or Engine Yard would be, although I’m sure you can make the argument that the architecture is a little more interesting. I’ll get into that in a second.

But the fact that they really did a reasonable job on a first cut product for supplying the infrastructure to build this in a pretty bulletproof way on your own infrastructure and so that is exciting. And then but I still had the concern of like, okay do I get really good at something like Chef and have only flexibility knowing that I’m going to use Chef or do I have to bring in a guy now who is a Cloud Foundry expert? Because having it open source doesn’t mean crap, right?

[0:04:54.1]

I mean it’s — you’re still — when you have that strategic opportunity meaning there is some technology you would like to use integrated with your infrastructure and some outsource provider is going to be gaining factor, whereas I mean knowing how to tweak the open-source source code could be a gaining factor as well.

But point is though the debate I had with Alex, Alex Honor who I work with, he is the architect at DTO. We spend a couple of hours debating this and the thing is, is that Cloud Foundry and I’m coming around, I’m still in the middle. Cloud Foundry offers an awful lot.

Michael Coté: And that’s part of what I wanted to ask you I mean now that you’ve messed around with it for a little bit. Like I think we all kind of know what it is on paper but what’s your description of what it is after actually using it for a little?

John Willis: Yeah, and I’ll blatantly say that I’m stealing this right from Alex. It’s just a container of containers and they’ve done a really, really, good job of isolating the infrastructure. I mean Engine Yard and Heroku do an excellent job of isolating infrastructure and giving you your application.

But what these guys have done is really build — it’s a second go around I guess, they got some of the SpringSource guys, you got some of the guys from whatever that acquisition that SpringSource made early on in cloud, it was that cloud PaaS company they bought before they bought —

Michael Coté: Oh yeah, sure.

John Willis: Remember those guys, I don’t remember who those guys were, but then — and you had the guy who was the original founder of Engine Yard, that Ezra guy.

So, you had all these guys on the team, I mean that you got them — and all these guys getting a second chance at this and the way they’ve architected Cloud Foundry is really I mean I think pretty much close to brilliant. They’ve got — I mean they have this cloud controller and it’s another interesting, they wrote like a cloud controller all in Rails.

And it was like first kind of to me — well I mean I shouldn’t say that because I don’t know enough, but a really serious enterprise grade architecture written with no Java, no — I mean at least the cloud controller, no java and no Erlang, all the things they say you have to have to be enterprise grade.

Even Chef, right, Chef is lot of Ruby but it’s a Rails app and it runs — it’s a Rails app that’s a cloud provider. It’s pretty clever and then it has kind of the built-in, again a load balancer but not just a load balancer, a loud balancer is that’s intelligent to know what container to work with. So, now you have like a no JS container, you have a Java container, you have a Ruby container and then they have isolation of the apps which is obviously, then they have Health Manager.

I mean that’s cool too. It’s got the heart beating and checking. But the other thing I think that is really, really, clever about it as well is they have like a service layer. I don’t know if it’s called container, but it’s another service which is the service layer, where now you can start. Now, all these different containers now can have these different services and now they supply like MySQL, Mongo, Redis, I’ve seen probably there’s plug-ins for everything now.

But the idea that you can actually — it really is that kind of promise that we always wanted, which is like all these different services being available to us in the cloud. So I got a lot more to learn, I’ve got — I mean actually to be perfectly honest with you I got my first app finally up but I’ve been trying to build the Chef cookbook with it and it uses — it just the classic Ubuntu, RubyGems and then they’ve got RVM which is Ruby Virtual Manager or whatever which is great for your desktop, but it’s kind of — I’m struggling trying to get — one can clean install all the way through repeatable that’s automated. But I’m —

Michael Coté: So, I mean you kind of mentioned a little bit of this but like what are the languages you say you need to know or be adapting to use it. Like there’s obviously a lot of Ruby running around in there, right?

John Willis: Well it’s going to fall into — and this is probably the way I think the industry is going anyway. So I mean like if you’re somebody who believes that Chef and Puppet are the way of the future for configuration management and then you look at this. So, from a system administrator’s point you need a Ruby guy for sure, RubyGems, and I think you’re in a good shape because the whole — most of the architecture is written in Ruby.

[0:09:52.1]

So and then — but the thing is — all right, so the guy who is the administrator, he has just got to be your classic Chef or Puppet sys admin, and the guy who knows a little about Ruby apps or Ruby on Rails apps and/or has dealt with that from a system administrator’s standpoint. But then from the developers and this is other thing Alex says and I agree is brain dead for the developers, drop your WAR file here, bang.

Michael Coté: So really, I mean the whole deploy cycle for development does seem pretty I guess drop WAR simple as if you will.

John Willis: Yeah, and you think like could the enterprise like this, yeah, hell yeah. I mean when I was with Conoco, I mean I’ve told this story before. You’d go to customers and talk about a cloud and that the development guys, the Java, the Enterprise Java guys would be like cloud 00:10:43. Give me a way to drop a WAR file into development, QA and production environment, without me having to do anything other than test my application. That’s what I want and —

Michael Coté: Then so also along those lines, oh man what was — oh I know what I was going to ask, so and then also like widening to — since you — I mean you’ve been at DTO for awhile and before in Opscode and everything, like I’m curious so when you’re looking at like Cloud Foundry, like what kind of — what kind of like hardware are you think people would – do you think people would run this on like, is it totally irrelevant what you run it on or is it sort of like — does there seem to be a certain profile of machine and networking gear and stuff that would go well with it?

John Willis: Yeah I don’t — I probably would start with saying, I’m probably not qualified to answer that question but that’s never stopped me before. Now so —

Michael Coté: That’s why I always ask you these questions.

John Willis: No, I mean as far as I can tell I don’t — I mean I think that’s the part of the beauty of it, it is truly abstracted that, that I mean right now it is kind of shift. I mean this is like early days of this. It’s shift with as installable, a store that installs everything on the same system.

So now you get yourself – I’ve been — what I’ve been doing is when I’m doing some serious testing I’ll just grab a big, you know one of the X large, the Amazon Extra Larges and pop it on there and it’s I think you can have a reasonable amount of effect and work with it.

I mean I think it would be fun to run, like well — I mean I wanted to talk about our DevOps workshop but I think it would be fun to run a full kind of service stack with everything, with Puppet and well the Chef or and 00:12:38 your apps, all within this container, this Cloud Foundry container. I think that — I mean the one thing I think about is imagine this is like the ultimate like clients, right?

Michael Coté: Yeah, like, and it’s not a virtual appliance necessarily but a soft appliance.

John Willis: Yeah, but it should be easily installable particularly with Chef Cookbooks, as a pop bang, now you have yourself a freaking running data center really. But back to your question, as far as I can tell it doesn’t matter and it looks like their — the drive that they’ve built this for is, I mean their motive, right, their drive as in motive is that it really does – shouldn’t matter, right? You run it on — you actually run it on your laptop. In fact supposedly there is a new release, it’s going to come out at some point, probably not in too far distance it’s going to have a micro version which is going to be very much like a Vagrant implementation. And so —

Michael Coté: Do you want to tell people what Vagrant is exactly in this context?

John Willis: Yeah, so Vagrant basically is, it’s being used — it’s pretty hot right now. It’s — largely you basically to — I think it was originally written with Chef in mind. So it’s basically — it basically sets up the — boy, my mind was just blank on the —

Michael Coté: Well, last I checked in with it. The Cool Kids or the Lunatic as it was called.

John Willis: Oh —

Michael Coté: It’s a way of using Chef to make — to do build around VirtualBox. So, it makes a big image.

John Willis: Yeah, the thing I was having a mind block on is VirtualBox, so basically it’s kind of an abstract layer around Sun’s or Oracle’s VirtualBox.

Michael Coté: And I was asking someone why it was that and I think the answer I got was that it was just the most open virtualization technology to build around, and so to —

John Willis: So, yes, so that’s the thing I was farting around trying to remember was the VirtualBox, but as you don’t — once you set it up, you don’t even really play with VirtualBox and it’s a config file. It was originally set-up with Chef where basically you define find the kind of image that you want and you basically define — and what you — originally you use Chef Solo which is standalone Chef you can’t — it’s no Chef server.

[0:14:58.8]

You basically point you point at the cookbooks and the actual role that you want to start and it would just fire up on your box the application. So, perfect for testing because again getting the developer even out of the Chef game and VirtualBox and the networking, you basically, you set up this config file and you just say Vagrant up and bang the thing comes up, right?

Michael Coté: Yeah.

John Willis: Well it sounds like — it sounds like the next release of Cloud Foundry is either — I don’t know if it’s going to use Vagrant or it’s going to have their own or whatever, but it’s going to be a way that you can basically run Cloud Foundry in a micro kind of instance. So, it’s like a virtualized environment that yes, probably not going to need VirtualBox, not that I think about it. Then it’s going to bring up Cloud Foundry, so the beauty is you run your Cloud Foundry on your laptop, you run your Cloud Foundry on some cloud, maybe Rackspace Amazon, you run your Cloud Foundry on VMware, you know what I mean?

Michael Coté: Right.

John Willis: So, you could take it from crawl to run in the application lifecycle and from the developers’ perspective everything is almost exactly the same. And that’s — so yeah, so back to your question I think that — I think that the design and then the architecture is such that it should be agnostic to wherever it winds up. So there — yeah no it’s — I’m pretty excited about it so using that quite a bit. I was also just kind of — just circle back on the whole — we had really — we talk about Amazon in a while really in general. Early on that’s all we ever talked about.

Michael Coté: Oh, yeah, there is so much other stuff going on now but they still send out their newsletter like once a week or month or something with — which I forget if I talked about it on this podcast, but there is a — every now and then there is a — so here is something, speaking of PaaS like, I think this was back at like structure or something. I’ve noticed that there is this idea that people are floating including the Werner Vogels or whatever. And that is that there is a blurring of the line between Platform as a Service and Infrastructure as a Service.

And — or at least that’s the idea that’s being tried out and I guess you sort of encounter this and from various people — for various legitimate and illegitimate, illegitimate is the wrong word, but not to credible reasons. And on the one hand when we’re just talking about it with kind of — like dipping into it with kind of the Vagrant thing. On the one hand there’s a lot of people who sort of build their own Platform as a Service using things like Chef and Puppet and other stuff to kind of wire together what they would have.

And they end up with kind of like a platform but it’s not really as a service, it’s as a customized platform. And then there is some other folks, kind of like Amazon is a good example who increasingly have sort of everything and I think to them there is the idea of having a Platform as a Service ultimately a Platform as a Service kind of makes Infrastructure as a Service not so important.

So if you’re not a PaaS guy, then you want to kind of like blow out the candle of PaaS stuffs so that your infrastructure stuff works out well.

John Willis: Well and that would be the Chef argument too, not that they make that. I make that more than they do really probably. But here is the thing, so I remember interviewing Michael Crandell on my old Cloud Café, it was a couple of years ago. And I asked him, I said, “You’re a PaaS RightScale,” right? And he kind of gave like a half-a-second delay and said, “Yeah, we are.” And then I realized, when I thought about that after some like, there’s a pretty long range of what can be considered a PaaS, right?

Michael Coté: Right.

John Willis: So, as you think about it, it’s like everything else, it’s kind of — I mean it’s great that we have a taxonomy that is SaaS, PaaS and Infrastructure as a Service, but the truth of the matter is like where does a PaaS start, where does it end, you know what I mean. I mean Heroku is probably as clear of a PaaS as you can get. I mean it’s on a PaaS that — and then something like using Chef with these — where they have their application deploy cookbooks. That’s — or RightScale even, those are like, are they really PaaSs, are they really, really mature applications on an Infrastructure as a Service?

But I will say that the army of people that — and I’ve been in this camp for the longest time which is build yourself or get yourself a good set of IISs, one or more, whether it’s a in-house public, private cloud or it’s a combination of RightScale and Amazon or it’s just Amazon, whatever floats your boats or floats your cloud.

[0:19:57.4]

But then, and then I’ve always been find a strategy to get — to go up the food chain as far as you can to be your own PaaS, and things like Puppet and maybe RightScale, whatever. Again, I think Cloud Foundry is more in the camp of what you would say this has got all the checkmarks of what you would want to call a clear PaaS. I mean Amazon has got a lot of great tools, they got Cloud Foundry, they got Beanstalk, they got the ELB, the Elastic Load Balancing, they’ve got the RDS, the Relational Database Service.

I mean, but still putting all that — in fact CloudFormation is a good first cut at — if you want to put on the table right now, I would put on the table RightScale, enStratus. If you look at what you might call Opscode’s application deploy, their QuickStart examples, they got Java, in fact I’m playing with that a lot today, their Java, their PHP, those are basically what they call their Application Cookbooks.

I would put the — again RightScale, enStratus, Application Cookbooks and CloudFormation all in that kind of same bucket of tools or products that take an Infrastructure as a Service pretty close or just over the PaaS line. Whereas clearly beyond the PaaS line is Cloud Foundry. And then going up the stream you could look at Engine Yard or Heroku, but again there is — some of the concerns that are losing control. I believe that you must — Cloud Foundry is where I get off the bus. Anything beyond that I’m not a fan, because I —

Michael Coté: Because you think it’s too specific I guess like 00:21:58 specific —

John Willis: I think having operational — I’ve had this arguing with people in discussion. I think that the thing that I say in my DevOps presentation is Tim O’Reilly in 2006 wrote an article said that operations is the elephant in the room and Jesse Robbins CEO, CTO then wrote an article a year later about operation is the secret sauce, using operations as a strategic weapon.

Michael Coté: Because we know that once you boil an elephant down to juice, it’s secret sauce.

John Willis: It may not taste well but by golly it sure is —

Michael Coté: It’s a secret because you don’t want to tell anyone you boiled an elephant.

John Willis: And then you get in all sorts of trouble.

Michael Coté: And then there is also that metaphor you don’t want to have to boil an elephant, you just want to do a little part. So therefore you hire a bunch of blind men to cut up the elephant —

John Willis: And then what’s the other metaphor, where they got the little guy riding the elephant and trying to make him go left, the elephant wants to go right.

Michael Coté: Yes, and then finally for — you can make the elephant dance and then you retire.

John Willis: And then the — yes, and then there’s to turn it completely around to a great discussion that I always love to talk about is Louis Gerstner. You can’t teach an elephant to dance.

Michael Coté: That’s right.

John Willis: Saved IBM, so there you go. No I think the —

Michael Coté: I read about him while we’re on the elephant topic. For some reason I didn’t realize that he used to be — he was like the CEO of Nabisco right or RJR Nabisco.

John Willis: Well he probably was one of those — he was — he is one of those McKinsey guys, so those McKinsey guys, that’s a whole — like they bake those. Those are like — McKinsey guys are Disney kids. They have the whole thing lined up man. They come out of Harvard, they go to McKinsey, they get their first job trying to help some failing horrible company that’s broken down and they put the guy in to be the quasi CEO. Then he gets his next job as a real CEO and the next thing he’s ruling the world.

Michael Coté: He also worked at AmEx which is —

John Willis: Yeah, in that book he talks about — we said this one — the great story, it’s worth saying again for just — in that book he talks about, his book that he wrote like you can’t teach an elephant dances, he was from AmEx and he wasn’t an IT guy. He was CEO, pure CEO.

And so he takes over the job of being a CEO at IBM and within the first couple of weeks he got to go down to to a shareholder’s meeting and they beat the shit out of him. What are you going to do with this? What 00:24:32 is going to do? How we’re going to do?” You know all this IT shit, he didn’t know nothing about, right?

And so he’s flying back up to Armonk or wherever they were going and he is in the corporate jet and he asked the flight attendant and he says, “Hey, can I get a scotch and water or something like that,” and she says, “Oh, there is no alcohol served on IBM jets and airplanes or whatever.” And he goes “Who do you got to ask to — who we got to talk to change that?” And she looked him and said, “I guess you.” He said, “Stop at Atlanta.”

[0:25:04.1]

Michael Coté: That’s fantastic. Good times.

John Willis: Well let me, I wanted to get back to the Amazon.

Michael Coté: Yeah, so much for Gerstner.

John Willis: No so, I think the thing — I don’t know if you saw recently the Amazon Jeff Barr put out there, they had like 449 billion objects in S3 in Q2 2011? And he has a chart, and so I was just looking at the chart and it’s like in Q4 2009 it was a 102 billion objects, in Q4 2010 there was 262, and here now in Q2 2011 it’s 449. That’s exponential growth. All these naysayer clouderowdies who keep saying, “Well, that’s ain’t going to last Amazon. They are going to be the standard and you just wait a couple of more years.”

Michael Coté: Well, let’s have a brief pause there. Who are these people saying that?

John Willis: Well there are people that say this?

Michael Coté: And would you say this is a frequent thing nowadays to —

John Willis: Well, it’s been going on for years. I think it’s some of the — it’s all like you said what were the — the kind of word — the magic words you were coming out trying to say that the Werner Vogels don’t want that — you know the way they want to position what a PaaS is?

Michael Coté: Oh right.

John Willis: The people — well you think of a good word to use for that while I finish this. It’s the people who I think have a lot of faith in private cloud. Agenda, there we go, they have their kind of own agenda, you know what I mean?

Michael Coté: Yeah.

John Willis: Like they want to push the idea that Amazon won’t be this unbelievable 900 pound Gorilla and Amazon just like just keeps chugging along. I mean in the truth that matters almost every enterprise in the world right now is doing something with Amazon. And the fact that they’re just the indicator of their growth of S3 objects is just showing it. This is — so I mean betting — I don’t think anybody bets against Amazon. Again I think there is people who have agendas that would like not to see this growth or put in the argument.

Michael Coté: Well you know what would — maybe someone has done numbers like this before but it’s always interesting to take like just take S3 or something, like take that total number of storage and then compare it to estimates for total storage overall. To get a sense of what’s going on there and to chart growth like — and to use an example, I’ve mentioned this on most every podcast and forum I’ve been on the in the past two weeks, but there is this really great chart by Horace 00:27:38 over at Asymco 00:27:40 which I’ll try to put a show note in the links too.

He basically did a chart. He tracks mobile numbers a lot and he did a chart where he took the total number of PCs in the world and again these are all estimates, so don’t you dear listener get your panties in a bunch about how you come up with these numbers, it’s not really important to that point.

John Willis: Well that — last name of 00:28:01, I mean I don’t think he got much credibility.

Michael Coté: Exactly. So you get the total number of PCs in the world over the past I don’t know three years and then you chart the growth of like Android and iOS devices and you kind of like put those up against the chart and obviously the number of PCs still dominates, but there is this in the past year or so there is this like really rapid growth at the top of the chart of iOS and Android devices eating into it.

And that kind of — those kind of charts are always nice and for example like seeing how many thing — how many megs or whatever is — whatever you want to measure it by are stored in S3, it would be handy to see it just — how many megs of storage there are globally. And you might want to throw out consumer desktops or maybe you don’t because there’s a lot of consumer stuff stored in there as well. But it would be curious to see what total — how many gigabytes of storage there are globally at any given time and then compare it to how many gigs are being stored on S3 or something like that.

John Willis: Well, I mean this is an area that probably I shouldn’t even step into because I don’t really have interest in it and I’m not — I don’t even know what I’m talking about but again that’s got to stop. I’ve notice that a couple of my son’s friends are getting Windows phones.

What’s interesting is that this is the problem I have with the Ubuntu 00:29:23. Every time I kind of jump up and say, “I’m tired of these Windows being all around, how can I get everything Ubuntu,” I mean they basically they tie me up like one of those TV shows, wrap me around with rope around my chair and don’t let me move and because they can’t — the guys, kids can’t run any your games — all the games

Michael Coté: On the Ubuntu?

John Willis: The real cool games that they love right now and this is the ages from 8 to about 13 or 12 are all on Windows. And they try to get on the work on even a virtual machine and just forget about it, just not worth the effort.

[0:30:00.0]
And I’m telling you and they’re all, like these Wizard101 and Roblox and these games that are just every kid in the planet is using right now. And I wonder if that’s going to have an effect over the next three to five years.

Michael Coté: Yeah, who knows man Windows Phone 7 is a — it’s dark course, in all the good and all the bad ways in the sense.

John Willis: Yeah, but all these kids that are like my son’s age now, they’re all going to their teens and they’re all like — all their games basically.

Michael Coté: Oh yeah, like their Penguin Palace and whatever.

John Willis: Yeah, I’m just saying all their games —

Michael Coté: This is an area where I don’t know what I’m talking about when it comes to the games and not the kids as in the people who have the —

John Willis: Yeah, I know that because my torture for having fun for just being a single crazy guy for like 25 or 30 years is now I’ve got to play these games every once in a while as my —

Michael Coté: Yeah, you know what, I like the — as I’ve said before, I think the Windows Phone 7 is actually a really good operating system, on phones.

John Willis: Well, I’m sorry.

Michael Coté: I’ll make my case briefly, not my cases that what I want to convince people but just what my thoughts are, so iOS and Android basically the same thing.

Now if you compare it sort of Android has much deeper features. I always like to think of Android and iOS as kind of like the ‘90s for Windows. Like do you remember like Windows used to always be more feature-full than like Mac and yet at some point in the ‘90s it crossed the line of it being too feature-full, there is just like too much things going on.

And at the moment Android is at that point where it hasn’t crossed line of being too feature-full. It’s just a much more open platform and it’s got more stuff and it’s more flexible and if they’re not careful pretty soon it’s just going to be another crapped up OS and iOS will keep it’s purity, which — that’s always been Apple’s thing is like, no, you don’t want that feature, trust me. And if you don’t trust me you still don’t want it.

And anyways whereas Windows Phone 7 is like — it’s not really to be — speak hyperbolically it’s not really anything like those two operating systems. It’s like genuinely different in the way it goes about doing things, and it still has apps and all that stuff but it is a lot — it’s different than those two things and that’s part of why I like it and also the little Samsung Focus I got I think is a fantastic piece of hardware and if I may go a little more consumer, I’ve used like three Samsung devices over the past like three months and those guys they’re doing pretty well like they make good hardware.

Like I’ve got a Galaxy Tab and a Samsung laptop and that phone and the trackpad though on this laptop that I have atrocious, terrible and I also have a Chromebook which is a Samsung, trackpad terrible. Like and it probably– actually it’s probably got nothing to do with the trackpad. It’s like I don’t know how to use those trackpads, they don’t have a button.

Like I have a MacBook Air but it’s the last one that has a button and I don’t understand buttonless. Like I don’t understand how to click and drag to select text, it drives me crazy. So, we’ll get out of that rat-hole.

John Willis: Well speaking of Amazon — no the one other thing I want to talk about are Spot Instances. For the first time I actually started using Spot Instances.

Michael Coté: This is the one where there is like a marketplace for —

John Willis: Yeah.

Michael Coté: Over time if you will.

John Willis: For like testing and development stuff it’s freaking awesome, you know what —

Michael Coté: It’s like they build them all for cloud.

John Willis: And it makes your life a little more exciting, like you’re going to just fire up like four instances, ah 00:33:37. Oh man, let me go and see what the highest price is right now, and like this adds a little of fun gambling if you will.

Michael Coté: Daring.

John Willis: Daring yeah so but it does — it saves money and what I found also is – I’ve always looked at it and said, “Ah, you know it’s going to be a pain in the ass.” But I mean if you willing to pay — yeah the nice thing is you go in and you look at the instance that you want and — or instances and it tells you what the going price is. And then you look — the little thing you just select and it’ll actually show you over a week or a month or just a day. And you can look — but usually just if I’m in a hurry I’ll just make it like — if it’s $0.24 is that the highest point in the last couple of weeks then I’ll just make it $0.25. And never wait more than about three or four minutes.

And you save, I mean it’s ridiculous like the — I mean like on the — like for example installing Cloud Foundry. If I install Cloud Foundry on a Small Instances Amazon it takes sometimes about 100 minutes, maybe even longer. I’ve actually — I can’t remember it was on one of the things on Micros or a small but I think one of the first couple of times I installed it took almost two hours. And well on average, well even when you look it will say beware when you start install process could take up to two hours.

[0:35:02.5]

I started up an m1.xlarge and basically the normal price is like $0.68 an hour. Again which isn’t a lot, but if you’re going to run it for a few hours and start using them over time. It’s probably worth doing the Spot Instance. It’s like Spot Instance is $0.24, and so I put up for $0.25 I wait like two-and-a-half minutes it comes up, like I get it started, two or three of them that I’m going to use and then literally the amount of time it takes to install like 20 minutes versus like a 100 minutes, or 110 minutes.

And I could have got up to $0.68 and 00:35:43 but I mean on a habit now I just anytime starting up a development or test systems that I’m only going to use for like a half a day or a day, or — and then if I forget to take them down, it’s going to be half the amount of money that I wasted, right so.

Michael Coté: It’s like a price line for cloud.

John Willis: Yeah, it’s not bad, it’s you know — and again if anything it adds a little more excitement in what you’re doing if you’re playing with this stuff. But I could see like if you got serious about it and I know companies do. I mean these companies will — they’ll build queues of servers and they’ll generate pools that can be available.

Michael Coté: So, now what’s the mechanics behind that? Like who is the supplier? Is Amazon supplying it or is it —

John Willis: Yeah, Amazon is the standard cloud supplier. I think Reuven Cohen has this — has some type of abstraction for multiple clouds but I don’t know I haven’t gotten — I’ll probably —

Michael Coté: So, correct me where I’m wrong here, but basically so there is idle cloud running around and at some point they make a decision all right we’re going to sell this Amazon being there — Amazon makes a decision we’re going to sell this top of the line cloud instance for a lot cheaper than list price just because no one is buying it at the moment. So, it’s typical like sales stuff right, like better to make some money than no money off of it. And do you — have you gotten a sense of like when they — how that the mechanics of that work out, like when do they decide they’re going to cut prices essentially?

John Willis: Well I think its available capacity. I think it goes — I think they take the highest bidder. I don’t know the exact mechanics, but I think they take the high — if they’ve got available capacity, they’ll look at what’s waiting. So when you go to start a Spot Instance it immediately goes into this Spot Queue and it’ll just say open, it’ll just sit there. And I think what will happen is they’ve got probably — they’ve got some engine.

Michael Coté: And then how much time do they give you on it?

John Willis: I don’t know. I played around the other night just to see like — I had a, it was like one of the small instances, so I had — I figured what the small instances was typically like — oh no the micro, I don’t know, maybe the small. So, typically like $0.03. They run $0.08 normally and you can get them for like $0.03 an hour. So, I played games where I put in $0.02; I put one in for a $0.03; I put one in for $0.05 and the one for $0.02 just, I just left it in the queue for like —

Michael Coté: But then do they give it to you for — do they guarantee an hour’s worth of time or like I mean can you reserve?

John Willis: I’m not still — I’ve heard stories that you could lose them so —

Michael Coté: Because that would be — I mean there’s got to be some sort of safety, you got to be like, I’m going to buy 200 hours at $0.02 an hour, right?

John Willis: I think they take them away. I think they take them away. I think when that reserve pricing or whatever that pricing — in other words if they get — I’m sure if they get to a capacity as to where they got people who are willing to pay the whole price, they probably, and they’ve got nothing left, they probably start yanking it, because I heard people have said that you can get on the yanked.

Michael Coté: So, the way you would want to use a Spot Cloud is mostly for sort of like processing and serving up things and you would want to be — you would want to be saving your data somewhere else because your Spot Cloud could just disappear, so it has to become obviously completely like stateless, right?

John Willis: Yeah, but I mean I use the — one thing I do a lot, which is cool is I use the EBS volumes and so the EBS boot volumes and so that it’s not a 00:39:25 storage. So everything you write on it, if you go ahead in Snapshot or if you create an image from it, then it basically saves it as your own image. So I’ll get like everything I need. So, I have like a bunch of set images for different environments that I’m testing and I’ll fire those up and then all I have to do is like copy my keys over or something like that and then I’m good to go.

[0:39:52.5]

Michael Coté: So let me ask you this question as I like to say. In your travels up until now, so now if you just take, only taking Amazon as a cloud, like there is a lot of things going on. We just went over like various strategies you would use for a Spot Cloud versus having your EBS and this, that and the other. Like how many levels of sophistication do you think there are for learning how to like deal with Amazon. And I don’t mean that in the bad way but like sort of the most naïve things is like I just sort of like get a server and some storage that I host, right?

John Willis: It’s actually a pretty high is that — there is a lot of bandwidth there and it goes. For example, I mean take for example that outage a couple of months ago, right?

Michael Coté: Yeah.

John Willis: I mean the people — I mean there was — that there’s levels of — all right, so there’s levels of like knowing how to use the APIs and all that and being able to basically manage, starting, stopping instances, the stuff I’m talking about It’s like 101, right?

Michael Coté: Yeah.

John Willis: And 102 is how do you start using the more advanced features like ELB, the RDS, the SNS, the queue if you’re going to use that. And then understanding whether you should be using them or not, and then you got CloudFormation and all that. And then I think that’s like 102, Amazon 102 and then you got kind of Amazon 103 is understanding the implications of how you use it, and that’s where all these people got in trouble.

And then I think that like it goes back to the whole thing you don’t just get into cloud and think, oh cloud, everything is done. I mean even if you learn everything about that cloud you still have to be a data center operations, you know what I mean? You still have to have your data center operations job, to know in what cases best to use.

But then beyond that then there is kind of folklore and people who have been doing it for a couple of years. In that outage only a few company survive. And the ones that survive are the ones that basically either — that either spent a lot of money to bring in a lot of people who had been doing it for many years. With Amazon at a large scale which is like for example Netflix. Netflix is like VMware, now they’re hiring like all the brightest guys out there.

It’s amazing, every other day I’ll see a guy like, that guy is going to Netflix, holy shit. But to have these guys that had been doing large scale cloud operations on Amazon for a couple of years that know where the bodies are buried. I mean that crap doesn’t come easy. And you listen to this — the guy who used to — there is another really, really sharp guy that gives presentations all the time and he — Jeremy and I can’t think of his last name right now, Reddit, he was the sys op at Reddit.

He gives a great presentation about crap that — Edberg, Jeremy Edberg, he used to be at Reddit. He just went to work for Netflix. And he has a great presentation, he was about like, oh all the bodies are buried and how a lot of people are switching away from EBS back to the 00:43:06 storage. So yeah, now there is a lot of layers for that onion to peel. I mean —

Michael Coté: I am always curious about the cloud complexity if you will, which with great complexity comes great power as they say.

John Willis: No I think Amazon — and yeah and with Amazon that complexity creates great opportunities, but again knowing how to — I think I might have said this once before in this podcast. If I hadn’t, I talked to a lot of people after that big outage and a lot of that outage was about like if you follow all the rules as Amazon laid them out you shouldn’t have been in trouble. Like you’ve used availability zones and you were supposedly in good shape if you spread like your database server and you had — you’ve done between zones, and then it turns out there was — I forgot what it was but it was some API that wasn’t available for EBS that wound up, 00:44:12 all those rules.

And the guys that survived were, they had actually multi-region DR set up, you know what I mean? And or — so that like the guys like Netflix, they were all, hey, we didn’t have any problems. Well I don’t — they brought in probably the best architects. They probably paid Amazon architects out and design it. I mean and yeah they were able to handle that outage without a blip, but then there was a lot of small guys that I talked to like oh, we know exactly what would happen if that happened but we never had a time to — you know who has the money and time to design multi-region DR.

[0:44:49.2]

Michael Coté: Yeah, that’s why we’re doing this cloud stuff.

John Willis: Yeah, that’s right. But and then I think one last point is, the other thing when I talk to people and this is again why I’m a strong believer of Infrastructure as Code, there were a lot of people that survived that to a certain extent where their outage might have been like eight hours and it was cut down to two or three hours because they had their whole infrastructure extracted through Chef or Puppet.

Because you still got the data movement problem and where to get that and your backups and all that but the point is that there were people that were able to like go into another region and pop and blow their infrastructure out. And there were other people that knew that and just said I’ve never gotten around to finish my whole infrastructure with Chef or Puppet.

This is why I tell people when we talk about DevOps, I mean, like it is another thing that’s driving me nuts these days, is I’m going to all these big companies and do these DevOps workshops. Get up, yeah I love it, come on get in here, tell us about it. And you do this; I do a first day kind of theory which I’m not going to do anymore. I’m tried of doing it because everybody is like yeah we already know that, yeah get on, move to the next thing, and then — so and the second day is hands-on. So, it’s almost hilarious how, it’s like everybody is like, yeah, we want DevOps but we can’t do that, that and that right now, we’re way too much in a hurry.

That is the core of the ideas, like get all this stuff right, let’s abstract all the infrastructure, let’s figure out the behavior definitions, let’s figure out all these things that are probably going to — they’re going to make you successful or more importantly the things that companies like ESTI and Wealthfront and those guys have done to be successful. Let’s do those things, oh no, we don’t have time for that. Let’s just get the CI server up.

And I’m like I want to pull my hair. It’s like, no, there is — you know I’m sorry, it’s painful, it’s a lot of work and it’s a lot of crap you have to do upfront.

Michael Coté: Well they used to call that cargo culting and I’m sure they still do. It’s like you don’t want to waste your time putting racing stripes on your car, you really need to put a new engine in it, like that’s — just because you have like — you’re doing continuous integration like it doesn’t mean you’re going to have anything. It’s just going to —

John Willis: You got that culture and you’ve got a bad foundation and bad plumbing, I mean this so — I mean so — but yeah I mean that’s the people that sit down and say, okay, there is a guy here in Atlanta. I met him recently and it took him a long time to get up on Chef and I finally met him. I met him at DevOps workshop we did in Atlanta a few months ago and he said, I said, he said, “Hey, I finally got everything up on Chef,” and I’m like that’s great and so what was – why did it take you so long?”

And he was like, “Well actually what we really wanted to do is we wanted to get it right so we did a kind of service modeling exercise where we modeled all our services in an abstract definition and then we use that as input, the drive shaft.” Like, Yeah, God, from your mouth to God’s ears. I mean you build your infrastructure like that you can move a data center in weeks.

Michael Coté: You got to drain swamps before you can build Disney World, that’s the problem.

John Willis: That’s — I love that. There you go — or else that castle will be floating.

Michael Coté: So you were mentioning you were doing DevOps workshops and so there is two things that I find interesting here. One, that there is enough demand to do DevOps workshops. So like — I mean DevOps has been — it’s been a pretty fast cycle of main streaming I guess. I mean it’s probably even about it a year-and-a-half maybe. I mean I know people talked about it for —

John Willis: Yeah, and I think it will be two years, I think this summer it’s about from when the first DevOps days.

Michael Coté: But it’s pretty quick, I mean the last thing we had was cloud which took a long time but not that DevOps is really on the magnitude of cloud. It’s more like a subset if you will but that’s a fast forward session. And so you think that you’re more or less you don’t have to do the definitional stuff, everyone is past that, that kind of thing you’re talking about.

John Willis: Well they’re not even passed it, it’s like — I think there is a couple of forums of either people are in denial and they’re like, oh no, no, we know that, that we got. It’s that whole like whenever I used to explain DevOps in the early podcast we were doing here and I talk about culture and you’re like, “Yeah, but not a bunch of crap?”

Michael Coté: Yeah, that’s my line, isn’t that a bunch of crap?

John Willis: Yeah, and now like you start talking about why it’s so important to understand behavior and why this works really well in some companies. People like will say, I already know that. You know what, we know that too, you can move on, move on, like it. And then you’ll go to the workshop day and they’ll — it will be like, well you kind of missed that whole point, like what’s — I know there are some fundamental things that I think are really important to understand.

And I think people — everybody is in such a God damn hurry that they just don’t want to hear that stuff. They just want to hit the pavement and run and so to them, to most people — so I’ve done a couple — I’ve done about – I’ve done four DevOps workshops. I’ve done two for pay and two freebies.

[0:50:08.4]

The freebies you get by pretty well because who is going to moan and groan about that. And the people are passion enough to go to a thing, they will listen it, but the pay for ones are being very painful because they tell you that, yeah, we want DevOps and We want to just be like ESTI and we want to be like — and then you go in there and start talking about the way why ESTI is what they — they is — why ETSI is the way they are.

Michael Coté: And they is what they is.

John Willis: Yeah, they is what they is, there brother, but they — and they don’t want to hear that. And first they are all like, oh we want to be like them, yeah we want to drink that Kool-Aid. Oh my God it’s going to be great, and then you start going into more than an hour of it, and they’re like, can we get on to the workshop stuff? And what people really want is — particularly the enterprises what they really want is they want to just go right to the cool stuff.

Let’s go install something, let’s install Jenkins and see how that works. Can we do that continuous delivery thing? Like yeah, we can but you want to talk about how that might change your developers and how things might work, and yeah, we’ll get it, we’ll do that later.

And it’s rather frustrating, in fact in some ways, this will be a bold, bold statement but I think the enterprise and — put it this way, I think cloud alone is doomed — cloud with DevOps you have a prayer, enterprise with cloud and DevOps the only way they’re going to get through this cleanly. I think is you’re going to have to go out and hire like 10 or 15 people that are completely from without any of the baggage of the organization and do the project.

Michael Coté: Sure, I think that’s one of your more successful change — like large organizational change strategies.

It’s you have to start out with a small groups that like is not encumbered by the current success with the organization and they have to build such a huge amount of success on their own that everyone else takes them seriously that and then this is the other, the final element that once the greater organization stumbles in a terrible way they get desperate enough and they look around and they see this small group of people has a good way of doing stuff.

And in failure is a certain amount of time enough to revisit core principles and I think it’s just — I mean part of what it is being an enterprise is it’s too risky to ensure your future because you’re making so much money in the present that you don’t really muck around with it and until there is sort of catastrophic failure, I mean and I’m not even putting this in a cynical way. Until there is catastrophic failure there is not much reason to change and now the longer view is that well sure — part of your job is to ensure the longevity of being able to produce revenue.

I’m speaking of very high terms now but — that’s just not how business runs. Like it’s very rare that a business especially when it gets to something that IT is involved in, that they really care that much about their long term future. I mean it’s more like — I guess this is what it is, it’s like the Viking School of Business. It’s like what’s the saying like party today, for tomorrow we may die and that’s the way a lot of it goes.

John Willis: But I mean one of the things I think that the cloud is a compelling event. So it’s compelling events that actually allow you to make these massive changes. But you can look and take stock of everything and say, oh, my God, like you have a massive catastrophic failure or I think again cloud is another — I think somebody said to me recently Wall Street is looking at that DevOps is really kicking into Wall Street.

And this was some — actually Simon Crosby, the guy –the Citrix who — this then Citrix guy who is now in a startup. He was saying that Wall Street is got kind of two mindsets now. There is the kind of legacy and there is the DevOps way, like a lot, really, yeah no and so there is this kind of I think cloud is the enabler or the compelling event that has long been, so you know what, since we’re doing this thing and we understand that there is a lot of changes, let’s do it.

I think the mistake that some companies make and this is a general statement, I mean some people can get away with it, some people may not, I think more people won’t get away with it, is to then cherry-pick all these guys from the legacy. Particularly a legacy that is primarily broken. We know, we’ve been, I’ve been and you’ve been in large companies, oh my God you say that place is so broken, and to thing that you’re just going to pick like 10 or 20 or 30 people from different teams and throw them in this new team and everything is going to work out all new and different, right?

[0:55:02.5]

Michael Coté: Sure.

John Willis: That again I think that, more often that not is going to be, like you’re saying it just going to be repeating failure, you’re just taking the same ideas, the same boundaries, the same limitations, the same way of thinking as opposed to let’s just go hire ten startup guys. Let’s find these guys out there and just like cherry-pick them and put them on this team and see where they get and then start bringing people in from — all the organizations to fit in, but —

Michael Coté: Yeah, I think that’s pretty much true. I mean the — I think Wall Street is always a — they tend to be a leader in anything that involves rapidly delivering technology.

I think for several reasons. One, they have a lot of money, and two, if — I get the sense that there is other industries that are like this, but the world of finance and Wall Street in particular sort of investing, not retail banking is — it’s a lot, not that it’s easy but compared to other industries it’s a lot easier to directly correlate IT work with making money versus I don’t know at a hospital, it’s probably more difficult to correlate IT with like making more money.

I mean I guess anyways, always people are fraught with problems but — and then I think the — I don’t know if there really is a third reason but it’s just — and also like a lot of the stuff that goes in finance now is about — it’s all about having something — information or something that someone else doesn’t have. Technology as IT being Information Technology is great at that kind of thing, like, you can actually use — if you — all you care about is information and being able to like do something based on an information, that’s kind of the whole point of IT, so —

John Willis: Yeah, and I think the thing about DevOps is the people who get with at the core DevOps is — unfortunately some people don’t get that it isn’t a button or it’s going to be a little painful to get there. You’re not going to do it with the rest of this year’s budget or you know what I mean, there’s got to be some changes made, but at the core DevOps is — the people who are getting it like Wall Street is the ability to innovate fast.

I mean — and that’s a loaded term, that’s been around forever but when we look at the way companies are the poster childs of DevOps right now, they are — the developers are pushing 30 to 50 times a day. They’re able to get things from the business people out to customers in hours.

Michael Coté: That’s like several months ago I wrote a thing on Cloud Marketing that cloud is just about speed, which is great oversimplification, but I mean that’s what — I think that’s the explanation that works with everyone. It’s like, look we’ll get you a bunch of jibber-jabber and fancy words and whatever in a little bit but basically we just want to speed up your delivery cycle and just make it so you can deploy stuff faster.

And but then the important thing for that is like as we’re going on a bit earlier is like if you don’t think — if you don’t like the speed of what you’re doing things now and the way you’re things is not working and you can’t do things that way so you need a new way of doing things and like so we can’t just install Jenkins and be done, like your whole thing is screwed.

And like you’re going to have to change things around, like you can’t win the Tour de France by eating like Rice Krispies snacks all day, you got to like change a lot of fundamentals about the way you run your life if you want to compete at that level. And I think it’s similar to whether you’re adapting Agile or DevOps or whatever, like you’re not going to be able to do things the same way.

John Willis: Yeah, change. No I mean, it was funny, whenever we were at Interop. I was on a panel with George Reese, and Shlomo Swidler. And it was a cloud operations thing and we were having a pretty good dialogue with the audience. And one women was talking about how — she didn’t see how her organization could ever get Agile to do deployments like multiple times a day and so at some point somebody asked her, well, why not? And she said well, “I mean we — you know our main application it used to be that we would deploy it every three months. Now, we have to deploy it every six months. So there is no way.”

And as she was answering that I’m like, God, I don’t want to ask her the next question, because it’s going to make her look like an idiot. And so finally everybody is kind of bouncing around, even the audience is trying to help her and then somebody else finally says, “Well, have you ever thought about breaking up the application and decomposing it?” Like “ah-ha”, yeah, but it’ll probably be a year from now, and year from now it will be a year to deploy. I mean so like yeah, no, yeah you 00:59:54. If the answer is I can’t do a DevOps or continuous deployment because it takes six months to deploy an application today, then like okay.

[1:00:04.5]

Michael Coté: No I think that’s there is our friend Israel Gat does a lot of stuff around technical debt and I think that’s like the —
John Willis: Ta-ta-da-da, every time I hear technical debt I want to say ta-ta-da-da.

Michael Coté: That’s right good old technical debt. I mean it really is like, that’s a fantastic metric to track because — and I like to think of it as just like the — you’re tracking when you ask development to do something and they’re like oh we can’t just add that feel, that’s going to take a few weeks, like there you go, you got some technical debt right there and —

John Willis: Ta-ta-da-da, so it’s like Monty Python I just have to say that.

Michael Coté: The problem with technical debt.

John Willis: Ta-ta-da, stop this.

Michael Coté: Is that you can only measure it once you’re in it, like no one appreciates it until they’re negatively affected by it. So it is a — and what I mean by that is you have to have enough experience and trust in your own experience to intuitively avoid it like —

John Willis: Yeah, but here is the thing right so —

Michael Coté: It’s not like a credit card where you’re like, well, I’m going to go $5,000 in debt, like you know what that means.

John Willis: So, I’m at — actually at DTO we’re working with some really large clouds right now, some massively large clouds.

So, we’re getting in the ground floor from on the kind of DevOps angle, and the thing is like as you go through this and go through these workshops and walkthrough what’s got to be done, you start realizing and you don’t have to Israel Gat who is an expert on this as opposed to me who is not, but you don’t have to be Israel Gat to understand all the branches that you start talking about that become obvious technical debt.

If you’re building a new cloud and you’re talking about well we’re going to use some things for CentOS and some things on Ubuntu, I’m like, Wooh-wooh, why do we care about operating system. This is a grand new — brand new scenario, like do we really give a crap what the operating — I mean if we care let’s pick one.

So, because immediately once you start talking about running — these are things that everybody else is battling like, when you talk to people like, yeah we’ve got another six months before we can get everything on one platform, we’re still working on that so we won’t be able to get to this until we done — like okay. Like, we know that now, you’re starting a cloud, let’s make the operating system as innocuous, I mean it just doesn’t really matter.

Then like I was talking about the guy from Silverpop, let’s service model, every service day one. All along the way we are like just chunking off potential technical debt, because the technical debt of installing everything manually and having multiple operating systems, multiple frameworks for operating systems. The developer will say, well it has to be that way. No, freaking we write you code, this the new cloud, this is the new infrastructure, because those are the positions when you have to move a data center or when CentOS version whatever is now really just way too old, it’s not just old, old, it’s old, old, old, old.

And now I got a — I mean it’s pretty easy on the ground floor. I think it’s much harder for what guys like Israel Gat and those guys do to go into large organizations and help them understand with all the legacy crap that’s lying around. But if you’re starting from a greenfield, I mean you can look down the field and say, holy crap man. What we probably want to do is put a big old fence around this thing is the first thing.

I would say it is like if you’re going to start a car company today. I’ve probably said this on this podcast before. If you’re going to start a car company today what’s the first thing you do? The first thing you do is not go start assembling your first car. The first thing you do is you build an assembly line, a factory. It’s going to get cars. Like you pretty much know that like you don’t want to get down to like the six months going, hey, you know what, this whole like manually building a car at a time is you know — and I guess something to say there fancy cars out there that they do it that way, but let’s just say you are going into that commodity car market, you would build.

And so — and I think that’s — a lot of that, just thinking about all this DevOps and all the things you have to do, I mean a lot of this is thinking of IT very much the way Toyota thought about manufacturing. I know I’m not the first one by a longshot to say that but when we have a greenfield cloud we actually can — it’s a little bit different than saying we need to be lean in our organization and we’re a — 20,000 server, 200,000 people organization saying that we need to adopt lean principles, is a lot different than sitting down and looking at a greenfield and say, hey folks, wouldn’t this be a great time to adopt lean principles?

Michael Coté: Yeah, like start with the ideal way you want to do stuff.

John Willis: Right, so anyway.

[1:05:01.8]

Michael Coté: Well that seems like a good place to wrap up. Did you have anything else you wanted to go over?

John Willis: No, I think that’s it right now.

Michael Coté: Yeah. Well, thanks as always to everyone for listening. We always appreciate it. And I guess we’ll see everyone next time.