In this episode, John calls in remote from Velocity with two of his buddies from Opscode. We talk about the conference and Opscode two announcements this week.
Download the episode directly right here, subscribe to the feed in iTunes or other podcatcher to have episodes downloaded automatically, or just click play below to listen to it right here:
Show Notes
- How’s Velocity?
- “Cultural things” – continuous deployment, a more Agile environment – tool chains, performance track.
- Overview of various of keynotes
- High scale and high-rate of change – the second is a subtle context requirement.
- What are the exact types of ops people there? “Web operations culture is now becoming the dominate operations culture.”
- The Opscode, Dyn, Zenoss BoF…
- What are people monitoring, exactly?
- The Opscode news: funding & platform beta launch. See my Quick Analysis on the topic.
- Why take more money on? More engineering needed, hire some. “Truthfully, that’s it,” says Adam. Make the product easier to start using.
- And then the Opscode Platform – “its like a hosted Chef server.” Any object in the Chef space can have an ACL… 20 nodes for $50/month, for free for 5 nodes.
- Example thinking of applying Opscode Platform…
- Who’s using this platform at the moment, what types of organizations? Some talk of people managing Windows desktops.
Transcript
As usual with these un-sponsored episodes, I haven’t spent time to clean up the transcript. If you see us saying something crazy, check the original audio first. There are time-codes where there were transcription problems – this one was esp. dicey for the transcript service.
Michael Coté: Well, hello everybody, it’s June 24, 2010, and this is the IT Management & Cloud Podcast. As always, I am one of your co-host Michael Coté and I’m joined on the Max speaker phone by the other co-host and two guests that we have. So why don’t you introduce yourself co-host?
John Willis: I am the co-host John M. Willis at johnmwillis.com and now I am at opscode.com.
Michael Coté: Who else do you have there with you?
Chris Brown: This is Chris Brown. I’m VP of Engineering at Opscode.
Adam Jacob: I am Adam Jacob; I’m the CTO of Opscode.
Michael Coté: So this is our calling in remote from Velocity 2010 of the Velocity 2010 Episode. Since we have that context, I mean you guys are over there and whatever, we are part of the area that’s going on at the movement like. So how’s the show going? How’s Velocity this year?
John Willis: Bigger than ever. It’s a huge community, but still a super high quality.
Chris Brown: Yeah, I was — that’s the thing that’s been most impressive to me it was a bit like – I’ve been coming since it started and there was 200 people and then 500 people, and now it’s 1,000 people. And what hasn’t changed is that feeling like you’re amongst this tribe that has this unified purpose and shared experience that is really unique.
I’ve never seen like – I’ve never seen communities grow that fast, where it was all so smooth and so clear and without having to branch out into some other vertical. There was clearly this very untapped need for that really focused operations conference. It’s really about building the culture and a practical utility. The culture tracks that are happening here are getting super well attended, because people just want to understand their own culture better, right? Listening to people who think about what we do and how you could be better at it. But aren’t just talking about how to configure your proxy to be faster, they’re still — they’re talking about how to make your organization faster. It’s very cool.
John Willis: Anthropologists should be here.
Chris Brown: Yeah.
John Willis: There’s something weird going on.
Michael Coté: That’s how important operations culture is, we need some people to study it. So, since you bring it up I mean what are the — beyond the configuring proxies as you were talking about, which is always fascinating and it’s a good bedtime reading if you want to stay up late, what — what are the cultural topics people are talking about? I mean is it, sort of, the same old why do people keep doing dumb stuff and things like that kind of stuff or are they more interesting — or are more — new things going on or what are the topics that are floating around?
Chris Brown: I think you’ve moved the past the why do people do dumb stuff, like we just accept that we do dumb stuff.
Adam Jacob: That was year one.
Chris Brown: Yeah, year one was like why do we do dumb shit and year two was like we don’t to have to dumb shit, and now year three is like we’re past dumb shit and now we’re talking culturally about what it means to no longer be doing dumb shit, which means that — essentially that there’s new dumb shit; it’s basically how that works like undiscovered new —
Michael Coté: So, you’ve moved on to wondering where shit comes from? A higher level of order of things —
Chris Brown: 00:02:49 and now it’s like where does it come from?
John Willis: This is a very existential conference.
Chris Brown: Yeah.
Michael Coté: So, what are – I mean like given the — although it is kind of funny to have the endless shit metaphors in the layers there, I mean what are things people are talking about?
Chris Brown: So certainly the conversation is about things like contiguous deployment, conversations about things like having a more agile operating environment and all those things that, that means both in process and in people, and then finally in tools.
There’s been plenty of talk about tool change – the performance, Track (ph) at Velocity has been really cool and that in previous years they’ve been really focused on front end’s performance and it still is, but there’s been a couple of really good back end conservations and a couple of really good back end performance talks.
There’s been a lot of really good talking about metrics and analytics, which is always really important. I’ve seen the cultural part of that coming to the forum more, where there’s more and more people talking about why you need to have great metrics, like almost every presentation is like, and you should be tracking this.
John Willis: And when people are geeking-up, comparing their metrics solution to each other, “No, look what I can see from my metric solution.” A couple of years ago it was all about heroism and now suddenly it’s about maturity.
Chris Brown: Yeah, and my favorite conversation in the hallway was with John Adams from Twitter where he popped open his laptop and the first thing he did was fire up MATLAB and he was — I wanted to hug him, you know, I mean it was like here are my people and it’s magic.
Michael Coté: And our people still like calling that stuff science. We started using science in the way we did operations or has it moved beyond that sling.
Adam Jacob: We’re certainly – and we’re certainly seeing math more often than we use to.
Chris Brown: And we’re saying engineering more often than we were before. Like systems administrator are part engineers, they’re part like a voodoo witchdoctor. And because so much of what you do when you learn a system is instinct, but the voodoo is like a shortcut for engineering. You’ve learned that because you’ve learned to do these engineering principles but you’ve made the magic obviously for our style.
Adam Jacob: Yeah, it’s a 00:05:04.
Chris Brown: Yeah. So now it’s going to be coming back and saying, “Okay, great; like we have these pretty skills to be able to interpret this data and you can fight, but not the logical sort of way, but how do we actually make that engineering again, how do we actually go back and apply some rigor (ph) and some process to ensure that our voodoo is correct.
John Willis: And John Adams is actually a really good example of that.
Chris Brown: Right, you know, he is one of those dudes who will sit down and use algorithms for financial trafficking in order to find better — how to how to optimize the systems, right? He’s doing – he’s taking stuff from sound engineering which will cut out the high and the low end and when cut all the low ends just to see where the peaks are and then you build a single line that shows you the highest bar and the duration of that loop between two edges, it’s very cool.
Adam Jacob: It’s funny that for me just talking to John Adams about — he was telling the thing he dreaded (ph) like – that was the day before the US scored that every time there’s a goal scored, he’s got to 00:06:03 out his laptop and start looking for failures because the 00:06:07 pattern is in and so that was before the U.S. scored a goal.
Chris Brown: It’s fun stuff.
Adam Jacob: Yeah, cool stuff.
John Willis: Which is clearly 00:06:11.
Adam Jacob: Yeah.
Chris Brown: Yeah, and there’s a lot more going on here. MATLAB is – it was a good day full of workshops, there’s all kinds of stuff, you could stay here 00:06:24.
John Willis: And actual fire drills.
Chris Brown: Yeah, there are actual fire drills.
Adam Jacob: Yeah, 00:06:28
Chris Brown: Yeah, during the Adams talk we actually was a — actually there was real fire drill, we all had to the leave the room, but — because it was almost over.
John Willis: We go this special 00:06:37 to like jump out from the ceiling to like 00:06:39.
Michael Coté: Did everyone successfully drill outside or get outside for the fire drill?
John Willis: Yeah. It was a lot of very hasty people on the 00:06:50.
Michael Coté: That’s right, so what have been – there’s — do they have like keynotes this year? Is it just separate tracks? What have been the big talks that people have had?
John Willis: Yeah, there’s always keynotes. So yesterday was Tim O’Reilly, and James Hamilton who is totally a rock star, looks like a rock star.
Chris Brown: Looks like a rock star.
John Willis: He looks like the dude from White Snake, if you’ve never seen him.
Michael Coté: Right, right.
Chris Brown: 00:07:19 James Hamilton.
Adam Jacob: IBM, Microsoft, Amazon
Chris Brown: Early, early days of SQL Server. 00:07:27 bunch of the internal 00:07:27 that makes that actually kind of cool.
Michael Coté: He was internally 00:07:31.
Chris Brown: He does, yeah, so James 00:07:37. Tim O’Reilly gave a great moving talk about the origin of Velocity and how that fits into his vision of how the future is here. It’s just not evenly distributed and has the role of operations people within that new future, and how and what we do is starting to become more evenly distributed, which is more 00:07:53 about the people. A decade ago if you had a web ops conference, you would have been like me and some dudes from AOL. And then today was Keynote from 00:08:03.
Adam Jacob: 00:08:05.
John Willis: He’s — what’s his background, he was very fast.
Michael Coté: Oh! Like you ever remember, that sounds right.
Chris Brown: And Amazon and a bunch of other places, but he gives really great talks, again, about culture and this one was about basically moving people forward with politeness and humor and using that as the tool for organizational change rather than passive aggression and 00:08:30 very short. I will be back 00:08:33.
John Willis: No I think that’s great, because — and he was talking about how you capture people into your community and how you entice. Frankly I thought that 00:08:40 how about you?
Chris Brown: 00:08:39.
John Willis: We’ve got a really good community and that’s — and those are the same principles that we have been using to —
Chris Brown: Be nice to people and they want to 00:08:48 back.
Adam Jacob: That’s right.
Chris Brown: Go crazy.
John Willis: Which means they contribute.
Chris Brown: Yeah, hard at work.
Michael Coté: Absolutely! So like you were saying there was — you know you were breaking out some of the topics like people were talking about continuous deployment and being more agile, and tool change, and performance, and stuff and so what — when people are talking about the experiences that they’ve had and things they are doing what are some of the more interesting, I don’t know, sort of, as we would say, cases or reports people are having about new, interesting ways they’re going about their operations job?
Adam Jacob: I don’t know that I’ve a heard that sounded new.
Michael Coté: Well, there is —
Chris Brown: Some big folks coming back to reporting for the second year 00:09:31 increase in scale. It’s just suddenly how much bigger everything is getting.
John Willis: And, I think, to a sort of mix then how much they’re learning about what it means to operate that way, do you know what I mean?
Michael Coté: Sure.
John Willis: So I don’t think it’s — in terms of like if you’ve been hanging out for the last two years then it’s not new. If you haven’t, then it certainly is. I think from technology point of view a lot of that stuff really comes down to the fact that for a lot of these people do still wind up having to build too many tools. You know, like the — then again to reference John Adams he — they wind up building their own deployment tool that uses BitTorrent, because they couldn’t get anything that could deploy fast enough.
Michael Cote: Right and then to set the context. I mean, the scale of what you are talking about, the context you people are operating in are like very, very high scale web hosted things or high scale IT that they have to deal with in most of the focused or the web things.
John Willis: And how the things have changed. So you’re talking about being able to adapt so quickly. Doing that deployment all the time implies that you’ve either got low latency to all of these systems or you’ve got your bits close at hand all the time with some sort of magical staging. And everybody builds a tool like that sooner or later.
Chris Brown: So, I think, a lot of what you hear that really will blow you away at Velocity is from those sorts of hallway conversations. It’s like you’ll talk to a guy who’ll come up me after the talk that I gave, I had people come up to describe how they manage their infrastructure with Shaft or with 00:11:00 or with CF Engine (ph) and I’m like, you know, we had a group of five or six guys standing around debugging someone’s 00:11:06 problems. I’m like, me included right, I was the one telling them how to fix these problems, like it was awesome and not only that happens because there’s this group of people who are really basically spending their day caring for each other for three days.
John Willis: And not afraid to dig right in right there, at the time.
Chris Brown: And this is real depth. If you had a lot of people asking many probing questions – I’m like trying to figure out exactly how to make his world better, and that’s awesome. And those are the ways that you learn what people are doing that’s really interesting. That’s where you hear people who are like 00:11:34 that was a really innovative answer, you know, the BitTorrent being one. Lots of people pulling out log systems, those sorts of things, very cool.
Michael Coté: So was there a sense that like most of these people were doing, sort of, a like public web sorts of things or were there other high scale, high rate of change environments people were doing things or was it he unusual traditional kind of consumer centric facing things that people were messing around with?
Chris Brown: I think it was still predominately web. And I mean there’s a reason for that, right? Where the DevOps thing, I was 00:12:11 I’m going to give later on the day that talks about how to back it, and what that really means. If you are a student of our culture, if you like the anthropology of it, what it actually comes down to is that web operations culture is now becoming the dominant operational culture.
There is no difference between — it used like — the traditional operation culture is now actually being replaced by the web operation culture and the way they think about the world, and the way they operate the way they move and the speed at which they do it. So, this conference winds up attracting primarily the vanguards of that movement and you see people creeping around the edges, I would say maybe 10% are from really traditional enterprises, where they look up to you and they are like, “Hey man, I work in big pharma,” or “I work in, –”
John Willis: Financial.
Chris Brown: Financial sectors or oil and like those are here, it’s just that they are just bleeding edge of their companies. Do you know what I mean? They are absolutely pushing the boundaries and more and more of that’s happening. And that’s really what that kind of moment is all about. So Velocity, I think, still very much is web performance focused, because that’s where the leading edge is. It’s certainly creeping out.
John Willis: Well and thank you cloud computing because some of those enterprises that are getting here because the cloud gives them that the ability to have high rate of change and adaptability that they just don’t have inside and when you are on the cloud every thing looks like web ops, suddenly you have the same sets of problems, the same set of techniques.
Chris Brown: Yeah, it doesn’t even matter what you are doing on the cloud with web ops.
John Willis: Exactly.
Chris Brown: You can’t fake it.
Adam Jacob: 00:13:46.
Michael Coté: No, no, I was just, I was just —
John Willis: You were in deep thought, is that right?
Michael Cote: Staring into the distance of operations culture, that’s all.
John Willis: 00:13:57 That’s right.
Michael Cote: So you — and then — so you guys also had like a — I don’t know what it’s called, 00:14:05 sort of thing last night with — what was it, we are talking about this in the last episode with you Opscode guys and I was thinking if it was Dynect or Dyn dean or Dyn whatever. The Dyn folks and the 00:14:18 whatever right. How did that go, what was that first off to recap for people?
Chris Brown: Yeah that went pretty good. Basically it was — we got this Zenoss. So we actually went down to Watergate and water flowed out and it turned out really well. We did a couple of different simulations where we went ahead and set-up like an Amazon east and west setup and then we knocked out a few machines and knocked out a few databases and then Zenoss picked it up. So it was build with Shaft to the recipes and what not. We knocked down some resources, Zenoss picked them up, Zenoss called the Chef to go ahead and repurpose the — add those machines and then from Chef we also called a provider that actually — actually Adam kind of wrote most of it, I don’t know which one they actually used, they wrote one and we wrote one, so —
Adam Jacob: Oh! They used mine.
Chris Brown: They did used yours and then Alan called there global DNS balancing and then we tried to do one through Multiple Zones where they use their global DNS sauce. It was cool, we had a QA, and we able to show the Zenoss screens of the machines and what was really cool is we — and it’s kind of nice I think we’re going to try to do it again in OSCON because if we were a little better prepared, we would have had like big screens and we were just getting out and everybody was kind of logging in through our website.
So the website was basically a sign-up for their DynTini drinks and so everybody signed in to the app and then we gave them the URL for Zenoss and then we gave them the ability to go look at the DNS entries. So they were actually playing with — there was a fair amount of people in the audience playing along on their laptop and that turned out really cool.
Adam Jacob: It was very cool. The Dyn guys are very impressive, like I thought that the API that they’ve built is really good and 00:16:02 for a really long time and the infrastructure that we’ve just been deploying. Being able to use those services and the chef recipes that takes our EC2 nodes, dynamically generates them a name that’s unique, had the role embedded it, the environments they’re in, the instance ID, and then updates dynamically Dine DNS. You know those recipes are — they’re 20 lines long, like it’s really, really simple to make it — be that dynamic.
If you think about spreading that app to the rest of the infrastructure like if you wanted to port your internal DNS over to Dyn and all your servers have the properly configured host names, then about four line of Chef recipe that you just apply to base and just deploy it on your whole infrastructure and you would port yourself to Dyn in the time that it takes for that converse to run and that’s pretty ethnic.
John Willis: So these guys are doing what we keep repeating over and over. Primitive (ph) simple composable services do one thing really, really well. 00:16:56 make it configurable and composable and let somebody else build policy on top.
Michael Cote: And then earlier like you guys were talking about how several people are like — whether it’s — Twitter needing to use BitTorrent to like as a custom way of cobbling together the tools they need to manage this and so forth and so on. Like going off the sort of, I guess, purposefully cobbled together thing that you guys were just talking about. Do you think it’s — is it sort of representative that you use a bunch of, I don’t know, sort of products and services like that to manage these various things or is there a lot more, I don’t know, custom coding still going on or fitting together things that aren’t necessarily meant to be fit-fat together.
Adam Jacob: And some of these services the ones that are too primitives that are purpose built, they — you don’t have to kick them to fit anymore, they don’t try to do more than one thing well, so there’s a lot less working around the edges. If it’s purpose built for that one thing then it’s actually pretty easy just to fit that API into whatever else you do. It used to be a lot worse. Yeah, you used to have to work around the edges of what they are assumed you wanted to do.
Chris Brown: And the thing about like — the thing that kind of drives you toward those services it’s quite frankly being lazy like.
John Willis: Totally.
Chris Brown: I am going to be modest. I’m an alright systems administrator and there’s a moment where I could have built the app infrastructure that updated the done files based on the things that built EC2 and published them and 00:18:23 items. Like I have those skills and my tool chest, I know how do it, I’ve done it a dozen times before in my life, I could do it again and 00:18:32 the whole thing. And it still would have been less awesome that the fact that I had an API I could call to Dynect and not have to care about it. That’s — the convenience factor alone makes it worth paying for their service. Then when you add in the fact that they add in a bunch of other services, they quite frankly are hard to build, global server —
John Willis: Global server – Global load docs (ph).
Chris Brown: Global load docs (ph), failover, short ETL monitoring, and there’s a bunch of stuff there that, that like I could do those things too but that’s even more work. And so I can always get it to — I can get it to 80% real fast and that last 20% is brutal or I won’t have it for months and that’s the stuff I want, like that’s the part I need fairly quickly. So I think there is a trend towards those services and I think there’ll continue to be one wherever you can find those primitives. When you can’t find those primitives I think people will continue to write extra code.
Adam Jacob: 00:19:19.
Michael Cote: And then also like related to that scenario like Zenoss was in there for cataloging what’s there and I would assume they do some amount of monitoring and things like that. And there has been more people getting in to the monitoring of — it’s more like skewed towards cloud stuff, but we’ll just put it in the context of the high scale, high rate of change environment and like looking around in the conversations you’ve been having, what’s the degree and type of monitoring that people are doing of these things?
I mean is it sort of just like you just monitor everything or that different things are monitoring or how do they handle that aspect of it or what are you sort of — when someone fires up there MATLAB thing, what exactly is it that they’re looking at? What’s the stuff they’re looking at to figure out what to do next?
Chris Brown: Yeah, I mean they’re looking at the rate of change, they’re looking at — they’re looking at jitter, they’re looking at standard deviation. So like lots and lots of people are starting to understand that if what you see on a graph is an average, but you don’t see the standard deviations you don’t know if you have a problem.
If the average is in middle, but your spread it like ten seconds wide, your average still might be two seconds, but some of those requests are brutalizing you and you won’t see it in the graph. I think there’s still a lot of people using open source modeling source. Most people using Ganglia, using Nagios, using you know — and even the commercial open source tools, I still see the less of when I started talking to people for 00:20:49 infrastructure and scale for as cool as they are.
I clearly see people who I’ve seen at large scale using those tools and I think it’s because there’s a bigger market and it’s not those guys. 00:21:03 there’s a wise choice there about the target course you do and those guys have some specialized needs. I think those new services that are coming out that started off a cloud 00:21:16 and a lot of that remains to be seen exactly how far they go — they’re both 00:21:21.
Adam Jacob: Their problems are so unique rather 00:21:23 John Adams where I was sitting next to the John Osfar during your session. And he’s sitting there, he’s laughing and I’m like I guess work never ends, he’s sitting there with a full screen and basically where he watches all days for 00:21:36 and he’s sitting there and I just can’t imagine that there’s a product that would just walk in and say, okay here’s what you need.
John Willis: Right, there’s no turnkey for that stuff.
Chris Brown: Which is why we harp on primitives so much; there are primitives. Now the reason that 00:21:51 building their own tools and using things like Ganglia; is Ganglia provides a really great primitive for putting metrics into the system and you can do it at a fairly high rate and you can do it a lot. So that’s their hardest problem and then graphing it is a different thing. They’re willing to put the work into graphic; they’re just not willing to put the work into 00:22:09 metrics back.
Michael Coté: What do you mean by that exactly?
Chris Brown: The hard part is instrumentation. The hard part is actually knowing what you want a watch and then being able to watch it no matter how much 00:22:23 you need to be and then there’s a second problem which doing that like actually displaying the data and we would all love to have better tools to give the display, but we’re willing to open MATLAB and extract the data from a RRG (ph) because we know that the thing we are trying to do is a little weird. But we’re not really willing to have to write to our own systems to get the data back to do the instrumentation of that part.
Michael Coté: Right, right.
John Willis: And then I think — I’ve seen a couple of examples too. It was like this morning some guy was showing us how he used to managing logs at scale and he’s using MapReduce and all that stuff that just — he use to get really existing about, he was looking for traffic who was like Cameroon and his sight and he was floating all his logs, Apache logs in to MapReduce and he had like using Peg (ph) and he was basically just showing us right on the fly that, look here some guy on Amazon just hammering us, he looked it up sure enough he was able to find out there was some Blackberry thing that was very — I don’t know it was — he was trying to use this to time out problem and —
Chris Brown: Folks who 00:23:24 for instance Twitter on their Blackberry, get the password and the Blackberry back into hammering (ph) Twitter and nobody notices but Twitter.
John Willis: So yeah where are you going to get a software product to do that, you know, what I mean?
Michael Coté: And it sounds like the kind of metrics is too narrow of a word for this, but it sounds like the types of things people are monitoring are sort of external forces that are negatively affecting their performance, if you will. So whether it’s like a Blackberry thing or some burst of traffic or goals being made on some field or whatever. They’re sort of — so you’re kind of monitoring, I don’t know, transactions or connecting user sessions and things like that. But I mean are people monitoring the more, I guess, state, if you will, storage available or I mean — I’m coming up with really stupid examples, but like what are those things that people pay attention to if at all?
Chris Brown: There are also lots and lots internal metrics too when you just think about, for instance, latencies between internal services. For the sophisticated services where you see one page and it’s actually composed of the results of possibly a 100 different internal services. This is huge mark off chain of what’s actually going on inside and then it gets all the way back to the first 00:24:40 of how much storage do I have left, and my database just gets slow because of something that’s happening on disk.
John Willis: Yeah, there’s a lot of — I mean all the basic stuff is almost always instrumental. And then — but certainly not, like I had a few conversations with people who are like, yeah it was really cool I — we had those presentations and we had this alarm going off, but they’re started using service like 00:25:03 to actually send out their pages, but notifications were more than a 154 characters long. So they were missing pages. They’re were like, “Holy crap the disk is full, holy crap the disk is full, and the holy crap the disk is full” and it said it was just too long so then they 00:25:17 my people back and we got that message though 00:25:20 and like they’re not the only ones who’ve had that problem, so there’s — it’s not all 00:25:26 in the service oriented world.
Michael Coté: Yeah, yeah I mean I ask — at that high scale some of the conversations I get into with monitoring people is, at some point you sort of sacrifice a certain level of detail and you kind of up the level of I don’t know what you would call it a badness if you will and so rather than worrying about little individual components or more atomic things you sort of wait for things to role up and you just kind of blow away whole sections of your system if you will in the hope that it some how fixes it, so it’s always that interesting to figure out how people are handling that.
Adam Jacob: And we used to call that OGF, the Overall Goodness Factor.
Michael Coté: Right that makes sense. So that you guys I mean Opscode also had a couple of big announcements this week I guess on Monday you announced that if I remember the date correctly or sometime earlier in the week and then one of them was around another round of funding that you guys got, the other one was the —
Chris Brown: Oh! No that was 00:26:26 buddy.
John Willis: In order to get the time.
Michael Coté: I know it; it’s all in there, I’ve double-checked several times.
John Willis: 00:26:36.
Michael Coté: And then there was also you guys launched the beta of the platform that you have, why don’t you guys go over that, what those announcements were?
Chris Brown: Sure. So let me — yeah we raised money from some really fabulous investors, Battery and our previous investors DFJ both came together and fueled up the process a little more, and we’re super proud and happy to have them along like, Battery has a really great history in the space so that 00:27:09 and they really understand infrastructure, they really understand companies who have, who have big visions and who really want to like build things that are awesome and they understands, we’ll try to help you do that and how to navigate some of the borders that come by. DFJ has been ridiculously fabulous this whole time.
Adam Jacob: From the beginning.
Chris Brown: From the very first moment they understood what we’re building and were willing to do whatever it took to help us and so too get us where we were going. I can’t say enough for the people that are on our Board for the people who can 00:27:45 diligence who kind of remain 00:27:50 talking to us, like those guys have so much deep knowledge about what’s happening in the world and in the industry and they’re still out there. These things — when you think about those guys you tend to think of them as not going to say this by getting myself 00:28:01, like you tend to — yeah I’m not going to say any of those things.
Michael Coté: Well, I’ll tell you other people tend to think if anyone who has a lot of money that invest in tech is really their only good for their money; that’s the general prevailing attitude.
Adam Jacob: Yeah and the thing about that, at least, with DFJ and with Battery that is absolutely not true. They’ve had quite a bit of insight both in the business and then also in the product and the way that they think it should be brought to market and it’s been incredibly valuable. I know that’s not true for everyone, for everyone of the investors. It’s absolutely true for us and so we love them and we’re really happy to have them as a part of the team. I’m also just — the team we’ve built at Opscode I couldn’t be proud about — the two thing that we done so far that that makes me all VP.
Chris Brown: 00:28:51.
Adam Jacob: Yeah, I actually did, it was 00:28:56 all these incredible people — a couple of years ago I quit my job and started this consulting company, because I was tired of it and then the next 00:29:08 and then the next thing you know, its like all these incredible people made this thing their own. It’s not mine I’m in the mine for years. There’s so many amazing people who like heard this little kernel of a thing and 00:29:19 that does sound like a pretty good idea but what — it would be better if we did this. Well, did you think about building — we could do that like a platform and what if we do this and I’m like and then all of a sudden there’s this snowball ridiculous engineers who are way beyond my caliber.
For the record, who have taken the thing and loved it and spent so much – most of their time and their life and then Chris Brown works with me what’s that about? It’s crazy talk and then you pile that with the open source that option that we’ve had, where there’s all these people who — there’s 150 people in the world who, not getting paid, just liked this and we’re like — and showed up.
Michael Coté: Quality contribution.
Adam Jacob: Yeah, high quality code that makes dramatic changes to the products and —
Chris Brown: And a lot of the guys are here this week.
Adam Jacob: Yeah, a lot of them came out this week and like — and they’re so great and they’re so kind and they’re so — they care about each other so much like it’s just — it’s amazing.
John Willis: It’s fun to watch two guys you’ve never met before argue with each other about how they ought to put code into your product.
Adam Jacob: Yeah.
Michael Coté: Yeah. You guys have attracted high profile and otherwise profile good contributions and given that you haven’t done a huge amount of like marketing push and I don’t know what you would call it, community relations like I mean there has been some of that, but it hasn’t been like huge mega steroid, it’s good that you’ve had that amount of contribution so far.
Adam Jacob: Yeah, and it’s just 00:30:48 I’m incredibly thankful for it. So when I think of the money, what I actually think about is all of those people now get even more validation and even more resources like continue to like make that snowball better like —
Michael Coté: Yeah, and I mean to that end, I mean what are the kind of like the general plans you have with that kind of cash? I mean people are, people sometimes say they’re just kind of like sit on it and hold it or whatever, and there is more marketing efforts you can always spend money on, and there is other product lines and acquisitions and all sorts of things, so like what, like why did you guys want to take money on?
Adam Jacob: One pretty straightforward answer is there is way more engineering than we can do right now. We got so much stuff we’ve been talking for the last year about embarking on and just not enough people. So a lot of it goes to straight to R&D, more engineers, more boots on the ground.
Chris Brown: Yeah, and that’s — truthfully that’s it. I mean we’ll spend money on marketing, we’ll spend — we’re going to lower the barrier to entry to the product, like it’s — right now it’s a great product, but it’s very 00:32:00 a little high for some people.
Adam Jacob: And Training and outreach for the folks who are, yeah who are 00:32:02 to get things up and running.
Chris Brown: Yeah and there’s a lot of things we can do with our cash that helps to lower that barrier that help make you either to adopt, that help get more support and more things in the hands of customers. And I mean that’s where the money is gone for truthfully and the reason it’s a big pile is there is a big pile of work. Like if it was done, if it was already 00:32:29 and there was nothing left to do, and this is the totality of Opscode, we wouldn’t have raised another round, we would have, or we would have raised a very small one, right? But it’s not, it’s a fun stuff to do. So we are not even near done, like this is the very beginning.
Michael Coté: And then so give us an overview of the Platform that you guys announced the beta of it, like what exactly is it?
Chris Brown: Sure, so it’s a couple of things, so at the simplest level, it’s like a hosted 00:33:00 provided to you as a service. You can go to the Opscode website, you can sign up the 00:33:05 form, install the Chef plan and just starting managing infrastructure 00:33:07. Behind the scenes, there’s a whole lot more going on in there. There’s — you should talk about it 00:33:14
Adam Jacob: Oh! Yeah, so one of the things we did to make it possible to host this for lots and lots of customers of scale and to make it secure as we got a fine grained access control system, so basically any object you can think about in the Chef space can actually have 00:33:29 associated with it and we’ve got identity for everything. Identities for users, client identities for the machines to check in and configure themselves and so you got a fairly good set of tools around say, well these users should have rights for these things, they can read cookbooks but possibly not write them.
So all across the system you can secure your data and secure the operations and then you build rules out of that, you got a groups to focus in finances or groups of system administrators to have root-level access. All of the things you’d expect from a grown up enterprise system we’re building into the platform now.
Chris Brown: One of things that is great about the platform is it continues to show that everything we do starts out life as an API. So what we actually have is this incredibly great API for doing 00:34:19 control and on top of that we use that API in our own services internally. And now without the public API and like as we grow, that’s a pattern we will continue to repeat. Like if you go to the Opscode website and you look at the Console, where you can log in and mange infrastructure from the web console, that console is nothing but API call, and it doesn’t call any private APIs. Everything that’s inside of that you can call and build your own, so if you think what we did is not so cool, that you could build a better one —
Adam Jacob: You can build a better one.
Chris Brown: You can build the better one and just point it to API and call it a day. So I am really proud of that. The other thing I am really proud of is that, when you start getting into this space, if you go look at the traditional big players, it’s incredibly expensive to get started, managing infrastructure. And then the support costs are outrageous like they make so much money doing support.
So when you sign up with Opscode Platform, basic supports are included in the package, right, you can have 20 nodes on the Opscode platform, fully managed for $50 a month, and you can do it for free for 00:35:26, and all of that always include basic support – you can ask a question and we will help you. John’s oracle (ph) will fly out and train you and the prices on those things, quite frankly, we are doing it way ridiculously cheaper. Like access to this sort of automation and that sort of knowledge like we are doing that in a way that nobody else.
Michael Coté: Yeah, I mean I know it’s a pretty painful price slash compared to bigger enterprise installs.
Chris Brown: Yeah, and for me the reason I am so excited about being here I was the co-host, if you will, is that infrastructure is growing like weeds, and when Adam says he is so proud to be working with Chris I just 00:36:12 bragging line, but it’s true 00:36:13 gentlemen that architect and design correct me if am wrong, EC2. And I like to joke that this is EC2, version 2. You know what I mean. He is the architect of a massively scalable infrastructure but that’s something few people in the planet can do.
Adam Jacob: I think we only have bunch of guys.
Chris Brown: Yeah and so if you take the combination of an infrastructure that’s ready to support a massive scale and if you leave any assumption that like infrastructure is growing as weeds right, I mean everybody can get 10, 20 or 1000 computers and everybody wants to analyze like a ridiculous amount of data, we are just going to see this phenomenal spike and people owning their own datacenters and they’re going to have to be managed. And so it just doesn’t get any more exciting than that and so —
Michael Coté: So what the platform, so let’s walk through like a theoretic scenario here, so let’s say I have like I don’t know, let’s say 500 nodes or whatever and I want to start using like the Opscode platform, so like never mind like go to the website and sign up or whatever, but what does it look like when I get, when I start using it? Like what’s kind of the – the interesting part of the initial setup and kind of the ongoing thing that happens month to month?
Adam Jacob: 0:37:25.8 Sure, I mean the kind of first thing you wind up doing is going and looking at which parts of your infrastructure already have public components, right. So there is this huge repository of hundreds of open source components and closed source components that you can already install in a kind of best practices way and just start reusing immediately right. So if you are an organization that is using Apache, that’s already taken care of, for you. If you are an organization that just wants to run MySQL — that’s taken care for, for you.
Chris Brown: And you 00:37:56 bleeding edge now, Riak, Cassandra.
Adam Jacob: Oh yeah, the Riak and Cassandra Cookbooks are really just — they’re just badass that has 00:38:03 zookeepers out there, like there is a ton of that technology. So the first thing you wind up doing is building you stuff with 00:38:11 and we have an example when we usually get started with, and then you just start importing cookbooks from that cookbook site, and run them and see what happens. And then as you need to make smaller, bigger changes you do that, and we provide tools to keep the changes you make in sync with the upstream.
So when they publish new versions like Cassandra cookbook, you can look at what changed in your infrastructure and merge them back together. You go through this really tight iterative loop of one step at a time, and you just starting bringing more and more and more automation to the infrastructure. Then once you get that baseline down, so now you’ve gone through a lot of the basic stuff that was tedious, or hardest, or whatever you kind of knocked out, now you start looking around if the integration type of problems. So you start looking around and you’re like, well stuffs that we have are normal an open LDAP install and an Active Directory and hard to keep the data synchronized.
So what you winded up doing with the Chef is you write a provider either to talk directly to Active directory which is still pretty easy, or you start taking that data and you put it into what we call data bags which are just arbitrary places to store data, they gets indexed in the Opscode Platforms.
So one example of that is all that user data you wind up exporting and then you write a recipe that 30 lines long and what it does is, search that data bag and provision the users that are supposed to have rights on the servers and next thing, you have a fully integrated end to end from 00:39:31 all the way through to the Unix hosts on the edge identity management solution that you paid nothing for. That’s pretty —
Michael Cote: Oh! Right, right, right, so you’re kind of treating all your identities just like configuration data for everything?
Adam Jacob: Just data, right, and so we provide these primitives we’re like just shift some data around, like you have application data that you need to use that can figure the infrastructure, right, throw it in that data bag, search for it, pull it out, and make infrastructure with it, right or hey, 00:40:00 there, great, just use the library, do it, and don’t —
Michael Coté: And then to the point of like the framing you guys were talking about earlier I mean you kind of start out as a, you kind of start out with all the junk you have you want to configure. And then you kind of, you figure out which recipes are out there, that will map to which junk you want and it sounds, it sounds like what you are seeing is people are starting small like they might target one thing to deal with, and just see how that goes and then they expand from that one, that one thing that they are doing and eventually getting to kind of more system wide things rather than just individual services on boxes or not on boxes but what I mean like individual little things.
Adam Jacob: Yeah, that’s right.
Michael Cote: Right, right and then the of course you have the, whatever little, whether you want to call it an agent, or proxy, or whatever thing, you got something on your network if it, that’s managing stuff for you and that’s kind of going back to the platform at some interval, and that is it – are you sort of like, are you enforcing the configuration or what kind of ongoing checks or anything are happening or what’s the ongoing stuff look like?
Adam Jacob: Yeah when you write these policies, what you are doing is you are basically declaring that at some point in time, this resource should be behaving at a particular way right. So when you get to this moment in the run, the service ought to be running. If it’s not, you start it. So if it’s stopped, it starts. If it’s already running, you do nothing. So when you write these policies, you don’t just run them once, you run them all the time, and they keep the system in that enforced state of always being correct. One of the things that’s cool about Chef is that its dynamic is that that state can change, right.
So the reality of modern systems is that they impact each other right, they are sometimes outside of the branch where one action one set of systems might impact the way you want to manage your stack and you don’t know in advance what those things are. So, again, to, kind of, harp on the data thing, you need to have ways that you can arbitrarily store and share that data across systems so that they can then writes codes that does the right thing in terms of changing the way the systems should be managed.
So, yeah, it absolutely has the long life cycle built in, where once you’ve written the initial automation, running that over again only does you, only does you good, right and we keep things in mind for great justice.
Michael Coté: Right, right, right.
Chris Brown: And actually, I was just going to say, sort of going back to the 00:42:22 example of that we also can make quick change on how things should be. So it’s easy to make a change in the infrastructure pretty quick for whatever reasons, this is reacting or if you add some things you just — if you’ve already got that role based for it you just base this that I need to add more of these and they become those – part of your existing infrastructure.
Michael Coté: And so with the platform, I mean, granted it’s in beta and all that but for people who are — for organizations that are using it like what types of — I mean what types of things are they managing with it? What types of organization or IT junk is being managed?
Chris Brown: Well, one of our IGN, what is it, International Gaming Networks they’ve just signed up on the platform and they were part of our announcement. And a big part of if — they’re actually migrating more to data centers, and so they’re seeing this as an excellent opportunity to just kind of build out a new data center, right. So I think that I was going to ask you to talk about goal time early –
Adam Jacob: Sure I mean —
John Willis: How do you — just like the people that have their structure in new Chef, there’s a lot of people that are either in the starting 00:43:30 and then it becomes a whole — the infrastructure is code idea where you actually code your infrastructure, first I mean —
Adam Jacob: Yeah, I mean the kind of people who are using that platform like the IGN is a great example, and one of the reasons why they decided to use the platform rather than using the infrastructure Chefs or others being, they’re we’re really good at running them at system at a scale and in a available way. It turns out if you have thousands of systems that’s actually quite a burden, like if you think about managing the config management system that’s supposed to run your infrastructure that needs to be super highly available like there’s actually a lot of expertise that goes in to making those systems highly available and we still have got 00:44:10 for them.
I think companies like GoTime are another example where they’re small startups that — well, what if they wanted to do is be able to manage their own infrastructure in a programmatic way and so their whole infrastructure is built on Chef and from boot strapping (ph) all the way through to app deployment – it configures everything, monitoring is preconfigured, trending is preconfigured, delivering is preconfigured, the app configures itself, their database fasters (ph) and MySQL servers they’re 00:44:41 stuff.
All that stuff is dynamic. If they bring new 00:44:45 servers out, the frontends reconfigure to add them in 00:44:49 cluster. That happens for them with no extra work, it shows up for free. They don’t have to think about monitoring other than what they want to monitor. They never have to go back and configure a system to add the service in.
So a lot of people who are using the platform right now are web shops. A lot of people aren’t and a lot of cases where they’re not web shops, what they’re building with it is usually pretty special case. It’s people who are building a new compute grid, people who are building — and for a lot of them, it’s because there’s a really clear value proposition there, which is that I have new infrastructure that needs to get billed (ph).
For some it’s going back managing old infrastructures, so there’s Windows support in Chef now, that’s coming along really well 00:45:31.
Chris Brown: It’s a bunch of work.
Adam Jacob: 00:45:37 and so that’s starting to creep up in terms of people wanting to like manage their Windows desktops. That’s a huge problem and a lot of existing solutions well, they’re not super grade, they’re not well — let’s put that way, — and so there’s a pretty wide variety of people doing a quite variety of things and quite frankly we don’t know a lot about some of the people who are using it right, we just started using it.
Michael Coté: Right, right. Well, that’s, I think that’s a pretty good summary of what’s going on there, is there anything else exciting going on Velocity that you guys want to mention any other people come out with announcements or things that they’ve shown off.
John Willis: Well, I think most of the announcements actually happened up at structure; we’ve been doing quite long enough there.
Adam Jacob: Yeah, yes.
Michael Coté: Yeah, that’s right. At the business side of all this business.
John Willis: Well, DevOps day is Friday, so maybe we can report about that next week.
Michael Coté: Yeah, definitely that should be thrilling.
John Willis: Completely sold out. 250 people, sold out. So it will be cool.
Michael Coté: Yeah, that will be tomorrow and are you guys doing various little panels or talks or anything at DevOps days, what are you doing there?
Adam Jacob: There’s going to be a smoking configuration management panels that has got me, they’ve got Luke Kanies, it’s got Theo Schlossnagle from OmniTI, and I like to say that Theo is both Michael Jordan and Mohammed Ali of systems administration in one guy and he also is so smart that he thinks automation is like a crust, because we’re all stupid. He’s also awesome, so for the record, like I don’t mean to make him sound like a 00:47:13 than he is, because he’s kind of a sweet little teddy bear on the inside.
Chris Brown: Adam just loves to hug him.
Adam Jacob: Do you think he’ll still like me? Probably not.
Chris Brown: That’s all good.
Adam Jacob: But OmniTI if you need help scaling your stuff —
Chris Brown: Smart guys.
Adam Jacob: Yeah, best of the best. So he’s going to be on that Panel and then the Moderator is Patrick Debois who is amazing.
Chris Brown: Yeah, and father of DevOps.
Adam Jacob: He’s kind of the grandfather of that I think.
Chris Brown: Looking forward to that panel.
Adam Jacob: He’s sure is and —
John Willis: 00:47:43 shaking in his boots.
Adam Jacob: It’s going to be great. So I’ve got my money that the first question he asks is does order matter or something equally contentious, so look for that 00:47:53 you involve snark (ph).
Chris Brown: It will be fun.
John Willis: 0:47:58.1 That will be fun.
Chris Brown: 00:47:58 popcorn.
Michael Coté: That’s right, well fantastic. Well I’m glad you guys ducked out of the conference to give us a little update and everything here I appreciate it, I’m sure that was good times.
Adam Jacob: Yeah, thanks for coming out.
John Willis: Thank you.
Michael Coté: Yeah, definitely and we’ll — John and I’ll be back next week and we’ll talk about what happened DevOps and go over just some of the little news items and what not instead of just focusing on a handful of things. But with that we’ll see everyone next time.
Chris Brown: Okay.
Adam Jacob: Bye.
Disclosure: Opscode is a client, as are Puppet Labs and Microsoft. See the RedMonk client list for others.
http://www.truereligions.in/