In the wake of last week’s annual Amazon Web Services conference, re:Invent, held at the Palazzo and Venetian in Las Vegas, the three monks who attended got together to chat about what we saw and the conference and what it means for AWS and the rest of the industry. The conversation is lightly edited for content and for incompatible emoji.
sogrady (Stephen O’Grady): So, we’re all back from re:Invent last week, so first question: what were the general impressions of the show?
fintanr (Fintan Ryan): High level, Amazon are executing amazingly well and remain focused even with a huge variety of new offerings.
Couple of things really stand out – focus around machine learning, from elastic GPUs to new compute instances, library support and all the way through to the Lex, Polly etc announcements
The hybrid story is being filled out, even if the direction of travel is ultimately one way
And lots of cool stuff around Lambda
sogrady: You’re actually answering my second question, as well @fintanr, which is what was the most important announcement.
fintanr: 🙂
sogrady: But before we get there, @monkchips, general thoughts?
monkchips (James Governor): Reinvent is a really big show now, it’s outgrown the location. The crowds were pretty crazy. Amazon is eating the world.
sogrady: That was one of my big takeaways as well. 32K or whatever the attendance was was too much for that venue. For the first time it was hard to get around, and you could essentially forget about meeting people for lunch or a beer at any of the main venues.
Moscone next?
monkchips: Moscone seems inevitable.
fintanr: Moscone would make sense to me.
sogrady: Sadly. Much as I like San Francisco the city, Moscone is not one of my favorite venues. With exception of West, anyway.
monkchips: Although they could zig when everyone else zags, and go for something like Javits in NYC, which is crazy big.
sogrady: True. I dislike the Javits more than Moscone, but at least it’s a quick flight for me.
fintanr: Quicker for @monkchips and I too
monkchips: Or they could just create a really big monster truck, capacity 45k people, and drive it from city to city.
sogrady: At this point, I wouldn’t put anything past them.
fintanr: The re:Invent mobile.
monkchips: Yeah i read these steam punk books with my son – a series called Mortal Engines, they have these cities they drive around in…
sogrady: Edgar Rice Burroughs too.
monkchips: Yeah Amazon does feel a lot like one of the Mortal engines though. They roam around the plains, gobbling up smaller cities and using them for parts.
sogrady: Ok, next question, which @fintanr already kind of answered: most important announcements/events at re:Invent?
fintanr: Actually the most important announcement in my mind wasn’t really announced to my mind @sogrady – it was the layering and interlinking of service upon service.
sogrady: Say more. Are you talking about things like pushing Lambda out to the edge?
fintanr: Lambda is one of the keys to everything that comes next.
Yes, Lambda on the edge, its use in the ML stuff, batch processing, step functions – it’s all so well thought out.
monkchips: Oh – you want us to get back on topic. Yeah for me it was AWSCloud goes On Prem. yeah lamdas at the edges, but also the new hardware form factors. For me it felt like AWS spent 3 years telling everyone they did hybrid cloud, and eventually got bored noone was listening, so they were like oh yeah – we’re going to sell servers and storage.
fintanr: +1 @monkchips
Selective servers and storage. I tweeted during the keynote “if I’m a storage vendor thinking data gravity wins, I am very, very scared by AWS Snowmobile.”
monkchips: Actually @fintanr, given you’re agreeing with me, I think I’ll disagree with you just a bit. i am not sure everything is exactly “well thought out” with AWS. It doesn’t have the feels of a classic master plan (the kind that usually ends in disaster). Remember the company is all about small teams building independent services. It seems a lot more organic, though obviously Werner and Andy have some kind of master plan.
sogrady: The Snowmobile is hilarious and scary in that way. And it’s funny, @fintanr, that you and I were reminded of the same thing:
But yes. I’m in agreement with @monkchips: AWS’ development generally is much less Five Year Plan than autonomous collective.
fintanr: Ah good old Project Black Box.
Oh I don’t think its a completely joined masterplan @monkchips but I think lots of the building blocks are now in place, its fast iteration on the layers above for new products now.
monkchips: Do we need to explain the Snowmobile thing, or has everyone on planet Earth now seen that?
sogrady: Good point. For those that haven’t seen it, it’s a shipping container full of storage that sits on the back of an eighteen wheeler.
Comes with private security and can ingest, what, a hundred Petabytes I think?
monkchips: With some local compute. It’s designed as a faster way to get absurd amounts of data into the cloud securely.
fintanr:
Yes that is a truck AWS Snowmobile 100Pb container, 45 foot container #reinvent pic.twitter.com/HSb4Fpcj8e
— Fintan Ryan (@fintanr) November 30, 2016
Yep, one hundred petabytes.
sogrady:
They weren’t specific on the actual number of these yet, is that right, just that there is “substantial” interest?
monkchips: What I found perhaps even more striking was that the Snowmobile’s little cousins, the snowballs, can now be clustered, offering sharding.
@sogrady note the PR story. Jassy was “annoyed” his team ordered too many snowballs, but then the company couldn’t meet demand.
sogrady: The other thing to me – and @fintanr and @monkchips will laugh about this – is that Snowmobile to me is very much a Jobs-to-be-Done offering.
fintanr: The snowballs are really impressive bits of kit and add in Greengrass, its a really compelling story.
@sogrady 100% Jobs-to-be-Done
sogrady: In one of the original JTBD case studies, they couldn’t sell condos to empty nesters because of concerns about storage. Take away that friction, and they sold.
monkchips: @fintanr say a little more about Greengrass – is that only available in Denver and certain selected markets?
sogrady: Same with Snowmobile. Is “I just can’t get all of my data up to the cloud” one of the last remaining objections? Problem solved.
fintanr: lol @monkchips, maybe in the Amsterdam edge network as well.
Quick summary Greengrass allows you to run Lambda on the edge, essentially embedded Lambda on connected devices.
monkchips: So Lamdas everywhere, basically. On the IoT side AWS announced new distribution partners such as Intel (which is obviously super-excited AWS is getting into hardware), Phillips etc.
sogrady: And offline, importantly.
monkchips: +1
fintanr: Offline is huge, remember James Gosling talking about the compute problems at sea that Liquid Robotics have to deal with?
sogrady: Indeed. And the data costs? It was something like $1K/MB for some of the satellite networks.
fintanr: And the cost of connectivity in remote locations…
snap 🙂
sogrady: Though my favorite part of Gosling’s talk was the shark teeth that his autonomous robots collect.
fintanr: To @monkchips point, Intel are feeling the love without doubt. I thought the mention of Arm Cortex was interesting as well.
monkchips: At one of its recent events, HPE, touted its new line of Edge servers and services. Feels like AWS is looking to cut them off at the pass. Use cases, closely tied to the snowball idea (to Fintan’s point and service complimentarity if that’s a word) – such as GE wind farms, where you want some local compute, and storage because the networks suck.
monkchips:
sogrady: That was a magnificent idea, but we’re getting off track.
Point is that AWS is extending its reach well beyond the traditional datacenter into local networks, be those storage arrays like Snowball or into connected devices via Greengrass.
monkchips: But essentially it’s kind of hard to stay on track. That was a meta narrative of re:Invent. think about it 1000 significant service or feature releases in 2016, say 3 a day. How on earth can anyone keep track of that. You could now be a full time AWS industry analyst, and still not keep up. Customers are feeling the difficulty, and it’s only going to get worse. 1500 services next year, perhaps?
sogrady: Do you get any sense that AWS has a problem there?
monkchips: We met super smart switched on people from startups, and they were like: “wait, AWS has that already”, or oh, we know AWS now has that feature, but we just haven’t re-architected to take advantage of it yet.
sogrady: We talk a lot about accelerating complexity and fragmentation at RedMonk, and it has clear negative implications outside of AWS. Could it be a real problem for them?
fintanr: I feel AWS may have a problem, particuarly where things are interlinked. And yes complexity and fragmentation could definitely be a problem @sogrady.
monkchips: We can end up in a situation like at IBM, where the customer ends up making architectural decisions based on what the Technical Sales Engineer recommends – you get very different answers based on whether they’re more oriented to Series i, p, x or z.
fintanr: I mentioned the compute options during the keynote as an area where choice was good, but further up, it is becoming very complex
monkchips: During the industry analyst summit you could almost feel the audible sighs of relief. Thank God! Complexity! We still have jobs…
sogrady: I think the IBM comparison is an interesting one, for the simple reason of accessibility. Do we think AWS has an advantage here over that traditional IBM acquisition process because developers are – frequently, anyway – the ones making the selection rather than having to rely on a salesperson?
lol @monkchips
fintanr: 100% @sogrady
monkchips: During the keynote, same, there was stuff for the server crowd – ooohhh check out the new instances. Storage People. Database people
fintanr: But thats not to say AWS don’t have a big, and expanding, salesforce. I’m intrigued as to how partners are give goals on using new services as well though.
sogrady: I’m also intrigued about what partners make of AWS at this point.
monkchips: @sogrady somewhat. But engineering bias is a thing – and the TSE really isn’t a salesperson so much as a customer architect.
Hahahah. “Partners”. Yeah was that definitely a meme at the show. Just wait six months, and you’re over.
sogrady: On the one hand, huge and growing platform. On the other, like all platforms, if your offering is popular enough, it inevitably becomes a feature – it’s how platforms work.
monkchips: +1 Slack, New Relic and others seem pretty confident.
It was just the same when Microsoft was in its pomp, although obviously distribution mechanisms massively favour replacement of third party services in a way they didn’t previously.
sogrady: Agreed. I was at New Relic’s conference a few weeks prior, and they are quite happy with where they are. And they will continue to have value because they’ll be able to traverse competitive environments in a way that’s probably not possible for AWS.
fintanr: For the medium term anyway @sogrady
sogrady: We’ll see. I can’t see Azure, GCE or whomever allowing full introspection by AWS, or vice versa.
fintanr: Something like X-Ray is very interesting if AWS can build on it. It’s primitive at the moment, but as architectures change.
That is very true @sogrady.
sogrady: You should explain what X-Ray is. And without using Zipkin to define it 🙂
monkchips: or DTrace.
fintanr: lol, okay, I’ll leave my love of other tracing frameworks at the door.
Amazon X-Ray is a new offering which allows you to trace end to end requests on AWS and chain them together. You can get visibility into a service.
sogrady: Yes, DTrace was a given. (And as an aside to Bryan Cantrill, note that we all capitalize DTrace properly.)
fintanr: More interestingly you can use it on Lambda, giving visibility that people using Lambda really wanted.
Also available as an agent for containers and AMIs (I haven’t dug into this yet though).
And it makes everything available in pretty pictures, simple ones for now, but I would be surprised if we don’t come back next year to see complex flame graphs etc available.
monkchips: I definitely see X-Ray (visibility) being used alongside Lambda Step Functions (new technology for managing Lambda proliferation, with a state machine and visual tooling for service flows).
fintanr: Absolutely @monkchips, and then there is step functions with retries, very powerful.
Actually on the whole Lambda area – C# support, not a surprise, but very welcome for a large customer base.
sogrady: Yes, definitely worth mentioning.
Of course there are several dozen other things worth mentioning that we won’t get to, so there is that.
fintanr: Several dozen squared it feels, so many announcements.
sogrady: But before we go down a Lambda rathole, let me ask about what seemed most important to me: AWS and OSS.
monkchips: @fintanr powerful but not cheap:
one last thing: #StepFunctions are 125x more expensive per invocation than #Lambda. That is…too much.
— Ben Kehoe (@ben11kehoe) December 3, 2016
It gets worse. Default throttling limit for each #StepFunctions state machine: *two invocations per second* https://t.co/Y3Rt7hAmB0
— Ben Kehoe (@ben11kehoe) December 3, 2016
fintanr: Whoa, thats a huge number @monkchips.
OSS: I’m concerned tbh @sogrady
sogrady: Historically, AWS has said very little about open source. They didn’t say that much more this time around, with the exception of Blox, but two new faces at AWS have Open Source as part of their mandate: Zaheda and Adrian.
So what do you two think this means for OSS and AWS?
monkchips: Well Amazon didn’t get to where it is today by making everything cheap, which I guess does provide a bridge to our OSS conversation…
fintanr: New faces are a definite plus, I do think there is a need to work on broadening a community for the OSS work, from the outset.
It feels a little too much like we have published it, and they will come. Governance matters as we see time and time again.
But it is very early days.
monkchips: Zaheda is great. She’s been on the open source thing for a while, and clearly Adrian understands that the value is not in the code, but the service, something that AWS obviously kind of struggles with. It is indeed early days. I think that Amazon’s main job is not outward facing, but internal. Getting the memo properly socialised
sogrady: Indeed, I think their success or failure to broaden AWS’s involvement in OSS will depend less on them and more on their mandate.
monkchips: +1M.
sogrady: Or, I suppose you could argue, their politicking skills.
But I think OSS is very important for AWS, because there are competitors like Azure – ironically – trying to use it as a competitive differentiator.
That’s why, the flood of announcements notwithstanding, I think the most important takeaway for me was the suggestion that AWS could be shifting its position on OSS.
monkchips: One reason I found this news, which seemingly didn’t make the cut for the conference, so interesting.
fintanr: Agreed @sogrady, there is definitely a change in the air
sogrady: One last question: what did re:Invent mean for AWS competitors?
You have < 3 minutes: go.
monkchips: Well like the other industry analysts, I am kind of amenable to having a job, so more OSS by Amazon is good for us, to be frank, them being a client and all.
fintanr: I’m actually listening to The Pixies as we write this, and I’m thinking of a lyric from The Sad Punk “I smell smoke that comes from a gun named extinction” – there are a number of competitors that aren’t going to survive what AWS is doing to the industry.
monkchips: For competitors the news was pretty much uniformly awful: more ass kicking in more directions, more collateral damage. It is what it is, AWS is not topping out yet.
sogrady: I’m unconvinced more or less OSS from them has an impact on our employment prospects, but I think more OSS from them is certainly bad for their competition.
monkchips:@sogrady Just my little joke. I really just meant there is a ton of work to do, and we’ll be helping, and we hadn’t made a client disclosure on this slack yet, had we :wink:.
sogrady: Funny, @fintanr, I’m listening to the Pixies channel on Pandora right now. Blood Gets Thin by Pete and the Pirates doesn’t have any obvious relevance, however.
@monkchips: Nope, but we’d do standard disclosure as always!
fintanr: The direction of travel, on lots of fronts, is into AWS – databases will only increase, storage stuff is disruptive (making storage easier to acquire is something CIOs will love, if the price point is correct), new openings for smaller hosting.
fintanr: The Lightsail announcement is not good for any small hosting company.
sogrady: And with that, I think this is a wrap. Thanks for joining.
Disclosure: Amazon is a RedMonk customer, and paid for T&E to re:Invent.
