Blogs

RedMonk

Skip to content

Last Year a Tech Event Saved My Life. why you should go to

We’re seeing crazy structural changes in tech, utterly changing how infrastructure is discovered, designed, built and used, redefining the relationship between tech companies, traditional clients and customers. Across all industry sectors, companies are joining the digital arms race, building technology to be leveraged by other firms, in the form of APIs, apps or open source technology. Companies with the most active communities are winning.

Cloud, Open Source, and Data – the underpinnings.

We’re all becoming more software-driven. Far from IT not being a competitive advantage, it is the only sustainable advantage. Enterprises increasingly want to act and look like Web companies. Fortune 500 companies are adopting software created by non-traditional software companies at an ever increasing rate – from Hadoop (created by Yahoo) to Chaos Monkey (Netflix) to Mesosphere (UC Berkeley). Business threats are exploding, driven by use of web scale technologies and smart data strategies by well-funded disruptive entrants like AirBnB, Transferwise, and Uber. Oh yeah – and things like Brexit (now that’s real disruption).

While social media and online collaboration help enable the new digital landscape, face to face interactions are arguably more important as ever. So much of the conversation is implicit. Silicon Valley companies, sometimes imagined as innovators in distributed management actually generally have campus mentalities and discourage working from home. San Francisco is the home of the meetup, as much as it is of the startup. Meetups underpin a shared culture. Evangelism is the new direct sales.

Tech is fragmenting, creating new opportunities, but also massive challenges in application development, procurement, management and and maintenance. Agile, DevOps, Noops, PaaS, Serverless. Events and meetups provide great opportunities to stay abreast of the latest technologies, development and management methods. 

People want to be inspired. You should encourage your staff to seek out highly concentrated developer talent pools, and hang out there. 

I have been thinking about all of this stuff as a prelude to a conversation I’ll be having with Pivotal’s Andrew Clay Shafer at SpringOne in Las Vegas August 1-4. Pivotal asked me to record a podcast series with Andrew as a preview – in this series of bite-sized podcasts we talk about the value of events, structural changes in the industry and even a little technology.

 


 

Categories: Uncategorized.

The serverless economy – why you should care about serverless

I recently helped out with the inaugural Serverless Conf, and gave the closing remarks. The event was held in a warehouse in Williamsburg, New York. Organiser Ant Stanley of ACloudGuru did a great job of organising his first conference.

There are plenty of smarter people than me explaining what “serverless” is – Paul Johnston, who spoke at the conference, has a great blog series on the topic. To me however serverless computing is primarily interesting for three key reasons:

  • On demand computing is taken to its natural conclusion – paying at the point of customer value. AWS Lambda, the first mainstream serverless platform, runs compute services in response to events – you write and upload code to AWS Lamda, associating it with particular AWS services, but what is kind of mind-blowing is that you don’t pay for the services until they are actually invoked. A startup can build out an entire infrastructure, without paying anything for it until the customer does.
  • The atomic unit of compute is the function. Just as we begin to understand the implications of container-based computing, in which the atomic unit of compute is the application, comes a new model to consider, in which the individual function is the service. As with microservices this kind of approach creates coordination overheads, but it also offers a great deal of flexibility.
  • You don’t need to think about running containers, any more than you do running an operating system when developing to a PaaS. Unlike containers, which have been associated closely with microservices, serverless is more like PaaS in abstracting all infrastructure required to run a service. Serverless certainly doesn’t obviate the need for ops, but it does exemplify a more PaaS like model.

The chart above suggests a couple of interesting things about the discontinuity in 2007. Deal size shrunk, while average number of deals dramatically increased. Why? The obvious answer is the launch of AWS, and the attendant creation of Y Combinator and the accelerator model, rather than traditional VC-driven company creation. Given that serverless hasn’t exploded so far, it might seem overblown to draw comparisons with the introduction of AWS. And yet. And yet. There is no doubt that today smaller teams can make a bigger impact than ever before. Consider Instagram or WhatApp, which sold to Facebook for $1.9bn with only 55 employees. Serverless attempts to industrialise developer impact- if we’re going to see the first single employee billion user multi-billion dollar valuation startup it’s likely they’ll be building on something like AWS Lambda. Could we see the another similar discontinuity to the graph above, with deal size cratering again, and many more flowers blooming? It’s possibly. Certainly Amazon’s serverless implementation would hope to enable that. Serverless is fundamentally about permissionless, the ongoing direction of travel for all IT decision-making.

As Sam Kroonenberg, CTO of ACloudGuru put it:

“AWS Lambda is to compute what s3 is to storage.”

AWS is pretty confident about serverless advantages. It’s Serverless GM Tim Wagner made it pretty clear that he expects serverless to put severe pressure on orthogonal markets – such as Docker-based infrastructure, partly because he expects the model to drive more testing into the cloud. Serverless is not solely an Amazon phenomenon however. IBM has adopted an open source model with OpenWhisk, while Google is building a set of services around Firebase, originally a database as a service it acquired. Microsoft has taken a hybrid hosted/open source approach with Azure functions.

Competition has been sufficient that Amazon took the surprising step of announcing it was to open source a serverless framework called Flourish. On that more details as they emerge.

Joe Emison works at DMGT building B2B applications and services. He gave a great presentation which was frankly pretty scathing about AWS.

“In this case i don’t think Amazon has the best option for any of the services out there. It’s about the front end. AWS serverless is largely about back end processing, which we have largely outsourced. To me, the advantage of serverless is the key is faster iterations, and a move to fatter clients”
Emison argued that AWS Lambda is focused on enterprise adoption, with the attendant complexity that necessitates, in order to boost revenues. For Emison serverless is simply the most effective way to establish product/market fitEmison recommends a stack of services designed from the ground as developer friendly service for high scale ops, including  Algolia (for search), Prerender (Javascript SEO), Sendgrid (email delivery), Cloudflare (CDN), Google Firebase (data storage and associated services), Auth0 (authentication).
Beyond product/market fit serverless potentially means not needing to penalise customers for being interested

That is why I think serverless could be a big deal. It’s the economy, stupid.
[update: While not intended to be completely comprehensive, one point I had originally intended to include in the post was that serverless may not be AWS only, but the approach clearly makes the most sense when you have a ton of resources at hand, with automation to support it. Mary Branscombe called me on it over the weekend and her comments are worth including verbatim:

Surprised you didn’t mention how serverless rewards massive operational efficiency of hyperscale. it’s a byproduct of cloud at a different unit of scale. by being that good at offering images you become able to offer a unit of compute at a much granular level. which means there’s an operational aspect to evaluating services, not just the technology. Can a business ever be efficient enough to make running serverless themselves economic? I’m doubtful, frankly. A model where essentially putting images on machines is something we do very fast and we don’t need to pass on the cost.

Serverless as a side effect of operations as scale. Nicely put. Kind of like AWS as a side effect of operations at scale.
Amazon AWS and IBM are both clients. Serverlessconf paid my T&E to be there.
Original Chart by Chris Tacy. He’s pretty awesome.

Categories: Uncategorized.

Auf Wiedersehen, Mate: Some Thoughts on the Tech-xit

Seeing as Laurel called me out- it’s basically double-edged swords all the way down. Yesterday I did some grief, but now i’m looking at acceptance. But politics is moving so fast right now it’s very hard to say with any certainty whether Brexit is even going to happen or not. For a TLDR video version see my vlog below.

 

Q: Will all the banks, financial services, accounting and legal firms move (with their IT workloads) to Frankfurt?

A: No. Some firms undoubtedly will move but we’re unlikely to see a “new” London emerge any time soon. There are going to be big businesses that up sticks and leave, for sure, but likely to a number of different EU cities, rather than just one. London has been a friendly half way house for US businesses, opening doors to expansion in the EU since we joined the institution. The UK has been the “market friendly” bit of the EU.  We won’t be able to play that role so well any more. And for professional services, the loss of “passporting” (you’ll hear that term a lot in the next few weeks, Ben Judah is all over it) if it happens, which allows for services business to operate in borderless fashion with the EU, could indeed see London decimated.  Let’s not forget, on the other hand, that Brexit could also help to solidify London’s position as the world’s preeminent centre for tax avoidance and yes, money laundering. Hot money loves London and I don’t see that changing any time soon.  The big US banks feel they have won already – their people aren’t about to move to Paris or Frankfurt and start paying significant tax if they can help it.

As my colleague Fintan points out, the one workload flight we can be assured of now is the one into the Cloud. I wrote a bit about cloud geo/data governance here recently.

 

Q: Will all the Web giants and Unicorns move to Ireland?

A: This was already happening to some extent – Apple, Google and Amazon see Ireland as a highly skilled English-speaking country with a low tax base, enabling the double Irish and the Dutch Sandwich tax avoidance for good measure. Apple is planning a vast data centre in Ireland, so big it’s said that it will increase the nations’s energy requirements by 8%. Google though has been building a massive complex in Kings Cross, in the heart of London and I don’t see it cancelling that plan anytime soon. I can see the massive new Amazon office from my window here in Shoreditch, and it’s important to remember that England is a nation of shopkeepers because England is a nation of shoppers. We consume things and businesses like consumers. After the Brexit we’re likely to see corporation tax further reduced, which will also be attractive to global firms.

 

Q: Will all the startups move to Berlin?

A: Brexit is definitely a shot in the arm for an already thriving Berlin startup scene. Again, like the move by Web giants to Ireland, this was already a thing. Why I’m Leaving London has been a trope for a while.  Software developers and entrepreneurs that already have kids are far less likely to move out in droves. Talking to startups in my coworking space – notably Andiamo and Weaveworks they see investments in Berlin as additional to, rather than instead of, investments here, at least for now. Weaveworks COO Matthew Lodge has some interesting thoughts on the Brexit subject in Quora here.

Also – Berlin commercial and residential property prices can only increase. It could be that Brexit on the other hand will force a correction in London’s overheated commercial property. Let’s not forget the currency slide. If a company is funded by American VCs the slide in Sterling means their US Dollars can now go further.

Hiring talented young people in London is about to get a lot harder. Why would an awesome 23 year old programmer or data scientist come here from the continent come where they’re apparently not wanted.  I know of at least one firm currently hiring, and CVs from the continent literally dried up overnight last Thursday. Advantage Berlin. Yesterday David Cameron announced there would be no amnesty for European immigrants that are already living and working here – which was both stupid and insensitive.

London needs to get its shit together fast – we live or die on our reputation for being open, tolerant, welcoming and fun-loving. We need more Gay Pride, less self-love.

 

Q: Why are you talking about London so much?

A: Because that’s where so much of the action is. I have no plans for a safari to Sunderland any time soon. I still remember the lump in my throat as I saw the first result come in.

 

Q: Why are you not talking about tech?

A: Because geopolitics and community are the things that actually matter. When it comes to impacts on the tech industry – open source will keep on doing its thing, being the way we build software. Margins will continue to be squeezed though – traditional Big Enterprise IT suppliers are going to find it harder to make sales in the UK, and Europe, owing to uncertainty, which we can expect to become the new excuse du jours for missed earnings over the next few quarters. Once we sign article 50 (if passporting gets nuked) we can expect some juicy systems integration contracts signed as European organisations build apps to take on, for example, Euro clearing. Cloud just gets another shot in the arm because of uncertainty – it’s a great excuse to pull the trigger on that AWS, Azure, GCP or SoftLayer plan for infrastucture, or transition to SaaS apps. Already anaemic Global growth will take a hit, so investment across the board will slow.

 

 

 

Categories: Uncategorized.

On Geography, Cloud, and Data Governance

europe sat

 

 

 

 

 

 

 

 

 

We’re seeing a significant shift in the Platform as a Service and Infrastructure as a Service markets as the cloud becomes a global phenomenon, with enterprise adoption and regulatory pressures driving the need for better data governance and control. Cloud providers can no longer simply tell customers not to worry about where their data is held.

On the consumer side for example, Germany in March instituted an antitrust probe of Facebook based on data protection questions.

The EU safe harbour agreement was declared invalid in October 2015 and is being strengthened (it’s a shield now, apparently), increasing obligations on US companies handling European data. Geopolitics isn’t becoming any more forgiving. European companies and governments are looking for more far more confidence about where and how data is held.

In May Microsoft trumpeted that it had passed the new Spanish national security framework. The company is even building its own undersea cabling network, MAREA, from the US to Bilbao, neatly bypassing joint US/UK national security infrastructure points of access.

Amazon Web Services has long allowed for logical partitioning in the shape of Virtual Private Cloud, but it now also multi-region choices for data hosting – it’s data center in Frankfurt has been a notable success.

Heroku, which is an AWS shop, is now taking advantage of Amazon functionality to offer it’s own geo-secure services. The Salesforce.com subsidiary recently asked me to contribute a quote to a white paper on security:

Today it’s not enough to simply offer hosting or black box PaaS. Developers need services that allow them to deploy private networks in the public cloud, making geographical choices about where to host data. Public cloud wins on convenience, but the market has made it very clear that data location and governance are not optional extras. Heroku is now addressing these security and compliance requirements across its portfolio, building on its heritage
of simplicity and convenience.

Security is hard, and needs to be baked in, rather than bolted on. Heroku and other cloud service providers are responding to market needs and upping their game in this area.

Categories: Uncategorized.

HPE breaking up to get lean

Breakin’ Up is Hard To Do, sang Neil Sedaka.

It seems nobody told HPE.

The company is clearly energised by its divestment strategy. Having already announced it would split into two businesses last year – HP Enterprise (servers, storage, services, systems) and HP, Inc (PCs and printers), HPE then recently surprised the market by announcing it was going to acquire CSC and spin out a newco combining the enterprise services assets of both companies.

The services roll up makes a great deal of sense – spinning out a “bad bank” of technical debt locked into long term outsourcing contracts. Markets certainly reacted favourably on the news.

HP_Enterprise_shares_surge_in_extended_trade-line_chart-ft-web-themeregular-300x288

To be fair to CSC it has done some good work in building out hybrid cloud offerings for its customers over the last few years, but the general direction of travel for traditional outsourcing is definitely a downswing. You could of course say the same thing for almost all enterprise IT businesses.

So to divest or not to divest? On that question Dell and HPE are moving in opposing directions. As Dell bulks up, taking on debt to acquire EMC and VMware/Pivotal, HP gets leaner.

I spent last week at HPE Discover, the company’s customer event in Las Vegas. Customers were pretty much unphased by the potential disruption caused by multiple demergers, and executives are seemingly revelling in it. Bottom line- nobody has anywhere to hide any more. Businesses are not going to cross subsidise each other and paper over cracks and faults in the the business.

HPE is now comprised of Software, Enterprise Systems and Financial Services (financing for customers). It would not be a surprise to see HP also spin out the financing business. Borrowing money to fund infrastructure acquisitions is of course not really how things are done in the age of the cloud. Make no mistake – Cloud is a forcing factor for pretty much all of the issues facing incumbent enterprise suppliers today. Cloud is putting pressure on all enterprise software markets – applications, hardware, networking, security, services, software, storage etc.

HPE’s response is to try and become more nimble, and focus in the near term on hardware and software:

  • HPE Software Group products fall into 4 main areas, ALM, IT Operations, Analytics and Security.
  • HPE Enterprise Group – Integrated Systems, Servers, Storage and Networking.

One response to the threat of cloud decimating on-premise IT workloads is to try and reframe the issue. So while it makes little or no sense for enterprises to run their own traditional data centres there are going to be areas where processing will move to the edge. Consider for example that a Boeing Dreamliner has a 42u rack server on board. Or in automotive cars are increasingly digital- the server in the boot. The idea behind the reframing is that rather than instantly push all data into the cloud as it is collected, which would require perfect, high performance cheap networking, it should be partially filtered and analysed “at the edge”. Data gravity is after all a thing. To support its edge story HPE last week pointed to a set of products originally announced in London in December called the Edgeline IoT Systems 10 and 20, running Microsoft Azure IoT Suite and Windows 10 – including ruggedised versions. HP has a partnership with GE around Edgeline, which is a decent start in building an ecosystem. Interestingly, HP also sees a potential role as an OEM, as its customers and partners roll their own edge offerings.

What if edge is the new on prem? HP isn’t the only pursuing this story. Also last week IBM and Cisco announced a new set of “edge” offerings to run IBM’s Watson for IoT software on Cisco network gear.

One argument in favour of the edge argument could be the rise of the smartphone. The internet didn’t kill native operating systems, it enabled them. We don’t run around using pure web apps on thin phones; rather the phones get thicker and slicker all the time. The internet created entirely new form factors for edge computing. The internet of things could repeat the trick.

But what about software and how we live now?

I presented at two customer sessions on Monday about how software development and management is changing. The cloud native companies have retaught us that engineering for scale requires trade-offs around consistency, availability and partition tolerance. Testing is part of the development process. Requirements can’t be fully known. Waterfall doesn’t allow for anything like continuous improvement. HP’s customers are very receptive to learning from web company methods and tools.

HP is retooling accordingly, and its customers are also beginning to. In both sessions, one for Quality Center customers, and one for LoadRunner and StormRunner (Performance Engineering), when I asked who was using Jenkins literally every customer in the room put their hands up. For more context you should also check out this post from my colleague Fintan CI: The Cloud Native On Ramp and The Dominance of Jenkins.

The company has got the memo around packaging (open source) – HPE ALM Octane embraces GIT, Jenkins, even Cucumber for specifications/requirements collaboration.

“Our perspective is no longer to replace functionality already available in the open source market”

Instead HPE is focusing on tagging and search, relationships, pipeline support and open APIs.

HPE is taking the same tack with public cloud (rather than looking to replace functionality already available there) it is trying to move up the stack, positioning Helion as a hybrid environment for running apps on prem on or in the public cloud. HPE is packaging up OpenStack and CloudFoundry for infrastructure and platform as a service, having acquired Stackato from ActiveState. The approach is not heavily differentiated, but it is standard – IBM and SAP are on similar tacks, and HPE can also sell into customers adopting Pivotal as a hedge.

In conclusion, HPE still has a ton of work to do to be a growth company going forward. RedMonk would argue the company needs a stronger more opinionated SaaS story. It’s one thing for customers to say they aren’t ready to be all in on cloud, but the consumability advantages of cloud delivery are inarguable. HP’s Haven On Demand machine learning APIs show the way here – easy to ease, well documented, lowering barriers to entry, consumable by developers – in terms of delivering the portfolio as HPE rebuilds it SaaS is easier than on prem, and customers are happy to pay for it; software licenses not so much at this point in time. On prem hardware is not a great market to be in, but it’s still worth tens of billions of dollars – HP, Cisco and Dell have a huge fight ahead.

HPE is getting lean for it, rather than bulking up.

Categories: Uncategorized.

Mi3M episode 4 making significant progress. airports and web scale renewables

Last week a few things caught my eye in the world of renewable energy – we’re making so much rapid progress in that respect there is real cause of optimism. An Indian airport running 100% on solar, Portugal making all the running as a global post Carbon leader, and new commitments from Microsoft to renewable energy.

On the other hand I was waxing on this stuff back in 2009, so progress is slower than needed.

It seems to that we’re facing an economy of scale problem here- and frankly if Goog and M$ didn’t know energy was an issue they wouldn’t be building on rivers, and investing in wind farms

Economies of scale come with a direct measurable cost – Apple’s data investment in Ireland, is projected to potentially increase the country’s energy consumption by 8%. It will be up Ireland to work with Apple to ensure investments in renewable energy to support the country’s new requirements.

Meanwhile on the minus side – it’s too damn hot. Last Friday India recorded its hottest temperature ever.

Categories: Uncategorized.

MOAR VIDEO: on moving Swiftlang forward and wearables, with explosions, charts and GIFs

In case you hadn’t noticed we’ve been stepping up our velocity of video production and packaging at RedMonk lately.

It turns out you can really have fun when you start using a green screen. We hit a new high point in production values and story telling with my last video about Swift lang – including charts, third party tweets, and an appearance from Taylor Swift.

 

It’s a pretty good show. But we just topped it, at least in timing and comedy with this take on Wearables, Nearables and Hearables – where I riff on building headless context-driven apps.

We’ve also developed a bit of a line in animated GIFs.

WearablesGIF

 

 

 

 

 

 

 

And some real time trolling

The Opinionated Infrastructure series is sponsored by IBM, and I give full production credit to Benny Crime and his brother Jack Heatseek.

Categories: Uncategorized.

I’m Satoshi

I'm-Satoshi

Categories: Uncategorized.

Monkchips in 3 Minutes Episode 2: Panama Papers, Gorilla, Financials.

The new format.

In which I talk about how The Panama Papers site, which is a great piece of work. You’ve read the news story, now check out the actual source material, which is powered by Neo4j.

I also talk about the morning paper by Adrian Colyer, in particular his take a scientific paper on Facebook’s absurd scale time series database Gorilla – it stores one trillion events per day in memory, as an in memory cache for Facebook’s logs for troubleshooting, then archived in HBase. Gorilla is optimised for writes and high availability.

Then filthy lucre – recent financial news from Apple, Amazon, Facebook and Google. Today the 4 Comma club is about trillions of events. One of these cloud companies however is going to be valued at over a trillion dollars at some point, possibly soon.

Anyway – hope you like Mi3M. Let me know what you think, and please do subscribe on Youtube.

 

Categories: Uncategorized.

Tags: , , ,

Dockerize vs Containerize. All the things.

dockerize

Of course Google Trends search volume comparisons are not exactly scientific, but they can be interesting and create useful data points. Thus is it with this search, showing how the containers conversation today is all about Docker.

Back in 2005 Solaris Zones was getting a fair bit of interest and containerize was performing in terms of Google Search volume. Has Solaris Turned the Corner? – nope, it never did. But the core technology idea was certainly relevant, which explains why and how Joyent and in particular Bryan Cantrill have become everpresents on the Docker conference scene – they’ve been doing containers since before they were a thing.

But Docker is at this point the name that really matters in containers – see The Docker Pattern. It’s the environment in which people are competing.

Docker is a client.

Feel free to point all the different searches I should have done. Like I say, present imperfect.

Categories: Uncategorized.