We had the opportunity to attend CloudNativeCon/KubeCon in Seattle earlier this month. As we recently noted the velocity of the most well-known project under the Cloud Native Compute Foundation banner, Kubernetes, is very impressive. Several of the more recent additions to the foundation, such as Prometheus and Fluentd also have very strong communities in place.
The wider Cloud Native ecosystem is, however, a very disparate and confused place. We anticipate a significant level of consolidation over the next twelve to eighteen months with some clear winners emerging. The emergence of several opinionated distributions of Kubernetes is hardly a surprise and this space will expand a little further before settling down. We will return to these opinionated distributions a little later in this piece.
The Journey to Cloud Native
During his keynote, Cloud Native Compute Foundation (CNCF) Executive Director Dan Kohn brought us on a journey through the recent history an evolution of computing, from hardware centric to ever higher levels of abstraction.
As always in computing the current generation is in the privileged position of standing on the shoulders of the giants that went before them. In looking at the journey to, and positioning of, the current cloud native wave, the CNCF is right to acknowledge this as well.
It is interesting to note the two key words that the CNCF choose to call out are microservices and orchestrating. There is no doubt that microservices are emerging as an incredibly important paradigm, and orchestration is vital to making microservices useful. When organisations have a suitable application to implement or update, we do strongly recommend cloud native approaches, starting at CI/CD.
But after highlighting what goes before, it is somewhat dismissive to limit the definition of cloud native applications to just those consisting of microservices – cloud native, as an architectural pattern, but far more importantly as an approach to developing software, needs to appeal to a broader church of users than just the early adopters. The transition period will take many years – distributed systems require a very different mind-set from the get go, and as we have noted more than a few times – sometimes monoliths are okay. That is not to say that cloud native does not need definition, it does – but being purist at this early stage runs the risk of alienating the very organisations that could benefit most from the approach.
Core Projects, Foundation Membership & Documentation
We had the opportunity to have a conversation with Dan Kohn and to attend an analyst session at the conference during which several projects were highlighted by the CNCF.
Projects
On the core Kubernetes project both the CNCF and Google were very keen to highlight the change in contributor profile, particularly in the 1.4 release and beyond. We have already seen the downward trend in Googles contributions versus the rest of the community, but this merited analysis on a release by release basis.
Looking across all releases we see just how much of a contribution Google has made, with version 1.0 being completely dominated by Googles contributions. Removing version 1.0 we get a clearer picture
This downward shift in Googles contributions is also reflected in the trends overtime that we have looked at in the past and bodes well for a more diverse community in the future.
The other projects discussed during the analyst session were Fluentd, Prometheus and Open Tracing. The growth in interest in Prometheus over the last year has been very impressive, and with good reason – how we view and use metrics is only growing in importance.
However, for me, the Open Tracing project has the potential to become an absolute key part of how we develop software in the future. It would be fair to say that we have a keen interest in all things debugging and telemetry information related here at RedMonk, and the overall goal of providing unified access to various tracing frameworks is one well worth pursuing.
Nice slide on @opentracing #kubecon pic.twitter.com/9DS3TShiWW
— Fintan Ryan (@fintanr) November 8, 2016
To understand the why of Open Tracing, you need to reframe your thinking about how we monitor software. A conceptual model we sometimes use to explain things is to think of each component of your application being an aggregation of individual instances, become an aggregation of services, and eventually the services make up your application. The individual instance is fundamentally irrelevant. More importantly failure of instances of a service is an absolute given, which in the old world of thinking about enterprise software was never considered
Open Tracing is very much in its early days, but we are watching the project with great interest.
Finally, Fluentd has emerged as a key component in the overall cloud native ecosystem. We have spoken about Fluentd in the wider logging context in the past, but its growth continues to be impressive, and its integration in places such as Kubernetes among others ensures its continued success.
Documentation
The best packaged software always wins in each revolution, and a key component of the packaging experience is documentation. Perhaps the most welcome announcement of the entire conference keynote was the $20K per year grant to improve documentation on any CNCF hosted project.
Documentation needs work @dankohn1 – highlighted this as an important in my #kubecon talk in London #CloudNativeCon
— Fintan Ryan (@fintanr) November 8, 2016
Documentation is absolutely key to driving adoption beyond an early adopter user base.
100% this, sharing experiences etc is key https://t.co/5hou0QW5h4
— Fintan Ryan (@fintanr) November 8, 2016
The documentation grant is an excellent initiative, and we look forward to watching the overall quality of documentation around CNCF hosted projects improve.
Foundation Membership
During the conference, a number new platinum (Samsung SDS) and silver tier (Canonical, Digital Ocean, Livewyer) members of the foundation were introduced, but perhaps the most notable part of this announcement was the massive drop in price for end user membership, depending on the membership tier the membership fee dropped from $7K to $1.7K, and $50K to $4.25K, with conference tickets thrown in.
There is no easy way of putting this, but that is a massive drop and more than a few people have expressed some concern as to the reasoning. In our conversation with Dan Kohn he was keen to emphasize that this is aimed at expanding the end user committee, and on one level you can see how – for most medium to large organisations any manager can get a PO of $4.25K approved.
Whether this proves to be a shrewd move remains to be seen. The CNCF are right to point out that they need far more end user involvement. Engagement needs to be at the right level and for the right reasons. A conference ticket is not, in and of itself, a good enough reason.
The Wider Commercial Ecosystem & The Gold Rush
Three areas are emerging as key battlegrounds for all the players in this space
- Security
- Networking
- Storage
All with an overlay across these areas of developer productivity. That leads to the obvious question – what is the gold rush we are speaking of? Right now, it is focused around two areas – opinionated distributions and developer workflows.
On the opinionated distributions front we see offerings such as Red Hat’s OpenShift, Tectonic from CoreOS, Kismatic from Apprendra and Navops from Univa among others appearing, alongside offerings from the cloud providers such as Azure Container Service from Microsoft and GKE from Google. Clearly there is significant space for consolidation here. As an aside, Gareth Rushgrove has taken an interesting look at the advent of Kubernetes distributions and an explanation as to why distribution may be too broad a description.
Developer workflows are embedded into many of these opinionated distributions, but we also see companies such as Shippable, Drone, Travis, Wercker and others playing in this space. Several these companies are taking a Kubernetes only approach, which is somewhat risky in the short to medium term.
But these offerings are, understandably, focused on shovels. Where is the actual gold? Like all things in the current wave, the actual gold is in the data – and this leads us to storage and networking. The storage story for containers is still relatively nascent, and is something we will cover in more detail quite soon – but suffice to say that where data is stored is incredibly important for production applications. Networking is somewhat more fully formed, and again is a subject we will return to in more detail soon.
Where Next?
During the keynote Dan Kohn highlighted several projects that the CNCF sees as potential candidates to become part of the foundation, these are
- Tracing (OpenZipkin)
- Messaging (NATS)
- Stream processing (Heron)
- Logging (Fluentd)
- Networking (Flannel, Calico, Weave, CNI)
- Configuration (etcd)
- RPC proxy (linkerd)
- Protocol buffers (GRPC)
- Naming (CoreDNS)
- Database (CockroachDB)
- Storage (Minio)
Each of these projects is interesting but, linkerd and gRPC are two which especially stand out as essential core components of the larger move to distributed computing which cloud native approaches is enabling.
Linkerd is a service mesh from cloud native applications, and is focused on solving the problem of communications between services at scale, while gRPC is a high-performance RPC framework for cloud native applications. In terms of core building blocks, these two projects will be essential.
Longer Term Trend
The direction of travel is all positive, the community is growing rapidly, and we continue to see tremendous interest among early adopters in many of the technologies that are falling under the CNCF banner, especially with Kubernetes.
As we have said several times, the size of the cloud native pie has yet to be defined, but it will be huge. There is space and time for a large, diverse, commercial ecosystem to become established. However, this will take time and time is not a luxury that many of the early companies in the space currently have.
On saying this, and despite the noise within the echo chamber that is cloud native, there is a distinct gap opening between early adopters and wider enterprises moving cloud native applications into production. The classic chasm that exists in every technology wave has appeared. The CNCF is a key part of bridging it.
Credits: Gold Rush Image – Dyea, beginning of Chilkoot trail, in March 1898, Public Domain Image, sourced on Wikipedia
_1.jpg){kind=link}
Disclaimers: The CNCF paid my T&E. Red Hat, IBM, Docker, Apprendra, Huawei, CoreOS, Meteor, Treasure Data, Univa, Microsoft, Pivotal and Engine Yard are current RedMonk clients.
No Comments