Docker, Maturity and DockerCon

DockerCon EU 2015 took place earlier this week in Barcelona, and I had the pleasure of attending. As you might imagine there is an awful lot going on in the container space at the moment, and the epic centers for this world is currently the bi-annual DockerCon events.

Paris and Dockers Heritage

While this is a post about DockerCon, it is far more appropriate to begin at the very start of the first days keynote. As many people reading this post will know Docker has its roots in a company called dotCloud, which started life in Paris.

#dockercon pic.twitter.com/GsOczWNUYi

— @[email protected] (@chanezon) November 16, 2015

Docker CEO Ben Golub asked everyone to spend a moment thinking about the events in Beirut and Paris last week, and Docker CTO Solomon Hykes gave a raw, honest and emotional response and stated what he plans to do, and asked everyone to join him in.

 “…what I’m going to do is, whatever level of positivity I had, I’m going to double that. If I was going to learn three things, I’m going to learn ten. If I was going to explain two things then I’m going to explain four, and I’m just going to try and do everything a little better, a little more positive and I’m thinking maybe you want to do that with me”.

Of all the things people can take away from DockerCon, Solomon’s request, in my opinion, is by far the most important.

A Maturing Company

On the business and technology side the key take away for me from DockerCon is that Docker is a fast maturing company. The overall product roadmap across the entire portfolio is becoming far more clearly aligned and focused, particularly to new comers to the Docker story – the Docker vision of Build, Ship, Run is becoming a reality as all the pieces get far more integrated across their tool chain.

Strategic partnerships have become much further developed and bedded in in the period since DockerCon US and the small incremental tasks necessary to be an enterprise company are happening across the board.

To put it somewhat more bluntly Docker as a company are growing up. It happens to everyone. But it is happening really, really fast for Docker.

Energy and Buzz

The conference was buzzing, and in a way that you just don’t see at some other vendor events. While the folks at Docker pointed out that the comparison around speed of growth of DockerCon to VMWorld is a bit clichéd at this point, it is an interesting reference point for many people looking at the evolution of the container space from a distance.

The ecosystem that has emerged around Docker is developer focused, and is creating a very diverse set of offerings. I had the opportunity to talk with a large number of companies during the conference; you can read a brief write up in my DockerCon company round up post.

More importantly the developers I talked with at DockerCon are still as enthusiastic about the technology as before, and are now asking far more difficult questions about using Docker in production.

Security, Swarm, and Control Planes

Swarm

What is interesting looking at Swarm is to try to understand where Docker see this functionality making the most impact. The scalability improvements are clearly aimed at negating criticisms from the top end of the market and under the hood significant chunks of Swarm have been rewritten. We are looking forward to hearing reports from users over the coming months.

However, what is clear, is that Swarm very much fits into “more than good enough for most users” camp. The longer-term implications for some of the entry-level use cases we see people turning to Kubernetes and Mesosphere for, and the adoption these drive, will be interesting to see.

Security

Security was a massive theme at DockerCon. The security journey that Docker has been on over the last year has been very interesting to watch. The investments in hiring engineers such as Diogo Mónica and the structured approach that he has brought, is paying dividends. While the longer term use of hardware image signing using Yubico Yubikey is something that really does not resonate with me beyond the coolness factor, the idea in and of itself has a lot of interesting applications and you can see a number of integration points for this technology.

We will touch on the Universal Control Plane again; it is worth taking a look at the secret management aspect here. Secret management is hard, and it is vital for any serious distributed system. The secret management feature is built upon KeyWhiz, the open source solution from Square. Comparisons with Hashicorps Vault are obviously going to be made, but it is good for the marketplace to see two commercially supported solutions available in the near term.

Docker Content Trust is a necessary development, one of single biggest concerns we have heard over the last six months has been around auditing and trusting images. On the commercial front Docker absolutely had to address this, and are actively doing so. At an enterprise level the integration of CVE scores is a very useful feature for risk assessments.

Control Planes

The development of the Universal Control Plane had been telegraphed via Project Orca, but seeing it pulled together with extra functionality as a beta commercial offering, it is clear that Docker want to ensure that they integrate easily into existing IT Ops workflows. Obviously this is far from the only control plane around the container eco-system, and this will be a very competitive space to watch.

There are, however, a lot of very sensible things in here such as integrating Active Directory and LDAP for authentication along the workflow. Collectively all of these announcements are a big step in the right direction for Docker.

The unglamorous, iterative stuff

Solomon spent a few minutes during the opening keynote talking about the various bits of boring, unglamorous fixes that had been worked on over the last few months. Items like corner case usability issues, minor, but annoying bugs and so forth.

When I talk about maturity, this to me was one of the strongest moments in the entire keynote. Fixing the unglamorous stuff, bit by bit, is not fun, but it is necessary. When you run commercial support organizations this kind of stuff matters – a lot.

Partnerships

I got the opportunity to sit down and talk with representatives from both Microsoft and Docker at a joint briefing during DockerCon. To say that both parties want to emphasize the depth and strength of their relationship would be somewhat of an understatement.

Both companies are very committed to addressing the demand they see in the marketplace for containers on windows, which is going across multiple sectors. What I found interesting was a divergence from some of the background noise we have been hearing the particular marketplace on the levels of adoption, especially in conservative markets such as insurance.

Update on @microsoft @docker partnership from @gossmanster, @Taylorb_msft and @scottcjohnson earlier #Dockercon << good things for customers

— Fintan Ryan (@fintanr) November 17, 2015

Now it is easy to dismiss all of this as lip service, and I am sure many will. The proof of such partnerships is in, to coin a phrase, the technical pudding. I sat through a session with John Starks of Microsoft and Arnaud Porterie of Docker on Windows kernel internals, and how they have gotten Docker to work natively on Windows (if you’re an OS geek like I am, hearing a core engineer from Microsoft talking about this is mana from heaven, you never, ever got this level of detail in the past).

Investment is a two way street, and Docker spent time highlighting that Docker tools now have parity across Windows and Mac during their keynote session. This may seem like a small thing to many people, it is far from it – while every conference you may go to now has Macbooks as the developer machine of choice, for many people in corporate environments Windows is the OS they use. Feature parity is very important for adoption.

Arnaud Porterie spent a lot of time emphasizing just how many contributions Microsoft have made to Docker in the last year, becoming the fourth largest contributor to Docker from a standing start.

There is a lot more to dig into here over the coming months, in particular with the upcoming release of Windows Server 2016, the integrations with Visual Studio Code and the rapidly expanding use of Docker on Azure.

The partnership that IBM are developing with Docker was highlighted by both Docker and IBM in my conversations, and with IBM now having developed a strong reseller partnership with Docker in recent times it will be interesting to see how this develops. In my discussions with IBM they also see use of the Docker toolchain as a very important deployment mechanism.

Unikernels

So this part of the post is a little bit of a geek out for me, and judging by twitter for quite a few others. By way of disclosure I know several of the team at Unikernel Systems, and I strongly believe unikernels are a very significant part of the future (feel free to call this my prediction for 2017, and possibly even late 2016).

Anil and Justin from Unikernel Systems demonstrated their work on using unikernels with docker tools. For developers this will be transparent, but in terms of the size of the images, this is a massive leap forward. I have written about creating small containers in the past, and to me the combination of reduced footprint along with a much, much smaller surface vector for attacks is an incredibly powerful capability to have. We are going to see unikernels in used for microservices, IoT and in a variety of other ways.

The demo from Anil and Justin was very, very cool. If you are really interested in what comes next spending some time on unikernel.org is not a bad way to pass some time.

As I said on twitter at the time

To me this this is public @unikernel unveiling by @avsm & @justincormack, unikernels & docker are part of the future #dockercon

— Fintan Ryan (@fintanr) November 17, 2015

and Solomon Hykes also commented

"With Docker, Swarm and unikernels we can bring scale and security to the world of IoT". #dockercon amazing!

— Solomon Hykes (@solomonstre) November 17, 2015

Come DockerCon 2016 in Seattle I fully expect to see sessions on unikernels, and expect them to be a very well attended section of the conference.

Disclosure: Docker provided my ticket for DockerCon. IBM is a RedMonk client. Docker, Microsoft, Hashicorp, Yubico and Unikernel Systems are not currently RedMonk clients.