Blogs

RedMonk

Skip to content

Driving the state of the art. Cloud natives and the appliance of science

Zanussi Dishwashers 1990s UK appliances slogans The Appliance of Science

“Applied science is a discipline of science that applies existing scientific knowledge to develop more practical applications, like technology or inventions.” – Wikipedia.

I was really pleased last week when Fintan dropped this post The Welcome Return of Research Papers to Software Craft. At RedMonk we’ve been talking about the trend for a while so it’s good to see the idea captured as a post.

“Over the last two to three years a small, but very important and noticeable trend, has started around the world – a growing appreciation of the importance of primary research and academic papers among software practitioners. Those that are crafting software are spending more and more time understanding, learning from, and reflecting on research from the past and present.”

The post is really good. You should read it in full. But here’s a bit more before I jump in

The level of practitioner interest in research papers has risen to a point that the opening keynote atQConLondon this week was delivered by Adrian Coyler, author of The Morning Paper and a venture partner at Accel. Adrian walked people through a number of his favourite papers and challenged people to think a little differently about what is coming in the future.

It is hard to pinpoint quite what caused this renewed interest, but it is safe to say that the emergence ofPapers We Love, with the associated meetup groups, frequent discussions on forums such as Hacker Newsand blogs such as Adrian’s has created a wonderfully curated entry point to research papers for the curious. When people such as Werner Vogels at AWS remind us of the importance of papers, people sit up and take notice. As an industry we have had, at times, a tendency to forget to look at problems in detail, and instead focus on the quickest time to getting a product out the door.

One of the most recent Papers We Love talks came from Bryan Cantrill, CTO of Joyent, where he talked about BSD Jails and Solaris Zones, and as he noted at the start of his talk, while reminiscing about soda at the journal club his Dad, a physician, hosted:

“I always felt that it was really interesting that medical practitioners did this, and I always try to draw lessons from other domains, and medicine is very good about this, and we are not very good about this. We in computer science and software engineering are not nearly as good about reading work, about discussing it.”

This. I ran a conference on exactly this theme a couple of years ago with Monki Gras 2014: Sharing Craft. My thesis is that as an industry we’re actually improving in how we learn and share across disciplines but there is a lot of work to be done. When explaining the current state of tech Netflix pretty much always features because of its leadership in multiple areas. It pays above market rate to technical staff as a matter of course, for example, in order that it only attracts top talent. Netflix has crystallized the new way of working at scale, a way of working with intellectual property not in theory but in open practice. Netflix is in effect applied science. It carries out experiments in computing at scale in order to drive the business forward. It then open sources the code it used, rinses and repeats. Netflix doesn’t theorise, file patents and then ring fence its work. On the contrary it open sources code in order to drive forward the state of the art.

[BONUS UPDATE. following an @acolyler link this morning I discovered that Netflix had made all of this utterly explicit in a talk at QCon recently: Monkeys in Lab Coats: Applying failure testing research @Netflix. I don’t believe the video is up yet but can’t wait to see it)

Industry and academia need each other. Far from the tire fires of production, university researchers have the time to ask big questions. Sometimes they get lucky and obtain answers that change how we think about large-scale systems! But detached from real world constraints, systems research in academia risks irrelevance: inventing and solving imaginary problems. Industry owns the data, the workloads and the know-how to realize large-scale infrastructures. They want answers to the big questions, but often fear the risks associated with research. Academics, for their part, seek real-world validation of their ideas, but are often unwilling to adapt their “beautiful” models to the gritty realities of production deployments. Collaborations between industry and academia — despite their deep interdependence — are rare.

In this talk, we present our experience: a fruitful industry/academic collaboration. We describe how a “big idea” — lineage-driven fault injection — evolved from a theoretical model into an automated failure testing system that leverages Netflix’s state-of-the-art fault injection and tracing infrastructure

Netflix isn’t alone in this approach, of course. It’s how smart companies get things done. Stephen has written extensively about the Rise and Fall of the Commercial Software Market, and the stages our industry has gone through. Software is no longer the product, but increasingly a by-product.

We’ve made significant progress since Google, rather than open sourcing its own code, published the MapReduce Paper in 2004. Yahoo got Doug Cutting to build its own implementation, Hadoop, which it did open source, and the rest is history. Here are 5 Google Projects that Changed Big Data forever. The Research at Google site is a thing of beauty.

reseach at google

By engineers for engineers.

Ever since Google was born in Stanford’s Computer Science department, the company has valued and maintained strong relations with universities and research institutes. In order to foster these relationships, we run a variety of programs that provide funding and resources to the academic and external research community. Google, in turn, learns from the community through exposure to a broader set of ideas and approaches.

But by 2015 Google realised that open sourcing the code itself, rather than just publishing papers about its approaches, made sense. Why watch somebody else create another Hadoop or Mesos when Google could build a community around stuff it actually built – and so Kubernetes was born. Things got really interesting when Google’s engineers met engineers at Red Hat they deeply respected. When we write the history of Google this will be seen as a seminal moment, when the appliance of science became properly a community-based activity. The decision to open source some of Google’s core machine learning technology – TensorFlow – followed naturally on the obvious and growing success of a better, more collaborative model for applied science.

So Netflix and Google do it. Twitter definitely does it. Apple got the memo and open sourced Swift. Facebook crowed about the success of React in 2015. Uber and Lyft are both adopting the model. Pivotal is picking up code like NetflixOSS and OpenZipkin from Twitter for distributed tracing. You can bet someone at one of the tech giants is currently reading this paper, Message-Passing Concurrency for Scalable, Stateful, Reconfigurable Middleware and considering its implications. Oh look – it’s science as code, check out Kompics on Github. Maybe we should check it out in production. Let’s not forget that Linux began life in academia. And oh yeah Walmart… is making distributed systems contributions too.

Github, just mentioned, is a fundamental building block of the new applied science. The combination of open source, social coding (a little GIT thrown in for forking and testing and recombining) has utterly changed the game in software and distributed systems. There is no advantage in proprietary approaches – only advancing the state of the art. Well Amazon might argue, but we’ll see.

Open, practical innovation isn’t just a software phenomenon – check out Facebook’s Open Compute project, which implements some computer science fundamentals. There is a reason Peter Winzer, Ph.D and Head of the Optical Transmissions Systems and Networks Research Department at Bell Labs gave a talk at its most recent meeting.

Obviously I need to be a little bit careful about Golden Era thinking, but the applied science approach of cloud technology, with associated information sharing, is so very different from other spaces, in which science seems to become ever more commodified, but not commoditised. Pharma for example wants government to fund all the research, while it keeps all the profits. Companies are trying quite successfully to make genetics private science – patenting genes that occur in the wild, with terrible implications if you have a marker for say, breast cancer. The very foundations of science are being privatised. Researchers try to prevent others from replicating their work, rather than hoping for replicated experiments. It makes no sense. Tech however is showing us showing us something important about how to advance the state of the art, and that’s good for all of us. Not everything is perfect in tech, and the Industry finds ways to harvest data that should be public (or should that be private) but at least in distributed systems something very very interesting is happening. The Appliance of Science.

Categories: Uncategorized.

Throwing The Phone Around, On Mobile Ecosystems


So we made another video. No green screen this time, just straight talk. In this episode I talk about control versus openness in mobile ecosystems, and the battle between Apple IoS and Google Android. The really cool new Business Cards+ from Moo gets a special mention. What no NFC? How about some Android-first development.

Sponsored by IBM MobileFirst, again. Creative freedom for the win.

Categories: Uncategorized.

Rise of The Docker Pattern

Sharp Green by Debbie Clapper

Once you’ve been in the industry for a while the patterns become clearer. Enterprise technology adoption has some fairly distinct shapes. In 2009 I wrote

“Amazon is the new VMware. The adoption patterns are going to be similar. Enterprises will see AWS as a test and development environment first, but over time production workloads will migrate there.”

I dubbed this “the VMware pattern”. New technologies generally don’t emerge as fully-fledged production environments. They are adopted first and grow into the role. Docker is currently on a fast track through this process.

In the same post I wrote:

“Amazon isn’t the de facto standard cloud services provider because it is complex – it is the leader because the company understands simplicity at a deep level, and minimum progress to declare victory.”

For docker replace “simplicity” with “convenience”. Why is Docker is so hot? The answer is simple. Developer-led adoption, or as Andrew Clay Shafer puts it:

“It’s the fastest path to developer dopamine”.

At RedMonk we have never seen a technology become ubiquitous so quickly. Docker makes it simple to spin up a container which contains everything needed to run an app – the code itself, the runtimes, systems tools etc. Develop on your laptop, then in theory deploy to any server. Unlike virtual machines, containers include the application and all of its dependencies, but share the kernel with other containers, an efficient model which maps cleanly to current development thinking in areas such as continuous integration and microservices. Stephen, in a thoughtful explanation of the Docker phenomenon, argues that:

The explosion of Docker’s popularity begs a more fundamental question: what is the atomic unit of infrastructure moving forward? At one point in time, this was a server: applications were conceived of, and deployed to, a given physical machine. More recently, the base element of an infrastructure was a virtual recreation of that physical machine. Whether you defined that as Amazon did or VMware might was less important than the idea that an image resembling a server, from virtualized hardware and networking interfaces to a full instance of an operating system, was the base unit from which everything else was composed.

Containers generally and Docker specifically challenge that notion, treating the operating system and everything beneath as a shared substrate, a universal foundation that’s not much more interesting the raised floor of a datacenter. For containers, the base unit of construction is the application. That’s the only real unique element.

So alongside many of the other micro and macro trends we currently see, notably infrastructure fragmentation, Docker basically just makes sense – it feels right and represents how developers live now. Next however comes the fun part – Docker will begin to reshape how operations and IT work, just as VMware did in the virtualisation wave.

Enterprises needs to find ways to deliver more digital services to market faster, which means not only becoming more adept at developing, but also consuming new technology. Docker can help with that. In the age of continuous deployment dev/test underpins the deployment process. Everything needs to be constantly tested, and constantly refactored, with an eye to disposability rather than reuse. See Microservices and Disposability: On Cattle, Pets, Prize Bulls, Wildebeests and Crocodiles

Docker is on an exceedingly well-funded mission to transform itself from developer favourite to Cloud Native production environment of choice for the enterprise, moving from Open Container format to “single virtual computer” of choice. The transition though from developer-led to enterprise production grade takes time. We’ve seen this before, from MySQL to Mongo to Spring… or for those with rather longer memories think the early versions of Oracle. Automation, backup, compliance, logging, monitoring, networking, scheduling, storage management, orchestration, security, and basic engineering solidity don’t happen overnight.

There is now an ecosystem of companies building tooling to support Docker in production- startups like ClusterHQ, Datadog, Rancher Labs, Server Density, Sysdig, Treasure Data and Weaveworks. More established players such as AppDynamics, CloudBees and New Relic. Also of course the cast of existing suppliers looking to embrace and extend Docker- including Amazon Web Services, IBM, Microsoft, Oracle, Pivotal etc. Then there are outright competitive plays for the bigger prizes, such as Google Kubernetes. There will be negative commentary – growing pains are par for the course.

Docker is not going to have everything it’s own way – but the path is now set clear for Docker to become an industry standard production platform. We can call the path it’s on the Docker Pattern.

For further reference see also
IBM, Red Hat adopt “VMware Pattern” for Cloud. Disruption Strategy Emerges
Amazon Web Services: an instance of weakness as strength

Our clients include Docker itself, Amazon Web Services, IBM, New Relic, Oracle, Pivotal and Treasure Data.

Debbie Clapper made the beautiful pattern above.

Categories: Uncategorized.

On IBM Ninja Moves at VMware, Swift Blank Space and Github

I’ve got a blank space, baby
And I’ll write your name – Taylor Swift

Last week at IBM’s Interconnect conference it made a series of announcements intended to demonstrate relevance in the cloud era – with the likes of VMware, Apple and Github. My colleague Fintan offers a summary here.

Helping enterprise customers make the transition to the cloud is job one for incumbent vendors so IBM announced a partnership with VMware to help customers migrate virtual workloads to IBM’s SoftLayer Infrastructure As a Service (IaaS) Platform. The deal signals that VMware is moving away from the idea of running its own platform, vCloud AIR. Competing on public cloud infrastructure build out is brutal from an investment perspective, and VMware has always been a channel play, so it certainly makes sense for VMware. The is not exclusive, but according to Barbara Darrow at Fortune it may have been more of a coup than was immediately obvious – How IBM Stole Google’s Thunder.

“Google really wanted to announce this and then all of a sudden IBM did a ninja move, and it was IBM’s deal,” said one source close to VMware who had knowledge of the process.

So why IBM and not Google? Well, for one thing, IBM has experience selling and implementing vSphere.

In advance of last week’s news, Jim Comfort, chief technology officer for IBM’s cloud unit, told Fortune that IBM’s services business is VMware’s largest distributor and that the company has already worked with VMware’s NSX network virtualization gear on SoftLayer.

That sort of experience is not something that Google, with its massive shared public cloud infrastructure, can claim—even though VMware co-founder Diane Greene has led Google’s enterprise group since November.

IBM experience being valuable. Whatever next… 😉 The deal isn’t exclusive, but it lays out a marker. It’s also worth mentioning the Github deal here – IBM and Github say they plan to offer Github Enterprise as a service on IBM Bluemix. Developer pipelines is a hot topic in tech right now – it’s kind of the new Application Lifecycle Management (ALM) – as cloud platforms integrate agile and continuous integration tools and approaches into their platforms.

The biggest bet and the most substantive Interconnect news from a developer perspective was an announcement with Apple – IBM is taking Swift onto the server side, in order to foster an open ecosystem for the new programming language. Apple open sourced Swift in late 2015, but it will need help if it wants to broaden it’s appeal beyond IOS developers.

Apple likes to be in control of the context in which it makes announcements and there was no doubting its commitment. Brian Croll, vice-president of product marketing at Apple, made the high level pitch, but then language creator Chris Lattner made an appearance too, to give a demo, which was pretty cool.

Swift is already the the fastest growing programming language we have tracked since we began our Programming Language Rankings but Apple wants Swift to become a dominant language. Many commentators have argued that Apple is giving IBM a leg up with the partnership, but IBM brings a great deal to the table.

Apple itself is at somewhat of a crossroads. Slowing global growth has hurt sales, which Apple forecasts are set to fall for the first time since 2003. Investors are now worrying about Peak Apple. Enterprise is an obvious potential source of new revenue growth, though CEO Tim Cook says it is already a $25bn business.

But as Tim Cook admits: ” “We don’t have deep knowledge of all the verticals that enterprise is in,” Cook acknowledged, referring to selling to specific sectors like the financial services and energy.”

IBM on the other hand… is of course all about those verticals – and finance, manufacturing, public sector.

It has been heads down developing IOS-specific apps using Swift – now over 100 – for those industries, and it has the sales channels and experience Apple clearly lacks.

One of the classic moves in the IBM playbook is to create a market by working closely with another major partner to define and carve out a space. Examples of the technique include working with Sun in the 1990s to define and push Java into the mainstream Enterprise, and also the Service Oriented Architecture wave with Microsoft, defining the WS-* stack. Both Java and SOA, while maligned by many for overcomplexity, defined the industry for a period of years. IBM is good at this stuff. See its similar move to carve out a space for Cloud Foundry in the enterprise, with Pivotal playing the role of BEA in a new app container wave. On that note IBM’s Bluemix PaaS also supports Swift with a Cloud Foundry buildpack.

swift at ibm

IBM isn’t just saying Swift is cool, it’s committed from an engineering perspective. Check out its Swift sandbox – the easiest way to take a look at the language, allowing you to write and run code from the browser. Sample code is so far fairly limited but we can expect IBM to move forward with this pretty quickly. But how does Swift work when it’s not running on an Apple device? On Ubuntu Swift takes advantage of glibc c and C++ libraries. One big question in my mind is graphics and windowing on non-Apple devices and what that might look like. The legacy of Objective-c is also a question. Swift makes developing for IOS a lot easier, but supporting Objective-c libraries so as not to cut off existing developers creates technical debt. The question is – can you actually be Swift developer at this point without also being an Objective-c developer?

At Interconnect IBM also announced a Swift framework/web server project called Kitura, inspired by Express.js. IBM is hoping that Swift will attract developers to fill in all the white space, in the same way as Javascript developers rushed in to create libraries and frameworks in Node.js. Nature abhors a vacuum.

Or as Taylor Swift puts it: “I got a blank space baby And I’ll write your name”.

It is very important however that in modern languages IBM keeps its focus on Node.js – which is a clearer immediate path to revenues, given current enterprise adoption. Swift is a language to turn heads, but Node.js is ready to settle in with.

IBM paid T&E for my trip to Interconnect and is a client. IBM Pivotal and VMware are all clients.

Categories: Uncategorized.

On Lightbend, Lagom, and Java is Dead is Dead

lang-plot-116-wm-e1455906685179

Java is Dead is Dead. It was fashionable for a while to comment on the idea Java has legs with a curt “oh no Java the language is dead but the JVM is decent”. The cool kids like Twitter were using Scala. But Java the language shows consistent ongoing strength. For data, Stephen just dropped our regular programming language rankings and Java is still very much a thing.

Today came news Typesafe is rebranding. A company built on the idea it could build a business around Scala will henceforth be called Lightbend, and is going after Java shops adopting microservices approaches with a new framework called Lagom – Swedish for “just enough”.

Lightbend is still about Reactive Applications, and will continue to develop in Scala, but it had to recognise that trying to replace Java outright was not going to work, no matter how productive its Play framework and Akka runtime toolkit could make developers.

I talked to Paul Krill at Infoworld about the news and said:

Analyst James Governor of RedMonk sees an opportunity for Lagom. “The Java community needs good tools for creating and managing microservices architectures,” he said. “Lagom is squarely aimed at that space.”

Lagom would compete with the Spring Boot application platform in some areas, according to Governor. “It is early days for Lagom, but the design points make sense,” he noted. Typesafe was focused on Scala, which was adopted in some industries, such as financial services, but never became mainstream, he argues. “So [the company now] is looking to take its experiences and tooling and make them more generally applicable with a Java-first strategy.

Categories: Uncategorized.

Swift Exploding, Monkchips flying, Imma Let You Finish

It’s been a while since we created any new Opinionated Infrastructure videos, which is a mistake. I feel that the series, gonzo though it is, represents some of the best work I have ever done in terms of immediacy and being engaging while talking about tech stuff. I was pretty happy therefore when IBM’s Mobile First group proposed that I got back in the game. I love to share stories and this commission comes at a good time. I want to do a fair bit more in solid video work this year, and we’re working on a site redesign to encourage that.

Next week is Mobile World Congress, a huge event which keeps getting bigger – because mobile is eating the world. As a marker consider that last year IBM CEO Ginni Rometty didn’t attend IBM Interconnect, but went to MWC instead. But that’s the business level conversation. What about developers? RedMonk’s regular programming language rankings {just updated!] show Apple’s new programming language SWIFT exploding – it’s one of the fastest growing technologies we’ve ever seen in terms of developer mindshare. More on that shortly.

Anyway we made this video, but totally forgot the most important thing – references to Taylor Swift.

Sorry for not being a Millennial. But I am pretty happy with how it came out anyway. I hope you enjoy it. And- did you know that IBM is leading a server-side SWIFT initiative, so you can run in on Ubuntu, and has this cool sandbox to play with?

update – for more reckons on rankings, while i was posting here, Stephen said this:

“Swift: Swift’s meteoric has predictably slowed as it’s entered the Top 20, but importantly has not stopped. For this ranking, Swift moves up one spot from #18 to #17. As always, growth is more difficult the closer you get to the top, and in passing Matlab, Swift now finds itself a mere two spots behind Go – in spite of being five years younger. It is also three spots behind Scala and only four behind R. Which means that Swift finds itself ranked alongside languages of real popularity and traction, and is within hailing distance of our Tier 1 languages (R is the highest ranking Tier 2). The interesting thing is that Swift still has the potential to move significantly; its current traction was achieved in spite of being a relatively closed alternative amongst open source alternatives. Less than four weeks before we took this quarter’s snapshot of data, Swift was finally open sourced by Apple, which means that the full effect of this release won’t be felt until next quarter’s ranking. This release was important for developers, who typically advantage open source runtimes at the expense of proprietary alternatives, but also because it allows third parties to feel comfortable investing in the community in a way they would not for a proprietary stack – see IBM’s enthusiastic embrace of Swift. This means that Swift has, uniquely, multiple potential new engines for growth. So it will be interesting indeed to see what impact the release has on Swift overall adoption, and whether it can propel it near or actually into the Top 10.”

Categories: Uncategorized.

Monki Gras – The developer conference about craft culture

I wanted a theme for 2016 that riffed off the idea of a Software and Crafts movement. Earlier this year Dave Letorey was talking about brewing beer for this year’s event, so Homebrew stuck.That is, I had the theme early.We’re in the age of the side project, in our industry because of open source dominance, so why not look at passion and sustainability and making beautiful things, not because its a job but because its a way of life.There is nothing more purpose driven than homebrew, or baking, or knitting, or becoming a wine distributor. You do it because you love it, not because it’s convenient. There is something incredibly powerful about that.Just as with software the brewing industry is being completely disrupted by the growth of microbreweries, many of which began as homebrew. Thus Evin O’Riordain, founder of The Kernel Brewery, one of our speakers this year, when he outgrew it gave his first set of equipment to the talented Mr Andy Smith at Partizan Brewing.

Source: Blog | Monki Gras | The developer conference about craft culture

Categories: Uncategorized.

A conversation about Continuous Deployment and Continuous Everything

When I was at HP Discover earlier this year we recorded a podcast about continuous deployment and DevOps with Dana Gardner of Interarbor Solutions and Ashish Kuthiala, Senior Director for Strategy at Hewlett Packard Enterprise. You can listen to the podcast here. Here are some edits from the transcript, with hopefully a couple of nuggets. I am fascinated for example by the convergence of social monitoring and product management and if necessary recall- as exemplified by the GM example.

 

Kuthiala: The continuous assessment term, despite my objections to the word continuous all the time, is a term that we’ve been talking about at HPE. The idea here is that for most software development teams and production teams, when they start to collaborate well, take the user experience, the bugs, and what’s not working on the production end at the users’ hands — where the software is being used — and feed those bugs and the user experience back to the development teams.

When companies actually get to that stage, it’s a significant improvement. It’s not the support teams telling you that five users were screaming at us today about this feature or that feature. It’s the idea that you start to have this feedback directly from the users’ hands.

We should stretch this assessment piece a little further. Why assess the application or the software when it’s at the hands of the end users? The developer, the enterprise architects, and the planners design an application and they know best how it should function. Whether it’s monitoring tools or it’s the health and availability of the application, start to shift left, as we call it.

 

Governor: One notion of quality I was very taken with was when I was reading about the history of ship-building and the roles and responsibilities involved in building a ship. One of the things they found was that if you have a team doing the riveting separate from doing the quality assurance (QA) on the riveting, the results are not as good. Someone will happily just go along — rivet, rivet, rivet, rivet — and not really care if they’re doing a great job, because somebody else is going to have to worry about the quality.

As they moved forward with this, they realized that you needed to have the person doing the riveting also doing the QA. That’s a powerful notion of how things have changed. Certainly the notion of shifting left and doing more testing earlier in the process, whether that be in terms of integration, load testing, whatever, all the testing needs to happen up front and it needs to be something that the developers are doing.

 

Governor: We’re making reference to manufacturing modes and models. Lean manufacturing is something that led to fewer defects, apart from (at least) one catastrophic example to the contrary. And we’re looking at that and asking how we can learn from that.

So lean manufacturing ties into lean startups, which ties into lean and continuous assessment.

What’s interesting is that now we’re beginning to see some interplay between the two and paying that forward. If you look at GM, they just announced a team explicitly looking at Twitter to find user complaints very, very early in the process, rather than waiting until you had 10,000 people that were affected before you did the recall.

Last year was the worst year ever for recalls in American car manufacturing, which is interesting, because if we have continuous improvement and everything, why did that happen? They’re actually using social tooling to try to identify early, so that they can recall 100 cars or 1,000 cars, rather than 50,000.

It’s that monitoring really early in the process, testing early in the process, and most importantly, garnering user feedback early in the process. If GM can improve and we can improve, yes.

 

Gardner: I remember in the late ’80s, when the Japanese car makers were really kicking the pants out of Detroit, that we started to hear a lot about simultaneous engineering. You wouldn’t just design something, but you designed for its manufacturability at the same time. So it’s a similar concept.

But going back to the software process, Ashish, we see a level of functionality in software that needs to be rigorous with security and performance, but we’re also seeing more and more the need for that user experience for features and functions that we can’t even guess at, that we need to put into place in the field and see what happens.

How does an enterprise get to that point, where they can so rapidly do software that they’re willing to take a chance and put something out to the users, perhaps a mobile app, and learn from its actual behavior? We can get the data, but we have to change our processes before we can utilize it.

 

Kuthiala: Absolutely. Let me be a little provocative here, but I think it’s a well-known fact that the era of the three-year, forward-looking roadmaps is gone. It’s good to have a vision of where you’re headed, but what feature, function and which month will you release so that the users will find it useful? I think that’s just gone, with this concept of the minimum viable product (MVP) that more startups take off with and try to build a product and fund themselves as they gain success.

It’s an approach even that bigger enterprises need to take. You don’t know what the end users’ tastes are.

I change my taste on the applications I use and the user experience I get, the features and functionality. I’m always looking at different products, and I switch my mind quite often. But if I like something and they’re always delivering the right user experience for me, I stick with them.

The way for an enterprise to figure out what to build next is to capture this experience, whether it’s through social media channels or engineering your codes so that you can figure out what the user behavior actually is.

The days of business planners and developers sitting in cubicles and thinking this is the coolest thing I’m going to invent and roll out is not going to work anymore. You definitely need that for innovation, but you need to test that fairly quickly.

Also gone are the days of rolling back something when something doesn’t work. If something doesn’t work, if you can deliver software really quickly at the hands of end users, you just roll forward. You don’t roll back anymore.

It could be a feature that’s buggy. So go and fix it, because you can fix it in two days or two hours, versus the three- to six-month cycle. If you release a feature and you see that most users — 80 percent of the users — don’t even bother about it, turn it off, and introduce the new feature that you were thinking about.

This assessment from the development, testing, and production that you’re always doing starts to benefit you. When you’re standing up for that daily sprint and wondering what are the three features I’m going to work on as a team, whether it’s the two things that your CEO told you you have to absolutely do it, because “I think it’s the greatest thing since sliced bread,” or it’s the developer saying, “I think we should build this feature,” or some use case is coming out of the business analyst or enterprise architects.

 

Gardner: For organizations that grok this, that say, “I want continuous delivery. I want continuous assessment,” what do we need to put in place to actually execute on it to make it happen?

 

Governor: We’ve spoken a lot about cultural change, and that’s going to be important. One of the things, frankly, that is an underpinning, if we’re talking about data and being data-driven, is just that we have new platforms that enable us to store a lot more data than we could before at a reasonable cost.

There were many business problems that were stymied by the fact that you would have to spend the GDP of a country in order to do the kind of processing that you wanted to, in order to truly understand how something was working. If we’re going to model the experiences, if we are going to collect all this data, some of the thinking about what’s infrastructure for that so that you can analyze the data is going to be super important. There’s no point talking in being data-driven if you don’t have architecture for delivering on that.

 

Kuthiala: You’re right. We have a very rich portfolio across the entire software development cycle. You’ve heard about our Big Data Platform. What can it really do, if you think about it? James just referred to this. It’s cheaper and easier to store data with the new technologies, whether it’s structured, unstructured, video, social, etc., and you can start to make sense out of it when you put it all together.

There is a lot of rich data in the planning and testing process, and all the different lifecycles. A simple example is a technology that we’ve worked on internally, where when you start to deliver software faster and you change one line of code and you want this to go out. You really can’t afford to do the 20,000 tests that you think you need to do, because you’re not sure what’s going to happen.

We’ve actually had data scientists working internally in our labs, studying the patterns, looking at the data, and testing concepts such as intelligent testing. If I change this one line of code, even before I check it in, what parts of the code is it really affecting, what functionality? If you are doing this intelligently, does it affect all the regions of the world, the demographics? What feature function does it affect? It’s narrowing it down and helping you say, “Okay, I only need to run these 50 tests and I don’t need to go into these 10,000 tests, because I need to run through this test cycle fast and have the confidence that it will not break something else.”

So it’s a cultural thing, like James said, but the technologies are also helping make it easier.

 

Kuthiala: We were talking about Lean Functional Testing (LeanFT) at HP Discover. The idea is that the developer, like James said, knows his code well. He can test it well before and he doesn’t throw it over the wall and let the other team take a shot at it. It’s his responsibility. If he writes a line of code, he should be responsible for the quality of it.

 

Governor: The RedMonk view of the world, is that, increasingly, developers are making the choices, and then we’re going to find ways to support the choices they are making. The term continuous integration began as a developer term, and then the next wave of that began to be called continuous deployment. That’s quite scary for a lot of organizations. They say, “These developers are talking about continuous deployment. How is that going to work?”

The circle was squared when I had somebody come in and say what we’re talking to customers about is continuous improvement, which of course is a term again that we saw first in manufacturing. But The Developer Aesthetic is tremendously influential here, and this change has been driven by them. My favourite “continuous” is a great phrase, continuous partial attention, which is the world we all live in now.

Categories: Uncategorized.

What is your Integrated Joy Strategy?

Categories: Uncategorized.

Thingmonk Update: crazy speaker line up, venue etc

OK we’re about a week away from Thingmonk and we have still have some places left. i wanted to ping you again because details are a lot clearer now, and the conference looks pretty banging, to be honest. You should come if you haven’t signed up already.

Our speakers list is pretty crazy – we’re flying Matt Biddulph in from the San Francisco to talk about Thington, a platform built from the design perspective of conversations between machines and machines and machines and people.  Mark Shuttleworth of Ubuntu fame will be talking about how open source will underpin IoT, Dave McCrory CTO of Basho will give us the ins and outs of design decisions in creating a time series data store, Jeremiah Stone will discuss GE Industrial Automation’s technical decisions in becoming a platform company.

As in previous years we also have a big focus on user experience with Sophie Riches and Sam Wimslet of IBM talking about Design Thinking at scale, with Claire Rowland updating us on her reckons from Designing Connected Products: UX design for the internet of things, published by O’Reilly. Amanda Brock will give us the low down on the legal implications of a world in which machines regularly make decisions on our behalf. Yodit Stanton will update on us opensensors.io – expect some Clojurey goodness, and we also have Natalia Oskina from the same firm explaining how it is monitoring air pollution levels near Heathrow (how unusual to bring some facts to the expansion debate).

Like AWS? Kyle Roche, who runs IoT at Amazon will be spending the conference with us, and you can expect a deeply technical talk from him.  Thomas Grassl and Craig Cmehil will be talking about how they’re retooling SAP to make it more developer friendly in order to increase the company’s relevance to IoT, bridging old school manufacturing and resource planning software into the new world. Boris Adryan is a PhD and squeaky wheel – he’ll be telling us what we’re all doing wrong 😉

We’re also very excited to have Moo involved this year, speaking on the technical, design and industrial challenges of rolling out an entirely new product category – NFC enabled business cards. Well be using these cards to track consumption of Coffee, drinks and food at the event and announcing a hack competition with the winners being showcased at sister conference Monki Gras in January.

Last year Andy Stanford Clark wowed us by running his presentation on a raspberry pi powered by a hydrogen fuel cell. We liked it so much that this year we’re running the conference at the Arcola Theatre in Dalston, which oddly enough is also the home of Arcola Energy, which provided the fuell cell in question.

Food and drink will be to the the usual RedMonk artisan standards- expect Arancini balls and other delights, and the finest craft ales known to humanity – oh yeah this year we’ll also be bringing you some natural wines. Breakfast will come from our media partners The New Stack, who are all about pancakes!

Anyway hope to see you there. There are some good discount codes flying around so just let me know if you want one, and haven’t already seen one out in the open.

Categories: Uncategorized.