Elasticsearch and Splunk: Google Trends

Google Trends – @elasticsearch, @splunk – http://t.co/0mMHCOJAS1 ELK going mainstream in search volume, steep curve pic.twitter.com/gulIFUhkj8

— Grumble Bundle (@monkchips) September 20, 2015

Last week while idly pondering the rise of Elasticsearch Logstash Kibana (ELK) stack I thought I would compare Elasticsearch with Splunk on Google Trends. The results were interesting.

There are obviously some significant caveats- for one, the search terms I used were dictated by what might actually be valid using Google Trends – “ElasticSearch” vs “Splunk”, rather than ELK vs Splunk, which is closer to what I had in mind, but might have been confused with large red deers.

The evolution of Elasticsearch has been interesting. From its beginnings as a full text search engine built on Lucene it has grown into something more far-reaching. Packaged by Elastic, a commercial company, as an analytics toolset bringing together ElasticSearch, Logstash as as a general purpose data collector (what began as a single purpose project, a tool for collecting and transforming logs, has now become a general purpose ETL tool), and Kibana for data visualisation and exploration. ELK is an easy developer friendly toolkit for data analysis.

It’s also probably worth mentioning Greylog, another commercial company building on Elasticsearch that has set itself to more directly target Splunk. Meanwhile fluentd is another open source project that performs similar functions to Logstash, developed by Treasure Data.

Splunk however is already a publicly traded company, IPO in 2012, with significant enterprise penetration that specialises in log management for operational analytics and security. 2015 total were $450.9 million, not bad for a company helping clients to mine data they have traditionally thrown away or ignored. Splunk is not primarily an open source company. Splunk is far ahead in revenues, and enterprise adoption model than any open source competitor. But the ELK stack is used in some similar spaces, and ELK is finding a role for general purpose analytics, not just analysis of logs for operational data specific.

Splunk vs Elasticsearch is very much an imperfect comparison and yet Elasticsearch is clearly a funnel in its own right, and ELK is finding its own DevOps ops led customer base. Enterprise ops teams are likely to choose Splunk, but Cloud developers and DevOps teams at this point are likely to favour ELK.

Unfortunately I wasn’t able to attend Splunk’s user conference this week, which might have helped me parse how the company will deal with the rise of open source competitors. This post isn’t intended as a negative against Splunk, but rather to note that in terms of Google Search Volume at least, Elasticsearch is the mix.

Elastic, Splunk and Treasure Data are all RedMonk clients.