The overriding theme of every disruption story I’ve ever heard is that firms thought they had more time than they did. So, I’ve been pondering the why. We can see disruption happening all around us, but why is it so difficult to get out in front of it?…
Forrester has seen unprecedented adoption of Hadoop in the last three years. We estimate that firms will spend $800 billion in Hadoop software and related services in 2017. Not surprisingly, Hadoop vendors have capitalized on this — Cloudera, Hortonworks, and MapR have gone from a “Who?” to “household” brands in the same period of time.
But like any good run, times change. And the major force exerting pressure on Hadoop is the cloud. In a recent report, The Cloudy Future Of Hadoop, Mike Gualtieri and I examine the impact the cloud is having on Hadoop. Here are a few highlights:
● Firms want to use more public cloud for big data, and Hadoop seems like a natural fit. We cover the reasons in the report, but the match seems made in heaven. Until you look deeper . . .
● Hadoop wasn’t designed for the cloud, so vendors are scurrying to make it relevant. In the words of one insider, “Had we really understood cloud, we would not have designed Hadoop the way we did.” As a result, all the Hadoop vendors have strategies, and very different ones, to make Hadoop relevant in the cloud, where object stores and abstract “services” rule.
● Cloud vendors are hiding or replacing Hadoop all together. AWS Athena lets you do SQL queries against big data without worrying about server instances. It’s a trend in “serverless” offerings. Google Cloud Functions are another example. DataBricks uses Spark directly against S3. IBM’s platform uses Spark against CloverSafe. See the pattern?
As more firms get tired of Hadoop’s on-premises complexity and shift to the public cloud, they will look to shift their Hadoop stacks there. This means that the Hadoop vendors will start to see their revenue shift from on-premises to the cloud.
Technology buyers have made it clear to us they want platforms for building data analytics applications. I call these insight platforms, and they were the No. 1 emerging technology of interest for enterprise architects in 2016. Understanding why is easy — insight platforms provide a common toolset and a place to run what you have built. They accelerate both time-to-value and agility, which are crucially important for keeping up with markets and customers. See Tame The Beast: Forrester’s Insight Platform Vendor Landscape and Want To Create Action From Big Data? Look At Enterprise Insight Platform Suites for more information.
Since using more public cloud is the No .1 big data priority, according to our 2016 survey of 3,000+ data and analytics decision makers, Insight Platforms-as-a-Service are next on my Forrester Wave™ agenda. We define Insight Platforms-as-a-service at multitenant platform-as-a-service cloud offerings that include tools for data management, several types of analytics and technology that help firms operationalize insight in other software and processes.
I already know the biggest players – Google, Amazon, Microsoft, IBM and Oracle – but I’m also looking for other providers who want to give them a run for their money. Who else I should look at? For example, should I include:
By now firms are deep into their big data investments — and frustrated. Too many new and rapidly evolving technologies are built on an open source and named after a bunch of zoo animals. The term insight platform has struck a chord with technology buy…
In 2014, I recognized something was a bit off with all the big data excitement and I started interviewing companies to get to the bottom of it. In 2015, Ted Schadler and I published the first of my ideas in the report “Digital Insights Are The New Currency Of Business.” In that report, we pointed out what was wrong – big data only focused on how to turn more data into more insight. It didn’t say anything about how to turn that insight into more action. In that report we defined a system of insight, which focused big data energy on implementing insights in software using closed loops that create action and continous learning. In this year’s Top Emerging Technologies To Watch report, we evaluated sytems of insight technologies that were creating the most change, and we found many. For example:
Insight Platforms: Data management and analytics are not separate technologies anymore. Open source and cloud have made it so easy for vendors to combine these technologies into a platform – but things don’t stop there. Add insight-to-execution technologies like predictive model runtimes and you get a platform that is ideal for building closed-loop systems of insight. Our latest vendor and user surveys indicate that insight platforms are hot, hot. Expect to see more data and analytics tools merge into platforms this year.
The mix of analysts who showed up to a recent Cambridge Semantics briefing illustrates a big problem data and analytics technology buyers have – too many data and analytics solutions and a ton of overlap. For example, of the five analysts who came:
Is your business digital? Like Domino’s Pizza, do you realize that you are not a product or service business, but that you are a software and data business that provides products or services? Do you exploit all of your customer’s data to know them insi…
Are you lost in a confusing soup of vendor-speak about what their data analytics stack actually offers? Big data, data platforms, advanced analytics, data lakes, real-time everything, streaming, the IoT, customer analytics, digital intelligence, real-t…
It’s been a while since I’ve blogged; not because I’ve had nothing to say, but rather because I’ve been busy with my colleagues Ted Schadler, James McCormick, and Holger Kisker working on a new line of research. We wanted to examine the fact that business satisfaction with analytics went down 21% between 2014 and 2015, despite big investments in big data. We found that while 74% of firms say they want to be “data-driven,” only 29% say they are good at connecting analytics to action. That is the problem.
Ted Schadler and I published some initial ideas around this idea in Digital Insights Are The New Currency Of Business in 2015. In that report, we started using the phrase digital insight to talk about what firms were really after ― action inspired by new knowledge. We saw that data and analytics were only means to that end. We also found that leading firms were turning data into insight and action by building systems of insight ― the business discipline and technology to harness insights and consistently turn data into action.
Here is a key figure from that report:
Three of four architects strive to make their firms data driven. But well-meaning technology managers only deal with part of the problem: How to use technology to glean deeper, faster insight from more data — and more cheaply. But consider that only 2…
I said that 2015 would be a tough year for enterprise data and analytics vendors in my spring report, “Brief: Turning Big Data Into Business Insights, 2015.” I thought two things would happen. First, open source would drag on vendors’ revenues as demand for big expensive products declined. Second, the cloud would create revenue headaches. Turns out, I was right. Teradata’s midyear earnings were down 8%, and IBM reported that Q2 revenue was down 12% from a year ago. As further proof, consider the rash of data management vendors running for private equity (e.g. Dell/EMC, Informatica, and TIBCO). It’s been tough times indeed, even though most vendors are keeping their messaging positive to reassure buyers and investors.
Over the past two weeks, I attended Teradata Partners in Anaheim and IBM Insight in Las Vegas — giving me a firsthand look at how two giants of the data and analytics industry are handling disruption. What I saw was a tale of two vendors that couldn’t be any more different: