Three of four architects strive to make their firms data driven. But well-meaning technology managers only deal with part of the problem: How to use technology to glean deeper, faster insight from more data — and more cheaply. But consider that only 2…
I said that 2015 would be a tough year for enterprise data and analytics vendors in my spring report, “Brief: Turning Big Data Into Business Insights, 2015.” I thought two things would happen. First, open source would drag on vendors’ revenues as demand for big expensive products declined. Second, the cloud would create revenue headaches. Turns out, I was right. Teradata’s midyear earnings were down 8%, and IBM reported that Q2 revenue was down 12% from a year ago. As further proof, consider the rash of data management vendors running for private equity (e.g. Dell/EMC, Informatica, and TIBCO). It’s been tough times indeed, even though most vendors are keeping their messaging positive to reassure buyers and investors.
Over the past two weeks, I attended Teradata Partners in Anaheim and IBM Insight in Las Vegas — giving me a firsthand look at how two giants of the data and analytics industry are handling disruption. What I saw was a tale of two vendors that couldn’t be any more different:
Enterprise architects face more exciting — and greater — challenges as the age of the customer takes off. But technology invention, innovation, and spending are notoriously cyclical. In fact, our first tech trends report in 2009 predicted a boom cycle through 2016. And we have seen this — with social, mobile, cloud, analytics, and big data, to name just the obvious ones. A big finding of our research however is this: The Age Of The Customer has changed the classic technology investment cycle. For example, technology management’s spend will grow about 5% in North America in 2016. This is a decent pace. However, spend on business technology — the things that let firms win, serve, and retain customers — will be double that!
All this new money will shift the focus of investment from point solution inventions toward “end-to-end innovation” by 2018. And by end-to-end, we mean across the customer life-cycle and customer journeys as opposed to classic ‘enterprise integration’. This shift will happen in three phases:
Enterprise architects, are you mired in a tangled web of data marts while your business pursues customer engagement without you? If you think a Hadoop-centric architecture is going to save the day, you may need to rethink. Your customers expect you to create systems of insight to deliver win-win engagement in real time. I’m seeing a new class of digital predators leverage the cloud to do just this. For example, Netflix designs cover graphics for its series based on subscriber viewing habits. They know their customers that well.
I call their technology approach an Elastic Analytics Platform in my recently published report. I formally define it as:
“A combination of data storage and middleware technology that allows the creation and dissolution of analytics components on demand, while provisioning these with data from one, or a few, distributed, virtualized data sources.”
That’s a mouthful. So here’s a rough picture:
Firms like Netflix, Stitch Fix (who? read the linked KDnuggets blog post), and LinkedIn are sourcing all their data, and I mean everything, into a few data stores in the cloud. Next, they are exploiting cloud to create analytic workloads on demand. This gives them elasticity two ways. First, they get scale-out storage; second, they get on-demand analytics components. For example, Netflix can spin up Hadoop, Spark, or Kafka clusters as they need them and provision these from Kafka or S3. They also have Teradata on Amazon. This gives them enormous flexibility to create as much of what they need when they need it.