Data intensity: the key to a data-driven future

What does the data-driven future look like?

It will consist of systems that:

• Highly automated and use data to make trusted, fair decisions in a split second.
• Personalized and situationally aware to meet user needs.
• Able to address data movement, geographic distribution, governance, privacy and security; and
• Decentralized, tackle data ownership and collaborate with centralized systems to enable data sharing for the greater good.

But we don’t have to wait for that.

The data-driven future is already here.

An autonomous vehicle is an intensive data-driven system that perceives its environment in real time and translates this into vehicle actions. At a level below autonomy, assistive technologies are also data-driven, relying on real-time data to gain insight – i.e. the blind spot detection system sends an alert – or to make decisions about when to use anti-lock brakes and crash avoidance systems.

Successfully enabling such applications and use cases to be more data driven is a journey that involves tackling complexity and adopting new approaches that allow you to better manage systems through maturity and sophistication. Think in terms of data intensity to assess digital maturity and resilience and take your data-driven business to the next level.

The data intensity is multivariable and changes dramatically as you move in more than one dimension. The data intensity of an application depends on data volume, query complexity, query latency, data ingestion speed, and user concurrency. Additional dimensions may include hybrid workloads (transactional and analytics), multimodal analytics (operational analytics, machine learning, search, batch, and real-time), elasticity, data movement requirements, and so on.

Data intensity is increasing

Data intensity isn’t just about data volume, it’s about what you do with your data. However, as the data volumes increase, the intensity increases. The intensity increases exponentially when the data also comes faster, requiring an application to handle 10 times more users while meeting the same (or better) latency SLAs. The intensity also increases when real-time operational data analysis is combined with natural language interaction and recommendations.

We live in a data-intensive era and the intensity is increasing as organizations increase their reliance on data to better understand their customers and shape experiences. How your organization responds in the data-intensive era can either create more complexity and friction for you and your customers, or give you new opportunities for differentiation and growth.

It is clearly counterproductive to opt for an approach that leads to more complexity and friction. But historically, many organizations have operated under the assumption that different workloads require different architectures and technologies, and that transactional and analytic workloads should be separate. Managing data intensity in this environment creates inherent complexityfriction and data movement that add latency and conflict with real-time insights.

Fortunately, you now have the chance to rethink and challenge traditional assumptions to embrace, enable and make the most of the data-intensive era. You can leverage cloud computing, which offers unprecedented scale and flexibility and empowers organizations to innovate and experiment; separation of storage and compute power, untangling storage and compute requirements; and modern solutions that combine transactional and analytic workloads into one engine for all workloads.

In a data-driven organization, day-to-day operations, business analytics insights, and customer experiences become one – in real time. That’s intense: data-intensive.

Oliver Schabenberger is the Chief Innovation Officer at SingleStore.