5 Top Trends Driving Data Infrastructure Strategies, According to Gartner

This is a busy year for many organizations when it comes to their data infrastructure. Many are implementing delayed upgrades and deployments necessitated by the pandemic. Some want to stay ahead of the competition with new investments. Others strive to improve relationships with customers and employees through technology that increases engagement.

Adam Ronthal, analyst in the data management practice at Gartnerdetailed, in an interview with VentureBeat, where most organizations will be investing in data infrastructure for the rest of 2022 and early 2023.

When we talk about investing in data infrastructure, it involves all the infrastructure needed to store and participate in a wide variety of data use cases, Ronthal said. These can be both operational and analytical use cases. It may also include operational or order fulfillment transaction systems.

“Any single use case for data requires successful management of that data to be successful. That goes for building applications as well as doing data science, machine learning, visualization, advanced analytics, data marketplaces, exchanges, etc. Facts supports every adjacent area and all business use cases that leverage that data,” Ronthal said.

Looking ahead, Ronthal sees the following as the key trends that will drive data infrastructure investment for the remainder of this year and towards 2023.

Trend 1: Moving from an on-premises to a cloud-based world

“The cloud will be the top trend underlying everything else. We are now seeing a shift in the market. In 2021 we were almost at the tipping point. That is, 50% of the revenue in the database management systems market is attributed to the cloud,” noted Ronthal. “This year we are very easily going over that 50%. That means we are moving from an on-premises world to the cloud. Hopefully in the meantime we will transform our systems and get ready for modernization.”

Trend 2: Cloud deployments are becoming more cohesive and holistic

“We’re starting to see cloud deployments as cohesive and increasingly holistic approaches to data ecosystems,” Ronthal said.

To illustrate his point, he shared an example:

“We first started working with the data ecosystem in 2019,” he explains. “At the time, Microsoft announced the next phase of analysis: synastry. In principle, [a] synapse is an analysis ecosystem. It tries to unite and merge different components of the analytic stack. This is done for exploratory, data lake-type components as well as for data warehousing. It provides common management and security tools to help you take advantage of these things and interact with a wide variety of components.

Since then, Ronthal said Microsoft has built things like natural links that make it easy to ingest data from operational sources.

“Then we have power business intelligence (BI). Other ecosystems are also emerging,” he added.

The entire ecosystem should enable an organization to understand how data is used and how it fits together. It should enable the organization to combine metadata, observability, governance, data integration and augmentation.

“So we have a very rich and diverse ecosystem that can be bought from one supplier. Customers expect it to just work. They don’t expect to spend a lot of time fiddling with the configuration,” Ronthal said.

He stressed that the ecosystem should not be closed.

“It should be open to third-party competitors,” Ronthal said. “If I decide I’m all-in with Amazon Web Services (AWS), and I really like Snowflake, I can do that. If I’m in Azure and I decide I’d rather use cleaver or delete instead of purview, I can do that. Or I could use Informatica instead of Azure Data Factory.”

“Basically, we have the ability to exchange third-party components. They operate within the confines of a broader writer-centric cloud services ecosystem and create a spider world for cloud services,” he added.

Trend 3: The rise of finops

He went on to explain that there is an increasing emergence of financial governance in a practice called “finops,” short for financial operations. This is a continuous and iterative approach to budget management, trying to gain predictability from budgets in the cloud, he explained.

“The costs of individual workloads are now more transparent than ever before,” says Ronthal. “It’s now possible for us to really look at a collection of work or set of workloads and say, ‘Hey, this cost me X dollars to run. Did I get any business value out of that?’”

“So we’re much more dynamic in the way we approach budgeting capabilities,” explains Ronthal. “Compare this to the on-premises world. Here we have a capital budget, we would invest at the beginning of the year, and that was about it.”

In the cloud, organizations have much more freedom to reallocate funds on the fly, Ronthal emphasized.

“We can do things that we may not have done last month, add things to our mix, take things away or change performance characteristics. It’s not so much about which service I have to run with which cloud supplier. It’s about whether I can get the work at the most optimal price,” says Ronthal.

Trend 4: A global approach to data fabric

The fourth trend that Ronthal noticed is an increased focus on data fabric.

Think of the data fabric as built on the building blocks that Ronthal mentioned earlier: metadata, integration, governance, observability and augmentation. Datafabric looks at the design point and purpose of a data environment. It also looks at the actual usage patterns, how the data was created, how the environment is actually used, and how the data is consumed.

“Then we look at alignment and assumptions,” explains Ronthal.

The data substance should also emerge from the data ecosystem, Ronthal said. It should enable an organization to build a number of business-oriented capabilities. That, in turn, allows the organization to connect all of these things together and build a holistic view.

“Ultimately, it will enable new business practices such as finops, dataops and devops, as well as collaborative behaviors and marketplaces,” he said.

Trend 5: Managing and mastering a connected world

“Today we look at connecting everything,” Ronthal said. “As a result, we create mountains of streaming data coming at us in real time. We may want to take action on it, such as running analytics or building predictive models for machine learning.”

“Then we want to push those models to the edge so we can act on that streaming data in real time,” Ronthal said. “There are components here that help us do that. There is an emerging class of databases, which we call the distributed database.”

There are now several suppliers that work in this environment.

“What they can do is implement databases that are spread across different geographic boundaries and all work as a cohesive whole,” he said. “They support the connected-everything approach, regardless of where the data is generated or consumed. We are looking at whether we can divide this over several environments and link everything together.”

How well an organization does at integrating new and emerging technologies into their systems and processes depends on the maturity level of that organization, Ronthal explains.

“Some organizations are good at it. They usually have entire teams looking at and evaluating emerging technology. Those organizations are probably well on their way with their cloud migration. They may not be quite there yet, but they are probably quite far on the right track,” he said.

“Other organizations are still trying to understand it all. Maybe they’re still trying to figure out how to build their first data warehouse, or they’re trying to figure out what a data lake is. Much of this remains tactical rather than strategic. This is especially true for less mature organisations,” notes Ronthal.

Another important factor, he said, is how well an organization is doing at retraining its workforce to handle new technological tools such as automation, artificial intelligence and machine learning technologies.

An important position that will arise is that of a cloud economist, Ronthal explains. This is someone who understands cloud deployment models under dense cloud pricing models. They also need to be able to work with multiple organizations to ensure cloud usage is adequate for the organization’s business model.

“There will also be a need for strong collaboration between the CEO, the analytics officer, the CIO, the CFO and the line-of-business directors. That will be absolutely critical to success in this new cloud world,” said Ronthal.