Don’t let your AI projects in the public cloud drain you

Last year, Andreessen Horowitz published a provocative blog post titled “The cost of the cloud, a trillion dollar paradox.” In it, the venture capital firm argued that out-of-control cloud spending is leading public companies to put billions of dollars in potential market capitalization on the table. An alternative, the company suggests, is to recalibrate cloud resources in a hybrid model. Such a model can boost operating profit and free up capital to focus on new products and growth.

Whether enterprises follow these guidelines remains to be seen, but one thing is certain: CIOs are demanding more flexibility and performance from their supporting infrastructure. This is mainly because they want to use advanced and computationally intensive artificial intelligence/machine learning (AI/ML) applications to improve their ability to make real-time, data-driven decisions.

To that end, the public cloud has been fundamental to making AI mainstream. But the factors that made the public cloud an ideal testing ground for AI (that is, elastic pricing, ease of bending up or down, among other factors) actually prevent AI from realizing its full potential.

Here are some considerations for organizations looking to maximize the benefits of AI in their environment.

For AI, the cloud is not a one-size-fits-all

Data is the lifeblood of the modern enterprise, the fuel that generates AI insights. And with many AI workloads constantly having to ingest large and growing amounts of data, it is imperative that the infrastructure can support these requirements in a cost-effective and robust manner.

When deciding how best to address AI at scale, IT leaders need to consider several factors. The first is whether colocation, public cloud, or a hybrid mix is ​​best suited to meet the unique needs of modern AI applications.

While the public cloud has been invaluable in bringing AI to market, it also faces challenges. Among which:

• Vendor lock in: Most cloud-based services are at risk of lock-in. However, some cloud-based AI services available today are highly platform-specific, each with its own distinct nuances and various partner-related integrations. As a result, many organizations tend to consolidate their AI workloads with a single vendor. That makes it difficult for them to switch suppliers in the future without incurring significant costs.
• elastic pricing: The ability to pay only for what you use is what makes the public cloud such an attractive option for businesses, especially those looking to lower their CapEx spend. And consuming a public cloud service through the drip often makes economic sense in the short term. But organizations with limited visibility into their cloud usage all too often find themselves consuming it by the bucket. At that point it becomes a tax that stifles innovation in the bud.
• Exit costs: With cloud data transfer, a customer does not have to pay for the data they send to the cloud. But to get that data out of the cloud, they have to pay outgoing costs, which can quickly add up. For example, disaster recovery systems will often be spread across geographic regions to ensure resilience. That means that in the event of an outage, data must be continuously duplicated between availability zones or to other platforms. As a result, IT leaders are beginning to understand that the more data that is pushed into the public cloud at any given time, the more likely it is to be painted into a financial corner.
• Data Sovereignty: The sensitivity and location of the data is another crucial factor in determining which cloud provider is best suited. Moreover, as a raft of new state-mandated data privacy rules going into effect, it will be important to ensure that all data used for AI in public cloud environments complies with applicable data privacy regulations.

Three questions to ask before moving AI to the cloud

The economies of scale that come with public cloud providers have made it a natural proving ground for today’s most demanding business AI projects. That said, before going all-in on public cloud, IT leaders should consider the following three questions to determine if it is indeed their best option.

When does the public cloud no longer make economic sense?

Public cloud offerings like AWS and Azure allow users to scale their AI workloads quickly and cheaply, as you only pay for what you use. However, these costs are not always predictable, especially since these kinds of data-intensive workloads tend to increase in volume as they voraciously ingest more data from various sources, such as training and refining AI models. While “paying through the drip” is easier, faster, and cheaper on a smaller scale, it doesn’t take long for this drip to pile up in buckets, pushing you into a more expensive price point.

You can limit the cost of these buckets by committing to long-term contracts with volume discounts, but the economics of these multi-year contracts are still rarely good. The emergence of AI Compute-as-a-Service beyond the public cloud provides options for those who want the convenience and cost predictability of an OpEx consumption model with the reliability of dedicated infrastructure.

Should all AI workloads be treated the same?

It is important to remember that AI is not a zero sum game. There is often room for both cloud and dedicated infrastructure or something in between (hybrid). Instead, look at the characteristics of your applications and data first, and invest the time up front to understand the specific technology requirements for the individual workloads in your environment and the desired business outcomes for each. Next, find an architectural model that allows you to match the IT resource delivery model that fits each stage of your AI development journey.

Which cloud model allows you to deploy AI at scale?

In the land of AI model training, new data must be regularly fed into the compute stack to improve the prediction capabilities of the AI ​​applications they support. As such, proximity to computing and data repositories have become increasingly important selection criteria. Of course, not all workloads need dedicated, persistent, high-bandwidth connectivity. But for those who do, excessive network latency can seriously hamper their potential. In addition to performance issues, there are a growing number of data privacy rules governing how and where certain data can be accessed and processed. These regulations should also be part of the cloud model decision-making process.

The public cloud has been key to making AI mainstream. But that doesn’t mean it makes sense for every AI application to run in the public cloud. Investing the time and resources to determine the right cloud model at the beginning of your AI project can go a long way in preventing failed AI projects.

Holland Barry is SVP and field CTO at Cyxtera.