AI offers great promise, but where do you host it?
The cloud provides the scalability and performance required for AI’s data- and compute-intensive workloads, but the best model may be hybrid.
  • Constantine von Hoffman
  • 11/04/2021
AI offers great promise, but where do you host it?
Artificial intelligence (AI) has become essential to the economy, and business leaders know it. PwC predicts AI will make a $15.7 trillion contribution to the global economy by 2030. In a recent IDG study, 51% of organizations say they are piloting or have deployed AI or machine learning (ML) technologies, and another 33% are actively researching the technology. This explains why IDC expects global spending on cognitive and artificial intelligence systems to top $57 billion this year.

Given how critical AI is, one of the big questions for IT leaders is what type of infrastructure do you need to support the nearly insatiable data- and compute-intensive demands of AI and ML? As importantly, do you host AI/ML infrastructure in the cloud, which offers more flexibility and scalability, or in your data center, where you have greater control?

We asked the IDG Influencer Network, a community of IT professionals, industry analysts, and other experts, about the best way to build an AI infrastructure. While there’s no perfect answer, it’s clear from their responses that both the cloud and the data center can play critical roles.

The one thing many of our experts agreed on is that the right answer is the one that best fits your organizational needs:

  • Joanna Young, executive digital advisor (@jcycio): “There is no ‘one size fits most/all’ here. There is a spectrum of answers to this as situational based on products, services, business model, and current state of tech.”
  • Ben Rothke, senior information security manager for Tapad (@benrothke): “When it comes to cloud or data centers, neither of them is intrinsically better or worse. Their benefit is if it supports the needs and goals of the organization.”
  • Arsalan Khan, a tech and transformation advisor (@ArsalanAKhan): “When it comes to AI, it depends on what your needs are and how fast you want access to data, how scalable it is, how secure it is, and how you plan to resolve issues.”
  • Howie Xu, vice president of machine learning/artificial intelligence at Zscaler (@H0wie_Xu): “Where the AI infrastructure and the data are hosted should totally depend on the nature, privacy and legal constraints, volume, use case, and obviously cost implications, too.”

Benefits of AI in the cloud

That said, respondents had some very strong arguments in favor of putting AI data and infrastructure in the cloud.

“AI infrastructure benefits most from cloud hosting compared to the data center,” said Scott Schober, president and CEO of Berkeley Varitronics Systems (@ScottBVS). “This is because AI relies on on-demand services that instantly cater to individual users or their devices. Data centers allow storage and collaboration of on-premises data but cannot scale well.”

Benjamin Ajibade, data analytics lead at SHIFT Nigeria (@Benni_aji), expanded on this point: “The cloud can provide developers with a high level of versatility while also ensuring an easy workflow. Although on-premises [systems] can be used for development and testing, it is recommended that a live application be deployed in the cloud for scalability and availability.”

Furthermore, the cloud promises significant cost savings. Using a service provider allows you to avoid buying and maintaining significant amounts of hardware and software. This, in turn, frees up IT teams to focus on strategic operations.

Moving infrastructure to the cloud “eliminates the costs of maintaining data centers on-site, such as hardware and maintenance,” said Audrey Desisto, founder of Digital Marketing Stream (@AudreyDesisto). “These upfront costs can prove prohibitive for AI projects, so using a cloud service provider can make research and development costs easier to manage.”

Will Kelly, technical marketing manager for a container security startup (@willkelly), seconded this point. “Like every data-intensive use case we have, data storage can be costly,” he said. “There are a lot of questions here that an organization needs to answer, especially when it comes to cloud costs. FinOps expertise is a must-have on this cross-functional team.”

However, while saving money is always important, it isn’t the primary factor when it comes to using the cloud. “Innovation and advanced capabilities are the big cloud drivers, not cost savings or even operational efficiency,” said Matt Turner, director of industry strategy at Alation (@matt_turner_nyc). “So, data and AI modeling and processing will move to take advantage of the accelerating innovation.”

Others agree that for organizations looking to take advantage of advanced AI capabilities, the cloud, along with edge computing, play a vital role.

“Keeping data at the edge is critical when companies have the right AI visualization tools and sensors for monitoring and predictive management solutions in place,” said Peggy Smedley, editorial director/president, Connected World (@ConnectedWMag). “AI is changing the game, and how companies respond to the edge and the cloud will also change accordingly.”

The appeal of a hybrid approach

In the end, the decision may come down to where AI and the cloud fit into your organization’s strategic IT plan.

“A cloud strategy requires talent and resources that may go deeper than the data center, especially in the areas of security, interoperability, analytics, and processing power,” said Frank Cutitta, CEO and founder of HealthTech Decisions Lab (@fcutitta). “It’s not that data centers don’t have the same challenges, it’s more a matter of how much IT leadership has a desire to offload these functions to a partner as opposed to investing in the resource to run it safely in a traditional data center.”

All this makes a compelling case for the cloud, but Scott Zoldi, chief analytics officer at FICO (@ScottZoldi), reminds us there is a lot to be said for controlling all your computing resources. “AI infrastructure needs to reflect operationalization requirements,” he said. “Far too often I hear from organizations that build in the cloud with open source but can’t operationalize at scale. Hosted infrastructure has a number of advantages when it comes to real-time decisioning, and this is often machine learning or AI that is purpose-built vs. commodity.”

The good news is it doesn’t have to be an either/or decision. Hybrid infrastructure, where resources are split between on-premises and cloud environments, can bring you the best of both.

“Why not use a combination of cloud and on-prem?” said Gene De Libero, chief strategy officer at (@GeneDeLibero). “Organizations can always leverage pre-made or custom AI cloud solutions when existing data center infrastructure is unsuitable for specific AI initiatives. However, creating and scaling cloud-like models on-prem is becoming easier through creative payment options and advances in technology.”

Which brings us back to the importance of picking a solution only after you’ve decided what works best for your organization.

“Where you put the infrastructure must factor in the use case needs, data proximity, performance, and security,” said Isaac Sacolick, president of StarCIO (@nyike).  “Many organizations are considering hybrid and multi-cloud architectures to provide the greatest flexibility.”



Watch the latest episodes of LNIT

Season 2 (8 episodes)
Season 1 (8 episodes)