Presented by

AI offers great promise, but where do you host it?
The cloud provides the scalability and performance required for AI’s data- and compute-intensive workloads, but the best model may be hybrid.
By Constantine von Hoffman
Nov 4, 2021
Artificial intelligence (AI) has become essential to the economy, and business leaders know it. PwC predicts AI will make a $15.7 trillion contribution to the global economy by 2030. In a recent IDG study, 51% of organizations say they are piloting or have deployed AI or machine learning (ML) technologies, and another 33% are actively researching the technology. This explains why IDC expects global spending on cognitive and artificial intelligence systems to top $57 billion this year.

Given how critical AI is, one of the big questions for IT leaders is what type of infrastructure do you need to support the nearly insatiable data- and compute-intensive demands of AI and ML? As importantly, do you host AI/ML infrastructure in the cloud, which offers more flexibility and scalability, or in your data center, where you have greater control?

We asked the IDG Influencer Network, a community of IT professionals, industry analysts, and other experts, about the best way to build an AI infrastructure. While there’s no perfect answer, it’s clear from their responses that both the cloud and the data center can play critical roles.

The one thing many of our experts agreed on is that the right answer is the one that best fits your organizational needs:

  • Joanna Young, executive digital advisor (@jcycio): “There is no ‘one size fits most/all’ here. There is a spectrum of answers to this as situational based on products, services, business model, and current state of tech.”
  • Ben Rothke, senior information security manager for Tapad (@benrothke): “When it comes to cloud or data centers, neither of them is intrinsically better or worse. Their benefit is if it supports the needs and goals of the organization.”
  • Arsalan Khan, a tech and transformation advisor (@ArsalanAKhan): “When it comes to AI, it depends on what your needs are and how fast you want access to data, how scalable it is, how secure it is, and how you plan to resolve issues.”
  • Howie Xu, vice president of machine learning/artificial intelligence at Zscaler (@H0wie_Xu): “Where the AI infrastructure and the data are hosted should totally depend on the nature, privacy and legal constraints, volume, use case, and obviously cost implications, too.”

Benefits of AI in the cloud

That said, respondents had some very strong arguments in favor of putting AI data and infrastructure in the cloud.

“AI infrastructure benefits most from cloud hosting compared to the data center,” said Scott Schober, president and CEO of Berkeley Varitronics Systems (@ScottBVS). “This is because AI relies on on-demand services that instantly cater to individual users or their devices. Data centers allow storage and collaboration of on-premises data but cannot scale well.”

Benjamin Ajibade, data analytics lead at SHIFT Nigeria (@Benni_aji), expanded on this point: “The cloud can provide developers with a high level of versatility while also ensuring an easy workflow. Although on-premises [systems] can be used for development and testing, it is recommended that a live application be deployed in the cloud for scalability and availability.”

Furthermore, the cloud promises significant cost savings. Using a service provider allows you to avoid buying and maintaining significant amounts of hardware and software. This, in turn, frees up IT teams to focus on strategic operations.

Moving infrastructure to the cloud “eliminates the costs of maintaining data centers on-site, such as hardware and maintenance,” said Audrey Desisto, founder of Digital Marketing Stream (@AudreyDesisto). “These upfront costs can prove prohibitive for AI projects, so using a cloud service provider can make research and development costs easier to manage.”

Will Kelly, technical marketing manager for a container security startup (@willkelly), seconded this point. “Like every data-intensive use case we have, data storage can be costly,” he said. “There are a lot of questions here that an organization needs to answer, especially when it comes to cloud costs. FinOps expertise is a must-have on this cross-functional team.”

However, while saving money is always important, it isn’t the primary factor when it comes to using the cloud. “Innovation and advanced capabilities are the big cloud drivers, not cost savings or even operational efficiency,” said Matt Turner, director of industry strategy at Alation (@matt_turner_nyc). “So, data and AI modeling and processing will move to take advantage of the accelerating innovation.”

Others agree that for organizations looking to take advantage of advanced AI capabilities, the cloud, along with edge computing, play a vital role.

“Keeping data at the edge is critical when companies have the right AI visualization tools and sensors for monitoring and predictive management solutions in place,” said Peggy Smedley, editorial director/president, Connected World (@ConnectedWMag). “AI is changing the game, and how companies respond to the edge and the cloud will also change accordingly.”

The appeal of a hybrid approach

In the end, the decision may come down to where AI and the cloud fit into your organization’s strategic IT plan.

“A cloud strategy requires talent and resources that may go deeper than the data center, especially in the areas of security, interoperability, analytics, and processing power,” said Frank Cutitta, CEO and founder of HealthTech Decisions Lab (@fcutitta). “It’s not that data centers don’t have the same challenges, it’s more a matter of how much IT leadership has a desire to offload these functions to a partner as opposed to investing in the resource to run it safely in a traditional data center.”

All this makes a compelling case for the cloud, but Scott Zoldi, chief analytics officer at FICO (@ScottZoldi), reminds us there is a lot to be said for controlling all your computing resources. “AI infrastructure needs to reflect operationalization requirements,” he said. “Far too often I hear from organizations that build in the cloud with open source but can’t operationalize at scale. Hosted infrastructure has a number of advantages when it comes to real-time decisioning, and this is often machine learning or AI that is purpose-built vs. commodity.”

The good news is it doesn’t have to be an either/or decision. Hybrid infrastructure, where resources are split between on-premises and cloud environments, can bring you the best of both.

“Why not use a combination of cloud and on-prem?” said Gene De Libero, chief strategy officer at (@GeneDeLibero). “Organizations can always leverage pre-made or custom AI cloud solutions when existing data center infrastructure is unsuitable for specific AI initiatives. However, creating and scaling cloud-like models on-prem is becoming easier through creative payment options and advances in technology.”

Which brings us back to the importance of picking a solution only after you’ve decided what works best for your organization.

“Where you put the infrastructure must factor in the use case needs, data proximity, performance, and security,” said Isaac Sacolick, president of StarCIO (@nyike).  “Many organizations are considering hybrid and multi-cloud architectures to provide the greatest flexibility.”



All Episodes (8)

Cybersecurity: Trust no one

Cybersecurity: Trust no one
Andy Ellis, Advisory CISO, Orca Security
Tim Brown, CISO, SolarWinds

Ransomware, a love story

Ransomware, a love story
Rahul Telang, Trustees Professor of Information Systems
Renee Guttmann, Global CISO and Risk Executive, Board Advisor

AI: Reality check

AI: Reality check
Olaf Groth, PhD, CEO, Prof. HULT IBS & US Berkeley, Author The AI Generation
Sanjay Srivastava, Chief Digital Officer, Genpact

AI: Automation nation

AI: Automation nation
Anima Anandkumar, Director of Machine Learning Research, NVIDIA
Brian Solis, Global Innovation Evangelist, Salesforce and Author

Flexible workforce: Hybrid vigor

Flexible workforce: Hybrid vigor
Kamila Sip, Neuroscience and Behavior Change Expert
Jon Levy, Behavioral Scientist, NYT Best-Selling Author

IT-as-a-service: A renter’s market

IT-as-a-service: A renter’s market
Tien Tzuo, Founder and CEO, Zuora
Matt Kimball, Principal Analyst, Moor Insights & Strategy

Edge computing: Distribute or die

Edge computing: Distribute or die
Stephanie Atkinson, Executive Thought Leader and Influencer
Satya Jayadev, VP and CIO, Skyworks

Green IT: The color of money

Green IT: The color of money
Rob Enderle, President and Principal Analyst, Enderle Group
Pamela Rucker, President, The Rucker Group, Instructor, Harvard Professional Development Programs