NVIDIA AI Instances Costing $2M in Power, Annually: Liftr Insights data

(US and Canada) Liftr Insights, a pioneer in market intelligence driven by unique data, demonstrated the on-going costs for running popular AI workloads from NVIDIA.

NVIDIA has been dominating news and the markets for its semiconductors. Their latest semiconductors, such as the Hopper H100, are considered essential to the rising demand for AI training and other artificial intelligence processes. Liftr data show that even their earlier types like the A100 remain in high demand.

For example, for a $33M investment in AI accelerator components and $2 M in electric consumption per year, a company could be running 1,000 H100s and 1,000 A100s in Dallas, Texas, which combined could be generating high performance in excess of 44.7 FP64 petaflops.

In comparison, Liftr Insights shows that running the same infrastructure in Houston would generate a larger power bill at approximately $2.1M per year. Annual costs would be less in San Antonio or Austin, at $1.9M and $1.6M, respectively.

"Despite the news of the delay in the Blackwell processes," says Tab Schadt, CEO of Liftr Insights, "major cloud providers like AWS, Azure, and GCP have been increasing their adoption of the latest NVIDIA semiconductors."

Liftr Insights, which tracks the top 6 cloud providers and 3 trending providers, shows the adoption growth of the Hopper brand. The H100 is the most widely discussed model within the Hopper series.

In addition to showing common configurations as well as adoption trends by the major consumers, Liftr data can provide deeper insight for data center operators.

"We help our customers understand the impact of these new chips on output and their financials," says Schadt. "When looking at new AI, it's more than knowing what's available. Rather, for specific configurations, it's understanding the performance, power consumption, and ultimately, the bottom line of on-going costs."