Amazon is buying billions in Nvidia AI systems it can't yet turn on

Amazon is racing to safe AI hardware before than before, equal as data center power shortages depart several GPU systems waiting in warehouses.

Under a new Early Rack Receipt process, AWS expected to obtain approximately $5 milliard in Nvidia's GB300 server racks at US accommodation before final deployment, according to August inner emails reviewed by Business Insider. The business projected approximately $3 milliard additional in GB300 racks all duration "for the foreseeable future," among the emails said.

Some AWS sites had area for additional racks but not adequate accessible power to rotate them on, group acquainted alongside the attempt said, leaving completed equipment to be staged at intermediary warehouses until the accommodation were ready.

At smallest one AWS squad did not have "a apparent forecasted path" for getting the incoming racks into data centers at the time, according to among the emails.

Dark GPUs

A deficiency of new power capability is one of multiple bottlenecks slowing the buildout of several AI data centers. This has led to a event known as dark GPUs, anywhere AI chips and servers are purchased but can sit idle because of power provision issues or another challenges. (Dark GPUs can additionally mention to scheme scheme flaws that average this equipment isn't completely used following being plugged in).

This is partially why financial force is mounting on big data center projects. Money must be spent upfront to safe supplies of Nvidia GPUs and another precious AI components. But if several data centers aren't prepared to run, those first investments can tie up huge amounts of cash. Debt financing can inhabit in the gaps, but the market has rotate into small consenting to contribute against several of these projects lately.

Trade-off

Amazon's early-rack program is one part of the haze leader's broader AI buildout, which is putting new force on its prosperity flow, as Business Insider exclusively reported this week. Amazon expects approximately $220 milliard in chief expenditures this twelvemonth — additional than any Big Tech peer. Its free-cash-flow turned negative final fourth for the archetypal period since 2023, and could remain negative for a while.

For AWS, the trade-off is securing scarce GPU racks before competitors do, equal if several data centers are not yet prepared to rotate them on. But taking ownership before can tie up billions in equipment before it starts generating haze revenue.

Previously, AWS took ownership of a completed rack lone upon shipment to a data center, according to inner documents. Under the new approach, it can pay for and document certain racks before in the provision chain.

An Amazon spokesman stated the business routinely stages completed hardware near deployment sites so it can instal it formerly capability becomes available, calling the custom "best endeavor practice." The spokesman added that Amazon has long-established systems for tracking infrastructure resources and disputed that the broader business lacked a way for getting racks into data centers.

Amazon CEO Andy Jassy earlier stated the business has a "clear row of sight" to powerful returns from its AI investments, calling the innovation a "once in a life opportunity."

"We've been through this sequence alongside the archetypal big AWS growth wave, and liked the results," Jassy wrote in Amazon's annual shareholder letter. "We anticipate to awareness likewise concerning this next wave."

Have a tip? Contact this newsman via email at [email protected] or Signal, Telegram, or WhatsApp at 650-942-3061. Use a individual email address, a nonwork WiFi network, and a nonwork device; here's our guide to sharing data securely.