Nvidia puts tokens per megawatt at the center of its AI factory pitch

Ian Buck, Nvidia's vice president of hyperscale and high-performance computing, made AI factory efficiency the centerpiece of his infrastructure keynote at the AI Infra Summit in Santa Clara on Tuesday. The size of the event says something about the moment: more than 8,000 attendees this year, up from 3,500 last year.

Nvidia's central claim is that the metric for AI infrastructure is shifting from peak performance to validated agentic tokens per megawatt. The company says DSX MaxLPS can deliver up to 1.4x more tokens per megawatt through factory-wide power optimization.

The hardest evidence came from GPU cloud provider Lambda, which released the first validation of DSX MaxLPS in a deployment environment on Nvidia Blackwell servers the same day. Running a five-rack, 19-node cluster, Lambda fit 19 nodes inside the power budget typically allocated to 16 full-power nodes, increasing cluster-wide token throughput by 24% from roughly 4 million to 5 million tokens per second, while improving performance per watt by 23%.

Based on Nvidia's projections, DSX MaxLPS can enable up to 40% more GPU capacity for next-generation Vera Rubin NVL72 AI factories within the same megawatt budget in suitable deployment environments. The mechanism is straightforward: the software continuously monitors GPU and rack-level power consumption and shifts available power where workloads need it most, reclaiming capacity that static provisioning leaves stranded, which matters most in factories running both training and inference. The DSX platform was first introduced at GTC Taipei in May.

In a second direction, Nvidia and partner Emerald AI demonstrated AI factories acting as grid resources. Emerald AI's Conductor platform receives signals from Silicon Valley Power's Flexible Load Interconnect Program and sheds load automatically against a predefined workload hierarchy: on an August evening, the factory's power fell from four megawatts to three, with lowest-priority jobs yielding while high-priority inference kept running and no operator required.

Nvidia says that deployment has now responded to more than 200 demand signals from Silicon Valley Power, answering in under a minute. It runs at Nvidia's Eos AI factory, part of the first commercial grid utility program designed to treat AI factories as dispatchable resources. Emerald AI plans to use Nvidia DSX Flex for Conductor, and the first dedicated DSX Flex commercial deployment will be a 96-megawatt Vera Rubin AI factory in Manassas, Virginia, at Nvidia's AI Factory Research Center, building on five prior demonstrations across two continents.

For agentic workloads, Nvidia also presented a combined platform with Groq 3 LPX, adding deterministic ultralow-latency inference to Vera Rubin. For 2-trillion-plus-parameter models at long context, the combination delivers up to 35X higher token throughput per megawatt than GB200 NVL72; on a 100K-context Qwen 3.8 27B workload, Groq 3 LPX hit 2,529 output tokens per second per user.

Nvidia also put agentic performance into SemiAnalysis's AgentX benchmark, which measures inference on recorded real-world agentic coding sessions with actual context growth, tool call delay and sub-agent spawning preserved, rather than on single-request benchmarks. On the DeepSeek V4 Pro model, Vera Rubin NVL72 delivers up to 30x higher throughput per megawatt than Nvidia GB300 NVL72, and the AgentX results show up to 45x lower cost per million tokens.

In power-constrained deployments, throughput per megawatt determines how much revenue an AI factory can generate, and cost per million tokens determines the margin on it. Nvidia is extending DSX into power delivery as well, folding an 800 VDC architecture into its reference designs to reduce conversion complexity. What to watch next is whether the 96-megawatt Manassas factory lands on schedule and whether grid flexibility turns from a pilot into a routine requirement for AI factory siting.