Microsoft to Deploy In-House Maia 300 AI Chip Next Year, Targeting 40% More Tokens per Megawatt

Microsoft is shifting the workhorse of its AI inference onto its own silicon. According to TradingKey, the company says it plans to deploy its next-generation in-house chip, Maia 300, in data centers starting next year and to expand the Maia series to a multi-gigawatt scale, with the aim of powering its AI software primarily with custom hardware and reducing reliance on Nvidia GPUs.

The Maia chips are purpose-built for AI inference — using existing models for tasks such as summarizing meetings and answering research questions. Microsoft expects Maia 300 to generate 40% more tokens per megawatt than mainstream GPU systems. A token is the basic unit of AI output, usually corresponding to a word or part of a word, so the metric measures output per unit of power.

Rani Borkar, who heads Microsoft's data center hardware and chip design team, said, “We are confident that we have built the best product for these workloads.” She added that Microsoft will back that conviction with action by deploying Maia on a gigawatt scale starting next year.

Today, most of Microsoft's AI inference is still handled by Nvidia GPUs. By customizing chips for its own data centers, the company hopes to improve inference efficiency and trim part of its spending on Nvidia hardware.

The previous generation is already in production. Microsoft began deploying Maia 200 earlier this year, and Borkar said it is already powering AI software such as Copilot, running models from OpenAI, Microsoft's AI team and open-source model families. Maia 200 has been deployed in data centers in the United States, Europe and Australia, providing AI capabilities for products including Microsoft Teams and Copilot.

Custom AI accelerators have become an important path for major cloud providers to reduce dependence on external hardware. Amazon and Google are advancing their own strategies and, having started earlier than Microsoft, are regarded as leaders in the custom-chip space. Microsoft's expanded deployment is meant to accelerate the shift of its internal AI services onto its own hardware.

Microsoft, however, currently does not plan to rent the underlying compute of Maia chips directly to cloud customers. Borkar noted that this differs from Google and Amazon, which already offer capacity from their in-house AI chips to customers. That makes Maia, at least for now, more of an internal efficiency play than a new cloud product line.

Two things are worth watching: whether Maia 300 actually lands next year as promised and delivers on the 40% efficiency claim, and whether Microsoft changes course and sells its custom compute externally the way its rivals do. Until then, Nvidia will still carry a substantial share of Microsoft's inference workload.