GPT-5.6 and Fable Team Up to Crack a 25-Year-Old Math Problem

A math problem that stood open for 25 years has been cracked with the help of two frontier AI models. Dimitris Papailiopoulos, a principal researcher at Microsoft Research, proved last week that a simple two-step algorithm can make MIMO detection exactly hit the maximum-likelihood threshold — in polynomial time, requiring only O(N³) operations. The whole proof took him seven days.

MIMO detection is a classic problem in wireless communications: a transmitter sends N bits through an N×N channel that scrambles the bits and adds noise, and the receiver must recover the original bits from the corrupted signal alone. The theoretically foolproof maximum-likelihood detector requires checking all 2^N combinations, which explodes exponentially as N grows.

The problem's history is full of reversals. In 1989, Sergio Verdú showed the worst case is NP-hard; in 2001, Hassibi and Vikalo believed the sphere decoder offered a polynomial-time answer, only to be disproved by Jaldén and Ottersten in 2005. Subsequent attempts — semidefinite relaxation, bit-flipping local search, AMP, statistical physics methods — got no closer than twice the theoretical threshold.

Last week, the gap was closed. Working with GPT-5.6 and Fable 5, Papailiopoulos proved that two simple steps — LMMSE rounding followed by greedy bit flipping — recover every bit exactly at signal-to-noise ratio 2logN in O(N³) time. The proof is bidirectional: it also shows that maximum-likelihood detection itself fails when the SNR dips just below the threshold.

The human-AI collaboration was anything but smooth. The two models proposed different routes — GPT-5.6 leaned on AMP, while Fable 5 suggested signed LMMSE plus greedy bit flipping, a workhorse algorithm used in industry but never rigorously proven. Papailiopoulos chose Fable's path, had GPT check and patch the holes, then spent days having the two models simplify each other's arguments, with one hard constraint: keep the 2logN threshold. He even declined to use Lean for formal verification, for a blunt reason — he doesn't know Lean.

The final proof can be checked line by line by hand, and the algorithm has just two core steps. Step one, LMMSE rounding, rounds a continuous estimate by sign so that the fraction of mis-guessed bits tends to zero as N grows. Step two, greedy bit flipping, flips the bit that most reduces a cost function each round; the proof shows the search can neither stall nor wander outside its guardrail, and can only stop at the true transmitted bit string.

For Papailiopoulos personally, it closes a 17-year loop. In 2009, as a first-year PhD student, he tried to solve MIMO detection with MCMC methods and failed. Now a principal researcher at Microsoft Research and an associate professor at the University of Wisconsin-Madison, he has taken the same problem down with AI by his side. When a workhorse algorithm that industry actually uses finally gets its rigorous proof, the value of AI-assisted mathematics extends from solving problems to completing theoretical proofs.

What to watch next: whether the proof survives peer review and generalizes to average-case analysis of other NP-hard problems; whether the division of labor seen here — Fable charting the route, GPT patching the holes — becomes a common pattern in human-AI mathematical research; and whether the missing Lean formal verification is supplied in follow-up work.