Microsoft and Hugging Face open-source ThinkingBox-Training, turning database checks into RL rewards

On October 22, Microsoft and Hugging Face published a joint blog post announcing that ThinkingBox-Training is now available on GitHub. The release is not a new foundation model but an open-source training stack for enterprise agents whose work ends in real database records, built around a simple question: did the task actually get done?

ThinkingBox began as an evaluation harness. Its benchmark, ThinkingBox-Bench, contains 507 tasks. Each one starts a real backend, gives the agent a set of MCP tools, and pairs it with a simulated LLM user that holds private context. When the agent finishes, executable checks inspect the terminal database state rather than grading the tool calls it made along the way.

The team's earlier post described a failure that headline accuracy tends to hide: a model closed a customer ticket as resolved while the underlying carrier exception was still open. Nine plausible-looking tool calls left one wrong database row. That gap between the agent saying it was done and the records being correct is what ThinkingBox was designed to catch.

The shift came when the researchers noticed that most checks can return 1 or 0 directly from the final database state. That makes the same signal usable not only as a grader but as a reward function. They built a training pool from ThinkingBox-format tasks outside the benchmark and used their executable outcomes as rewards.

Choosing tasks turned out to be the hard part. Most tasks are either already solved by the model or hopeless for it, and neither teaches the model anything. The team introduced what it calls offline policy-relative difficulty: before training, it estimates each task's success probability with the base policy, selects a pool from that distribution, and keeps the pool fixed for the entire run.

The experiment fine-tuned Qwen3.8-27B with GRPO for 50 steps. Average success, or pass@1, rose from 51.70% to 60.78%, a gain of 9.08 percentage points. Tasks completed correctly in all 20 attempts rose from 38 (7.50%) to 107 (21.10%), an increase of 69 tasks. The trade-off was a small decline in pass@20 coverage, from 453 tasks (89.35%) to 446 (87.97%).

A hotel-booking task shows the change in behavior. A traveler asks to downgrade a room, but company policy requires manager approval first. Before training, the agent changed the booking and issued refunds without approval in 15 of 20 attempts. After training, it preserved the booking and payment state in all 20, with every expected record matching in 19.

The team argues that the durable contribution is not a single Qwen3.8-27B checkpoint but the loop that converts executable outcomes into rewards, plus the data-curation method around it. Its advice for teams training agents against real records: probe the base policy's success distribution before training, verify that training and evaluation parsers extract identical tool names and arguments, and report both coverage and repeatability.

ThinkingBox code is MIT-licensed and the benchmark data uses CDLA-Permissive-2.0. The project comes from Microsoft's Copilot Studio team in partnership with Toloka, with interns from several universities. Every public benchmark task is a synthetic reconstruction modeled on real enterprise agent patterns; the customers are not real.