Realtime AI News
Terminal X's Eval Agent Matches Senior Investor Judgment 88.6% of the Time With GPT-6.1
Terminal X has built an Eval Agent on GPT-6.1 Sol that grades investment outputs against a 54-criterion rubric, matching senior investor judgment 88.6% of the time, according to OpenAI. The company says the agent also catches 25% more deviations from a fund's mandate.
Terminal X has published a case study with OpenAI describing an Eval Agent built on GPT-6.1 Sol that scores investment work against a 54-criterion rubric. The headline number: its judgments agree with senior investor judgment 88.6% of the time.
The second number matters as much as the first. Terminal X says the agent catches 25% more deviations from a fund's mandate, which reframes the product from a drafting assistant into a control layer rather than a faster way to produce documents.
Rubrics are the mechanism. Instead of asking a model whether an analysis "looks right", the Eval Agent checks a stated list of 54 criteria and returns a graded verdict on each one. That structure is what makes an automated judgment auditable: a disagreement with a human reviewer points to a specific criterion rather than a vague objection.
For funds, mandate compliance is exactly the kind of task where generic model quality is not enough. A memo can be fluent and still recommend a position the fund is not allowed to hold. Catching that is a rule-shaped problem, and rules are what a rubric encodes.
The 88.6% figure should be read carefully. Agreement with senior investors is a measure of imitation, not of correctness; if the senior reviewers share a blind spot, the agent will reproduce it efficiently. The vendor's own framing is about matching expert judgment, not about outperforming it.
The economics are the quiet argument. Senior investment review is expensive, scarce, and slow, and it does not scale with deal flow. An agent that reproduces most of that judgment at machine speed changes how much work a small team can supervise, even if a human still signs off.
What to watch is whether the rubric-based approach generalizes beyond one fund's mandate. Rubrics are portable in principle, but the criteria that define a strategy are often tacit, and the hard part of deploying this pattern elsewhere will be writing the 54 lines, not running the model.
Why it matters
Rubric-driven evaluation turns model review into an auditable checklist, a template for high-stakes settings where a wrong answer is a compliance breach. The 88.6% figure signals imitation of expert judgment, not superior judgment.
Nearby Updates
All10/20, 15:03
Anthropic pitches an AI safety bill that kicks in only at $1B in damage or 50 deaths
Anthropic is pitching lawmakers a bill that would let the government block an AI model only after it causes more than $1 billion in damage or 50 deaths, according to a report by Pasquale Pillitteri. The proposal replaces capability-based oversight with an extremely high harm threshold, turning the frontier-AI debate into an argument over two concrete numbers.
10/20, 15:37
BYD launches Didixia AI agent, claiming an early move into the agent era
BYD has launched an AI agent product called Didixia, framing the release as the company entering the AI agent era. The public information covers only the launch itself, with product form, capability boundaries and target vehicles still undisclosed.
10/20, 16:03
Beyond Now launches Wave AI Framework for agent collaboration across the CSP revenue lifecycle
Beyond Now has released Wave AI Framework, a product aimed at getting multiple AI agents to collaborate across the revenue lifecycle of communications service providers. StreetInsider reports the launch, which pushes agentic automation from standalone tools into the back-office systems that determine how telecom revenue is booked.
10/20, 16:52
OpenAI says GPT-6.1 Sol hits 97.9% on its COBOL migration test
Dataconomy reported on October 20 that OpenAI's GPT-6.1 Sol scores 97.9% on the company's COBOL migration evaluation, up from 86.6% at low reasoning effort and 5.4% for the previous generation. The numbers come from a new OpenAI benchmark that requires byte-for-byte parity on hidden tests.