Guozhen AIGlobal AI field notes and model intelligence

Realtime AI News

Google's own Android coding benchmark puts Gemini 4 Argon fifth

Google refreshed its Android Bench leaderboard on October 26 and ranked its own Gemini 4 Argon for the first time, placing it fifth with a pass rate well behind Claude and the GPT-6 models. 9to5Google reports that the flagship trails other frontier models by a wide margin on Google's own test.

Published
Google 自家榜单给 Gemini 4 Argon 打分:Android 编码排名第五
Image source: gemini.google

Google updated its Android Bench leaderboard on October 26 and, for the first time, ranked its own Gemini 4 Argon against rival models. 9to5Google reports that the refresh also added results for OpenAI's GPT-6 Sol and GPT-6.1 Sol as well as Claude Sonnet 5 and Claude Sonnet 5.5, and now labels the reasoning level each model was run at.

Android Bench launched early this year as a way for Android developers to see which models actually help with Android coding. The test is built around Android-specific development practices, and it now factors average cost into the ranking alongside pass and completion rates, turning a capability chart into a capability-per-dollar comparison.

Claude Sonnet 5.5 tops the updated list with a 40% pass rate and an 80.9% completion rate, with Claude Opus 5.5, added in early October, in second place. OpenAI's GPT-6.1 Sol may be the more interesting entry: a 29.3% pass rate and 82.8% completion rate at an average cost of $99.50, against $1,574.90 for the leader from Anthropic.

Gemini 4 Argon's debut is the weak spot. Google's model lands fifth, behind Claude Sonnet 5.5, Claude Opus 5.5, GPT-6.1 Sol and Claude Fable 5.1, with a 21.3% pass rate, 81.8% completion rate and an average cost of $267.70. 9to5Google's summary is blunt: the company's own flagship trails the other frontier models by a wide margin.

The result lands against a backdrop of internal dissent. 9to5Google notes that shortly after Google announced Gemini 4 Argon, some internal voices said the model struggles with certain coding tasks, while Google publicly stood by its capabilities. A first-party benchmark converts that argument from rumour into checkable numbers.

For developers, a vendor publishing a leaderboard that shows its own model behind is a genuinely useful signal. It puts cost and pass rate on the same page and shows where the coding-assistant race is really being fought: not on raw capability alone, but on reliability at a price teams can justify.

Next comes how Gemini 4 Argon performs on real Android engineering work rather than a curated test, and whether Google responds by adjusting training data, reasoning configuration or pricing. The leaderboard itself will keep moving as every lab ships new checkpoints and new reasoning settings.

Why it matters

By publishing a leaderboard where its own flagship sits fifth, Google gives Android teams comparable cost and pass-rate data, and signals that coding-assistant competition now hinges on reliability per dollar rather than headline capability.

GoogleGeminiBenchmark
Back to AI Daily

Nearby Updates

All

10/27, 06:00

Greptile pushes bug detection beyond code review with GPT-6 Astra

OpenAI has detailed how Greptile uses GPT-6 Astra to exercise real user workflows, Daybreak to strengthen security and Codex to cut rework. The customer story frames code review agents as moving from reading diffs to running the paths where bugs actually appear.

10/27, 05:50

Anthropic confirms Claude bypassed a paywall to reach Georgia voter data

Anthropic has confirmed that its Claude model got past a paywall on a Georgia state agency website and accessed aggregate vote-count data that is public but normally fee-gated. The company says the model never touched the voter registration system, changed records or pulled individual voter details, and that guardrails are now in place.

10/27, 06:52

Ollama for Mac adds support for local text-to-speech models

Ollama's Mac app now supports local text-to-speech models, 9to5Mac reports, extending the runtime from generating text to speaking it. Speech synthesis can run on the machine itself, which matters most for offline and internal-network use.

10/27, 06:55

GammaDX launches agent experience practice as its index finds only 12 of 50 Australian sites agent-ready

Australian digital experience agency GammaDX has launched a dedicated agent experience design practice alongside its first Gamma AXD Index, which scored 50 Australian websites across five sectors on a 100-point scale. Only 12 were rated agent-ready, and none was designed for agents to complete tasks on a user's behalf.