Google refreshes Android Bench for the agentic coding era
Google has updated Android Bench, its benchmark and leaderboard for measuring how large language models handle Android development tasks. The original Android Developers Blog post, published on July 8, 2026 by Zoe Lopez-Latorre, says the July release adopts the Harbor framework, adds new model results, and opens more of the benchmark workflow to developer feedback.
Android Bench was introduced in March as a way to test models on real-world Android engineering work rather than on generic coding prompts. Google says the refresh is meant to keep the evaluation aligned with newer agentic development systems, where models call tools, inspect repositories, modify code, and iterate through multi-step fixes.
What changed
The main methodological change is the move to Harbor, which describes itself as a framework for specifying sandboxed agent tasks for evaluation and optimization. Google says Android Bench now uses Harbor alongside an updated benchmarking agent, and that it re-ran the benchmark across all models to establish a new baseline.
That matters because a benchmark migration can move scores even when the underlying models have not changed. Google explicitly says historical scores remain available in the archive, while the current leaderboard reflects the refreshed evaluation path.
The July release also adds eight models to the leaderboard: Claude Fable 5, Claude Sonnet 5, Claude Opus 4.8, GLM 5.2, Kimi K2.7 Code, MiniMax M3, Qwen 3.7 Plus, and Qwen 3.7 Max. On the live Android Bench leaderboard, Google lists results as of July 8 and defines the score as the average percentage of 100 test cases solved across 10 runs. The same table includes confidence ranges, average latency, and average cost per full benchmark run.
According to Google's July 8 leaderboard data, Claude Fable 5 leads the overall table with 84.5%, followed by GPT 5.5 at 80.2% and Claude Sonnet 5 at 76.2%. Among open-weight models, Google lists GLM 5.2 at 72.2% and Kimi K2.7 Code at 70.4%.
Why Android developers should care
Generic software benchmarks can miss mobile-specific problems. Google's methodology page says Android Bench is built around real-world issues and pull requests from open-source projects, with tasks tied to Android development patterns such as Jetpack Compose, Coroutines and Flows, Room, Hilt, navigation migrations, Gradle configuration, SDK changes, media, camera, foldables, and granular runtime permissions.
That focus makes the benchmark more useful for Android teams evaluating AI coding assistants. A model that performs well on broad algorithmic tasks may still struggle with Compose state, lifecycle-sensitive code, Gradle edge cases, or platform API migrations. Android Bench is not a guarantee that a model will work well in a specific codebase, but it gives teams a more relevant signal than a leaderboard built only around general programming puzzles.
The cost and latency columns are also important. In production developer workflows, a model's pass rate is only one part of the decision. Teams also need to know whether a coding agent can finish tasks fast enough, whether repeated runs are affordable, and whether higher scores come with higher operational cost.
More community input, but still curated
Google says developers can now submit Android development tasks for review and run or share benchmark evaluations. The Android Bench GitHub repository describes the project as a framework for benchmarking LLMs on Android development tasks, including whether models can understand mobile codebases, generate patches, and solve Android-specific engineering problems. The repository is Apache-2.0 licensed and includes documentation for downloading results and generating summaries.
The key caveat is that community task submission is not the same as immediate inclusion. Google says submitted tasks will be reviewed and assessed before being added. That is the right constraint for a benchmark that developers may use to compare tools: broader input can improve coverage, but the dataset still needs consistent quality control.
What to watch next
The bigger story is that AI coding evaluation is becoming more domain-specific. Android Bench now sits closer to the actual work mobile developers do: editing repositories, handling platform conventions, and measuring trade-offs beyond a single score. If Google keeps publishing methodology changes, archives, and cost metrics, Android teams will have a clearer way to compare AI assistants without treating general-purpose coding leaderboards as the whole picture.