RealSWE

Real-SWE (version 1.1, published by Specific) is a software-engineering benchmark that evaluates AI coding agents on private, enterprise codebases whose code and solutions are not available on the public internet. Tasks reflect work with business consequences, such as billing logic, tax calculations and customer migrations, often spanning multiple services.

Models are evaluated using their native harnesses, to reflect how enterprise engineers work in practice, so results measure model-and-harness combinations rather than models in isolation. The leaderboard reports resolution rate with one standard error either side, and a Pareto chart plots resolution rate against estimated cost per rollout.

On the leaderboard as scraped, Claude Opus 5.5 ranks first at 56.8%, followed by GPT 6 Astra (56.0%) and Gemini 4 Carbon (55.2%), with overlapping error bands among the top entries. The lowest of the 20 listed models scores 4.8%. A YouTube creator cited different, lower figures for an earlier version of the board, so scores vary by benchmark version. The page does not document the task count, methodology details or a public dataset.