METR Time Horizons

METR Time Horizons is a research page and dataset from METR that reports the "task-completion time horizon" of public frontier language models. The time horizon is the task length, measured by how long a human expert takes, at which an AI agent is predicted to succeed with a given reliability. The page shows 50% and 80% time horizons, computed from agent performance on over a hundred diverse software tasks.

Method: METR estimates human completion times for each task, mostly by contracting skilled professionals and taking the geometric mean of successful completion times. For each agent it fits a logistic curve of success probability against human task duration, then reads off where the curve crosses 50% or 80%. Tasks come from RE-Bench, HCAST and a set of shorter novel tasks, mainly in software engineering, machine learning and cybersecurity. METR notes that measurements above 16 hours are unreliable with the current task suite.

The page stresses that a time horizon is a measure of task difficulty for humans, not the length of time an AI agent can act autonomously. It also says the results do not imply AI can automate all jobs or all intellectual tasks, since the tasks are self-contained and low-context. The page says it is no longer being updated, and that some models were measured long after release or skipped because of limited capacity. It is a freely accessible web page, with analysis code and raw data linked from it.