EdgeBench: When AI Evaluation Enters the Marathon Era
ByteDance Seed's EdgeBench introduces 134 real-world tasks with 12-hour horizons, revealing a log-sigmoid scaling law for agent learning and a doubling of learning speed every three months.
Introduction: The Marathon of AI Evaluation
For years, AI model evaluation has been a sprint—give a model a task, see if it gets it right on the first try. Benchmarks like SWE-Bench, HumanEval, and Aider Polyglot all measure a model's out-of-the-box capability.
But they fail to answer a more important question: given enough time and feedback, can an AI agent continuously learn and improve from its environment?
On July 6, 2026, ByteDance's Seed team released EdgeBench, officially ushering AI agent evaluation into the marathon era.
What Is EdgeBench
EdgeBench is an ultra-long-horizon evaluation benchmark for AI agents, featuring 134 real-world tasks across six domains:
- Scientific Discovery
- Software Engineering
- Combinatorial Optimization
- Professional Knowledge Work
- Formal Mathematics
- Interactive Games
Each task is designed for at least 12 hours of continuous agent operation—far beyond the single-shot evaluation of traditional benchmarks. According to the paper, each task required an average of 57.2 hours of expert human effort to construct.
Core Finding: The Log-Sigmoid Scaling Law
EdgeBench's most striking finding comes from analyzing approximately 38,000 hours of agent-environment interaction data.
The research team discovered that agent learning performance follows a precise log-sigmoid curve. When aggregating all 134 tasks by interaction time, the performance trajectories of five frontier models fit with an R² of 0.998.
What does this curve mean? Agents don't improve linearly—they learn faster initially, then gradually plateau. This regularity provides a mathematical foundation for predicting long-term model performance in real-world scenarios.
Learning Speed Doubles Every Three Months
Even more surprising are the cross-generational trends. The paper compared frontier models from September 2025 to May 2026 and found that agent learning speed roughly doubles every three months.
This suggests we may be witnessing a new kind of Moore's Law for agent capabilities—not a doubling of compute, but an exponential increase in the efficiency of learning from experience.
The 12-Hour Leaderboard
The paper evaluated five major models under a 12-hour budget:
| Model | 2h Score | 12h Score | Code 12h | Math 12h |
|---|---|---|---|---|
| Claude Opus 4.8 | 39.0 | 51.3 | 67.4 | 55.0 |
| GPT-5.5 | 36.8 | 48.4 | 65.0 | 50.0 |
| GPT-5.4 | 29.7 | 39.3 | 54.1 | 40.8 |
| GLM-5.1 | 26.0 | 37.4 | 50.9 | 24.6 |
| DeepSeek-V4-Pro | 23.3 | 31.0 | 43.0 | 14.1 |
Claude Opus 4.8 leads the pack at the full 12-hour mark, with GPT-5.5 close behind. Interestingly, while DeepSeek-V4-Pro scores lower on formal mathematics, its learning curve steepness (β=0.93) matches Claude Opus 4.8, indicating that its environmental learning efficiency is not low.
Why EdgeBench Matters
Traditional AI evaluation is like a final exam—it only cares how many questions you get right. EdgeBench is more like an apprenticeship assessment—it measures whether you can learn from feedback and continuously improve in real work.
This shift in perspective matters because it aligns more closely with how AI is actually used in the real world. No one gives AI a single attempt and calls it done—we provide feedback, iterate, and optimize. EdgeBench is the first to systematically quantify this process.
Additionally, 51 tasks and the full evaluation framework have been open-sourced on GitHub, providing the entire AI community with a standardized long-horizon evaluation platform.
Limitations and Reflections
EdgeBench has caveats worth noting. While the aggregated curve is remarkably clean, individual task learning trajectories are far less smooth. The paper also acknowledges significant variance across domains—DeepSeek-V4-Pro, for instance, scores 43 on code tasks but only 14.1 on formal mathematics.
Another question: is a 12-hour budget sufficient for real-world applications? The research team has already conducted extended experiments on 28-hour and 72-hour task subsets, with fitting precision remaining above 0.993. Future marathons may well be even longer.
Conclusion
EdgeBench's release marks a pivotal shift in AI agent evaluation—from "can it do it" to "can it learn." As agents increasingly enter real-world workflows, this kind of evaluation that measures learning capability rather than static knowledge will become ever more critical.
If the trend of learning speed doubling every three months holds, we may be closer than we think to AI agents that can truly evolve autonomously in complex environments.