RIP Classic Reasoning Benchmarks. What’s Next?
Give up at least one of: text only, short time horizon, easy to grade, and expert human superiority.
MirrorCode: Evidence AI can already do some weeks-long coding tasks
epoch.aiEarly results from MirrorCode benchmark with METR: AI agents can complete weeks-long coding tasks, including reimplementing a 16,000-line codebase.
Import AI 453: Breaking AI agents; MirrorCode; and ten views on gradual disempowerment
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv and feedback from readers. If you’d like to support this, please subscribe. A shorter issue than usual as I was attendi…
Import AI 453: Breaking AI agents; MirrorCode; and ten views on gradual disempowerment
Was fire equivalent to a singularity for people at the time?