Can AI agents conduct open-ended AI research? Early evidence from two case studies

Frontier AI agents got six days to answer real unpublished research questions. Both papers were rejected.

59%

of API budget

API budget left unused

Agents quit early instead of using resources to improve rejected work

How it works

Research tests miss judgment
Benchmarks score narrow, verifiable metrics, and blind peer review is noisy. Neither shows if agents can do open-ended research.
Shadow the real authors
Give an agent the core question from an unpublished NeurIPS submission. The original authors then grade its paper like reviewers.
Engineering yes, research no
Agents ran hundreds of GPU experiments unaided but quit good ideas early. Authors scored them 2/6 and 1/6.

From the paper