computer sciencepaper
Max Zimmer, Nico Pelleriti, Christophe Roux, Sebastian Pokutta
More in maths and computing
- Exploring a question in knot theorymathematics, 27 Oct 2026
- Terence Taomathematics, 13 Aug 2026
- Robert Lipshitzmathematics, Jul 2026
- Tim Dettmerscomputer science, 13 Jan 2026
What they did
The authors present a framework of methodological rules, written as agent prompts, that turns CLI coding agents into research assistants running inside a sandboxed container. Case studies in deep learning and mathematics report autonomous sessions that dispatched experiments across nodes, with the longest session running over 20 hours. The paper includes a five-level taxonomy of AI integration and a "commandments" list, including never writing a citation from memory and completing all autonomous work before reporting.
In their words
"In practice, our longest autonomous session ran for over 20 hours, dispatching independent experiments across multiple nodes without human intervention."
"Natural-language proofs remain difficult to verify and require manual inspection."
"researchers must invest substantial effort in verifying agent outputs"
Lessons they draw
- Failure mode found: the agent kept stopping to ask whether to continue, so a rule requires finishing all autonomous work before reporting (Zimmer et al.).
- One-variable-at-a-time testing was critical; testing normalisation and weight decay separately showed independent, nearly additive gains (Zimmer et al.).
- Compute the oracle baseline early; in one case it showed 92 percent of the optimum was already achieved, which prevented wasted effort (Zimmer et al.).
- Audit formulas through numeric tests on small matrices before benchmarking; negative results (9 of 24 experiments) were more informative than positive ones (Zimmer et al.).
- Never write a citation from memory; agents hallucinate references, and verification is the researcher's responsibility (Zimmer et al.).
Limits of this account
A methods-and-framework paper by authors who built the tool; case studies are the authors' own, and I read the abstract, framework and case-study lessons but not the full related-work section.