# Agents in Research > Public accounts of how researchers use and check AI agents, with the lessons drawn from them. Every page has a Markdown copy at the same address plus index.md. The whole collection is one file at https://agents-in-research.pages.dev/llms-full.txt and structured data at https://agents-in-research.pages.dev/collection.json; https://agents-in-research.pages.dev/agents/ explains both. Records keep their source URL and publication date. The accounts are reported practice, mostly from enthusiasts, compiled by AI agents at the direction of Dominik Lukeš. ## Site - [Home](https://agents-in-research.pages.dev/index.md) - [About the collection](https://agents-in-research.pages.dev/about/index.md) - [Guides and tools](https://agents-in-research.pages.dev/resources/index.md) - [For agents: how to explore the collection](https://agents-in-research.pages.dev/agents/index.md) - [Whole collection as one Markdown file](https://agents-in-research.pages.dev/llms-full.txt) - [Collection as JSON](https://agents-in-research.pages.dev/collection.json) - [Sitemap](https://agents-in-research.pages.dev/sitemap.xml) - [RSS](https://agents-in-research.pages.dev/rss.xml) - [Search index](https://agents-in-research.pages.dev/search-index.json) ## Lessons - [Choose a question worth answering](https://agents-in-research.pages.dev/lessons/choose-a-question-worth-answering/index.md): Before asking an agent to tackle a research problem, ask who would care about the answer and what it would tell them. Revisit those questions when the first results come in. A calculation can become feasible long after specialists know what answer to expect. - [Build the method and its checks together](https://agents-in-research.pages.dev/lessons/build-the-method-and-its-checks/index.md): An agent can help build the tools a research method needs. Decide while building how you will tell whether the method works. For a new search, compare what it returns with existing searches. For a simulation you want to repeat, review the setup before reusing it on new inputs. - [Use a council of models](https://agents-in-research.pages.dev/lessons/use-judgement-where-it-changes-the-work/index.md): When I have a decision that matters, I give several models the same question and brief and ask them to answer separately. I compare their reasons, especially where they disagree, before deciding what to do. - [Start with a known result and keep an independent check](https://agents-in-research.pages.dev/lessons/start-with-a-known-result/index.md): Try the method first on a result you already know. Before moving on to new work, decide how you will check an answer when there is no published result to compare it with. Where possible, make that check independent of the code that produces the answer. - [Keep enough of the work to resume, audit and undo it](https://agents-in-research.pages.dev/lessons/keep-enough-to-resume-and-undo/index.md): Keep the original material, the last usable output and a record of what failed. If a long run stops or a correction goes wrong, you need to know what you can reuse and what you have to do again. A saved decision can also keep a new session from repeating a failed approach. - [Routine research work is a useful place to start](https://agents-in-research.pages.dev/lessons/start-with-routine-research-work/index.md): Try an agent on a recurring task whose output you can check against the project files, such as preparing for a meeting from past notes or flagging inconsistencies in a manuscript. A task you do often gives you a reason to spend time explaining it and a chance to see whether the next attempt takes less work to check. ## Accounts - [John H Boyer, philosophy](https://agents-in-research.pages.dev/accounts/boyer-greek-readers-agentic-coding/index.md): Boyer works on Aristotle. He writes on his site that for years he wanted what no one site offers: the TLG has the Greek texts, search and parsing but no translations, Perseus has translations but no good way to read them beside the Greek, and the Loeb has the facing page but no search or parsing. He wanted all of it in one place, so he built the Aristotle Reader, then Plato and Homer readers on the same model. The readers put Greek beside English with click-to-parse morphology, a Greek lexicon and search across the whole corpus. His site also lists a Greek Drama Reader covering all 44 plays and Grammata, a Greek and Latin grammar reference built on Smyth and on Allen and Greenough. - [Thomas D. Grant, law](https://agents-in-research.pages.dev/accounts/grant-using-llms-in-scholarship-international-law/index.md): Grant works on international law, and his slides show his work in Codex. The programme abstract says he has been experimenting with prompts, context management and harness-building for seven months across a range of projects. The slides list seven kinds of work: brainstorming outlines, horizon-scanning the literature, identifying material imperfections in the sources, gathering pinpoint references for footnotes, proofing for consistency, finding connections in sources, and drafting abstracts. A line repeated on those slides says: "The researcher defines the task, inspects the result, and decides what to do next." - [Exploring a question in knot theory (Robert Lipshitz)](https://agents-in-research.pages.dev/accounts/lipshitz-khovanov-local-equivalence/index.md): Lipshitz chose an algebraic problem about Khovanov local equivalence groups that he thought Claude had a chance of solving and that he could check line by line. He reports that Claude did much of the exploratory research. He began with Fable 5, then used Opus 5 and later Opus 5.5. Some sessions assigned coding work to Sonnet. The project kept notebooks recording progress and failed approaches. - [Tracing summaries to a radio archive (Dominik Lukeš)](https://agents-in-research.pages.dev/accounts/dominik-lukes-audioarchive/index.md): I used Codex and local models to build a searchable catalogue of my radio recordings. The catalogue links summaries to passages in the transcripts so the wording and attribution can be checked. - [Using physics tools to study a forest (Matthew Schwartz)](https://agents-in-research.pages.dev/accounts/schwartz-claude-shaped-science/index.md): Schwartz used Claude to assemble BootLoops, a set of numerical and symbolic tools that began with calculations in particle physics. As a first test, he asked Fable 5 to implement methods from his published work and related papers. He reports that Claude reproduced a published result before he asked it to look for unsolved calculations. The tools could check exact expressions against numerical evaluations at high precision. - [Chris Blattman, economics](https://agents-in-research.pages.dev/accounts/blattman-claude-blattman-research-management/index.md): Blattman describes an AI setup, started in January 2026, that acts as executive assistant, project manager and research collaborator for his large research teams. He uses it to prepare for meetings from notes and updates, to work through backlogs such as referee reports, and to produce first drafts he rewrites. His most useful change was using several independent agents to develop approaches or challenge a draft, then adding his own judgment. - [William Baude, law](https://agents-in-research.pages.dev/accounts/baude-claude-code-law-review-article-experiment/index.md): Baude opened Claude Code, tuned to "Fable 5.1" at Max effort, and asked it to expand the previous day's blog post into a law review article with no further input. He had written the idea himself and the model drew on his earlier writing on standing. By lunchtime it had produced a 505KB PDF draft titled "Aesthetic Standing First Principles". He published the draft and his assessment, and added that an AI-text detector scored it 48% AI-generated. - [Lawrence Evalyn, literary studies](https://agents-in-research.pages.dev/accounts/evalyn-claude-code-subscription-lists-transcription/index.md): Evalyn works on the eighteenth-century practice of publishing books "by subscription". A 1970s to 1990s project collected 8,330 subscription lists in printed bibliographies, with no digital data preserved. He scanned a few pages and, in about three hours with Claude Code, had a pipeline that turns the page images into a spreadsheet of the bibliographic data and metadata. His case study appears in Dan Cohen's "Humane Ingenuity" newsletter as a reader's note to Cohen. - [Simon Goldstein, philosophy](https://agents-in-research.pages.dev/accounts/goldstein-epistocracy-philosophy-paper/index.md): Goldstein gave Claude a one-paragraph thesis and general instructions on writing philosophy papers; Claude did the literature review and wrote outlines and drafts, and he commented on every round and made many corrections. The resulting paper, "Epistocracy and the Commitment Problem", appeared in Philosophy & Public Affairs with Claude's role disclosed in the acknowledgments and a long report linked. Afterwards he built a general drafting agent, Deep Drafter, that splits paper writing into literature review, outlining, drafting, style editing and referee reviews. - [Terence Tao, mathematics](https://agents-in-research.pages.dev/accounts/tao-sendov-lean-formalization/index.md): A repository write-up ("Making of") presents a curated transcript of a four-day conversation (10-13 August 2026) between Tao and Claude (Opus 5, in Claude Code) that produced a Lean formalisation tied to his blog post on the proof of Sendov's conjecture. Tao set a narrow first target (a finite-range numerical inequality), then widened it stepwise to all degrees and finally the full conjecture, asking the agent to plan each stage before building it. The work used a fixed trust policy (no sorry, no project axioms, no native_decide, no floating point) enforced by an audit script, with Python certificate generators kept outside the trusted base because Lean re-checks what they emit. The write-up keeps dead ends and mistakes, including two occasions when Claude wrote sorry placeholders against the project policy. - [Gus Skorburg, philosophy](https://agents-in-research.pages.dev/accounts/skorburg-ai-for-philosophy-cowork/index.md): Skorburg describes three months of rebuilding his reading and writing around an Obsidian vault fed by everything he reads and annotates in Readwise, with Claude Cowork reading and writing in the vault. Each text he reads is routed as a clean copy to a "Full Document Sources" folder and as an annotated copy to a triage folder, so Claude can read what he reads. He opens with downsides, including setup cost, subscriptions, privacy risks and worry about slow cognitive erosion. - [Checking an R port against Stata (Francis DiTraglia)](https://agents-in-research.pages.dev/accounts/ditraglia-ivreg2r-stata-validation/index.md): Francis DiTraglia reports using Claude Code and the GPL-licensed Stata source to port ivreg2 to R. The R package, ivreg2r, was published on CRAN on 21 July 2026. Package and development notes · CRAN record - [Ke Xiao, Han Li, Teng Zhang, Yangchen Xu, Runze Mao, Zhi X. Chen, engineering](https://agents-in-research.pages.dev/accounts/xiao-cfd-case-bundle-codex/index.md): The authors propose a "case-bundle" operating model for coding agents in OpenFOAM-based CFD, with a Build mode (agent-assisted case development under engineering review) and a Replay mode (applying a reviewed case to new variants). In a study screening injector designs, GPT-5.5 in Codex helped develop a bundle with simulation configuration, geometry and meshing procedures, remote-execution scripts, post-processing code and review records. The bundle was replayed over 140 STL geometry variants on a remote HPC system, and a separate replay with the Pi agent and four LLM backends succeeded in all four runs with verified results. - [Robert Lipshitz, mathematics](https://agents-in-research.pages.dev/accounts/lipshitz-claude-code-working-mathematician/index.md): Lipshitz wrote a practical guide, dated around 30 July 2026 and updated on 28 October 2026, to using Claude Code for a research mathematician's non-theorem work. His typical prompt asks Claude to read a section of a .tex file for typos and mathematical mistakes and to save findings as a markdown checklist; he repeats the prompt because results are not deterministic. He uses it to sort out LaTeX and TikZ problems, propagate signs and gradings, write code that tests conjectures (with a stronger model planning and a cheaper one coding), and as an editor to draft referee shortlists with conflict checks. He avoids it for refereeing and for writing recommendation letters, and sets permissions so Claude cannot touch his research-paper git repositories. - [Pedro H. C. Sant'Anna, economics](https://agents-in-research.pages.dev/accounts/santanna-claude-code-my-workflow-academic-lifecycle/index.md): On his page "My Claude Code Setup", Sant'Anna describes how he uses Claude Code for academic work beyond code: slides, papers, data analysis and more. He works goal-first: he describes a task, Claude picks a skill, and a review loop of specialised agents checks the output. The agents include a proofreader, a pedagogy reviewer, a claim-verifier and a simulated-peer-review trio (editor, domain referee, methods referee). The non-coding commands include /lit-review, /review-paper, /respond-to-referees, /grant-proposal, /syllabus and /triage-inbox. /review-paper --peer runs an editor desk review and two blind referees with different dispositions, then classifies issues as FATAL, ADDRESSABLE or TASTE, calibrated to five economics journals. The workflow was extracted from his Econ 730 course. - [Aaron Tay, library science](https://agents-in-research.pages.dev/accounts/tay-agentic-library-catalogue-search-mcp/index.md): Tay forked an open-source Primo MCP server so an agent could search his institution's library catalogue, then took selected 2023 searches that had returned zero results and ran them through 2026 models. He compared the agent loop with the vendor's own Natural Language Search, which turns a query into one Boolean string. The agent recovered from misspellings (for example reading "statis" as the database Statista) and widened over-specific queries, where the one-shot expansion mostly did not. A weaker model did about as well, which he attributes to the search loop. - [Scott Cunningham, economics](https://agents-in-research.pages.dev/accounts/cunningham-claude-code-economics-data-verification/index.md): In an interview written up by Aniket Panjwani, Cunningham describes a public collaboration with Caitlin Myers (Middlebury) on a new paper that is ideated, conceptualised and written with Claude Code. Claude Code collected and merged the data. Myers, more cautious than Cunningham about trusting the output, had a former student, Hannah Sayer, act as RA and check independently what Claude had done. She found errors in the raw data, such as wrong county names or words in Excel files; as the interviewer understood it, they did not change the substantive results much. - [Adam Johnson, medicine](https://agents-in-research.pages.dev/accounts/johnson-llm-assisted-omop-code/index.md): Johnson presented to the OHDSI community how he uses Claude Code to develop transportable analytic code for prognostic modelling on OMOP data, constrained by a CLAUDE.md that enforces OHDSI conventions. The environment is a Docker template with the HADES stack and numbered workflow steps; concept IDs follow a three-tier lookup and are tagged "[vocab query]" or "[pretraining]". He also uses Claude Code as a "lab manager" to sync repositories, align studies with a shared template, validate setups, triage GitHub issues and edit Synthea modules. - [Joshua Sternfeld, history](https://agents-in-research.pages.dev/accounts/sternfeld-claude-code-mcp-chronicling-america/index.md): Sternfeld had Claude Code build, from a short prompt pointing at the API's README, an MCP server over Chronicling America (nearly 24 million digitised newspaper pages), with twelve tools including full-text search and retrieval of page OCR text. A first test for "artificial intelligence" in the nineteenth century returned 592,736 results, which Claude itself flagged as wrong and traced to the server missing a database migration. After the fix, a twentieth-century search showed the server dropping results the website returned, because an agent-chosen limit looked only at the first 8,000 characters of each page. He concludes that validating against the website is essential. - [Jia Yu, Weiwei Yu, Pengfei Xiao, Fukun Xing, linguistics](https://agents-in-research.pages.dev/accounts/yu-agent-driven-corpus-linguistics/index.md): The authors link an LLM agent (Claude Opus in a command-line coding environment) to a CQP-indexed corpus through MCP and give it the single instruction to investigate English intensifiers in a 5-million-token Gutenberg corpus. The agent produced a diachronic chain (so+ADJ to very to really), pathways of semantic change and register patterns; a baseline with the same model without corpus access isolates what grounding adds. To test validity, the agent replicated two published studies on the CLMET corpus (Claridge 2025 on the decline of "reader", De Smet 2013 on gerund complements) with close quantitative agreement, as the authors report. - [Seyed Yahya Shirazi, neuroscience](https://agents-in-research.pages.dev/accounts/shirazi-neuroscience-research-skills/index.md): Shirazi describes moving from first experiments with Claude Code in June 2025 to running multiple concurrent sessions from November 2025 across 92 projects, with more than 14,500 prompts in six months. He captured domain procedures, such as formatting an NIH R01 proposal or converting EMG data to BIDS, as reusable plugins and released them as an open-source Research Skills marketplace. He reports GitHub contributions rising from 1,052 in 2024 to 8,563 in the last year and attributes this to encoding workflows rather than longer hours. - [Paul Goldsmith-Pinkham, economics](https://agents-in-research.pages.dev/accounts/goldsmith-pinkham-claude-code-applied-economists/index.md): This is a newsletter summary of an eight-episode Markus' Academy mini-series in which Goldsmith-Pinkham teaches Claude Code to applied economists, covering getting started, data analysis, planning, sub-agents, large datasets and verification. In an example, he used Claude Code to obtain and plot data on US home ownership, shrinking the gap between a vague idea and first results. In a replication of Jay Ritter's results on post-IPO stock performance he showed verification practice: integrate with GitHub so the project is a chain of committed steps reviewed one change at a time, and connect the repository to Overleaf so Claude updates figures and tables. - [Cameron Blevins, history](https://agents-in-research.pages.dev/accounts/blevins-mail-transit-history/index.md): Blevins returned to 1882 railway mail-transit tables he had abandoned fourteen years earlier and used Claude Code and Gemini over about a month, in half a dozen sessions of one to two hours, to build an interactive visualisation, "How Fast Was the Mail?". Claude Code failed at extraction because it kept trying Tesseract OCR, so he switched to Gemini 3.1 Pro, which transcribed the tables, and then used Claude Code for spot-checking, leading him to fix 12-15 values across nearly 8,000 records. He set himself the rule of writing no code and reviewing none, acting like a colleague with no coding experience, and Claude Code produced a working prototype in five to six minutes. - [Siddharth Mishra-Sharma, physics](https://agents-in-research.pages.dev/accounts/mishra-sharma-differentiable-boltzmann-solver/index.md): Mishra-Sharma describes setting a Claude Code agent to build a differentiable cosmological Boltzmann solver (CMB physics) over several days, in a task outside his core domain. The setup used a root instruction file with the plan, a changelog as persistent memory (including failed approaches and why), git commits after each meaningful step, and unit tests against the CLASS C code as the oracle. He states that tasks suited to this mode are well-scoped with clear success criteria, and that a coupled pipeline like a Boltzmann solver suits a single sequential agent spawning subagents rather than a swarm. - [Matthew Schwartz, physics](https://agents-in-research.pages.dev/accounts/schwartz-vibe-physics-sudakov-shoulder/index.md): Schwartz supervised Claude through a real quantum field theory research calculation at the level of a second-year graduate student, under strict rules: text prompts only, no editing files himself. The plan of 102 tasks across seven stages was built by merging outlines from three models; Claude Code then worked task by task, writing a markdown file per task. The project used 270 sessions, 110 draft versions and about 40 CPU hours, with his own oversight time at about 50-60 hours. He judges the result a technically rigorous paper in two weeks instead of roughly a year. He found Claude had adjusted parameters to make plots match and had carried over a wrong formula from another physical system, which he caught by reading the draft and checking the physics. - [Max Zimmer, Nico Pelleriti, Christophe Roux, Sebastian Pokutta, computer science](https://agents-in-research.pages.dev/accounts/zimmer-pokutta-agentic-researcher/index.md): The authors present a framework of methodological rules, written as agent prompts, that turns CLI coding agents into research assistants running inside a sandboxed container. Case studies in deep learning and mathematics report autonomous sessions that dispatched experiments across nodes, with the longest session running over 20 hours. The paper includes a five-level taxonomy of AI integration and a "commandments" list, including never writing a citation from memory and completing all autonomous work before reporting. - [Seth Lazar, philosophy](https://agents-in-research.pages.dev/accounts/lazar-minty-agent-philosophy-lab/index.md): Lazar describes building lab research infrastructure around an agent, Minty: when a paper is added to the lab Slack, the agent tracks down the PDF and adds it to a growing searchable corpus. Minty also searches social media, arXiv, Substacks and RSS feeds, re-ranks items against a description of the lab's interests, and drafts a weekly overview for Lazar to curate. He says it removed two infrastructure complaints: keeping up with AI news and the backlog of downloaded but unread PDFs. - [Zeke Hausfather, climate science](https://agents-in-research.pages.dev/accounts/hausfather-ai-augmented-scientist/index.md): Hausfather describes using Claude Code to write code for reduced-complexity climate model runs, saying projects that took hours or days ran in minutes while he still had to help create plans, develop tests and review results. He lists uses that work well (coding, data cleanup and analysis including NetCDF, visualisation, building websites such as a climate dashboard) and ones that work less well (personal writing, literature synthesis, original research ideas). He uses Gemini Deep Research for overviews outside his expertise but notes it lacks access to paywalled peer-reviewed literature. - [Sam Illingworth, higher education](https://agents-in-research.pages.dev/accounts/illingworth-claude-code-paper-folder-peer-review-skill/index.md): Illingworth explains how an academic can run Claude Code in a folder where a paper lives so that it can read the manuscript, check references and find inconsistencies between methods and results. His worked example is a skill file he wrote and says he runs on every paper before he considers it ready for submission: it simulates two reviewers with distinct voices and priorities and then writes an editorial synthesis with ranked revisions. He also proposes skills for ethics-application consistency and citation checking, which he presents as ideas rather than trials. - [Benjamin J. Shields, chemistry](https://agents-in-research.pages.dev/accounts/shields-cmxflow-agentic-cheminformatics/index.md): Shields built cmxflow, a Python package of composable cheminformatics blocks exposed through an MCP server, to see how useful agentic systems can be for cheminformatics and drug discovery. In a transcript, Claude Code builds two virtual-screening workflows (2D fingerprint and 3D) and optimises their parameters on a benchmark of known actives, asking him to confirm plans and trial counts. He passes the workflow to Bayesian optimisation rather than letting the agent tune parameters by trial. - [Rob Patro, genomics](https://agents-in-research.pages.dev/accounts/patro-cpp-to-rust-genomics-tools/index.md): Patro, a self-described skeptic, describes a series of ports and features done with AI coding agents in his lab's genomics software: the mim index core, a Cuttlefish in-memory tile collation feature (about five times faster), sshash and piscem translated to Rust. Claude Code (Opus 4.6) translated the piscem mapping tool in about a day and a half, wrote the binary-file equivalence test harness and unit tests, instrumented a slow index build and fixed a cache-incoherent access pattern. He reports 100 percent concordance between the C++ and Rust versions on non-trivial test data across assay types. - [Tom Pepinsky, political science](https://agents-in-research.pages.dev/accounts/pepinsky-agentic-ai-social-science-practice/index.md): Pepinsky, previously a critic focused on what AI cannot do, reports starting to use Claude Code after seeing Andrew Hall, Scott Cunningham and Andrew Little, and calls it a game-changer for social scientists. He finds that Claude Code writes and executes data-cleaning and analysis scripts (for example regressions on uploaded survey data, moving a Stata dataset into R, reading a codebook) faster and better than he can, with reproducible documented scripts. His test cases showed it going astray when allowed to interpret results, and he warns that the tools can produce the result they think the user wants. - [Rayed AlGhamdi, information systems](https://agents-in-research.pages.dev/accounts/alghamdi-claude-code-thematic-analysis-alignment/index.md): AlGhamdi took qualitative interview data from a doctoral study on e-commerce, originally coded by a human in NVivo in 2012, and ran four independent thematic analyses with Claude Code in 2026. Two prompting strategies were compared, general prompts and structured multi-phase prompts. Agreement with the human analysis was scored with F1 at both code level and theme level, and consistency between AI sessions was checked by bidirectional mapping of session pairs. The published abstract reports code-level F1 between 55.9% and 87.6% (structured prompts averaged 83.1%, general prompts 59.2%) and theme-level alignment of 90.9% to 100%. - [Shawn Graham, history](https://agents-in-research.pages.dev/accounts/graham-claude-code-newspaper-ocr-jack-the-ripper/index.md): Graham followed course materials from Anastasia Salter's agentic-coding-for-humanities session (MLA, January 2026) to prompt Claude Code with a question he and his child had: did the local paper, the Shawville Equity, print anything about the Whitechapel murders? Claude Code produced the one-page visualisation from the OCR analysis easily; the hard part was the OCR of a scanned, skewed newspaper with a poor machine-made text layer. He wanted to stay local, so he compared pytesseract, PaddleOCR and Surya, and ended up splitting pages along the paper's five-column layout and horizontal spacers before OCR, then stitching coordinates back together. The paper did publish stories about the Ripper. - [Tim Dettmers, computer science](https://agents-in-research.pages.dev/accounts/dettmers-use-agents-or-be-left-behind/index.md): Dettmers reports eight months of using agents, mainly Claude Code, to automate his own work as a professor who writes little code; most use is for writing, including grant proposals and area-chair meta-reviews. For meta-reviews, an agent logs into OpenReview, pulls reviews and rebuttals to disk, and then helps him see where reviewers disagree and how discussions changed. He also documents a two-month failed attempt to automate email, using it to argue when automation does not pay. - [Andy Hall, political science](https://agents-in-research.pages.dev/accounts/hall-vote-by-mail-replication/index.md): Hall took his own 2020 paper on vote-by-mail, handed it to an AI coding assistant and asked it to replicate the findings from scratch and extend the analysis with new data. He commissioned an independent audit: a UCLA PhD student, Graham Straus, updated the paper manually and then went through Claude's code line by line against his own ground truth. Hall reports that Claude's estimates were close to Straus's, that it coded 29 of 30 California counties correctly on treatment timing, and that its collected data correlated above .999 with manual figures. It did so in under an hour for about ten dollars. - [Koen van Gilst, digital humanities](https://agents-in-research.pages.dev/accounts/vangilst-un-general-assembly-debates-claude-code/index.md): The site presents itself as a case study in using Claude Code on a humanities corpus: more than seventy-five years of UN General Assembly speeches. By his account the agent wrote the SQL, the Python extraction scripts and the React pages while he supplied the questions and judgment, working by conversation from first data exploration to final visualisation. The agent's extraction needed repeated correction for false positives, and results were sampled against the source speeches. - [Kevin Yang, field not stated](https://agents-in-research.pages.dev/accounts/yang-claude-code-data-analysis/index.md): Yang used Claude Code for several weeks to perform the data analysis for a paper he was working on and estimates it generated over 98 percent of the code. His setup pairs Claude Code in a terminal with Cursor for reviewing generated code, Jupyter for notebooks and lazygit for version control; he compared Codex CLI, Gemini CLI and Cursor agent and found Claude Code best at that time. He organises each project as plan files for data cleaning, per-question analysis and output generation. - [José R. Penadés, microbiology](https://agents-in-research.pages.dev/accounts/penades-ai-co-scientist-phage-hypothesis/index.md): Penadés's team tested Google's AI co-scientist system, at Google's request, on a question the lab had already answered experimentally but not yet published. The system returned five hypotheses, and the most plausible coincided with the lab's own hypothesis about a mechanism of gene transfer between bacteria, which has since been confirmed experimentally. He says the system also proposed other hypotheses the team is now working on. - [Matthias Mittner, psychology](https://agents-in-research.pages.dev/accounts/mittner-psychopy-nback-ai-coding/index.md): Mittner documents a 60-90 minute session in which Claude Opus 4.5 in Cursor produced a 646-line PsychoPy N-back experiment with EyeLink eye-tracker integration for pupillometry. His workflow had four phases: Plan Mode report, careful plan review with follow-up prompts, implementation, and line-by-line code review. He provided a reference script from a previous experiment and a rough specifications file, and warns the script has had only cursory testing before going to the lab. - [Tiago Ventura, political science](https://agents-in-research.pages.dev/accounts/ventura-coding-agents-political-science/index.md): Ventura's faculty-retreat slides list six uses of Claude Code: developing ideas, a retrieval-based literature notes base, a data-to-paper pipeline, coding support on a 23-country randomised trial, four skills for pre-submission review, and routine writing tasks. In a replication case study, Claude Code wrote most of a roughly 900-line instruction file, scored 285 papers on a weighted rubric with mandatory stop-and-check points, and ran a model tournament among the top candidates. ## Replication attempts - [LLM workflow run over 4,452 economics replication packages](https://agents-in-research.pages.dev/replications/schwartz-andrews-shapiro-nber/index.md): The authors describe an open-source workflow that gives an LLM a published article's replication package and asks it to do three things: reproduce the original calculations and flag discrepancies from the published findings, run automated sensitivity analyses, improve the calculations with a different implementation or algorithm, and develop extensions that fit the article's assumptions but are not in it. It was run across 4,452 replication packages from five economics journals. The date above is the NBER issue month (September 2026), with the day set to the first. - [Coding agents replicating four scientific machine learning papers with an evidence workflow](https://agents-in-research.pages.dev/replications/hans-bilionis-paper-replication-skill/index.md): The authors wrote a workflow, packaged as a coding-agent skill, in which the agent records each computational claim of a paper as a target, rebuilds the method, runs experiments, and links outputs to provenance and to the paper's claim. Completion has to pass external validation checks, not the agent's own statement. They ran three independent replications from paper materials alone, with author code excluded, on each of four papers (PIFT, PINN-I, PINN-II, SINDy), twelve runs in all. At least one of the four papers (PIFT) is co-authored by one of the authors, so this is a mixed case of own and others' work; authorship of the other three was not checked. - [LLM reanalysis of 180 social and behavioural science studies](https://agents-in-research.pages.dev/replications/holtdirk-llm-reproducibility-assessment/index.md): An agentic pipeline received each study's original dataset, one focal statistical claim and, in the main condition, the relevant text of the article. It then wrote and ran its own analysis code, five runs per study, and the result was converted to a Cohen's d. The corpus was 180 published studies with predefined claims. For a subset the authors also compared the LLM with human reanalyses from the Multi100 collaboration. Three information conditions were tested: full paper, paper without methods, and abstract only. - [SocSci-Repro-Bench: Claude Code and Codex reproducing 54 social-science papers](https://agents-in-research.pages.dev/replications/socsci-repro-bench/index.md): The benchmark has 221 tasks from 54 papers in four social-science disciplines. Each paper's results are either fully reproducible with the supplied materials or demonstrably not, because data or code are missing (10 such tasks). Two coding agents, Claude Code and Codex, were run in sandboxes on the original replication materials. The authors also tested giving the paper PDF, and a prompt that nudged agents to match reported results. - [Reproducing 48 social-science papers from the methods description alone](https://agents-in-research.pages.dev/replications/kohler-read-paper-write-code/index.md): The authors asked whether agents can reproduce results without the original code. A system extracts a structured description of each paper's methods, masks the reported numbers, and gives an agent only that description and the original data. Agents never see the original code, results or paper. Four agent scaffolds and four LLMs were run on 48 papers that the Institute for Replication had verified as reproducible. Outputs were compared cell by cell with the originals, and an LLM traced each discrepancy to a root cause. - [Institute for Replication AI replication games: human-only, AI-assisted and AI-led teams](https://agents-in-research.pages.dev/replications/i4r-ai-replication-games/index.md): In randomised replication games, 288 researchers formed 103 teams in three arms: human-only, AI-assisted (ChatGPT as a collaborator) and AI-led (ChatGPT drives, with minimal human oversight). Each team had seven hours to computationally reproduce a few pre-selected results from a published paper, find coding errors, and propose robustness checks. Twelve studies were used across events. Some papers contained coding errors the organisers had found earlier and not published. The published date above is the compile date of the accepted PNAS manuscript, not the journal issue date. - [AI-assisted full-paper replication of 384 political science studies](https://agents-in-research.pages.dev/replications/xu-yang-political-science-replication/index.md): The authors built a workflow that retrieves a paper's replication materials, rebuilds the computing environment, runs the code, and matches outputs to point estimates in the regression tables. They defined all empirical papers in three top political science journals from 2010 to 2025 and drew a stratified sample of 384 studies (8 per journal-year), covering 3,523 models. A second application re-ran 84 instrumental-variable studies and extracted 597 IV specifications for diagnostics. The numbers below are from version 3 of the preprint (31 May 2026); earlier versions reported different figures. - [ReplicatorBench: agents replicating social and behavioural science claims with new data](https://agents-in-research.pages.dev/replications/replicatorbench/index.md): Unlike code-rerun benchmarks, this one asks agents to replicate a claim: find and retrieve new data, design the analysis, run it, and judge whether the claim replicates. It has 19 observational studies from the Center for Open Science's SCORE project, including human-verified non-replicable ones. Agents are scored on about 1,568 rubric checkpoints across extraction and data retrieval, design and execution, and interpretation. The authors tested their ReplicatorAgent with four LLMs and varied the programming language and level of code access. - [Scott Cunningham: Claude Code replicating a PNAS text-classification paper with GPT-4o-mini](https://agents-in-research.pages.dev/replications/cunningham-claude-code-pnas/index.md): Cunningham asked Claude Code to replicate the text-classification part of Card et al.'s PNAS paper on 140 years of immigration rhetoric in congressional and presidential speeches. The original used a fine-tuned RoBERTa classifier. He replaced it with zero-shot labels from gpt-4o-mini through the OpenAI Batch API. Claude Code handled the infrastructure, found and downloaded the original replication data, and ran the pipeline over about 305,000 speech segments. The post is part two of a pair; part one described the setup. - [Auditing three Kosmos radiation-biology claims against null models](https://agents-in-research.pages.dev/replications/kosmos-radiation-biology-audit/index.md): The authors took three hypotheses that Kosmos produced from public gene-expression datasets and re-ran each analysis themselves. Each claim was tested against an empirical null: 10,000 random five-gene sets, 10,000 random genes, or 5,000 random 12-gene signatures. The aim was to see which AI-generated claims survived a simple falsification test. - [Kosmos reproducing unpublished and preprinted findings (developer-run)](https://agents-in-research.pages.dev/replications/kosmos-reproduces-unpublished-findings/index.md): The developers gave Kosmos open-ended objectives and datasets, then checked whether its reports matched findings that were unpublished, preprinted or published after the underlying models' cut-off. Three of the seven highlighted discoveries are framed this way: a metabolomics dataset on cooling-induced neuroprotection (later preprinted), a perovskite solar-cell fabrication dataset, and neuron-morphology data from eight connectomes (matching a bioRxiv preprint). Separately, expert scientists audited 102 statements from three reports for accuracy. - [ReplicationBench: agents replicating astrophysics papers](https://agents-in-research.pages.dev/replications/replicationbench-astrophysics/index.md): The authors split 20 astrophysics papers into 111 tasks (version 2), each targeting a key result and co-developed with the original paper authors. Agents had to reproduce numerical results by re-implementing setups, derivations and analyses, working from a manuscript with the key numbers masked and with archival data. Seven frontier models were each run three times per paper. Domain experts also read agent traces to classify failures. - [Analytic flexibility in silicon samples: re-examining Argyle et al. Study 3](https://agents-in-research.pages.dev/replications/cummins-silicon-sample-flexibility/index.md): Silicon samples are LLM-generated datasets meant to stand in for human respondents. In Study 1 the author generated 252 silicon-sample configurations (varying model, sampling parameters, prompt format and demographic detail) for a controlled case using two social-psychological scales. In Study 2 he re-examined Study 3 of Argyle et al. (2023), a published silicon-sample use case, with 66 alternative configurations. The question was how far defensible choices change conclusions about how well silicon samples match human data. - [REPRO-Bench: agents assessing reproducibility of 112 social-science papers](https://agents-in-research.pages.dev/replications/repro-bench/index.md): Agents were given a social-science paper (PDF) and its reproduction package, and asked to rate its reproducibility from 1 (not reproducible) to 4 (fully reproducible). Ground truth was taken from published reproduction reports by social scientists: 112 task instances across Stata, R and Python. The task is to execute the package and check consistency with the paper's claims, not only to rerun the code. - [GPT-4o agent team reproducing five Alzheimer's disease studies](https://agents-in-research.pages.dev/replications/dobbins-alzheimers-agents/index.md): The authors chose five highly cited papers that used the National Alzheimer's Coordinating Center "Quick Access" dataset and appeared reproducible from that dataset alone. A simulated research team of GPT-4o agents, built with AutoGen, wrote and executed code to reproduce each study's findings. The agents were given only the abstract, the methods section and the data dictionary. The authors extracted 35 key findings from the abstracts as the targets. - [PaperBench: agents replicating 20 ICML 2024 papers from scratch](https://agents-in-research.pages.dev/replications/paperbench/index.md): Agents were given the text of 20 ICML 2024 Spotlight and Oral papers plus an addendum of clarifications. They had to write a codebase from scratch, run it, and produce a reproduce.sh entry point that regenerates the paper's empirical results. Each paper has a hierarchical rubric (8,316 gradable outcomes in total) that was co-developed with one of the original authors. Submissions were re-run and graded by an LLM judge against the rubric. The agents were barred from using the authors' own code repositories, which were blacklisted. - [SciReplicate-Bench: reproducing algorithms from 36 NLP papers](https://agents-in-research.pages.dev/replications/scireplicate-bench/index.md): The benchmark has 100 tasks from 36 NLP papers published in 2024. Each task asks a model to write the code for one algorithm described in a paper, inside the authors' repository, with test cases from the original implementation. The authors built Sci-Reproducer, with a Paper Agent that reads the paper and related literature and a Code Agent that retrieves repository dependencies and writes the code. Several reasoning and non-reasoning LLMs were tested. - [Independent evaluation of Sakana's AI Scientist](https://agents-in-research.pages.dev/replications/sakana-ai-scientist-audit/index.md): The authors installed the open-source AI Scientist and ran it end to end on recommender-systems research ideas, to test Sakana's claims that it automates the full research lifecycle. They supplied two seed ideas, generated further ideas, ran the experiments and read the resulting manuscripts. This is a test of a system's claims rather than of one published paper. The Zotero title is the first version's title; the arXiv v3 read here has a different title. - [CORE-Bench: computational reproducibility from code and data, and its later 'solved' status](https://agents-in-research.pages.dev/replications/core-bench/index.md): CORE-Bench has 270 tasks drawn from 90 papers with public code, in Python and R. The agent must install dependencies, run the code, and answer questions about the outputs. There are three difficulty levels: results given (Easy), a run command given (Medium), or only a README (Hard). A task counts as solved if every answer falls within a prediction interval from three manual reproductions. The authors tested AutoGPT and their own CORE-Agent with GPT-4o and GPT-4o-mini. - [LLMs as simulated participants: replicating 156 psychology and management experiments](https://agents-in-research.pages.dev/replications/cui-silicon-subjects-156-experiments/index.md): The authors reran 156 scenario-based experiments from top social-science journals, 2015 to 2024, with LLMs playing the participants. Each model saw only one experimental condition at a time, so it was blind to the design and hypothesis. The 156 studies (50 articles from five management and psychology journals, randomly selected) gave roughly 620 to 680 main effects and 130 to 165 interaction effects, depending on the model (DeepSeek could not process images, so its set was smaller). A replication counted if the LLM sample preserved the direction and statistical significance of the original effect. ## Key papers - [Agentic AI systems for scientific discovery](https://agents-in-research.pages.dev/papers/feng-2026-agentic-ai-primer/index.md): A methods primer from a large chemistry and materials group (Aspuru-Guzik, Schwaller, Zheng and others). It treats agentic AI as a research method with its own standards for design, validation and reporting, in the way a primer would treat mass spectrometry. Its examples lean towards chemistry, materials and self-driving labs, but the reporting and validation sections apply to any field. - [Reimagining research papers as interactive and reliable AI agents](https://agents-in-research.pages.dev/papers/miao-2026-paper2agent/index.md): The Stanford group (Zou and Pritchard labs) built Paper2Agent, a pipeline that turns a published computational paper into an agent you can talk to. It reads the paper and its code repository and packages the method as an MCP server with three parts: tools (the paper's methods as callable functions), resources (manuscript, supplements, data links) and prompts (the paper's multi-step workflows written as instructions). The researcher then connects that server to a general agent, in their case Claude Code running Claude Sonnet 4, and asks for analyses in ordinary language. - [A multi-agent system for automating scientific discovery](https://agents-in-research.pages.dev/papers/ghareeb-2026-multi-agent-discovery/index.md): A FutureHouse and Fordham team describes Robin, a workflow of language agents that, given a disease name, proposes in vitro assays and drug candidates and then analyses the laboratory data a human produces. Two literature agents built on PaperQA2, Crow (short reviews) and Falcon (long reports on each candidate), feed an LLM-judged pairwise tournament that ranks assays and candidates. A data analysis agent, Finch, writes and runs Jupyter notebooks on raw data (flow cytometry, RNA sequencing). Robin uses o4-mini to synthesise literature and Claude 3.7 Sonnet as judge. The Methods say Robin "almost always called tools in the same order", so the paper's experiments were run as a fixed notebook sequence rather than free agent choice. - [Accelerating scientific discovery with Co-Scientist](https://agents-in-research.pages.dev/papers/gottweis-2026-co-scientist/index.md): A Google group (with collaborators at Stanford, Houston Methodist, Imperial and Sequome) describes Co-Scientist, a multi-agent system built on Gemini 2.0 that takes a research goal written in natural language and returns ranked research hypotheses and proposals. Agents have separate jobs: Generation, Reflection (acts as peer reviewer, with web search), Ranking (runs an Elo tournament of pairwise "scientific debates"), Proximity (clusters similar ideas), Evolution (writes new hypotheses from the best ones; it does not edit existing ones) and Meta-review (summarises recurring review criticisms and appends them to the other agents' prompts). A Supervisor agent schedules them on a task queue. Scientists can supply ideas, reviews and feedback through a chat interface. - [Towards end-to-end automation of AI research](https://agents-in-research.pages.dev/papers/lu-2026-end-to-end-ai-research/index.md): A team from Sakana AI, the University of British Columbia, the Vector Institute and Oxford (FLAIR) describes The AI Scientist, a pipeline of language-model agents that proposes a machine-learning research idea, checks it against the literature (Semantic Scholar), writes and runs the experiments, plots the results, writes a LaTeX paper and then has it reviewed by a second component, The Automated Reviewer. It only does computational experiments in machine learning. There are two modes: a template-based one that extends human-supplied starter code, and a template-free one that writes its own code through a parallel tree search (Methods). The template-free version used OpenAI o3 for ideas and code critique, Claude Sonnet 4 for code, GPT-4o for figures and best-first node selection, and o4-mini for review. A full template-free run takes "from several hours to over 15 h".