Lesson 2 of 6
Build the method and its checks together
An agent can help build the tools a research method needs. Decide while building how you will tell whether the method works. For a new search, compare what it returns with existing searches. For a simulation you want to repeat, review the setup before reusing it on new inputs.
Aaron Tay built a way for an agent to search a library catalogue repeatedly, changing the query when it got nowhere. On selected old searches that had returned zero results, he compared that loop with the catalogue’s single-pass query expansion. The loop recovered some searches by fixing spelling or broadening the query. Tay also found that he still needed to validate links in the final answer. His examples came from a selected set of catalogue failures.
Ke Xiao and colleagues used Codex to help prepare a package of simulation settings, meshing procedures and analysis scripts. They kept review records with the package, then reused it on 140 versions of an injector design. This is a report on one engineering study.
In an earlier project, Joshua Sternfeld compared his agent-built newspaper search with the collection’s own website. The agent had set a limit on how much of each page it searched, so the new tool missed matches that the website found.