Lesson 5 of 6
Keep enough of the work to resume, audit and undo it
Keep the original material, the last usable output and a record of what failed. If a long run stops or a correction goes wrong, you need to know what you can reuse and what you have to do again. A saved decision can also keep a new session from repeating a failed approach.
In my radio archive project, records with invalid source references were held back. The recovery run left out claims with invalid references and rebuilt summaries from the evidence that remained. We kept the original failures and saved editorial corrections separately from the model output.
Thomas D. Grant starts each project with a set of files that have separate jobs: one holds what was agreed, one the open questions, and one the sources found and what has been checked in each. His slides say each file answers a different question when the project resumes. He saves corrections to the model’s prose as rules with examples, and he split the rules file when it grew to 44 rules.
John H Boyer’s instruction files for his Greek readers keep a dated list of failures and what each taught. They also keep hand-over notes per track of work, after one session overwrote another’s notes in a shared file.
The write-up of Terence Tao’s Lean formalisation records dead ends, including times the agent inserted proof placeholders against the project’s rules. Chris Blattman saves corrections that should affect future work in the instructions or examples the agent will use next time. He also removes old rules when they start getting in the way.