Lesson 4 of 6

Add a note

Start with a known result and keep an independent check

Try the method first on a result you already know. Before moving on to new work, decide how you will check an answer when there is no published result to compare it with. Where possible, make that check independent of the code that produces the answer.

Matthew Schwartz started the work that became BootLoops by asking Claude to implement methods from his papers and related work. He reports that it reproduced a result from one of his papers before he asked it to look for unsolved calculations. For the new calculations, the method let him compare exact formulas with high-precision numerical evaluations.

In Terence Tao’s formalisation of the proof of Sendov’s conjecture, Python code generated certificates and Lean checked them. Lean can reject a faulty certificate without relying on the generator to detect the fault. The project also tested the check itself by deliberately altering certificates. Part of that test turned out to be weaker than expected.

Francis DiTraglia used Stata’s ivreg2 as the reference for his R port. The R tests read results that Stata had exported to CSV and compared coefficients, covariance matrices and diagnostics. The expected results came from Stata, so the same checks could be rerun after changes to the R code. Validation guide

Search the collection

↑ ↓ move · Enter open · Shift+Enter original source · Esc close · Search index for agents

Keyboard shortcuts

Single keys work when you are not typing and no dialog is open. Turn them off if stray key presses move the page. ⌘K / Ctrl+K and Esc keep working.

/
Search the collection
⌘K / Ctrl+K
Open or close search
?
Show these shortcuts
h
Home
l
Lessons
a
Accounts
r
Replications
p
Papers
g
Guides and tools
d
Timeline
b
Bookmark the record you are reading (press again to remove it)
s
Your saved list and notes
← →
Previous and next record on an account, lesson, replication or paper page, and in a quick look
m
Switch between system, light and dark theme
t
Back to the top of the page
Open full page