library scienceblog

Add a note

Aaron Tay

What they did

Tay forked an open-source Primo MCP server so an agent could search his institution's library catalogue, then took selected 2023 searches that had returned zero results and ran them through 2026 models. He compared the agent loop with the vendor's own Natural Language Search, which turns a query into one Boolean string. The agent recovered from misspellings (for example reading "statis" as the database Statista) and widened over-specific queries, where the one-shot expansion mostly did not. A weaker model did about as well, which he attributes to the search loop.

In their words

"I was vibe coding almost the whole way: run it, look at what broke, ask Codex or Claude to fix it."

"to see how far one-shot query expansion gets without a loop. The answer is: not far."

"nothing in my current setup forces the final answer to contain only fields returned by Primo"

Lessons they draw

  • The gain came from the iterative search loop rather than model tier (Tay).
  • Grounding in tool output holds at retrieval, not in the final answer; generated links should be validated (Tay).
  • The test is exploratory: non-random 2023 queries replayed on 2026 models (Tay).

Limits of this account

First post of a series; read in the opening sections only. The coding was done with the agents and the research object is a search system, so part of the account is software building. The page names no Claude model. Quotes checked against the fetched page text on 2026-11-05.

Search the collection

↑ ↓ move · Enter open · Shift+Enter original source · Esc close · Search index for agents

Keyboard shortcuts

Single keys work when you are not typing and no dialog is open. Turn them off if stray key presses move the page. ⌘K / Ctrl+K and Esc keep working.

/
Search the collection
⌘K / Ctrl+K
Open or close search
?
Show these shortcuts
h
Home
l
Lessons
a
Accounts
r
Replications
p
Papers
g
Guides and tools
d
Timeline
b
Bookmark the record you are reading (press again to remove it)
s
Your saved list and notes
← →
Previous and next record on an account, lesson, replication or paper page, and in a quick look
m
Switch between system, light and dark theme
t
Back to the top of the page
Open full page