# Tom Pepinsky

- Researcher: Tom Pepinsky
- Role: Political scientist; blog on research practice that mentions papers seen at the Journal of East Asian Studies; institution not stated on the page
- Field: political science
- Tools: Claude Code
- Published: 2026-01-23
- Source: https://tompepinsky.com/2026/01/23/agentic-ai-and-social-science-research-practice/
- Older account (before March 2026). Latest flagship models when written: Claude Opus 4.5 and GPT-5.2. Latest now: Claude Opus 5.5 and GPT-6.1 Sol. Agent tools have also changed since, so check whether a limitation reported here still applies.

## What they did

Pepinsky, previously a critic focused on what AI cannot do, reports starting to use Claude Code after seeing Andrew Hall, Scott Cunningham and Andrew Little, and calls it a game-changer for social scientists. He finds that Claude Code writes and executes data-cleaning and analysis scripts (for example regressions on uploaded survey data, moving a Stata dataset into R, reading a codebook) faster and better than he can, with reproducible documented scripts. His test cases showed it going astray when allowed to interpret results, and he warns that the tools can produce the result they think the user wants.

## In their words

> "Upload a CSV file containing survey data to Claude Code and ask it to run a regression analysis and plot the quantities of interest, and it works."

> "In several test cases, Claude Code seemed to be guessing what I wanted to hear, interpreting basic regression results incorrectly but provocatively."

> "use agentic AI for tasks that involve following rules. Do not use agentic AI for tasks that generate answers, arguments, or interpretations."

## Lessons they draw

- Use agentic AI for rule-following tasks (cleaning, scripts, visualisation), not for answers, arguments or interpretation (Pepinsky).
- Citation hallucination remains a problem for LLM literature claims, whereas code can be verified by executing it (Pepinsky).
- Risk of p-hacking: the tool may generate what it "thinks" you want, and if execution is handed over the author may not know what was chosen or left out (Pepinsky).
- As coding costs fall, the relative value of reading, interpreting and understanding rises; he reports seeing AI-written submissions with interpretation problems (Pepinsky).

## Limits of this account

Early-use impression (about one month with the tool) with test cases described rather than documented; the interpretation claim rests on his stated hunch.
