Ai2 tests LLMs as judges of research novelty: rewording the prompt drops GPT-5.4 from 93.5% to 40.9% accuracy
A study of how fragile LLM-as-judge setups are, released with a benchmark, data and code.
// Key points
- Data comes from ICLR 2026 reviews: 154 high-novelty and 145 lower-novelty human ideas, plus the 154 high-novelty ideas paired with 154 from a simple LLM generator.
- On human-versus-generated pairs, rewording the prompt alone moved GPT-5.4 from 93.5% to 40.9% accuracy, below chance; on the human-only set, Claude Opus 4.6 shifted by 21 points.
- A simple baseline (Claude Opus 4.6, low reasoning, majority vote over three calls) scored 0.854 macro-F1 at $2.59, versus 0.642 at $30.66 for Idea Novelty Checker and 0.772 at $82.05 for the Sakana AI Scientist evaluator.
- The benchmark, data (allenai/novelty-judge-bench) and code (allenai/novelty-eval) are public; the post does not state a license.
Builder's takeThis is a warning for any product that has a model assign scores: interview scoring or automated content-quality review can swing when the prompt is merely reworded. I treat scoring prompts as code and rerun a fixed human-labeled set on every change. Cheap majority voting over a few calls is often steadier and cheaper than an elaborate agentic judge.