Research Nov 12, 2026 · Originally published Nov 11

Hugging Face shows pooling-aware training: multi-vector retrieval stores 74.8% fewer document vectors while keeping 97.8% of its NanoBEIR score, up from 94.6%

Multi-vector models such as ColBERT retrieve well but store a lot. How training the encoder for token pooling buys better retrieval at the same storage budget.

Image: Hugging Face blog
Primary source Hugging Face blog · huggingface.co Read the original ↗

// Key points

  • Multi-vector models keep one vector per token, so a corpus of millions of documents can hold hundreds of millions of vectors; token pooling merges similar vectors to shrink storage.
  • At pool_factor=4, a normally trained model retains 94.6% of its unpooled nDCG@10, and pooling-aware training raises that to 97.8%, with nDCG@10 rising from 49.03 to 51.23 while storing 74.8% fewer document vectors.
  • With adaptive pooling at an average of 29.58 vectors per document (84.7% fewer), the pooling-aware model retains 97.3% versus 93.3% for the normally trained one.
  • The method is implemented as a loss wrapper in Sentence Transformers, with a short reproducible training example.

Builder's takeIf AI Cloud Drive's semantic search moves to multi-vector models, storage cost is the main worry, and this post gives a reproducible trade-off. I would estimate vector counts and storage cost at pool_factor=4 first, then check recall on our own documents rather than relying on NanoBEIR.

Every Monday, the week in AI in one email

Every Monday morning: the 6 AI stories that mattered most in the past seven days, with my take, in your inbox.

Your email is only used to send the weekly and is never shared. Every email has a one-click unsubscribe link. See the latest issue →