Whoosh for RAG: BM25 keyword retrieval & hybrid search in pure Python

Live demo · All guides · GitHub · Docs

TL;DR. Vector search alone misses exact terms — names, IDs, error codes, acronyms, rare keywords. Add a lexical BM25 retriever alongside your embeddings and fuse the two rankings. Whoosh gives you production-grade BM25 in pure Python: no server, no native wheels, the index is just files. This guide shows a complete, runnable hybrid retriever using Whoosh + Reciprocal Rank Fusion (RRF) — with any vector store you already use.

Why keyword search still matters for RAG

Retrieval-Augmented Generation lives or dies on retrieval. If the right chunk isn't in the context, the model can't use it. Dense (vector) retrieval is great at meaning — "car" matches "automobile" — but it has a well-known blind spot: it can quietly fail on the exact tokens that matter most.

Rare / out-of-vocabulary terms: product SKUs, function names, error codes (ERR_2043), gene symbols, ticket IDs.
Proper nouns and acronyms the embedding model never saw often enough to place well in vector space.
Exact-match intent: when a user pastes a literal phrase, they usually want that phrase, not its nearest neighbor.

Lexical BM25 is the classic complement: it rewards documents that contain the query's actual terms, weighted by rarity and length. Combining lexical and dense retrieval — hybrid search — reliably beats either one alone on real corpora. The two methods fail on different queries, so fusing them recovers the misses.

Whoosh is a lexical engine — it does not compute embeddings. That's exactly the point: it handles the keyword half so you can pair it with whatever vector store you like (FAISS, Chroma, pgvector, a hosted API). No GPU, no server, no native dependencies — pip install whoosh3 and the index is a folder on disk.

1. Index your chunks with Whoosh

In RAG you retrieve chunks, not whole documents. Store a stable id for each chunk (so you can join back to your vector store) plus the chunk text. Using an OrGroup parser keeps recall high — a chunk matches if it contains any of the query terms, and BM25 still ranks the best-matching chunks first.

from whoosh.index import create_in
from whoosh.fields import Schema, TEXT, ID
from whoosh.qparser import MultifieldParser, OrGroup
from whoosh import scoring
import os

# 1. Define a schema: a stable chunk id + the searchable text.
schema = Schema(
    id=ID(stored=True, unique=True),
    text=TEXT(stored=True),
)

os.makedirs("rag_index", exist_ok=True)
ix = create_in("rag_index", schema)

# 2. Add your chunks. `id` should match the key in your vector store.
chunks = [
    ("c1", "The mitochondria is the powerhouse of the cell and produces ATP energy."),
    ("c2", "Python is a popular programming language for data science and ML."),
    ("c3", "Cellular respiration converts glucose into usable energy in the mitochondria."),
    ("c4", "Vector databases store embeddings for semantic similarity search."),
]
writer = ix.writer()
for chunk_id, text in chunks:
    writer.add_document(id=chunk_id, text=text)
writer.commit()

2. A BM25 keyword retriever

Wrap the search in a function that returns a ranked list of chunk ids — the same shape your vector retriever returns, so they're easy to fuse.

def keyword_search(query, k=10):
    """Return up to k chunk ids ranked by BM25 relevance."""
    with ix.searcher(weighting=scoring.BM25F()) as s:
        parser = MultifieldParser(["text"], schema=ix.schema, group=OrGroup)
        q = parser.parse(query)
        hits = s.search(q, limit=k)
        return [hit["id"] for hit in hits]

print(keyword_search("energy in the mitochondria"))
# -> ['c1', 'c3']

BM25F() is Whoosh's default, well-tuned ranking function. Because we parse with OrGroup, a query like "energy mitochondria cell" retrieves every chunk that mentions any of those words, ordered so the most on-topic chunk comes first.

3. Fuse lexical + vector results with RRF

Reciprocal Rank Fusion is the simplest robust way to merge two ranked lists. It ignores raw scores (which live on incompatible scales — BM25 vs. cosine similarity) and uses only rank position, so you never have to normalize. Each result contributes 1 / (k + rank) to its document's fused score; documents that rank well in both lists rise to the top.

def reciprocal_rank_fusion(rankings, k=60):
    """Fuse several ranked id-lists into one. `k` is the RRF constant (60 is standard)."""
    scores = {}
    for ranking in rankings:
        for rank, doc_id in enumerate(ranking):
            scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank + 1)
    return sorted(scores, key=scores.get, reverse=True)


def hybrid_search(query, vector_search, k=10):
    """Combine Whoosh BM25 with your existing vector retriever."""
    lexical = keyword_search(query, k=k)      # ['c1', 'c3', ...]
    dense = vector_search(query, k=k)         # your embeddings, same shape
    return reciprocal_rank_fusion([lexical, dense])[:k]

vector_search is whatever you already have — a FAISS index, a Chroma collection, pgvector, a hosted embeddings API. It just needs to take a query and return a list of chunk ids. The fused ids are your final context; look the text back up (from Whoosh's stored fields or your own store) and pass it to the LLM.

4. Putting it together

# A stand-in for your real embedding retriever.
def vector_search(query, k=10):
    # In production: embed(query), search FAISS/Chroma/pgvector, return chunk ids.
    return ["c3", "c1", "c4"]

final_ids = hybrid_search("energy mitochondria", vector_search)
print(final_ids)  # -> ['c1', 'c3', 'c4']

# Fetch chunk text back for the LLM context window:
with ix.searcher() as s:
    id_to_text = {}
    for cid in final_ids:
        doc = s.document(id=cid)
        if doc:
            id_to_text[cid] = doc["text"]
context = "\n\n".join(id_to_text[cid] for cid in final_ids if cid in id_to_text)

▶ Try Whoosh live in your browser — it runs the real library via WebAssembly, builds an index, and ranks results with BM25, entirely client-side. Then star it on GitHub if it saves you a dependency — that's the main signal that keeps this revival going.

When hybrid is (and isn't) worth it

Situation	Recommendation
Corpus full of names, codes, IDs, jargon	Hybrid — lexical catches exact terms dense retrieval fumbles.
Users paste literal phrases / error messages	Hybrid, or even lexical-first.
Purely conceptual, paraphrased queries	Vector may suffice; measure before adding complexity.
You want zero infra for the keyword side	Whoosh — pure Python, index is just files, no server.

A practical rule: start with one retriever, build an evaluation set of real queries with known-good chunks, and only add the second retriever if it measurably improves recall. RRF makes adding Whoosh cheap — a few lines and no score tuning.

Why Whoosh for the keyword half

Pure Python. No compiler, no native wheels, no server process. Deploys anywhere CPython runs, including serverless and notebooks.
Embedded. The index is a directory of files — trivial to ship alongside your app or rebuild in CI.
Real BM25F ranking, plus phrase, boolean, range, and fielded queries when you need more control than a bag of terms.
Actively maintained again. pip install whoosh3 installs the current release with Python 3.9–3.14 support.

Install: pip install whoosh3 · Docs: whoosh docs · Related: Whoosh as a LangChain retriever, autocomplete & prefix search, Whoosh vs Elasticsearch.

Every code block above was run against whoosh3 3.16.4 before publishing. "Whoosh" is the pure-Python search library originally by Matt Chaput; this is an actively maintained continuation. RRF is a standard, model-agnostic fusion method — the vector half is left to whatever embedding stack you prefer.