Cookbook

This page collects short, runnable recipes for common search tasks. Each one maps to a self-contained script in the examples/ directory of the repository, so you can run it end to end:

git clone https://github.com/priya-sundaram-dev/whoosh
cd whoosh
pip install -e .
python examples/quickstart.py

All recipes use only the standard library plus Whoosh itself — there are no extra runtime dependencies.

Quick start

examples/quickstart.py — the shortest path from an empty directory to a working search: define a Schema, add a couple of documents, parse a user query with QueryParser, and print results with keyword highlighting.

A guided tour

examples/tutorial.py — a longer, commented walk-through that builds a small product catalogue in an in-memory index. It covers:

  • an in-memory index with RamStorage

  • upserting records with writer.update_document

  • single-field and multi-field parsing (QueryParser / MultifieldParser)

  • combining a parsed query with an exact-term filter

  • sorting results by a numeric field

  • grouping results with a FieldFacet

  • highlighting matched terms in stored text

The prose version of the same material lives in TUTORIAL.md at the repo root.

“Did you mean …?” spelling correction

examples/did_you_mean.py — Whoosh has spelling correction built in, with no external dependencies. The recipe shows both single-word suggestions via searcher.suggest and whole-query correction via searcher.correct_query, and prints both a plain-text and an HTML “did you mean” prompt. See also “Did you mean… ?” Correcting errors in user queries.

Autocomplete / search-as-you-type

examples/autocomplete.py — three pure-Python approaches to search-as-you-type:

Pick the one that fits your latency and index-size budget.

Whoosh vs. SQLite FTS5

examples/benchmark_vs_sqlite.py — an honest, reproducible micro-benchmark comparing build time, on-disk size, and average query latency against SQLite’s FTS5 extension over the same corpus. Use it to decide when Whoosh’s pure-Python, zero-dependency, deeply programmable model is the right trade-off for your project, and when an embedded C engine is a better fit.

Faceted navigation (filter sidebar with counts)

examples/faceted_search.py — the pattern behind the “filter sidebar” on almost every shopping or catalogue site. Alongside the results you show each facet (brand, category, price band…) with a count of how many matching documents fall into each bucket, and clicking a bucket narrows the result set.

Whoosh does this natively: pass a facet — or a dict of them — as the groupedby argument to Searcher.search and read the per-bucket counts from Results.groups(). The counts come from the same search call that produces your results, so they always reflect the current query. The recipe covers:

  • FieldFacet for single-valued fields

  • FieldFacet with allow_overlap=True for multi-valued KEYWORD fields

  • RangeFacet for numeric buckets

  • “drill down” by AND-ing a chosen facet value onto the current query

See also Sorting and faceting for the full faceting reference.

Highlighting and snippets

examples/highlighting.py — turn raw matches into the “keyword in context” snippets you see on a real search-results page. Call Hit.highlights(fieldname) and Whoosh finds the best-scoring passages, trims them to a readable length, and wraps each matched term in markup. The recipe covers:

  • the one-liner: hit.highlights("body") off a stored field

  • choosing where snippets are cut — ContextFragmenter (a window around each match) vs SentenceFragmenter (whole sentences)

  • choosing how matches are marked — HtmlFormatter with your own tag and CSS class, or UppercaseFormatter for plain text

  • fast “pinpoint” highlighting: index the field with chars=True and use PinpointFragmenter so long documents are highlighted without being re-tokenized

  • highlighting a field you did not store, by passing the original text to hit.highlights("body", text=...)

See also How to create highlighted search result excerpts for the full highlighting reference.

Custom analyzers (build your own text pipeline)

examples/custom_analyzers.py — the feature that sets Whoosh apart: instead of a fixed set of “language modes”, you compose your own text-processing pipeline from a tokenizer and a chain of filters using the | operator:

from whoosh.analysis import RegexTokenizer, LowercaseFilter, StopFilter
analyzer = RegexTokenizer() | LowercaseFilter() | StopFilter()

The first item must be a tokenizer; everything after it is a filter. Attach the analyzer to a field (TEXT(analyzer=analyzer)) and Whoosh runs the same pipeline at index time and query time, so the two always agree. The recipe covers:

  • watching a pipeline take shape one stage at a time — tokenize, lowercase, drop stop words, then stem with StemFilter

  • accent folding with CharsetFilter and the bundled accent_map so cafe matches café

  • normalising tokens with SubstitutionFilter so wi-fi, wi_fi and wifi collapse to one term

  • character NgramFilter for substring / “matches anywhere” search

  • wiring a custom analyzer onto a field and confirming, with a real index, that run finds running/runner/ran and ZURICH finds Zürich

See also About analyzers for the full analysis reference.

Custom scoring & sorting (control the ranking)

examples/scoring_and_sorting.py — ranking is where a search library earns its keep. Whoosh gives you several independent levers, and this recipe runs each one against a real index so you can see the ranking change:

  • tuning the default BM25F model — B controls document-length normalisation and K1 controls term-frequency saturation; per-field values use a <field>_B keyword (for example BM25F(B=0.75, body_B=0.2))

  • swapping the model entirely for TF_IDF or Frequency

  • mixing models per field with MultiWeighting (for example TF_IDF for titles, BM25F everywhere else)

  • scoring with your own function via FunctionWeighting, which receives (searcher, fieldname, text, matcher) and returns a float — ideal for experiments and business rules

  • skipping relevance altogether and sorting by a stored, sortable field with search(q, sortedby="views", reverse=True) — faster than scoring and often exactly what “newest first” / “most viewed” UIs need

Pass any weighting model to the searcher:

from whoosh import scoring
with ix.searcher(weighting=scoring.BM25F(B=0.0, K1=2.0)) as s:
    results = s.search(q)

See also scoring module and Sorting and faceting for the full reference.

Closing indexes cleanly (and avoiding Windows file-lock errors)

Whoosh keeps an index’s on-disk files open while a reader or searcher is alive, so it can answer queries without re-opening files each time. If you let those objects be cleaned up by the garbage collector instead of closing them, the files stay open until the object is actually collected.

On POSIX systems that is usually harmless. On Windows, an open file handle prevents the file from being deleted or replaced, so deleting or rebuilding an index while a reader is still open surfaces as PermissionError: [WinError 32] The process cannot access the file because it is being used by another process. The robust fix is not to sprinkle gc.collect() calls around — it is to close what you open.

Every reader and searcher is a context manager, so a with block releases the handles deterministically as soon as the block exits, even on error:

from whoosh.qparser import QueryParser

qp = QueryParser("body", ix.schema)
q = qp.parse("pure AND search")

with ix.searcher() as searcher:          # searcher closes on exit
    results = searcher.search(q, limit=10)
    titles = [hit["title"] for hit in results]

with ix.reader() as reader:              # readers are context managers too
    total = reader.doc_count()

When you are completely finished with an index object, call ix.close() to release any cached readers it is holding on your behalf:

ix.close()

After everything is closed, the index directory can be deleted or rebuilt immediately — including on Windows — with no gc.collect() workaround.

If you use AsyncWriter, remember that its background thread must finish (via commit()) before the segment’s files are released. Track any writers you create and join them before tearing down the index.

A complete, runnable version of this pattern lives in examples/resource_management.py.

A command-line folder search tool

examples/search_cli.py — a tiny, dependency-free command-line program that indexes a folder of text, Markdown, reStructuredText, or source files and lets you search it straight from your terminal. No server, no external service:

# Index the current directory (creates ./.whoosh_index/)
python examples/search_cli.py index .

# Search it, with highlighted snippets
python examples/search_cli.py search "full text search"

# Re-index only changed/new files and drop deleted ones (fast; uses mtimes)
python examples/search_cli.py index . --update

# Choose which extensions to index, or emit HTML <mark> highlights
python examples/search_cli.py index ~/notes --ext .md,.txt
python examples/search_cli.py search "ranking" --html

It demonstrates several everyday patterns in one place: a Schema with a unique ID path, writer.update_document for idempotent upserts, writer.delete_by_term to prune deleted files, incremental indexing driven by a stored NUMERIC mtime, field-boosted MultifieldParser queries, and result highlighting with ContextFragmenter. It is a single file you can copy into your own project and adapt.

A full-text search API with FastAPI

examples/fastapi_app.py — a small, production-shaped REST API that adds full-text search to a web service. It exposes PUT /documents/{id} (an idempotent upsert), DELETE /documents/{id}, and GET /search with pagination and highlighted snippets:

pip install "whoosh3" fastapi "uvicorn[standard]"
uvicorn fastapi_app:app --reload

curl -X PUT localhost:8000/documents/1 \
    -H 'content-type: application/json' \
    -d '{"title": "Getting started with Whoosh", "body": "pure-python search"}'

curl 'localhost:8000/search?q=python&page=1&page_size=10'

The search logic lives in a small, framework-free SearchIndex class so it is easy to unit-test without an HTTP server (run python fastapi_app.py for a self-contained demo). It shows the pattern you actually need in a service: a persistent on-disk index opened once at startup and closed at shutdown (so file handles are released — important on Windows), writer.update_document upserts keyed on a unique ID, BM25F ranking, searcher.search_page for pagination, and highlighted snippets via HtmlFormatter. See Adding full-text search to your Python app for the broader “adding search to your app” guide, including a Django variant.

Migrating from Whoosh 2.x / whoosh-reloaded

Already using the original Whoosh or Whoosh-Reloaded? The MIGRATING.md guide at the repo root explains what changed: the import package is still whoosh, the on-disk index format is unchanged, and the public API is the same. In most cases the only change you make is the package you install.