Highlighted search-result snippets with Whoosh

A search box isn't finished when it returns the right documents — users still have to see why each result matched. The familiar answer is the snippet: a short excerpt from the document with the searched words highlighted, like the grey text under every Google result. Whoosh ships this out of the box in its whoosh.highlight module, so you don't need a separate library or an Elasticsearch highlight block.

pip install whoosh3

1. Index some text worth highlighting

Highlighting needs the original text of the field. The simplest path is to store the field (stored=True) so Whoosh can read it straight back from each hit. We'll also index a title and a unique id.

import os, shutil
from whoosh import index
from whoosh.fields import Schema, TEXT, ID
from whoosh.qparser import QueryParser
from whoosh import highlight

d = "hl_index"
shutil.rmtree(d, ignore_errors=True); os.mkdir(d)

schema = Schema(
    id=ID(stored=True, unique=True),
    title=TEXT(stored=True),
    body=TEXT(stored=True),          # stored so we can highlight it
)
ix = index.create_in(d, schema)

w = ix.writer()
w.add_document(id="1", title="Getting started with Whoosh",
    body="Whoosh is a fast, pure-Python full-text search library. "
         "You can build a complete search engine in Python without running any external service. "
         "It supports BM25 ranking, faceting, spelling correction, and a rich query parser.")
w.add_document(id="2", title="Fast indexing",
    body="Batch your documents and commit once so indexing stays fast. "
         "Whoosh writes segments to disk and merges them in the background.")
w.commit()

2. Get a highlighted snippet for every hit

Search as usual, but pass terms=True so Whoosh remembers which query terms matched — that's what lets it highlight them. Then call hit.highlights("body") on each result.

with ix.searcher() as s:
    q = QueryParser("body", ix.schema).parse("python search engine")
    results = s.search(q, terms=True)
    for hit in results:
        print(hit["title"])
        print(hit.highlights("body"))
        print("-")

Verbatim output:

Getting started with Whoosh
is a fast, pure-<b class="match term0">Python</b> full-text <b class="match term1">search</b> library. You can build...a complete <b class="match term1">search</b> <b class="match term2">engine</b> in <b class="match term0">Python</b> without running
-

Whoosh chose the most relevant window of text, stitched two fragments together with an ellipsis, and wrapped each matched term in a <b> tag. The term0/term1 classes let you colour different query words differently if you like. Rendered, that snippet looks like this:

3. Use <mark> tags and control the excerpt

Two knobs shape the output: the formatter (how matches are wrapped) and the fragmenter (how the excerpt is chosen and how long it is). Set them on the results object before calling highlights(). Here we emit semantic <mark> tags and widen the excerpt with a ContextFragmenter.

with ix.searcher() as s:
    q = QueryParser("body", ix.schema).parse("python search engine")
    results = s.search(q, terms=True)

    # how the excerpt is chosen: up to 120 chars, ~40 chars around each match
    results.fragmenter = highlight.ContextFragmenter(maxchars=120, surround=40)
    # how matches are wrapped:  ...
    results.formatter = highlight.HtmlFormatter(tagname="mark", classname="match")

    hit = results[0]
    print(hit.highlights("body", top=2))   # up to 2 fragments

Verbatim output:

Whoosh is a fast, pure-<mark class="match term0">Python</mark> full-text <mark class="match term1">search</mark> library. You can build a complete <mark class="match term1">search</mark> <mark class="match term2">engine</mark> in <mark class="match term0">Python</mark> without running any external service. It

Style it with one CSS rule — mark.match { background:#fff2a8; } — and you have a production snippet. Useful fragmenters:

4. Highlight a field you didn't store

Storing large bodies bloats the index. If body is not stored=True, Whoosh can't read the text back on its own — but you can hand it the text with the text= argument (fetch it from your database, files, or a pandas DataFrame keyed by the stored id).

with ix.searcher() as s:
    q = QueryParser("body", ix.schema).parse("python search")
    results = s.search(q, terms=True)
    hit = results[0]
    original_text = load_body(hit["id"])   # your own lookup by id
    print(hit.highlights("body", text=original_text))

Verbatim output (passing the text back in):

is a fast, pure-<b class="match term0">Python</b> full-text <b class="match term1">search</b> library. You can build...a complete <b class="match term1">search</b> engine in <b class="match term0">Python</b> without running

Gotchas

• No highlights come back? Make sure you searched with terms=True — without it Whoosh doesn't track which terms matched, and highlights() returns an empty string.
• Empty string for a real hit? The field must be stored=True or you must pass text=. Highlighting needs the source text.
• Set the formatter/fragmenter on results, not on each hit — the hit reads them from its parent results object.
• Escape or trust? HtmlFormatter escapes the surrounding text for you, so the snippet is safe to drop into a page.

The same highlights() call works inside a Flask or FastAPI endpoint and pairs naturally with faceted search. For tokenisation that affects what counts as a "term", see the analyzer recipes.