Whoosh vs Tantivy (tantivy-py): which full-text search for Python?

← All posts · Live demo · GitHub · PyPI

Short answer: If raw indexing and query throughput on large corpora is your priority — and you're happy to depend on a compiled Rust engine — reach for Tantivy via its Python bindings, tantivy-py. If you want a search engine you can read, extend, and debug entirely in Python, that installs with a plain pip install on any platform with no compiler and no native code, and that runs on PyPy and even in the browser via WebAssembly, choose Whoosh. They're both good; they optimise for different things.

Once you've decided you don't need a separate service like Elasticsearch or OpenSearch, the two most common embeddable, in-process options in the Python world are Whoosh and Tantivy (through tantivy-py). This is a practical guide to choosing between them.

The one-line difference

Whoosh is a full-text search library written entirely in Python. The whole engine — analysis, indexing, scoring, query parsing — is Python you can step through in a debugger.
Tantivy is a full-text search engine written in Rust, inspired by Apache Lucene. tantivy-py is a thin Python binding over that Rust library, so the heavy lifting happens in compiled native code.

At a glance

	Whoosh (`whoosh3`)	Tantivy (`tantivy-py`)
Implementation	Pure Python	Rust core + Python bindings
Install	`pip install whoosh3` — pure wheel, works everywhere CPython runs	`pip install tantivy` — prebuilt wheels for common platforms; falls back to building from source (needs a Rust toolchain) when no wheel matches
Runtime deps	None	Compiled extension module
Raw speed	Good for thousands to low-millions of docs; it's Python, so it won't match native code	Very fast; built for high indexing and query throughput on large corpora
Ranking	BM25F (default), TF-IDF, pluggable in Python	BM25
Query language	Rich built-in parser: boolean, phrase, range, wildcard, fuzzy, fields	Query parser plus programmatic query building
Analysis / stemming	Composable analyzers, stemming, stop words, n-grams — all in Python and easy to customise	Configurable tokenizers/filters exposed by the Rust engine
Highlighting	Yes, built in	Snippet generation available
Faceting / grouping	Yes	Yes (aggregations)
Spelling / "did you mean?"	Yes, built in	Not a built-in library feature
PyPy	Runs (pure Python)	Depends on the compiled extension working under PyPy
Browser / WebAssembly (Pyodide)	Yes — Whoosh runs in the browser; there's a live demo	Not a typical target for the bindings
Best fit	Embedded search you program in Python; portability matters more than peak speed	Large corpora and high query volume where native performance matters

On benchmarks — an honest note. I'm not going to hand you a made-up head-to-head number. Tantivy is a Rust engine and, for large indexes and heavy query loads, it will comfortably out-perform any pure-Python engine, including Whoosh — that's the expected outcome and it's the whole point of binding to native code. If throughput at scale is your deciding factor, that answer is already clear. What Whoosh trades that speed for is portability and programmability, which is what the rest of this page is about. If you want a reproducible Whoosh baseline on your own data, the repo ships examples/benchmark_vs_sqlite.py you can adapt.

Where Whoosh wins

Zero-friction install, everywhere. A pure-Python wheel has no ABI, no platform matrix, and no build step. If a prebuilt Tantivy wheel doesn't exist for your OS/architecture/Python combination, pip will try to compile from source, which needs a Rust toolchain installed — a real friction point in locked-down CI, minimal containers, and unusual platforms. Whoosh sidesteps that entirely.
You can actually read and change it. Custom scoring, a bespoke analyzer, a new storage backend — you do all of that in Python, set a breakpoint, and see exactly what happens. With a Rust core, customising internals means writing Rust.
Exotic runtimes. PyPy and the browser (via Pyodide/ WebAssembly) are first-class for Whoosh. The browser demo on this site is the whole search engine running client-side.
Batteries included. Built-in spelling correction and highlighting mean fewer moving parts to assemble yourself.

Where Tantivy wins

Speed and scale. Native code plus a Lucene-style design means fast indexing and low-latency queries over large document sets.
Throughput under load. If you're serving many queries per second or reindexing millions of documents, the Rust engine has real headroom that a pure-Python one does not.
Backed by a search-focused team. Tantivy comes out of the Quickwit ecosystem and is used as the core of larger search systems.

Same task, both libraries

Indexing two documents and running a query looks broadly similar in each. Whoosh:

from whoosh.fields import Schema, TEXT, ID
from whoosh.index import create_in
from whoosh.qparser import QueryParser
import os, tempfile

d = tempfile.mkdtemp()
schema = Schema(title=TEXT(stored=True), body=TEXT)
ix = create_in(d, schema)

w = ix.writer()
w.add_document(title="First", body="the quick brown fox")
w.add_document(title="Second", body="a lazy brown dog")
w.commit()

with ix.searcher() as s:
    q = QueryParser("body", ix.schema).parse("brown")
    for hit in s.search(q):
        print(hit["title"])

Tantivy (tantivy-py), sketched from its documented API:

import tantivy

builder = tantivy.SchemaBuilder()
builder.add_text_field("title", stored=True)
builder.add_text_field("body", stored=False)
schema = builder.build()

index = tantivy.Index(schema)
writer = index.writer()
writer.add_document(tantivy.Document(title="First", body="the quick brown fox"))
writer.add_document(tantivy.Document(title="Second", body="a lazy brown dog"))
writer.commit()

index.reload()
searcher = index.searcher()
query = index.parse_query("brown", ["body"])
for _score, addr in searcher.search(query, 10).hits:
    print(searcher.doc(addr)["title"][0])

How to choose

Pick Tantivy if:

Your corpus is large (many millions of documents) or your query volume is high.
Peak indexing/search performance is the deciding factor.
A compiled dependency and building from source when needed are acceptable in your environment.

Pick Whoosh if:

You want a dependency-free pip install that works on any platform, in minimal containers, and in locked-down CI.
You want to customise scoring, analysis, or storage in plain Python.
You need PyPy or in-browser (WebAssembly) support.
Your corpus is in the thousands-to-low-millions range — comfortably within Whoosh's sweet spot — and you value programmability over raw speed.

It isn't strictly either/or. Plenty of projects prototype and ship with Whoosh for its simplicity, and only reach for a native engine if and when scale actually demands it.

Whoosh is maintained again. It's pure Python, zero runtime dependencies, BSD-2-Clause, with a live browser demo and modern packaging.

⭐ Star Whoosh on GitHub · Try the live demo · pip install whoosh3