tinysearch is a lightweight, fast, full-text search engine. It is designed for static websites.
tinysearch is written in Rust, and then compiled to WebAssembly to run in a
browser.
It can be used together with static site generators such as
Jekyll, Hugo,
Zola,
Cobalt, or
Pelican.
The test index file of my blog with around 40 posts creates a WASM payload of
99kB (49kB gzipped, 40kB brotli).
That is smaller than the demo image above; so yes.
tinysearch is a Rust/WASM port of the Python code from the article "Writing a full-text search engine using Bloom filters". It can be seen as an alternative to lunr.js and elasticlunr, which are too heavy for smaller websites and load a lot of JavaScript.
Under the hood it uses a Xor Filter â a datastructure for fast approximation of set membership that is smaller than bloom and cuckoo filters. Each blog post gets converted into a filter that will then be serialized to a binary blob using bincode. Please note that the underlying technologies are subject to change.
- Only finds entire words. As a consequence there are no search suggestions (yet). This is a necessary tradeoff for reducing memory usage. A trie datastructure was about 10x bigger than the xor filters. New research on compact datastructures for prefix searches might lift this limitation in the future.
- Since we bundle all search indices for all articles into one static binary, we recommend to only use it for small- to medium-size websites. Expect around 2 kB uncompressed per article (~1 kb compressed).
wasm-pack is required to build the WASM module. Install it with
cargo install wasm-pack
To optimize the JavaScript output, you'll also need terser:
npm install terser -g
If you want to make the WebAssembly as small as possible, we recommend to install binaryen as well. On macOS you can install it with homebrew:
brew install binaryen
Alternatively, you can download the binary from the release page or use your OS package manager.
After that, you can install tinysearch itself:
cargo install tinysearch
A JSON file, which contains the content to index, is required as an input. Please take a look at the example file.
âšī¸ The body
field in the JSON document is optional and can be skipped to just
index post titles.
Once you created the index, you can run
tinysearch fixtures/index.json
This will create a WASM module and the JavaScript glue code to integrate it into
your website. You can open the demo.html
from any webserver to see the result.
For example, Python has a built-in webserver that can be used for a quick test:
python3 -m http.server
then browse to http://0.0.0.0:8000/demo.html to run the demo.
You can also take a look at the code examples for different static site generators here.
For advanced usage options, run
tinysearch --help
Please check what's required to host WebAssembly in production -- you will need to explicitly set gzip mime types.
If you don't have a full Rust setup available, you can also use our nightly-built Docker images.
Here is how to quickly try tinysearch with Docker:
# Download a sample blog index from endler.dev
curl -O https://raw.githubusercontent.com/tinysearch/tinysearch/master/fixtures/index.json
# Create the WASM output
docker run -v $PWD:/app tinysearch/cli --engine-version path=\"/engine\" --path /app/wasm_output /app/index.json
By default, the most recent stable Alpine Rust image is used. To get nightly, run
docker build --build-arg RUST_IMAGE=rustlang/rust:nightly-alpine -t tinysearch/cli:nightly .
WASM_REPO
: Overwrite the wasm-pack repositoryWASM_BRANCH
: Overwrite the repository branch to useTINY_REPO
: Overwrite repository of tinysearchTINY_BRANCH
: Overwrite tinysearch branch
To integrate tinysearch in continuous deployment pipelines, a github action is available.
- name: Build tinysearch
uses: leonhfr/tinysearch-action@v1
with:
index: public/index.json
output_dir: public/wasm
output_types: |
wasm
The following websites use tinysearch:
Are you using tinysearch, too? Add your site here!
- Matthias Endler (@mre)
- Jorge-Luis Betancourt (@jorgelbg)
- Mad Mike (@fluential)
tinysearch is licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.