pgvecto.rs is a Postgres extension that provides vector similarity search functions. It is written in Rust and based on pgrx. Read more at πour blog.
| Feature Category | Feature | |
|---|---|---|
| Search Capabilities | π Vector Search | Ultra-low-latency, high-precision vector search. |
| 𧩠Sparse Vector Search | Keyword-based vector search using SPLADE or BM25 algorithms. | |
| π Full-Text Search | Comprehensive text search across any language, powered by tsvector. | |
| Data Handling | β Complete SQL Support | Full SQL support, enabling joins and filters without limitations or extra configuration. |
| π Async indexing | Non-blocking inserts with up-to-date query readiness. | |
| π Easy Data Management | No need for syncing vectors and metadata with external vector DB, simplifying development. | |
| Data Types | π’ FP16/INT8 Data type | Supports FP16 and INT8 data types for improved storage and computational efficiency. |
| π Binary vector support | Vector indexing with binary vectors, and Jaccard distance support. | |
| πͺ Matryoshka embeddings | Subvector indexing, like vector[0:256], for enhanced Matryoshka embeddings. | |
| β¬οΈ Extended Vector Length | Vector lengths up to 65535 supported, ideal for the latest cutting-edge models. | |
| System Performance | π Production Ready | Battle-tested database ecosystem integrated with PostgreSQL. |
| βοΈ High Availability | Logical replication support to ensure high availability. | |
| π‘ Resource Efficient | Efficient attribute storage leveraging PostgreSQL. | |
| Security & Permissions | π Permission Control | Easy access control like read-only roles, powered by PostgreSQL. |
- Getting Started
- Usage
- Administration
- Developers
For new users, we recommend using the Docker image to get started quickly.
docker run \
--name pgvecto-rs-demo \
-e POSTGRES_PASSWORD=mysecretpassword \
-p 5432:5432 \
-d tensorchord/pgvecto-rs:pg16-v0.2.1Then you can connect to the database using the psql command line tool. The default username is postgres, and the default password is mysecretpassword.
psql -h localhost -p 5432 -U postgresRun the following SQL to ensure the extension is enabled.
DROP EXTENSION IF EXISTS vectors;
CREATE EXTENSION vectors;pgvecto.rs introduces a new data type vector(n) denoting an n-dimensional vector. The n within the brackets signifies the dimensions of the vector.
You could create a table with the following SQL.
-- create table with a vector column
CREATE TABLE items (
id bigserial PRIMARY KEY,
embedding vector(3) NOT NULL -- 3 dimensions
);Tip
vector(n) is a valid data type only if vector(3) of vector is also a valid data type. However, you cannot still put vector for a column or there is some values mismatched with dimension denoted by the column, you won't able to create an index on it.
You can then populate the table with vector data as follows.
-- insert values
INSERT INTO items (embedding)
VALUES ('[1,2,3]'), ('[4,5,6]');
-- or insert values using a casting from array to vector
INSERT INTO items (embedding)
VALUES (ARRAY[1, 2, 3]::real[]), (ARRAY[4, 5, 6]::real[]);We support three operators to calculate the distance between two vectors.
-
<->: squared Euclidean distance, defined as$\Sigma (x_i - y_i) ^ 2$ . -
<#>: negative dot product, defined as$- \Sigma x_iy_i$ . -
<=>: cosine distance, defined as$1 - \frac{\Sigma x_iy_i}{\sqrt{\Sigma x_i^2 \Sigma y_i^2}}$ .
-- call the distance function through operators
-- squared Euclidean distance
SELECT '[1, 2, 3]'::vector <-> '[3, 2, 1]'::vector;
-- negative dot product
SELECT '[1, 2, 3]'::vector <#> '[3, 2, 1]'::vector;
-- cosine distance
SELECT '[1, 2, 3]'::vector <=> '[3, 2, 1]'::vector;You can search for a vector simply like this.
-- query the similar embeddings
SELECT * FROM items ORDER BY embedding <-> '[3,2,1]' LIMIT 5;vecf16 type is the same with vector in anything but the scalar type. It stores 16-bit floating point numbers. If you want to reduce the memory usage to get better performance, you can try to replace vector type with vecf16 type.
Please check out ROADMAP. Want to jump in? Welcome discussions and contributions!
- Chat with us on π¬ Discord
- Have a look at
good first issue πissues!
We welcome all kinds of contributions from the open-source community, individuals, and partners.
- Join our discord community!
- To build from the source, please read our contributing documentation and development tutorial.
Thanks goes to these wonderful people (emoji key):
This project follows the all-contributors specification. Contributions of any kind welcome!
Thanks to the following projects:
![allcontributors[bot] avatar](https://avatars.githubusercontent.com/in/23186?v=4 "allcontributors[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent