Code Monkey home page Code Monkey logo

vector-admin's Introduction

VectorAdmin logo

VectorAdmin: The universal interface for vector databases..
A production-ready vector database management system that doesn't hurt to use.

Twitter | Discord | License | AnythingLLM | Docs [WIP]

Quick! Can you tell me exactly what information is embedded in your Pinecone or Chroma vector database? I bet you can't. While those teams are focusing on building the underlying architecture we made it easier for you to manage vector data without the headaches and API calls.

We call it VectorAdmin and we want to be the best universal GUI for vector database management.

Managing VectorData view more screenshots

Watch the demo!

Watch the video

Product Overview

VectorAdmin aims to be a full-stack application that gives you total control over your otherwise unwieldy vector data that you are embedding via an API or using tools like LangChain, which don't show you want you just saved into your database.

VectorAdmin is a full capable multi-user product that you can run locally via Docker as well as host remotely and manage multiple vector databases at once.

VectorAdmin is more than a single tool. VectorAdmin is a suite of tools that make interacting with and understanding vectorized text easy without compromise for the controls you would expect from a traditional database management system.

Some cool features of VectorAdmin

  • Multi-user instance support and oversight
  • Atomically view, update, and delete singular text chunks of embeddings.
  • Copy entire documents or even whole namespaces and embeddings without paying to re-embed.
  • Upload & embed new documents directly into the vector database.
  • Migrate an entire existing vector database to another type or instance. still in progress
  • Manage multiple concurrent vector databases at once.
  • Permission data and access to data
  • 100% Cloud deployment ready.
  • Automated regression testing that run as namespaces or collections are updated with new documents to ensure response quality. still in progress
  • Full API, Javascript, and Python standalone client and LangChain integration. still in progress
  • Extremely efficient cost-saving measures for managing very large documents. You'll never pay to embed a massive document or transcript more than once.

Technical Overview

This monorepo consists of three main sections:

  • document-processor: Flask app to digest, parse, and embed documents easily.
  • frontend: A viteJS + React frontend that you can run to easily create and manage all your content.
  • backend: A nodeJS + express server to handle all the interactions and do all the vectorDB management.
  • workers: An InngestJS instance to handle long-running processes background tasks for snappy performance.
  • docker: Run this entire arch in a single command as a docker instance recommended.

Requirements

  • yarn and node on your machine
  • python 3.9+ for running scripts in document-processor/.
  • access to an OpenAI API key if planning to update embeddings or upload new documents.
  • a Pinecone.io free account or a running ChromaDB instance.

How to get started (Docker - simple setup)

Get up and running in minutes with Docker

How to get started (Development environment)

The below instructions will not work on Windows.

  • yarn dev:setup from the project root directory.
  • cd document-processor && python3.9 -m virtual-env v-env && source v-env/bin/activate && pip install -r requirements.txt

In separate terminal windows from project root:

  • yarn dev:server
  • yarn dev:frontend
  • yarn dev:workers
  • cd document-processor && flask run --host '0.0.0.0' --port 8888

On first boot of the system you will be prompted to login. Consult the backend/.env.development and set or use the SYS_EMAIL and SYS_PASSWORD values. Once your new account is setup the root credentials will no longer work and you can use your admin account.

Contributing

  • create issue
  • create PR with branch name format of <issue number>-<short name>
  • yee haw let's merge

vector-admin's People

Contributors

timothycarambat avatar shatfield4 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.