patrickcurl / sanechain Goto Github PK

View Code? Open in Web Editor NEW

3.0 2.0 0.0 33.43 MB

Filling in the missing gaps with langchain, and creating OO wrappers to simplify some workloads.

License: MIT License

JavaScript 13.50% TypeScript 86.50%

agent ai artificial-intelligence cohereai gpt gpt3 gpt4 inference-engine langchain language-model

sanechain's Introduction

Sane Chain

An attempt to make langchainjs easier to work with

WIP - ~~nothing works yet, just saving the name~~ Some things work, just um - not tested, no warranties 🥇

Adds the following loaders:

Utility Classes
1. DocumentLoader
Loaders

Utility Classes

DocumentLoader

This class essentially packages up all of langchainjs (plus sanechain) and creates a class: DocumentLoader that can basically load up all your documents regardless of type.

Example:

const filesAndDirectories = [
  'path/to/somefile.md',
  'path/to/somefile.pdf',
  'path/to/somefile.text',
  'path/to/somefile.html',
  'path/to/somedirectory',
  'https://github.com/some/repo',
  'https://github.com/some/other_repo',
  'path/to/chatgpt.json'
]

const documentLoader = new DocumentLoader(filesAndDirectories)
const documents = documentLoader.loadDocuments()
const splitDocuments = documentLoader.splitDocuments()
// Might take time, probably gonna implement a queue system to speed things up, already using async though.
// also @todo add full parity with all langchain python loaders.

Loaders

ChatGPT Loader

import { ChatGPTLoader } from './chat_gpt_loader.js';

const loader = new ChatGPTLoader('path/to/chat/log.json', 10);
const documents = await loader.load();

Simpler GithubRepoLoader

Insert github link, get repo documents.

  import {GithubRepoLoader} from 'sanechain'
  const loader = new GithubRepoLoader("https://github.com/owner/repo", { /*params*/ });
  const documents = await loader.load();