Code Monkey home page Code Monkey logo

gguf.js's Introduction

GGUF.js

Download, manage, and run Llama GGUF files easily with Faraday.dev

gguf

A Javascript library (with Typescript types) to parse metadata for GGML-based GGUF files.

Supported Architectures

  • Llama
  • MPT
  • GPTNeoX
  • GPTJ
  • GPT2
  • Bloom
  • Falcon
  • RWKV
  • Gemma

This library goal is to be 1-to-1 with the spec. PRs welcome!

Install

yarn add gguf

or

npm install gguf

Usage

import gguf, { isLLamaMetadata } from 'gguf'

// pass in a file path, gguf.js will only load in what is needed for the metadata
// not the whole file
const { metadata, error } = await gguf('./llama2.gguf')

if (error) {
  throw error
}

// helper function to give full type safety
// see more in `src/index.ts`
if (isLLamaMetadata(metadata)) {
  console.log(`context length: ${metadata.llama.context_length}`)
}

TypeScript

Typescript is supported by default and all definitions are in src/metadataTypes

License

MIT © Ahoy Labs, Inc.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.