Skip to content

A Javascript library (with Typescript types) to parse metadata of GGML based GGUF files.

License

Notifications You must be signed in to change notification settings

ahoylabs/gguf.js

Repository files navigation

GGUF.js

Download, manage, and run Llama GGUF files easily with Backyard.ai

gguf

A Javascript library (with Typescript types) to parse metadata for GGML-based GGUF files.

Supported Architectures

  • Llama
  • MPT
  • GPTNeoX
  • GPTJ
  • GPT2
  • Bloom
  • Falcon
  • RWKV
  • Gemma

This library goal is to be 1-to-1 with the spec. PRs welcome!

Install

yarn add gguf

or

npm install gguf

Usage

import gguf, { isLLamaMetadata } from 'gguf'

// pass in a file path, gguf.js will only load in what is needed for the metadata
// not the whole file
const { metadata, error } = await gguf('./llama2.gguf')

if (error) {
  throw error
}

// helper function to give full type safety
// see more in `src/index.ts`
if (isLLamaMetadata(metadata)) {
  console.log(`context length: ${metadata.llama.context_length}`)
}

TypeScript

Typescript is supported by default and all definitions are in src/metadataTypes

License

MIT © Ahoy Labs, Inc.