CREATORS

@crtrs/llm

The toolkit for apps that think locally.
Run LLMs on your own machine, from Node.

Local

Nothing leaves your machine.

Typed

Replies shaped by a zod schema.

Streamed

Token by token, stoppable at any point.

Llama 3.2Qwen 3Gemma 3Phi-4Mistral 7BDeepSeek R1SmolLM 3gpt-oss 20BMinistral 3BGranite 3.3OLMo 2Hermes 3@crtrs/llm

Usage

Talk to it. Point to a GGUF file on disk and prompt it. The reply streams in token by token.

hello.ts
import { load } from '@crtrs/llm'
const ai = await load('./model.gguf', { system: 'Answer briefly.' })
await ai.prompt('Hi', { onToken: (t) => process.stdout.write(t) })
await ai.unload()
Output

API

Docs
  • load()
  • LoadOptions
  • Model
  • prompt()
  • PromptOptions
  • SchemaPromptOptions<T>
  • getTokenCount()
  • unload()
GitHubCREATORS-INDUSTRIES/llmnpm@crtrs/llm