Skill · em Dados, IA e pesquisa

gguf-quantization

GGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.

Procedência

Antes de instalar

3 arquivos · 29,4 KB · só texto, nenhum script

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/gguf-quantization/.

d=".claude/skills/gguf-quantization"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gguf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Global: instala em ~/.claude/skills/gguf-quantization/.

d="$HOME/.claude/skills/gguf-quantization"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gguf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Codex

Neste projeto: instala em .agents/skills/gguf-quantization/.

d=".agents/skills/gguf-quantization"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gguf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Global: instala em ~/.agents/skills/gguf-quantization/.

d="$HOME/.agents/skills/gguf-quantization"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gguf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Antigravity

Neste projeto: instala em .agents/skills/gguf-quantization/.

d=".agents/skills/gguf-quantization"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gguf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Global: instala em ~/.gemini/antigravity-cli/skills/gguf-quantization/.

d="$HOME/.gemini/antigravity-cli/skills/gguf-quantization"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gguf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.gguf-quantization

Prévia do SKILL.md

---
name: gguf-quantization
description: GGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [GGUF, Quantization, llama.cpp, CPU Inference, Apple Silicon, Model Compression, Optimization]
dependencies: [llama-cpp-python>=0.2.0]
---

# GGUF - Quantization Format for llama.cpp

The GGUF (GPT-Generated Unified Format) is the standard file format for llama.cpp, enabling efficient inference on CPUs, Apple Silicon, and GPUs with flexible quantization options.

## When to use GGUF

**Use GGUF when:**
- Deploying on consumer hardware (laptops, desktops)
- Running on Apple Silicon (M1/M2/M3) with Metal acceleration
- Need CPU inference without GPU requirements
- Want flexible quantization (Q2_K to Q8_0)
- Using local AI tools (LM Studio, Ollama, text-generation-webui)

**Key advantages:**
- **Universal hardware**: CPU, Apple Silicon, NVIDIA, AMD support
- **No Python runtime**: Pure C/C++ inference
- **Flexible quantization**: 2-8 bit with various methods (K-quants)
- **Ecosystem support**: LM Studio, Ollama, koboldcpp, and more
- **imatrix**: Importance matrix for better low-bit quality

**Use alternatives instead:**
- **AWQ/GPTQ**: Maximum accuracy with calibration on NVIDIA GPUs
- **HQQ**: Fast calibration-free quantization for HuggingFace
- **bitsandbytes**: Simple integration with transformers library
- **TensorRT-LLM**: Production NVIDIA deployment with maximum speed

## Quick start

### Installation
…

Ver todo o marketplace