Skill · em Dados, IA e pesquisa

quantizing-models-bitsandbytes

Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.

Procedência

Antes de instalar

4 arquivos · 44 KB · só texto, nenhum script

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/quantizing-models-bitsandbytes/.

d=".claude/skills/quantizing-models-bitsandbytes"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-bitsandbytes"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/memory-optimization.md" "$u/references/memory-optimization.md" \
  -o "$d/references/qlora-training.md" "$u/references/qlora-training.md" \
  -o "$d/references/quantization-formats.md" "$u/references/quantization-formats.md"

Global: instala em ~/.claude/skills/quantizing-models-bitsandbytes/.

d="$HOME/.claude/skills/quantizing-models-bitsandbytes"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-bitsandbytes"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/memory-optimization.md" "$u/references/memory-optimization.md" \
  -o "$d/references/qlora-training.md" "$u/references/qlora-training.md" \
  -o "$d/references/quantization-formats.md" "$u/references/quantization-formats.md"

Codex

Neste projeto: instala em .agents/skills/quantizing-models-bitsandbytes/.

d=".agents/skills/quantizing-models-bitsandbytes"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-bitsandbytes"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/memory-optimization.md" "$u/references/memory-optimization.md" \
  -o "$d/references/qlora-training.md" "$u/references/qlora-training.md" \
  -o "$d/references/quantization-formats.md" "$u/references/quantization-formats.md"

Global: instala em ~/.agents/skills/quantizing-models-bitsandbytes/.

d="$HOME/.agents/skills/quantizing-models-bitsandbytes"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-bitsandbytes"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/memory-optimization.md" "$u/references/memory-optimization.md" \
  -o "$d/references/qlora-training.md" "$u/references/qlora-training.md" \
  -o "$d/references/quantization-formats.md" "$u/references/quantization-formats.md"

Antigravity

Neste projeto: instala em .agents/skills/quantizing-models-bitsandbytes/.

d=".agents/skills/quantizing-models-bitsandbytes"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-bitsandbytes"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/memory-optimization.md" "$u/references/memory-optimization.md" \
  -o "$d/references/qlora-training.md" "$u/references/qlora-training.md" \
  -o "$d/references/quantization-formats.md" "$u/references/quantization-formats.md"

Global: instala em ~/.gemini/antigravity-cli/skills/quantizing-models-bitsandbytes/.

d="$HOME/.gemini/antigravity-cli/skills/quantizing-models-bitsandbytes"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-bitsandbytes"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/memory-optimization.md" "$u/references/memory-optimization.md" \
  -o "$d/references/qlora-training.md" "$u/references/qlora-training.md" \
  -o "$d/references/quantization-formats.md" "$u/references/quantization-formats.md"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.quantizing-models-bitsandbytes

Prévia do SKILL.md

---
name: quantizing-models-bitsandbytes
description: Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Optimization, Bitsandbytes, Quantization, 8-Bit, 4-Bit, Memory Optimization, QLoRA, NF4, INT8, HuggingFace, Efficient Inference]
dependencies: [bitsandbytes, transformers, accelerate, torch]
---

# bitsandbytes - LLM Quantization

## Quick start

bitsandbytes reduces LLM memory by 50% (8-bit) or 75% (4-bit) with <1% accuracy loss.

**Installation**:
```bash
pip install bitsandbytes transformers accelerate
```

**8-bit quantization** (50% memory reduction):
```python
from transformers import AutoModelForCausalLM, BitsAndBytesConfig

config = BitsAndBytesConfig(load_in_8bit=True)
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-2-7b-hf",
    quantization_config=config,
    device_map="auto"
)

# Memory: 14GB → 7GB
```

**4-bit quantization** (75% memory reduction):
```python
config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16
…

Ver todo o marketplace