Skill · em Dados, IA e pesquisa

gptq

Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA…

Procedência

Antes de instalar

4 arquivos · 23,9 KB · só texto, nenhum script

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/gptq/.

d=".claude/skills/gptq"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gptq"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/calibration.md" "$u/references/calibration.md" \
  -o "$d/references/integration.md" "$u/references/integration.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Global: instala em ~/.claude/skills/gptq/.

d="$HOME/.claude/skills/gptq"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gptq"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/calibration.md" "$u/references/calibration.md" \
  -o "$d/references/integration.md" "$u/references/integration.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Codex

Neste projeto: instala em .agents/skills/gptq/.

d=".agents/skills/gptq"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gptq"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/calibration.md" "$u/references/calibration.md" \
  -o "$d/references/integration.md" "$u/references/integration.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Global: instala em ~/.agents/skills/gptq/.

d="$HOME/.agents/skills/gptq"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gptq"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/calibration.md" "$u/references/calibration.md" \
  -o "$d/references/integration.md" "$u/references/integration.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Antigravity

Neste projeto: instala em .agents/skills/gptq/.

d=".agents/skills/gptq"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gptq"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/calibration.md" "$u/references/calibration.md" \
  -o "$d/references/integration.md" "$u/references/integration.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Global: instala em ~/.gemini/antigravity-cli/skills/gptq/.

d="$HOME/.gemini/antigravity-cli/skills/gptq"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/optimization-gptq"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/calibration.md" "$u/references/calibration.md" \
  -o "$d/references/integration.md" "$u/references/integration.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.gptq

Prévia do SKILL.md

---
name: gptq
description: Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Optimization, GPTQ, Quantization, 4-Bit, Post-Training, Memory Optimization, Consumer GPUs, Fast Inference, QLoRA, Group-Wise Quantization]
dependencies: [auto-gptq, transformers, optimum, peft]
---

# GPTQ (Generative Pre-trained Transformer Quantization)

Post-training quantization method that compresses LLMs to 4-bit with minimal accuracy loss using group-wise quantization.

## When to use GPTQ

**Use GPTQ when:**
- Need to fit large models (70B+) on limited GPU memory
- Want 4× memory reduction with <2% accuracy loss
- Deploying on consumer GPUs (RTX 4090, 3090)
- Need faster inference (3-4× speedup vs FP16)

**Use AWQ instead when:**
- Need slightly better accuracy (<1% loss)
- Have newer GPUs (Ampere, Ada)
- Want Marlin kernel support (2× faster on some GPUs)

**Use bitsandbytes instead when:**
- Need simple integration with transformers
- Want 8-bit quantization (less compression, better quality)
- Don't need pre-quantized model files

## Quick start

### Installation

```bash
# Install AutoGPTQ
pip install auto-gptq
…

Ver todo o marketplace