Skill · em Dados, IA e pesquisa

speculative-decoding

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques. Use when optimizing inference speed (1.5-3.6× speedup), reducing latency for real-time applications, or deploying models with limited compute. Covers draft models, tree-based attention,…

Procedência

Antes de instalar

3 arquivos · 32 KB · só texto, nenhum script

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/speculative-decoding/.

d=".claude/skills/speculative-decoding"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-speculative-decoding"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/lookahead.md" "$u/references/lookahead.md" \
  -o "$d/references/medusa.md" "$u/references/medusa.md"

Global: instala em ~/.claude/skills/speculative-decoding/.

d="$HOME/.claude/skills/speculative-decoding"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-speculative-decoding"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/lookahead.md" "$u/references/lookahead.md" \
  -o "$d/references/medusa.md" "$u/references/medusa.md"

Codex

Neste projeto: instala em .agents/skills/speculative-decoding/.

d=".agents/skills/speculative-decoding"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-speculative-decoding"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/lookahead.md" "$u/references/lookahead.md" \
  -o "$d/references/medusa.md" "$u/references/medusa.md"

Global: instala em ~/.agents/skills/speculative-decoding/.

d="$HOME/.agents/skills/speculative-decoding"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-speculative-decoding"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/lookahead.md" "$u/references/lookahead.md" \
  -o "$d/references/medusa.md" "$u/references/medusa.md"

Antigravity

Neste projeto: instala em .agents/skills/speculative-decoding/.

d=".agents/skills/speculative-decoding"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-speculative-decoding"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/lookahead.md" "$u/references/lookahead.md" \
  -o "$d/references/medusa.md" "$u/references/medusa.md"

Global: instala em ~/.gemini/antigravity-cli/skills/speculative-decoding/.

d="$HOME/.gemini/antigravity-cli/skills/speculative-decoding"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-speculative-decoding"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/lookahead.md" "$u/references/lookahead.md" \
  -o "$d/references/medusa.md" "$u/references/medusa.md"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.speculative-decoding

Prévia do SKILL.md

---
name: speculative-decoding
description: Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques. Use when optimizing inference speed (1.5-3.6× speedup), reducing latency for real-time applications, or deploying models with limited compute. Covers draft models, tree-based attention, Jacobi ite…
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Emerging Techniques, Speculative Decoding, Medusa, Lookahead Decoding, Fast Inference, Draft Models, Tree Attention, Parallel Generation, Latency Reduction, Inference Optimization]
dependencies: [transformers, torch]
---

# Speculative Decoding: Accelerating LLM Inference

## When to Use This Skill

Use Speculative Decoding when you need to:
- **Speed up inference** by 1.5-3.6× without quality loss
- **Reduce latency** for real-time applications (chatbots, code generation)
- **Optimize throughput** for high-volume serving
- **Deploy efficiently** on limited hardware
- **Generate faster** without changing model architecture

**Key Techniques**: Draft model speculative decoding, Medusa (multiple heads), Lookahead Decoding (Jacobi iteration)

**Papers**: Medusa (arXiv 2401.10774), Lookahead Decoding (ICML 2024), Speculative Decoding Survey (ACL 2024)

## Installation

```bash
# Standard speculative decoding (transformers)
pip install transformers accelerate

# Medusa (multiple decoding heads)
git clone https://github.com/FasterDecoding/Medusa
cd Medusa
pip install -e .

# Lookahead Decoding
git clone https://github.com/hao-ai-lab/LookaheadDecoding
cd LookaheadDecoding
pip install -e .
…

Ver todo o marketplace