Skill · em Dados, IA e pesquisa

sglang

Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.

Procedência

Antes de instalar

4 arquivos · 44,8 KB · só texto, nenhum script

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/sglang/.

d=".claude/skills/sglang"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-sglang"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/deployment.md" "$u/references/deployment.md" \
  -o "$d/references/radix-attention.md" "$u/references/radix-attention.md" \
  -o "$d/references/structured-generation.md" "$u/references/structured-generation.md"

Global: instala em ~/.claude/skills/sglang/.

d="$HOME/.claude/skills/sglang"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-sglang"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/deployment.md" "$u/references/deployment.md" \
  -o "$d/references/radix-attention.md" "$u/references/radix-attention.md" \
  -o "$d/references/structured-generation.md" "$u/references/structured-generation.md"

Codex

Neste projeto: instala em .agents/skills/sglang/.

d=".agents/skills/sglang"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-sglang"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/deployment.md" "$u/references/deployment.md" \
  -o "$d/references/radix-attention.md" "$u/references/radix-attention.md" \
  -o "$d/references/structured-generation.md" "$u/references/structured-generation.md"

Global: instala em ~/.agents/skills/sglang/.

d="$HOME/.agents/skills/sglang"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-sglang"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/deployment.md" "$u/references/deployment.md" \
  -o "$d/references/radix-attention.md" "$u/references/radix-attention.md" \
  -o "$d/references/structured-generation.md" "$u/references/structured-generation.md"

Antigravity

Neste projeto: instala em .agents/skills/sglang/.

d=".agents/skills/sglang"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-sglang"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/deployment.md" "$u/references/deployment.md" \
  -o "$d/references/radix-attention.md" "$u/references/radix-attention.md" \
  -o "$d/references/structured-generation.md" "$u/references/structured-generation.md"

Global: instala em ~/.gemini/antigravity-cli/skills/sglang/.

d="$HOME/.gemini/antigravity-cli/skills/sglang"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-sglang"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/deployment.md" "$u/references/deployment.md" \
  -o "$d/references/radix-attention.md" "$u/references/radix-attention.md" \
  -o "$d/references/structured-generation.md" "$u/references/structured-generation.md"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.sglang

Prévia do SKILL.md

---
name: sglang
description: Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Inference Serving, SGLang, Structured Generation, RadixAttention, Prefix Caching, Constrained Decoding, Agents, JSON Output, Fast Inference, Production Scale]
dependencies: [sglang, torch, transformers]
---

# SGLang

High-performance serving framework for LLMs and VLMs with RadixAttention for automatic prefix caching.

## When to use SGLang

**Use SGLang when:**
- Need structured outputs (JSON, regex, grammar)
- Building agents with repeated prefixes (system prompts, tools)
- Agentic workflows with function calling
- Multi-turn conversations with shared context
- Need faster JSON decoding (3× vs standard)

**Use vLLM instead when:**
- Simple text generation without structure
- Don't need prefix caching
- Want mature, widely-tested production system

**Use TensorRT-LLM instead when:**
- Maximum single-request latency (no batching needed)
- NVIDIA-only deployment
- Need FP8/INT4 quantization on H100

## Quick start

### Installation

```bash
# pip install (recommended)
pip install "sglang[all]"
…

Ver todo o marketplace