Skill · em Dados, IA e pesquisa

llama-cpp

Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.

Procedência

Antes de instalar

4 arquivos · 14,4 KB · só texto, nenhum script

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/llama-cpp/.

d=".claude/skills/llama-cpp"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-llama-cpp"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/optimization.md" "$u/references/optimization.md" \
  -o "$d/references/quantization.md" "$u/references/quantization.md" \
  -o "$d/references/server.md" "$u/references/server.md"

Global: instala em ~/.claude/skills/llama-cpp/.

d="$HOME/.claude/skills/llama-cpp"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-llama-cpp"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/optimization.md" "$u/references/optimization.md" \
  -o "$d/references/quantization.md" "$u/references/quantization.md" \
  -o "$d/references/server.md" "$u/references/server.md"

Codex

Neste projeto: instala em .agents/skills/llama-cpp/.

d=".agents/skills/llama-cpp"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-llama-cpp"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/optimization.md" "$u/references/optimization.md" \
  -o "$d/references/quantization.md" "$u/references/quantization.md" \
  -o "$d/references/server.md" "$u/references/server.md"

Global: instala em ~/.agents/skills/llama-cpp/.

d="$HOME/.agents/skills/llama-cpp"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-llama-cpp"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/optimization.md" "$u/references/optimization.md" \
  -o "$d/references/quantization.md" "$u/references/quantization.md" \
  -o "$d/references/server.md" "$u/references/server.md"

Antigravity

Neste projeto: instala em .agents/skills/llama-cpp/.

d=".agents/skills/llama-cpp"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-llama-cpp"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/optimization.md" "$u/references/optimization.md" \
  -o "$d/references/quantization.md" "$u/references/quantization.md" \
  -o "$d/references/server.md" "$u/references/server.md"

Global: instala em ~/.gemini/antigravity-cli/skills/llama-cpp/.

d="$HOME/.gemini/antigravity-cli/skills/llama-cpp"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/inference-serving-llama-cpp"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/optimization.md" "$u/references/optimization.md" \
  -o "$d/references/quantization.md" "$u/references/quantization.md" \
  -o "$d/references/server.md" "$u/references/server.md"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.llama-cpp

Prévia do SKILL.md

---
name: llama-cpp
description: Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Inference Serving, Llama.cpp, CPU Inference, Apple Silicon, Edge Deployment, GGUF, Quantization, Non-NVIDIA, AMD GPUs, Intel GPUs, Embedded]
dependencies: [llama-cpp-python]
---

# llama.cpp

Pure C/C++ LLM inference with minimal dependencies, optimized for CPUs and non-NVIDIA hardware.

## When to use llama.cpp

**Use llama.cpp when:**
- Running on CPU-only machines
- Deploying on Apple Silicon (M1/M2/M3/M4)
- Using AMD or Intel GPUs (no CUDA)
- Edge deployment (Raspberry Pi, embedded systems)
- Need simple deployment without Docker/Python

**Use TensorRT-LLM instead when:**
- Have NVIDIA GPUs (A100/H100)
- Need maximum throughput (100K+ tok/s)
- Running in datacenter with CUDA

**Use vLLM instead when:**
- Have NVIDIA GPUs
- Need Python-first API
- Want PagedAttention

## Quick start

### Installation

```bash
# macOS/Linux
brew install llama.cpp
…

Ver todo o marketplace