Skill · em Dados, IA e pesquisa

grpo-rl-training

Expert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training

Procedência

Antes de instalar

4 arquivos · 37,5 KB · inclui 2 scripts que executam: examples/reward_functions_library.py, templates/basic_grpo_training.py

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/grpo-rl-training/.

d=".claude/skills/grpo-rl-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-grpo-rl-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/examples/reward_functions_library.py" "$u/examples/reward_functions_library.py" \
  -o "$d/README.md" "$u/README.md" \
  -o "$d/templates/basic_grpo_training.py" "$u/templates/basic_grpo_training.py"

Global: instala em ~/.claude/skills/grpo-rl-training/.

d="$HOME/.claude/skills/grpo-rl-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-grpo-rl-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/examples/reward_functions_library.py" "$u/examples/reward_functions_library.py" \
  -o "$d/README.md" "$u/README.md" \
  -o "$d/templates/basic_grpo_training.py" "$u/templates/basic_grpo_training.py"

Codex

Neste projeto: instala em .agents/skills/grpo-rl-training/.

d=".agents/skills/grpo-rl-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-grpo-rl-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/examples/reward_functions_library.py" "$u/examples/reward_functions_library.py" \
  -o "$d/README.md" "$u/README.md" \
  -o "$d/templates/basic_grpo_training.py" "$u/templates/basic_grpo_training.py"

Global: instala em ~/.agents/skills/grpo-rl-training/.

d="$HOME/.agents/skills/grpo-rl-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-grpo-rl-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/examples/reward_functions_library.py" "$u/examples/reward_functions_library.py" \
  -o "$d/README.md" "$u/README.md" \
  -o "$d/templates/basic_grpo_training.py" "$u/templates/basic_grpo_training.py"

Antigravity

Neste projeto: instala em .agents/skills/grpo-rl-training/.

d=".agents/skills/grpo-rl-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-grpo-rl-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/examples/reward_functions_library.py" "$u/examples/reward_functions_library.py" \
  -o "$d/README.md" "$u/README.md" \
  -o "$d/templates/basic_grpo_training.py" "$u/templates/basic_grpo_training.py"

Global: instala em ~/.gemini/antigravity-cli/skills/grpo-rl-training/.

d="$HOME/.gemini/antigravity-cli/skills/grpo-rl-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-grpo-rl-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/examples/reward_functions_library.py" "$u/examples/reward_functions_library.py" \
  -o "$d/README.md" "$u/README.md" \
  -o "$d/templates/basic_grpo_training.py" "$u/templates/basic_grpo_training.py"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.grpo-rl-training

Prévia do SKILL.md

---
name: grpo-rl-training
description: Expert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Post-Training, Reinforcement Learning, GRPO, TRL, RLHF, Reward Modeling, Reasoning, DPO, PPO, Structured Output]
dependencies: [transformers>=4.47.0, trl>=0.14.0, datasets>=3.2.0, peft>=0.14.0, torch]
---

# GRPO/RL Training with TRL

Expert-level guidance for implementing Group Relative Policy Optimization (GRPO) using the Transformer Reinforcement Learning (TRL) library. This skill provides battle-tested patterns, critical insights, and production-ready workflows for fine-tuning language models with custom reward functions.

## When to Use This Skill

Use GRPO training when you need to:
- **Enforce specific output formats** (e.g., XML tags, JSON, structured reasoning)
- **Teach verifiable tasks** with objective correctness metrics (math, coding, fact-checking)
- **Improve reasoning capabilities** by rewarding chain-of-thought patterns
- **Align models to domain-specific behaviors** without labeled preference data
- **Optimize for multiple objectives** simultaneously (format + correctness + style)

**Do NOT use GRPO for:**
- Simple supervised fine-tuning tasks (use SFT instead)
- Tasks without clear reward signals
- When you already have high-quality preference pairs (use DPO/PPO instead)

---

## Core Concepts

### 1. GRPO Algorithm Fundamentals

**Key Mechanism:**
- Generates **multiple completions** for each prompt (group size: 4-16)
- Compares completions within each group using reward functions
- Updates policy to favor higher-rewarded responses relative to the group

**Critical Difference from PPO:**
…

Ver todo o marketplace