Skill · em Dados, IA e pesquisa

fine-tuning-with-trl

Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.

Procedência

Antes de instalar

5 arquivos · 23 KB · só texto, nenhum script

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/fine-tuning-with-trl/.

d=".claude/skills/fine-tuning-with-trl"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-trl-fine-tuning"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/dpo-variants.md" "$u/references/dpo-variants.md" \
  -o "$d/references/online-rl.md" "$u/references/online-rl.md" \
  -o "$d/references/reward-modeling.md" "$u/references/reward-modeling.md" \
  -o "$d/references/sft-training.md" "$u/references/sft-training.md"

Global: instala em ~/.claude/skills/fine-tuning-with-trl/.

d="$HOME/.claude/skills/fine-tuning-with-trl"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-trl-fine-tuning"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/dpo-variants.md" "$u/references/dpo-variants.md" \
  -o "$d/references/online-rl.md" "$u/references/online-rl.md" \
  -o "$d/references/reward-modeling.md" "$u/references/reward-modeling.md" \
  -o "$d/references/sft-training.md" "$u/references/sft-training.md"

Codex

Neste projeto: instala em .agents/skills/fine-tuning-with-trl/.

d=".agents/skills/fine-tuning-with-trl"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-trl-fine-tuning"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/dpo-variants.md" "$u/references/dpo-variants.md" \
  -o "$d/references/online-rl.md" "$u/references/online-rl.md" \
  -o "$d/references/reward-modeling.md" "$u/references/reward-modeling.md" \
  -o "$d/references/sft-training.md" "$u/references/sft-training.md"

Global: instala em ~/.agents/skills/fine-tuning-with-trl/.

d="$HOME/.agents/skills/fine-tuning-with-trl"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-trl-fine-tuning"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/dpo-variants.md" "$u/references/dpo-variants.md" \
  -o "$d/references/online-rl.md" "$u/references/online-rl.md" \
  -o "$d/references/reward-modeling.md" "$u/references/reward-modeling.md" \
  -o "$d/references/sft-training.md" "$u/references/sft-training.md"

Antigravity

Neste projeto: instala em .agents/skills/fine-tuning-with-trl/.

d=".agents/skills/fine-tuning-with-trl"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-trl-fine-tuning"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/dpo-variants.md" "$u/references/dpo-variants.md" \
  -o "$d/references/online-rl.md" "$u/references/online-rl.md" \
  -o "$d/references/reward-modeling.md" "$u/references/reward-modeling.md" \
  -o "$d/references/sft-training.md" "$u/references/sft-training.md"

Global: instala em ~/.gemini/antigravity-cli/skills/fine-tuning-with-trl/.

d="$HOME/.gemini/antigravity-cli/skills/fine-tuning-with-trl"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-trl-fine-tuning"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/dpo-variants.md" "$u/references/dpo-variants.md" \
  -o "$d/references/online-rl.md" "$u/references/online-rl.md" \
  -o "$d/references/reward-modeling.md" "$u/references/reward-modeling.md" \
  -o "$d/references/sft-training.md" "$u/references/sft-training.md"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.fine-tuning-with-trl

Prévia do SKILL.md

---
name: fine-tuning-with-trl
description: Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Post-Training, TRL, Reinforcement Learning, Fine-Tuning, SFT, DPO, PPO, GRPO, RLHF, Preference Alignment, HuggingFace]
dependencies: [trl, transformers, datasets, peft, accelerate, torch]
---

# TRL - Transformer Reinforcement Learning

## Quick start

TRL provides post-training methods for aligning language models with human preferences.

**Installation**:
```bash
pip install trl transformers datasets peft accelerate
```

**Supervised Fine-Tuning** (instruction tuning):
```python
from trl import SFTTrainer

trainer = SFTTrainer(
    model="Qwen/Qwen2.5-0.5B",
    train_dataset=dataset,  # Prompt-completion pairs
)
trainer.train()
```

**DPO** (align with preferences):
```python
from trl import DPOTrainer, DPOConfig

config = DPOConfig(output_dir="model-dpo", beta=0.1)
trainer = DPOTrainer(
    model=model,
    args=config,
…

Ver todo o marketplace