Skill · em Dados, IA e pesquisa

openrlhf-training

High-performance RLHF framework with Ray+vLLM acceleration. Use for PPO, GRPO, RLOO, DPO training of large models (7B-70B+). Built on Ray, vLLM, ZeRO-3. 2× faster than DeepSpeedChat with distributed architecture and GPU resource sharing.

Procedência

Antes de instalar

5 arquivos · 51,2 KB · só texto, nenhum script

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/openrlhf-training/.

d=".claude/skills/openrlhf-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-openrlhf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/algorithm-comparison.md" "$u/references/algorithm-comparison.md" \
  -o "$d/references/custom-rewards.md" "$u/references/custom-rewards.md" \
  -o "$d/references/hybrid-engine.md" "$u/references/hybrid-engine.md" \
  -o "$d/references/multi-node-training.md" "$u/references/multi-node-training.md"

Global: instala em ~/.claude/skills/openrlhf-training/.

d="$HOME/.claude/skills/openrlhf-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-openrlhf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/algorithm-comparison.md" "$u/references/algorithm-comparison.md" \
  -o "$d/references/custom-rewards.md" "$u/references/custom-rewards.md" \
  -o "$d/references/hybrid-engine.md" "$u/references/hybrid-engine.md" \
  -o "$d/references/multi-node-training.md" "$u/references/multi-node-training.md"

Codex

Neste projeto: instala em .agents/skills/openrlhf-training/.

d=".agents/skills/openrlhf-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-openrlhf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/algorithm-comparison.md" "$u/references/algorithm-comparison.md" \
  -o "$d/references/custom-rewards.md" "$u/references/custom-rewards.md" \
  -o "$d/references/hybrid-engine.md" "$u/references/hybrid-engine.md" \
  -o "$d/references/multi-node-training.md" "$u/references/multi-node-training.md"

Global: instala em ~/.agents/skills/openrlhf-training/.

d="$HOME/.agents/skills/openrlhf-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-openrlhf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/algorithm-comparison.md" "$u/references/algorithm-comparison.md" \
  -o "$d/references/custom-rewards.md" "$u/references/custom-rewards.md" \
  -o "$d/references/hybrid-engine.md" "$u/references/hybrid-engine.md" \
  -o "$d/references/multi-node-training.md" "$u/references/multi-node-training.md"

Antigravity

Neste projeto: instala em .agents/skills/openrlhf-training/.

d=".agents/skills/openrlhf-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-openrlhf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/algorithm-comparison.md" "$u/references/algorithm-comparison.md" \
  -o "$d/references/custom-rewards.md" "$u/references/custom-rewards.md" \
  -o "$d/references/hybrid-engine.md" "$u/references/hybrid-engine.md" \
  -o "$d/references/multi-node-training.md" "$u/references/multi-node-training.md"

Global: instala em ~/.gemini/antigravity-cli/skills/openrlhf-training/.

d="$HOME/.gemini/antigravity-cli/skills/openrlhf-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/post-training-openrlhf"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/algorithm-comparison.md" "$u/references/algorithm-comparison.md" \
  -o "$d/references/custom-rewards.md" "$u/references/custom-rewards.md" \
  -o "$d/references/hybrid-engine.md" "$u/references/hybrid-engine.md" \
  -o "$d/references/multi-node-training.md" "$u/references/multi-node-training.md"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.openrlhf-training

Prévia do SKILL.md

---
name: openrlhf-training
description: High-performance RLHF framework with Ray+vLLM acceleration. Use for PPO, GRPO, RLOO, DPO training of large models (7B-70B+). Built on Ray, vLLM, ZeRO-3. 2× faster than DeepSpeedChat with distributed architecture and GPU resource sharing.
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Post-Training, OpenRLHF, RLHF, PPO, GRPO, RLOO, DPO, Ray, vLLM, Distributed Training, Large Models, ZeRO-3]
dependencies: [openrlhf, ray, vllm, torch, transformers, deepspeed]
---

# OpenRLHF - High-Performance RLHF Training

## Quick start

OpenRLHF is a Ray-based RLHF framework optimized for distributed training with vLLM inference acceleration.

**Installation**:
```bash
# Launch Docker container
docker run --runtime=nvidia -it --rm --shm-size="10g" --cap-add=SYS_ADMIN \
  -v $PWD:/openrlhf nvcr.io/nvidia/pytorch:25.02-py3 bash

# Uninstall conflicts
sudo pip uninstall xgboost transformer_engine flash_attn pynvml -y

# Install OpenRLHF with vLLM
pip install openrlhf[vllm]
```

**PPO Training** (Hybrid Engine):
```bash
ray start --head --node-ip-address 0.0.0.0 --num-gpus 8

ray job submit --address="http://127.0.0.1:8265" \
  --runtime-env-json='{"working_dir": "/openrlhf"}' \
  -- python3 -m openrlhf.cli.train_ppo_ray \
  --ref_num_nodes 1 --ref_num_gpus_per_node 8 \
  --reward_num_nodes 1 --reward_num_gpus_per_node 8 \
  --critic_num_nodes 1 --critic_num_gpus_per_node 8 \
  --actor_num_nodes 1 --actor_num_gpus_per_node 8 \
…

Ver todo o marketplace