Skill · em Dados, IA e pesquisa

training-llms-megatron

Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies. Use when training models >1B parameters, need maximum GPU efficiency (47% MFU on H100), or require tensor/pipeline/sequence/context/expert parallelism. Production-ready framework used…

Procedência

Origem: davila7/claude-code-templates
Caminho: cli-tool/components/skills/ai-research/distributed-training-megatron-core
Versão fixada: 57f899e5394bb8ca166f38eacae8f0853cbfe033
Licença: MIT
Espelhado em 25/09/2026
1 download no Claude Code Templates (lido em 25/09/2026)

Antes de instalar

5 arquivos · 48 KB · só texto, nenhum script

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/training-llms-megatron/.

d=".claude/skills/training-llms-megatron"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/distributed-training-megatron-core"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/benchmarks.md" "$u/references/benchmarks.md" \
  -o "$d/references/parallelism-guide.md" "$u/references/parallelism-guide.md" \
  -o "$d/references/production-examples.md" "$u/references/production-examples.md" \
  -o "$d/references/training-recipes.md" "$u/references/training-recipes.md"

Global: instala em ~/.claude/skills/training-llms-megatron/.

d="$HOME/.claude/skills/training-llms-megatron"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/distributed-training-megatron-core"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/benchmarks.md" "$u/references/benchmarks.md" \
  -o "$d/references/parallelism-guide.md" "$u/references/parallelism-guide.md" \
  -o "$d/references/production-examples.md" "$u/references/production-examples.md" \
  -o "$d/references/training-recipes.md" "$u/references/training-recipes.md"

Codex

Neste projeto: instala em .agents/skills/training-llms-megatron/.

d=".agents/skills/training-llms-megatron"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/distributed-training-megatron-core"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/benchmarks.md" "$u/references/benchmarks.md" \
  -o "$d/references/parallelism-guide.md" "$u/references/parallelism-guide.md" \
  -o "$d/references/production-examples.md" "$u/references/production-examples.md" \
  -o "$d/references/training-recipes.md" "$u/references/training-recipes.md"

Global: instala em ~/.agents/skills/training-llms-megatron/.

d="$HOME/.agents/skills/training-llms-megatron"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/distributed-training-megatron-core"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/benchmarks.md" "$u/references/benchmarks.md" \
  -o "$d/references/parallelism-guide.md" "$u/references/parallelism-guide.md" \
  -o "$d/references/production-examples.md" "$u/references/production-examples.md" \
  -o "$d/references/training-recipes.md" "$u/references/training-recipes.md"

Antigravity

Neste projeto: instala em .agents/skills/training-llms-megatron/.

d=".agents/skills/training-llms-megatron"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/distributed-training-megatron-core"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/benchmarks.md" "$u/references/benchmarks.md" \
  -o "$d/references/parallelism-guide.md" "$u/references/parallelism-guide.md" \
  -o "$d/references/production-examples.md" "$u/references/production-examples.md" \
  -o "$d/references/training-recipes.md" "$u/references/training-recipes.md"

Global: instala em ~/.gemini/antigravity-cli/skills/training-llms-megatron/.

d="$HOME/.gemini/antigravity-cli/skills/training-llms-megatron"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/distributed-training-megatron-core"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/benchmarks.md" "$u/references/benchmarks.md" \
  -o "$d/references/parallelism-guide.md" "$u/references/parallelism-guide.md" \
  -o "$d/references/production-examples.md" "$u/references/production-examples.md" \
  -o "$d/references/training-recipes.md" "$u/references/training-recipes.md"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.training-llms-megatron

Prévia do SKILL.md

---
name: training-llms-megatron
description: Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies. Use when training models >1B parameters, need maximum GPU efficiency (47% MFU on H100), or require tensor/pipeline/sequence/context/expert parallelism. Production-ready framework used for Nemo…
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Megatron-Core, Large-Scale Training, NVIDIA, Tensor Parallelism, Pipeline Parallelism, Model Parallelism, H100, Distributed Training, Production]
dependencies: [megatron-core, torch, apex, transformer-engine]
---

# Megatron-Core - Large-Scale LLM Training

## Quick start

Megatron-Core trains LLMs from 2B to 462B parameters with up to 47% Model FLOP Utilization on H100 GPUs through advanced parallelism strategies.

**Installation**:
```bash
# Docker (recommended)
docker run --gpus all -it --rm nvcr.io/nvidia/pytorch:25.04-py3

# Or pip
pip install megatron-core
```

**Simple distributed training**:
```bash
# Train with 2 GPUs using data parallelism
torchrun --nproc_per_node=2 examples/run_simple_mcore_train_loop.py

# Or LLaMA-3 8B training
./examples/llama/train_llama3_8b_fp8.sh
```

## Common workflows

### Workflow 1: Train LLaMA-style model with 3D parallelism

Copy this checklist:
…

Ver todo o marketplace