Skill · em Dados, IA e pesquisa

moe-training

Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase.…

Procedência

Antes de instalar

4 arquivos · 45,9 KB · só texto, nenhum script

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/moe-training/.

d=".claude/skills/moe-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-moe-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/architectures.md" "$u/references/architectures.md" \
  -o "$d/references/inference.md" "$u/references/inference.md" \
  -o "$d/references/training.md" "$u/references/training.md"

Global: instala em ~/.claude/skills/moe-training/.

d="$HOME/.claude/skills/moe-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-moe-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/architectures.md" "$u/references/architectures.md" \
  -o "$d/references/inference.md" "$u/references/inference.md" \
  -o "$d/references/training.md" "$u/references/training.md"

Codex

Neste projeto: instala em .agents/skills/moe-training/.

d=".agents/skills/moe-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-moe-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/architectures.md" "$u/references/architectures.md" \
  -o "$d/references/inference.md" "$u/references/inference.md" \
  -o "$d/references/training.md" "$u/references/training.md"

Global: instala em ~/.agents/skills/moe-training/.

d="$HOME/.agents/skills/moe-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-moe-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/architectures.md" "$u/references/architectures.md" \
  -o "$d/references/inference.md" "$u/references/inference.md" \
  -o "$d/references/training.md" "$u/references/training.md"

Antigravity

Neste projeto: instala em .agents/skills/moe-training/.

d=".agents/skills/moe-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-moe-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/architectures.md" "$u/references/architectures.md" \
  -o "$d/references/inference.md" "$u/references/inference.md" \
  -o "$d/references/training.md" "$u/references/training.md"

Global: instala em ~/.gemini/antigravity-cli/skills/moe-training/.

d="$HOME/.gemini/antigravity-cli/skills/moe-training"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/emerging-techniques-moe-training"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/architectures.md" "$u/references/architectures.md" \
  -o "$d/references/inference.md" "$u/references/inference.md" \
  -o "$d/references/training.md" "$u/references/training.md"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.moe-training

Prévia do SKILL.md

---
name: moe-training
description: Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers Mo…
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Emerging Techniques, MoE, Mixture Of Experts, Sparse Models, DeepSpeed, Expert Parallelism, Mixtral, DeepSeek, Routing, Load Balancing, Efficient Training]
dependencies: [deepspeed, transformers, torch, accelerate]
---

# MoE Training: Mixture of Experts

## When to Use This Skill

Use MoE Training when you need to:
- **Train larger models** with limited compute (5× cost reduction vs dense models)
- **Scale model capacity** without proportional compute increase
- **Achieve better performance** per compute budget than dense models
- **Specialize experts** for different domains/tasks/languages
- **Reduce inference latency** with sparse activation (only 13B/47B params active in Mixtral)
- **Implement SOTA models** like Mixtral 8x7B, DeepSeek-V3, Switch Transformers

**Notable MoE Models**: Mixtral 8x7B (Mistral AI), DeepSeek-V3, Switch Transformers (Google), GLaM (Google), NLLB-MoE (Meta)

## Installation

```bash
# DeepSpeed with MoE support
pip install deepspeed>=0.6.0

# Megatron-DeepSpeed for large-scale training
git clone https://github.com/microsoft/Megatron-DeepSpeed
cd Megatron-DeepSpeed
pip install -r requirements.txt

# Alternative: HuggingFace Transformers
pip install transformers accelerate
```

## Quick Start
…

Ver todo o marketplace