Skill · em Dados, IA e pesquisa

blip-2-vision-language

Vision-language pre-training framework bridging frozen image encoders and LLMs. Use when you need image captioning, visual question answering, image-text retrieval, or multimodal chat with state-of-the-art zero-shot performance.

Procedência

Antes de instalar

3 arquivos · 48,3 KB · só texto, nenhum script

Instalar na sua CLI

O comando baixa a versão fixada (commit 57f899e) direto da origem, para a pasta que a CLI lê. Precisa de curl (macOS e Linux); no Windows não há comando, porque o Rook Labs é para macOS.

Claude Code

Neste projeto: instala em .claude/skills/blip-2-vision-language/.

d=".claude/skills/blip-2-vision-language"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/multimodal-blip-2"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Global: instala em ~/.claude/skills/blip-2-vision-language/.

d="$HOME/.claude/skills/blip-2-vision-language"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/multimodal-blip-2"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Codex

Neste projeto: instala em .agents/skills/blip-2-vision-language/.

d=".agents/skills/blip-2-vision-language"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/multimodal-blip-2"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Global: instala em ~/.agents/skills/blip-2-vision-language/.

d="$HOME/.agents/skills/blip-2-vision-language"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/multimodal-blip-2"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Antigravity

Neste projeto: instala em .agents/skills/blip-2-vision-language/.

d=".agents/skills/blip-2-vision-language"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/multimodal-blip-2"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Global: instala em ~/.gemini/antigravity-cli/skills/blip-2-vision-language/.

d="$HOME/.gemini/antigravity-cli/skills/blip-2-vision-language"
u="https://raw.githubusercontent.com/davila7/claude-code-templates/57f899e5394bb8ca166f38eacae8f0853cbfe033/cli-tool/components/skills/ai-research/multimodal-blip-2"
curl -fsSL --create-dirs \
  -o "$d/SKILL.md" "$u/SKILL.md" \
  -o "$d/references/advanced-usage.md" "$u/references/advanced-usage.md" \
  -o "$d/references/troubleshooting.md" "$u/references/troubleshooting.md"

Peça ao Rook

Já usa o Rook Labs? Cole no chat do Rook: instale a skill https://rooklabs.sh/marketplace/cct.blip-2-vision-language

Prévia do SKILL.md

---
name: blip-2-vision-language
description: Vision-language pre-training framework bridging frozen image encoders and LLMs. Use when you need image captioning, visual question answering, image-text retrieval, or multimodal chat with state-of-the-art zero-shot performance.
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Multimodal, Vision-Language, Image Captioning, VQA, Zero-Shot]
dependencies: [transformers>=4.30.0, torch>=1.10.0, Pillow]
---

# BLIP-2: Vision-Language Pre-training

Comprehensive guide to using Salesforce's BLIP-2 for vision-language tasks with frozen image encoders and large language models.

## When to use BLIP-2

**Use BLIP-2 when:**
- Need high-quality image captioning with natural descriptions
- Building visual question answering (VQA) systems
- Require zero-shot image-text understanding without task-specific training
- Want to leverage LLM reasoning for visual tasks
- Building multimodal conversational AI
- Need image-text retrieval or matching

**Key features:**
- **Q-Former architecture**: Lightweight query transformer bridges vision and language
- **Frozen backbone efficiency**: No need to fine-tune large vision/language models
- **Multiple LLM backends**: OPT (2.7B, 6.7B) and FlanT5 (XL, XXL)
- **Zero-shot capabilities**: Strong performance without task-specific training
- **Efficient training**: Only trains Q-Former (~188M parameters)
- **State-of-the-art results**: Beats larger models on VQA benchmarks

**Use alternatives instead:**
- **LLaVA**: For instruction-following multimodal chat
- **InstructBLIP**: For improved instruction-following (BLIP-2 successor)
- **GPT-4V/Claude 3**: For production multimodal chat (proprietary)
- **CLIP**: For simple image-text similarity without generation
- **Flamingo**: For few-shot visual learning

## Quick start
…

Ver todo o marketplace