Claude Skill MartBrowse skillsQuick linesLearn by videoTerminal guideWhat is a Skill?
 
← Back to list

RWKV Architecture Guide

A code-first skill for installing, running, and fine-tuning RWKV — the RNN+Transformer hybrid with O(n) inference and constant-memory infinite context.

Dev & CodingAdvanced★ 13,448⑂ 948AI score 7/10Last updated: Oct 11, 2026

What it does

  • Provides install steps and working snippets for RWKV inference in both GPT (parallel) and RNN (sequential) modes.
  • Shows streaming token generation and state-passing patterns for processing million-token documents with O(1) memory (no KV cache).
  • Gives a PyTorch Lightning + DeepSpeed fine-tuning template plus fixes for OOM, slow inference, and lost state.
  • Compares RWKV against Transformers, Mamba, RetNet and Hyena, with per-model VRAM requirements.

Who it's for

  • ML engineers deploying long-context models under tight memory budgets.
  • Developers building streaming or real-time generation apps.
  • Researchers evaluating alternatives to attention-based architectures.

Examples

  1. "Load the 1.5B RWKV checkpoint with cuda fp16 and write a token-by-token streaming generator."
  2. "Build a pipeline that chunks a 1M-token document and carries state across forward calls."
  3. "Set up an 8-GPU DeepSpeed stage-2 fine-tuning script for RWKV and reduce OOM errors."

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on Claude Skill Mart.
Copy the 01-model-architecture/rwkv folder from the GitHub repo Orchestra-Research/AI-Research-SKILLs into my ~/.claude/skills/rwkv-architecture/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/Orchestra-Research/AI-Research-SKILLs.git && mkdir -p ~/.claude/skills && cp -r AI-Research-SKILLs/01-model-architecture/rwkv ~/.claude/skills/rwkv-architecture

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open a terminal.
  2. Clone the repository: git clone https://github.com/Orchestra-Research/AI-Research-SKILLs.git
  3. Create the skills folder: mkdir -p ~/.claude/skills
  4. Copy this skill: cp -r AI-Research-SKILLs/01-model-architecture/rwkv ~/.claude/skills/rwkv-architecture
  5. Restart Claude Code, then ask something like "help me run long-context inference with RWKV".
  6. For real runs you also need an NVIDIA GPU (CUDA 11.6+) and pip install torch rwkv.
View source on GitHub ↗License: MIT