Claude Skill MartBrowse skillsQuick linesLearn by videoTerminal guideWhat is a Skill?
 
← Back to list

Experiment Suite

Turns a research question into a complete experiment package: design doc, runnable code, results.json with provenance, publication-grade figures, and a structured report.

Data & AnalyticsAdvanced★ 176⑂ 17AI score 9/10Last updated: Jul 28, 2026

What it does

Builds an end-to-end experiment package from a single research question:

  • experiment_design.md — motivation → hypothesis → datasets → baselines → metrics → ablations → compute budget (≥ 700 words, every choice justified)
  • data_contract.md — data source, access route, version, split, and reuse boundary
  • experiment/ — model.py, data.py, train.py, evaluate.py, config.yaml, requirements.txt, launchable via python train.py --config config.yaml
  • results.json — per-seed entries, per-method/per-metric mean & std, ablation block, explicit provenance (measured / simulated / illustrative)
  • figures/ — 3–6 publication-grade charts plus their make_*.py sources and a basename-only manifest.json
  • experiment_report.md — problem → design → method → setup → results → analysis → limitations

It also enforces an honesty policy: simulated numbers stay labelled in the JSON, figure captions, report disclosure, and any downstream paper. Reference playbooks under references/ cover incremental execution, figure QA, and a final quality gate.

Who it's for

  • Researchers and grad students who repeatedly design, run, and write up experiments
  • ML / data science engineers who need reproducible, documented experiment packages
  • Anyone comparing multiple methods and needing conference-quality figures
  • Users who want to hand results off to a paper-writer pipeline

Examples

  1. "Does a Transformer beat LightGBM on retail demand forecasting?" → data contract, PyTorch skeleton, 3-seed results.json, comparison + ablation figures, full report
  2. You already have measured logs → run in measured mode; simulated: false and a provenance path are recorded, figures and report rebuild from real numbers
  3. Early planning with no compute → simulated mode produces the whole package shape with "simulated" watermarks on every figure, usable as a proposal skeleton

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/ai4s-research/ai4s-skills/HEAD/skills/experiment-suite/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on Claude Skill Mart.
Copy the skills/experiment-suite folder from the GitHub repo ai4s-research/ai4s-skills into my ~/.claude/skills/experiment-suite/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/ai4s-research/ai4s-skills.git && mkdir -p ~/.claude/skills && cp -r ai4s-skills/skills/experiment-suite ~/.claude/skills/

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open a terminal.
  2. Clone the repository: git clone https://github.com/ai4s-research/ai4s-skills.git
  3. Create the skills directory: mkdir -p ~/.claude/skills
  4. Copy the skill: cp -r ai4s-skills/skills/experiment-suite ~/.claude/skills/
  5. Verify that references/ and figure_examples/ came along — they drive design depth and figure quality.
  6. From your project folder run claude and ask: "Use the experiment-suite skill to build an experiment package for <your research question>."
  7. Outputs land in output/experiment-suite/<slug>/latest/. Install Python 3, matplotlib, and your ML framework (e.g. PyTorch) if you plan to actually run the generated code.
View source on GitHub ↗License: MIT