Claude Skill MartBrowse skillsQuick linesLearn by videoTerminal guideWhat is a Skill?
 
← Back to list

A/B Test Planner

Produces statistically rigorous A/B test plans with hypothesis, sample size, duration, safety metrics, and a results-interpretation guide.

Data & AnalyticsIntermediate★ 1,437⑂ 257AI score 9/10Last updated: Oct 11, 2026

What it does

  • Triggers on requests like "design an A/B test", "calculate sample size", "set up an experiment", or "interpret test results" and returns a complete test plan.
  • Enforces a directional hypothesis template (change → metric → direction → magnitude → segment → evidence-based rationale) plus explicit Control/Treatment definitions, traffic split, and exclusions.
  • Estimates required sample per variant from baseline rate and MDE at 80% power / 95% confidence, then converts it to a duration using daily eligible traffic (2-week minimum, 4-week maximum).
  • Requires safety metrics protecting revenue, retention, or core engagement, with a concrete rollback threshold, and maps four outcomes: ship / iterate / reject / inconclusive.
  • Flags anti-patterns (peeking and p-hacking, multi-variant comparisons, proxy metrics) and self-scores the deliverable against a 40-point rubric.

Who it's for

  • Product managers and growth leads who run experiments regularly.
  • Startup teams testing onboarding flows, pricing, or UI changes.
  • Analysts who want a consistent experiment design document.

Examples

  1. "Design an A/B test for new signup button copy — baseline 10%, 3,000 daily visitors" → sample per variant, expected weeks, rollback trigger.
  2. "Calculate sample size for cutting onboarding to 3 steps" → estimate with assumptions labeled, plus qualitative alternatives if traffic is under ~1,000/day.
  3. "Treatment is +3% but not significant — should we ship?" → treated as inconclusive with guidance on extending vs. redesigning.

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/mohitagw15856/pm-claude-skills/HEAD/i18n/es/skills/ab-test-planner/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on Claude Skill Mart.
Copy the i18n/es/skills/ab-test-planner folder from the GitHub repo mohitagw15856/pm-claude-skills into my ~/.claude/skills/mohitagw15856-ab-test-planner/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/mohitagw15856/pm-claude-skills.git && mkdir -p ~/.claude/skills && cp -r pm-claude-skills/i18n/es/skills/ab-test-planner ~/.claude/skills/

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open a terminal.
  2. Clone the repo: git clone https://github.com/mohitagw15856/pm-claude-skills.git
  3. Create the skills directory: mkdir -p ~/.claude/skills
  4. Copy the skill (Spanish version): cp -r pm-claude-skills/i18n/es/skills/ab-test-planner ~/.claude/skills/
    • For the English original: cp -r pm-claude-skills/skills/ab-test-planner ~/.claude/skills/
  5. Verify ~/.claude/skills/ab-test-planner/ contains SKILL.md plus the references/ and templates/ folders.
  6. Restart Claude Code and try: "Design an A/B test for our checkout page."
View source on GitHub ↗License: MIT