Claude Skill MartBrowse skillsQuick linesLearn by videoTerminal guideWhat is a Skill?
 
← Back to list

A/B Test Planner

Produces a statistically rigorous A/B test plan with hypothesis, sample size, duration, guardrails, and a results interpretation guide.

Data & AnalyticsIntermediate★ 1,420⑂ 252AI score 8/10Last updated: Oct 3, 2026

What it does

  • Builds a complete experiment plan for feature, UI, onboarding, or pricing tests.
  • Forces a directional hypothesis template (change → metric → direction → magnitude → segment → rationale).
  • Estimates required sample size from baseline rate and MDE at 80% power / 95% confidence, then converts it into a duration (2-week minimum, 4-week maximum).
  • Requires guardrail metrics protecting revenue, retention, or core engagement, plus a concrete rollback threshold.
  • Pre-commits four outcomes: ship / iterate / reject / inconclusive — so results aren't argued about afterwards.
  • Calls out anti-patterns: peeking, multiple comparisons, bundled changes, proxy-metric misuse.

Who it's for

  • Product managers and growth teams standing up an experimentation practice
  • Non-statisticians who still need trustworthy test design
  • Leads standardizing how A/B tests get reviewed and approved

Example uses

  1. "Design an A/B test for new checkout button copy; baseline conversion 5%, 4,000 eligible users/day" → returns sample size, duration, guardrails, and rollback trigger.
  2. "Help me write a hypothesis for cutting onboarding from 3 steps to 2" → asks for supporting data and fills the template.
  3. "We only get 500 users/day — can we test this?" → flags insufficient power and recommends qualitative alternatives like moderated tests or interviews.

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/mohitagw15856/pm-claude-skills/HEAD/exports/openclaw/ab-test-planner/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on Claude Skill Mart.
Copy the exports/openclaw/ab-test-planner folder from the GitHub repo mohitagw15856/pm-claude-skills into my ~/.claude/skills/ab-test-planner/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/mohitagw15856/pm-claude-skills.git /tmp/pm-claude-skills && mkdir -p ~/.claude/skills/ab-test-planner && cp -r /tmp/pm-claude-skills/exports/openclaw/ab-test-planner/* ~/.claude/skills/ab-test-planner/

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open your terminal.
  2. Clone the repo: git clone https://github.com/mohitagw15856/pm-claude-skills.git
  3. Create the skill folder: mkdir -p ~/.claude/skills/ab-test-planner
  4. Copy the files: cp -r pm-claude-skills/exports/openclaw/ab-test-planner/* ~/.claude/skills/ab-test-planner/
  5. Restart Claude Code and ask something like "design an A/B test for…" to trigger the skill.
  6. Have real numbers ready (baseline conversion rate, daily eligible traffic) for the most accurate plan.
View source on GitHub ↗License: MIT