Claude Skill MartBrowse skillsQuick linesLearn by videoTerminal guideWhat is a Skill?
 
← Back to list

A/B Test Plan Builder

Produces a statistically rigorous A/B or multivariate test spec — hypothesis, sample size, guardrails, and stopping rules.

Marketing & SEOIntermediate★ 728⑂ 124AI score 8/10Last updated: Aug 9, 2026

What it does

Give it the element you want to test (headline, CTA button, pricing layout, checkout flow), your baseline conversion rate, and daily traffic — and it returns a complete, pre-registered experiment specification.

  • If/Then/Because hypothesis statement grounded in data or UX principle
  • Sample size calculation at 95% significance and 80% power by default
  • Absolute vs. relative MDE clarification — the most common sample-size mistake; at a 5% baseline the two readings differ by roughly 200×
  • Test duration estimate (7-day minimum business cycle, validity warning past 8 weeks)
  • Primary, secondary, and guardrail metric definitions
  • Monitoring plan with SRM (sample ratio mismatch) detection and technical QA checkpoints (not statistical peeking)
  • Stopping rules and Go/No-Go criteria, plus a protocol for inconclusive results
  • Traffic feasibility check with low-traffic alternatives

Who it's for

  • CRO and growth practitioners running a real experimentation program
  • Marketers who have been eyeballing "is this significant?"
  • Low-traffic startups that need to know whether a test is even feasible
  • Product teams that want experiment learnings documented as an asset

Examples

  1. Landing page headline test: "3.2% conversion, 1,800 visitors/day, want to catch a +0.5pp lift" → per-variant sample size, estimated run days, and a flag if it exceeds 8 weeks.
  2. Checkout flow shortening: designs a 2-step variant against the 3-step control, sets completion rate as primary, and monitors refund rate and AOV as guardrails.
  3. Multivariate design: builds a 2 CTA copies × 2 button colors matrix, and if traffic is thin, recommends cutting variants, raising the MDE, or switching to qualitative research.

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/indranilbanerjee/digital-marketing-pro/HEAD/skills/ab-test-plan/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on Claude Skill Mart.
Copy the skills/ab-test-plan folder from the GitHub repo indranilbanerjee/digital-marketing-pro into my ~/.claude/skills/ab-test-plan/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/indranilbanerjee/digital-marketing-pro.git && mkdir -p ~/.claude/skills && cp -r digital-marketing-pro/skills/ab-test-plan ~/.claude/skills/

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open a terminal.
  2. Clone the repo: git clone https://github.com/indranilbanerjee/digital-marketing-pro.git
  3. Create the skills directory: mkdir -p ~/.claude/skills
  4. Copy the skill: cp -r digital-marketing-pro/skills/ab-test-plan ~/.claude/skills/
  5. This skill calls scripts/sample-size-calculator.py, significance-tester.py, and campaign-tracker.py. Copy the repo's scripts/ folder too, or install the whole repo as a Claude Code plugin so ${CLAUDE_PLUGIN_ROOT} resolves correctly.
  6. Python 3 is required; if imports fail, run pip install scipy.
  7. Restart Claude Code and try: "Plan an A/B test for my landing page CTA button."
  8. For brand voice, run /digital-marketing-pro:brand-setup first to create a brand profile. Without one, the skill falls back to defaults.
View source on GitHub ↗License: MIT