AI Skill MartBrowse skillsPromptsProgramsLearnWhat is a Skill?
 
← Back to list

A/B Test Plan Builder

Builds a full experiment spec — hypothesis, per-variant sample size, run length, guardrails and stopping rules — using real statistical scripts.

Marketing & SEOIntermediate★ 936⑂ 153AI score 9/10Last updated: Nov 7, 2026

What it does

  • Frames the hypothesis in an If / Then / Because format backed by data, research or a UX principle.
  • Computes sample size by running sample-size-calculator.py with your baseline rate, MDE, significance (default 95%) and power (default 80%). It forces you to confirm whether the MDE is absolute or relative — the single most common error, which can change required sample size by roughly 200× at a 5% baseline.
  • Estimates duration and feasibility from daily traffic, enforcing a minimum 7-day business cycle and flagging anything over 8 weeks as a validity risk.
  • Defines monitoring and stopping rules: interim QA checkpoints, sample ratio mismatch (SRM) detection, guardrail alerts, and go/no-go thresholds for winner, loser and inconclusive outcomes.
  • Pre-registers the plan before launch and, after the test, evaluates results with significance-tester.py.

Who it's for

  • CRO and growth practitioners optimising landing pages, CTAs and checkout flows
  • Marketers who keep getting asked "how many visitors per variant do we need?"
  • PMs and designers who want tests called by statistical criteria, not gut feel
  • Teams designing multivariate tests across email subject lines or pricing layouts

Examples

  1. Headline test: 5% baseline, 2,000 visitors/day — compute the sample size needed to detect a +1.0 percentage-point absolute lift and the resulting run length.
  2. Checkout form simplification: primary metric is completion rate, with bounce rate and AOV as guardrails that trigger early stoppage if revenue degrades.
  3. Low-traffic reality check: a page with 150 visitors/day would need 12 weeks, so the skill recommends fewer variants, a larger MDE, or qualitative research instead.

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/indranilbanerjee/digital-marketing-pro/HEAD/skills/ab-test-plan/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on AI Skill Mart.
Copy the skills/ab-test-plan folder from the GitHub repo indranilbanerjee/digital-marketing-pro into my ~/.claude/skills/ab-test-plan/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/indranilbanerjee/digital-marketing-pro.git && mkdir -p ~/.claude/skills && cp -r digital-marketing-pro/skills/ab-test-plan ~/.claude/skills/

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open a terminal and check Python: python3 --version (3.10 or newer required).
  2. Clone the repo: git clone https://github.com/indranilbanerjee/digital-marketing-pro.git
  3. Create the skills directory: mkdir -p ~/.claude/skills
  4. Copy the skill: cp -r digital-marketing-pro/skills/ab-test-plan ~/.claude/skills/
  5. Keep the repo's scripts/ folder available (the calculator and significance tester live there) and, if your host doesn't set it, export CLAUDE_PLUGIN_ROOT to the repo path.
  6. Optional: run /digital-marketing-pro:brand-setup to create a brand profile under ~/.claude-marketing/brands/ for voice and compliance context.
  7. Restart Claude Code and ask something like: "Plan an A/B test for my landing page headline — 5% current conversion rate, 2,000 visitors per day."
View source on GitHub ↗License: MIT