Claude Skill MartBrowse skillsQuick linesLearn by videoTerminal guideWhat is a Skill?
 
← Back to list

A/B Test Setup

An experimentation expert skill that turns vague ideas into statistically valid A/B tests and builds a repeatable growth experimentation program.

Marketing & SEOIntermediate★ 53,594⑂ 7,972AI score 9/10Last updated: Oct 7, 2026

What it does

  • Hypothesis framework: converts "let's see what happens" into a structured, falsifiable prediction with a named audience and measurable outcome.
  • Sample size & duration: quick-reference table by baseline conversion rate and expected lift, plus links to Evan Miller and Optimizely calculators and a deeper references/sample-size-guide.md.
  • Metric design: separates primary, secondary, and guardrail metrics so you don't win a test while hurting the business.
  • Execution discipline: pre-launch QA checklist, traffic allocation options (50/50, 90/10, ramping), client- vs server-side implementation notes, and an explicit warning about the peeking problem.
  • Growth program: ICE prioritization, velocity targets (4–8 tests/month, 20–30% win rate), an experiment playbook template for capturing reusable patterns, and a weekly/bi-weekly/monthly/quarterly cadence.

Who it's for

  • Growth and performance marketers optimizing landing, pricing, or signup pages
  • PMs and founders validating a change before rolling it out
  • Teams trying to institutionalize experimentation rather than run one-off tests

Example uses

  1. "I want to test two CTA headlines on the pricing page — how long do I run it?" → it asks for baseline rate and traffic, then computes required sample per variant and expected duration.
  2. "Would a bigger button increase signups?" → it rewrites the idea as a strong hypothesis and proposes primary, secondary, and guardrail metrics.
  3. "I have 20 experiment ideas queued" → it builds an ICE score table, orders the backlog, and sets a monthly review cadence.

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/coreyhaines31/marketingskills/HEAD/skills/ab-testing/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on Claude Skill Mart.
Copy the skills/ab-testing folder from the GitHub repo coreyhaines31/marketingskills into my ~/.claude/skills/ab-testing/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/coreyhaines31/marketingskills.git /tmp/marketingskills && mkdir -p ~/.claude/skills && cp -r /tmp/marketingskills/skills/ab-testing ~/.claude/skills/

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open a terminal (Terminal on macOS, Git Bash on Windows).
  2. Clone the repo: git clone https://github.com/coreyhaines31/marketingskills.git
  3. Create the skills directory: mkdir -p ~/.claude/skills
  4. Copy just this skill: cp -r marketingskills/skills/ab-testing ~/.claude/skills/
  5. Confirm the references/ folder (sample-size-guide.md, test-templates.md) came along.
  6. Restart Claude Code and try: "Design an A/B test for my pricing page CTA." The skill triggers on phrases like "A/B test", "split test", "statistical significance", or "ICE score".
View source on GitHub ↗License: MIT