AI Skill MartBrowse skillsPromptsProgramsLearnWhat is a Skill?
 
← Back to list

A/B Test Setup

Designs statistically valid A/B tests end to end — hypothesis, sample size, launch checklist, and results analysis.

Marketing & SEOIntermediate★ 28,022⑂ 3,926AI score 9/10Last updated: Aug 30, 2026

What it does

  • Turns vague ideas into strong hypotheses using a "Because [data], we believe [change] will cause [outcome] for [audience]" template.
  • Runs the bundled sample_size_calculator.py (two-proportion z-test, α=0.05, 80% power) to get sample size per variant and estimated test duration.
  • Separates primary, secondary, and guardrail metrics, and recommends traffic allocation (50/50, 90/10, ramping).
  • Provides a pre-launch QA checklist, explicit do/don't rules during the test (no peeking, no mid-test edits), and a 6-step results analysis checklist.
  • Produces artifacts: experiment brief, results report, and a prioritized test backlog.

Who it's for

  • Growth and performance marketers, CRO specialists, product managers.
  • Anyone who wants landing page, pricing page, or CTA changes validated by data instead of opinion.
  • Teams using PostHog, Optimizely, VWO, LaunchDarkly, or Split.

Examples

  1. "Baseline conversion is 5%, I want to detect a 20% lift" → calculator returns ~8.2k per variant plus a duration estimate from daily traffic.
  2. Two headline variants are being debated → the skill designs a single-variable A/B test with a documented hypothesis.
  3. After a test ends → generates an analysis report covering significance, effect size, segments, and guardrails.

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/alirezarezvani/claude-skills/HEAD/.gemini/skills/ab-test-setup/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on AI Skill Mart.
Copy the .gemini/skills/ab-test-setup folder from the GitHub repo alirezarezvani/claude-skills into my ~/.claude/skills/ab-test-setup/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/alirezarezvani/claude-skills.git /tmp/claude-skills && mkdir -p ~/.claude/skills && cp -r /tmp/claude-skills/.gemini/skills/ab-test-setup ~/.claude/skills/

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open your terminal.
  2. Clone the repo: git clone https://github.com/alirezarezvani/claude-skills.git
  3. Create the skills folder: mkdir -p ~/.claude/skills
  4. Copy the skill: cp -r claude-skills/.gemini/skills/ab-test-setup ~/.claude/skills/
  5. Verify ~/.claude/skills/ab-test-setup/ contains SKILL.md, scripts/, and references/.
  6. Check Python with python3 --version so the sample size calculator can run.
  7. Restart Claude Code and ask something like "design an A/B test for my pricing page" to trigger the skill.
View source on GitHub ↗License: MIT