# A/B test setup Human Guide

## What This Is For
When the user wants to plan, design, or implement an A/B test or experiment. It gives the agent a clearer input/output frame for A/B test setup: what context to ask for, what decisions to make, and what usable artifact to return.

Use this as a human-readable version of the A/B test setup agent skill. It is meant for marketers, operators, founders, and other non-coders who want the workflow without reading agent-specific implementation instructions.

## When To Use This
- Use this when you need a repeatable process for A/B test setup.
- Use this when the task needs judgment, examples, constraints, or a clear output format rather than a one-off prompt.
- Use this when you want to hand an AI assistant enough context to produce a usable marketing artifact.

## When Not To Use This
- Do not use this when you only need a quick factual answer.
- Do not use this when the work depends on private data you cannot share with the assistant.
- Do not use this as a replacement for legal, compliance, financial, or medical review.

## What You Need Before Starting
- The goal or business outcome you want.
- The audience, customer segment, or market context.
- Any source material the assistant should respect, such as notes, briefs, examples, URLs, or brand guidance.
- Constraints such as tone, length, channel, deadline, region, or approval requirements.
- A clear definition of what a good final answer should look like.

## Step-By-Step Workflow
1. State the job clearly: "Use the A/B test setup guide to help me with..."
2. Add context: audience, goal, offer, channel, source material, and constraints.
3. Ask the assistant to identify missing inputs before producing the final output.
4. Have the assistant follow the skill-specific guidance below.
5. Review the result against the final checklist and ask for revisions where needed.

## Skill-Specific Guidance
- **Test Context** - What are you trying to improve? What change are you considering?
- **Current State** - Baseline conversion rate? Current traffic volume?
- **Constraints** - Technical complexity? Timeline? Tools available?
- Not just "let's see what happens"
- Specific prediction of outcome
- Otherwise you don't know what worked
- Primary metric tied to business value
- Secondary metrics for context
- Guardrail metrics to prevent harm
- [Evan Miller's](https://www.evanmiller.org/ab-testing/sample-size.html)
- [Optimizely's](https://www.optimizely.com/sample-size-calculator/)
- Single metric that matters most

## Decision Points And Nuance
The original skill emphasizes: Workspace Context, Operating Contract, Initial Assessment, Core Principles, Start with a Hypothesis, Test One Thing, Statistical Rigor, Measure What Matters, Hypothesis Framework, Structure.

Use these questions to steer the work:
- What is the intended audience or buyer?
- What source material must be preserved?
- What should the assistant optimize for: clarity, persuasion, accuracy, speed, creativity, or conversion?
- What examples represent the desired quality bar?
- What should the assistant avoid?

## Common Mistakes
- Otherwise you don't know what worked
- Don't peek and stop early
- Avoid running tests longer than 4-8 weeks:
- **Fix**: Be realistic about MDE, get more traffic, or don't test
- **Don't test**: Make decision based on qualitative data instead
- If you must check results before reaching sample size:
- For simple tests that don't need full documentation:

## Copy-And-Paste Prompt
```text
Use the A/B test setup human guide.

My goal:
[Describe the business outcome]

Audience:
[Describe who this is for]

Context and source material:
[Paste notes, examples, links, or existing copy]

Constraints:
[Tone, length, channel, timeline, must-include items, must-avoid items]

Before producing the final output, ask me for any missing information that would materially improve the result.
```

## Final Checklist
- [ ] The output matches the original goal.
- [ ] The audience and context are reflected in the answer.
- [ ] Important constraints and source material were preserved.
- [ ] The assistant made the relevant decisions explicit.
- [ ] The final artifact is ready to use, review, or hand to the next person.

## Source
This guide was generated from the manojbajaj95/claude-gtm-plugin skill entry for `ab-test-setup`.

## Source Skill Notes
These notes preserve the nuance from the original skill. Use them as supporting reference when the workflow above feels too generic.

# A/B Test Setup

## Workspace Context

Read bootstrap context before asking questions: `strategy/brand.md` for brand, audience, offer, channels, tools, constraints, and metrics; `about/me.md` for personal voice; `content/ideas.md` and `content/calendar.md` for content planning. Use legacy product-marketing context files only as fallback. Save generated drafts to `content/<platform>/drafts/YYYY-MM-DD_short-topic-slug.md`, and route durable learnings back to `strategy/brand.md`, `about/me.md`, or `content/ideas.md`.

## Operating Contract

This skill is self-contained for its frontmatter scope: use its local instructions, references, scripts, and assets as the playbook; ask only for missing task-specific inputs; hand off to adjacent skills instead of expanding scope; and return an actionable artifact, decision, plan, draft, or diagnostic.


You are an expert in experimentation and A/B testing. Your goal is to help design tests that produce statistically valid, actionable results.

## Initial Assessment

Before designing a test, understand:

1. **Test Context** - What are you trying to improve? What change are you considering?
2. **Current State** - Baseline conversion rate? Current traffic volume?
3. **Constraints** - Technical complexity? Timeline? Tools available?

---

## Core Principles

### 1. Start with a Hypothesis
- Not just "let's see what happens"
- Specific prediction of outcome
- Based on reasoning or data

### 2. Test One Thing
- Single variable per test
- Otherwise you don't know what worked

### 3. Statistical Rigor
- Pre-determine sample size
- Don't peek and stop early
- Commit to the methodology

### 4. Measure What Matters
- Primary metric tied to business value
- Secondary metrics for context
- Guardrail metrics to prevent harm

---

## Hypothesis Framework

### Structure

```
Because [observation/data],
we believe [change]
will cause [expected outcome]
for [audience].
We'll know this is true when [metrics].
```

### Example

**Weak**: "Changing the button color might increase clicks."

**Strong**: "Because users report difficulty finding the CTA (per heatmaps and feedback), we believe making the button larger and using contrasting color will increase CTA clicks by 15%+ for new visitors. We'll measure click-through rate from page view to signup start."

---

## Test Types

| Type | Description | Traffic Needed |
|------|-------------|----------------|
| A/B | Two versions, single change | Moderate |
| A/B/n | Multiple variants | Higher |
| MVT | Multiple changes in combinations | Very high |
| Split URL | Different URLs for variants | Moderate |

---

## Sample Size

### Quick Reference