Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c4cb25cb6c | |||
| 7c13fccdd9 |
@@ -1,8 +1,8 @@
|
||||
---
|
||||
name: ab-test-setup
|
||||
description: When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," or "how long should I run this test." Use this whenever someone is comparing two approaches and wants to measure which performs better. For tracking implementation, see analytics-tracking. For page-level conversion optimization, see page-cro.
|
||||
description: When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," "how long should I run this test," "growth experiments," "experiment velocity," "experiment backlog," "ICE score," "experimentation program," or "experiment playbook." Use this whenever someone is comparing two approaches and wants to measure which performs better, or when they want to build a systematic experimentation practice. For tracking implementation, see analytics-tracking. For page-level conversion optimization, see page-cro.
|
||||
metadata:
|
||||
version: 1.1.0
|
||||
version: 1.2.0
|
||||
---
|
||||
|
||||
# A/B Test Setup
|
||||
@@ -229,6 +229,93 @@ Document every test with:
|
||||
|
||||
---
|
||||
|
||||
## Growth Experimentation Program
|
||||
|
||||
Individual tests are valuable. A continuous experimentation program is a compounding asset. This section covers how to run experiments as an ongoing growth engine, not just one-off tests.
|
||||
|
||||
### The Experiment Loop
|
||||
|
||||
```
|
||||
1. Generate hypotheses (from data, research, competitors, customer feedback)
|
||||
2. Prioritize with ICE scoring
|
||||
3. Design and run the test
|
||||
4. Analyze results with statistical rigor
|
||||
5. Promote winners to a playbook
|
||||
6. Generate new hypotheses from learnings
|
||||
→ Repeat
|
||||
```
|
||||
|
||||
### Hypothesis Generation
|
||||
|
||||
Feed your experiment backlog from multiple sources:
|
||||
|
||||
| Source | What to Look For |
|
||||
|--------|-----------------|
|
||||
| Analytics | Drop-off points, low-converting pages, underperforming segments |
|
||||
| Customer research | Pain points, confusion, unmet expectations |
|
||||
| Competitor analysis | Features, messaging, or UX patterns they use that you don't |
|
||||
| Support tickets | Recurring questions or complaints about conversion flows |
|
||||
| Heatmaps/recordings | Where users hesitate, rage-click, or abandon |
|
||||
| Past experiments | "Significant loser" tests often reveal new angles to try |
|
||||
|
||||
### ICE Prioritization
|
||||
|
||||
Score each hypothesis 1-10 on three dimensions:
|
||||
|
||||
| Dimension | Question |
|
||||
|-----------|----------|
|
||||
| **Impact** | If this works, how much will it move the primary metric? |
|
||||
| **Confidence** | How sure are we this will work? (Based on data, not gut.) |
|
||||
| **Ease** | How fast and cheap can we ship and measure this? |
|
||||
|
||||
**ICE Score** = (Impact + Confidence + Ease) / 3
|
||||
|
||||
Run highest-scoring experiments first. Re-score monthly as context changes.
|
||||
|
||||
### Experiment Velocity
|
||||
|
||||
Track your experimentation rate as a leading indicator of growth:
|
||||
|
||||
| Metric | Target |
|
||||
|--------|--------|
|
||||
| Experiments launched per month | 4-8 for most teams |
|
||||
| Win rate | 20-30% is common for mature programs (sustained higher rates may indicate conservative hypotheses) |
|
||||
| Average test duration | 2-4 weeks |
|
||||
| Backlog depth | 20+ hypotheses queued |
|
||||
| Cumulative lift | Compound gains from all winners |
|
||||
|
||||
### The Experiment Playbook
|
||||
|
||||
When a test wins, don't just implement it — document the pattern:
|
||||
|
||||
```
|
||||
## [Experiment Name]
|
||||
**Date**: [date]
|
||||
**Hypothesis**: [the hypothesis]
|
||||
**Sample size**: [n per variant]
|
||||
**Result**: [winner/loser/inconclusive] — [primary metric] changed by [X%] (95% CI: [range], p=[value])
|
||||
**Guardrails**: [any guardrail metrics and their outcomes]
|
||||
**Segment deltas**: [notable differences by device, segment, or cohort]
|
||||
**Why it worked/failed**: [analysis]
|
||||
**Pattern**: [the reusable insight — e.g., "social proof near pricing CTAs increases plan selection"]
|
||||
**Apply to**: [other pages/flows where this pattern might work]
|
||||
**Status**: [implemented / parked / needs follow-up test]
|
||||
```
|
||||
|
||||
Over time, your playbook becomes a library of proven growth patterns specific to your product and audience.
|
||||
|
||||
### Experiment Cadence
|
||||
|
||||
**Weekly (30 min)**: Review running experiments for technical issues and guardrail metrics. Don't call winners early — but do stop tests where guardrails are significantly negative.
|
||||
|
||||
**Bi-weekly**: Conclude completed experiments. Analyze results, update playbook, launch next experiment from backlog.
|
||||
|
||||
**Monthly (1 hour)**: Review experiment velocity, win rate, cumulative lift. Replenish hypothesis backlog. Re-prioritize with ICE.
|
||||
|
||||
**Quarterly**: Audit the playbook. Which patterns have been applied broadly? Which winning patterns haven't been scaled yet? What areas of the funnel are under-tested?
|
||||
|
||||
---
|
||||
|
||||
## Common Mistakes
|
||||
|
||||
### Test Design
|
||||
|
||||
+2
-47
@@ -2,7 +2,7 @@
|
||||
name: ai-seo
|
||||
description: "When the user wants to optimize content for AI search engines, get cited by LLMs, or appear in AI-generated answers. Also use when the user mentions 'AI SEO,' 'AEO,' 'GEO,' 'LLMO,' 'answer engine optimization,' 'generative engine optimization,' 'LLM optimization,' 'AI Overviews,' 'optimize for ChatGPT,' 'optimize for Perplexity,' 'AI citations,' 'AI visibility,' 'zero-click search,' 'how do I show up in AI answers,' 'LLM mentions,' or 'optimize for Claude/Gemini.' Use this whenever someone wants their content to be cited or surfaced by AI assistants and AI search engines. For traditional technical and on-page SEO audits, see seo-audit. For structured data implementation, see schema-markup."
|
||||
metadata:
|
||||
version: 1.2.0
|
||||
version: 1.1.0
|
||||
---
|
||||
|
||||
# AI SEO
|
||||
@@ -226,50 +226,6 @@ AI systems don't just cite your website — they cite where you appear.
|
||||
- Create YouTube content for key how-to queries
|
||||
- Answer relevant Quora questions with depth
|
||||
|
||||
### Machine-Readable Files for AI Agents
|
||||
|
||||
AI agents aren't just answering questions — they're becoming buyers. When an AI agent evaluates tools on behalf of a user, it needs structured, parseable information. If your pricing is locked in a JavaScript-rendered page or a "contact sales" wall, agents will skip you and recommend competitors whose information they can actually read.
|
||||
|
||||
Add these machine-readable files to your site root:
|
||||
|
||||
**`/pricing.md` or `/pricing.txt`** — Structured pricing data for AI agents
|
||||
|
||||
```markdown
|
||||
# Pricing — [Your Product Name]
|
||||
|
||||
## Free
|
||||
- Price: $0/month
|
||||
- Limits: 100 emails/month, 1 user
|
||||
- Features: Basic templates, API access
|
||||
|
||||
## Pro
|
||||
- Price: $29/month (billed annually) | $35/month (billed monthly)
|
||||
- Limits: 10,000 emails/month, 5 users
|
||||
- Features: Custom domains, analytics, priority support
|
||||
|
||||
## Enterprise
|
||||
- Price: Custom — contact sales@example.com
|
||||
- Limits: Unlimited emails, unlimited users
|
||||
- Features: SSO, SLA, dedicated account manager
|
||||
```
|
||||
|
||||
**Why this matters now:**
|
||||
- AI agents increasingly compare products programmatically before a human ever visits your site
|
||||
- Opaque pricing gets filtered out of AI-mediated buying journeys
|
||||
- A simple markdown file is trivially parseable by any LLM — no rendering, no JavaScript, no login walls
|
||||
- Same principle as `robots.txt` (for crawlers), `llms.txt` (for AI context), and `AGENTS.md` (for agent capabilities)
|
||||
|
||||
**Best practices:**
|
||||
- Use consistent units (monthly vs. annual, per-seat vs. flat)
|
||||
- Include specific limits and thresholds, not just feature names
|
||||
- List what's included at each tier, not just what's different
|
||||
- Keep it updated — stale pricing is worse than no file
|
||||
- Link to it from your sitemap and main pricing page
|
||||
|
||||
**`/llms.txt`** — Context file for AI systems (see [llmstxt.org](https://llmstxt.org))
|
||||
|
||||
If you don't have one yet, add an `llms.txt` that gives AI systems a quick overview of what your product does, who it's for, and links to key pages (including your pricing).
|
||||
|
||||
### Schema Markup for AI
|
||||
|
||||
Structured data helps AI systems understand your content. Key schemas:
|
||||
@@ -353,7 +309,7 @@ Monthly manual check:
|
||||
- Feature comparison tables (you vs. category, not just competitors)
|
||||
- Specific metrics ("processes 10,000 transactions/sec" not "blazing fast")
|
||||
- Customer count or social proof with numbers
|
||||
- Pricing transparency (AI cites pages with visible pricing) — add a `/pricing.md` file so AI agents can parse your plans without rendering your page (see "Machine-Readable Files" above)
|
||||
- Pricing transparency (AI cites pages with visible pricing)
|
||||
- FAQ section addressing common buyer questions
|
||||
|
||||
### Blog Content
|
||||
@@ -402,7 +358,6 @@ Monthly manual check:
|
||||
- **Ignoring third-party presence** — You may get more AI citations from a Wikipedia mention than from your own blog
|
||||
- **No structured data** — Schema markup gives AI systems structured context about your content
|
||||
- **Keyword stuffing** — Unlike traditional SEO where it's just ineffective, keyword stuffing actively reduces AI visibility by 10% (Princeton GEO study)
|
||||
- **Hiding pricing behind "contact sales" or JS-rendered pages** — AI agents evaluating your product on behalf of buyers can't parse what they can't read. Add a `/pricing.md` file
|
||||
- **Blocking AI bots** — If GPTBot, PerplexityBot, or ClaudeBot are blocked in robots.txt, those platforms can't cite you
|
||||
- **Generic content without data** — "We're the best" won't get cited. "Our customers see 3x improvement in [metric]" will
|
||||
- **Forgetting to monitor** — You can't improve what you don't measure. Check AI visibility monthly at minimum
|
||||
|
||||
Reference in New Issue
Block a user