Semantic Topic Clustering (v1.9.0)

SERP-overlap-driven keyword clustering for content architecture. Groups keywords by how Google actually ranks them (shared top-10 results), not by text similarity. Designs hub-and-spoke content clusters with internal link matrices and generates interactive cluster map visualizations.

Scripts: Located at the plugin root scripts/ directory.

---

Quick Reference

---

Planning Workflow

Step 1: Seed Keyword Expansion

Expand the seed keyword into 30-50 variants using WebSearch:

1. Related searches — Search the seed, extract "related searches" and "people also search for"
2. People Also Ask (PAA) — Extract all PAA questions from SERP results
3. Long-tail modifiers — Append common modifiers: "best", "how to", "vs", "for beginners", "tools", "examples", "guide", "template", "mistakes", "checklist"
4. Question mining — Generate who/what/when/where/why/how variants
5. Intent modifiers — Add commercial modifiers: "pricing", "review", "alternative", "comparison", "free", "top"

Deduplication: Normalize variants (lowercase, strip articles), remove exact duplicates. Target: 30-50 unique keyword variants. If under 30, run a second expansion pass with the top PAA questions as seeds.

Step 2: SERP Overlap Clustering

This is the core differentiator. Load references/serp-overlap-methodology.md for the full algorithm.

Process:

1. Group keywords by initial intent guess (reduces pairwise comparisons)
2. For each candidate pair within a group, WebSearch both keywords
3. Count shared URLs in the top 10 organic results (ignore ads, featured snippets, PAA)
4. Apply thresholds:

Optimization: With 40 keywords, full pairwise = 780 comparisons. Instead:

• Pre-group by intent (4 groups of ~10 = 4 x 45 = 180 comparisons)
• Only cross-check group boundary keywords
• Skip pairs where both are long-tail variants of the same head term (assume same cluster)

DataForSEO integration: If DataForSEO MCP is available, use serp_organic_live_advanced instead of WebSearch for SERP data. Run python3 scripts/dataforseo_costs.py check serp_organic_live_advanced --count N before each batch. If "status": "needs_approval", show cost estimate and ask user. If "status": "blocked", fall back to WebSearch.

Step 3: Intent Classification

Classify each keyword into one of four intent categories:

Remove navigational keywords from clustering. Flag borderline cases for manual review. Keywords can have mixed intent (e.g., "best CRM software" is both commercial and informational) -- classify by dominant intent.

Step 4: Hub-and-Spoke Architecture

Load references/hub-spoke-architecture.md for full specifications.

Design the cluster structure:

1. Select the pillar keyword — Highest volume, broadest intent, most SERP overlap with other keywords
2. Group spokes into clusters — Each cluster is a subtopic area (2-5 clusters per pillar)
3. Assign posts to clusters — Each cluster gets 2-4 spoke posts
4. Select templates per post — Based on intent classification:

1. Set word count targets:

   • Pillar page: 2500-4000 words
   • Spoke posts: 1200-1800 words

2. Cannibalization check — No two posts share the same primary keyword. If SERP overlap is 7+, merge those keywords into a single post targeting both.

Step 5: Internal Link Matrix

Design the bidirectional linking structure:

Rules:

• Every post must have minimum 3 incoming internal links
• No orphan pages (every post reachable from pillar in 2 clicks)
• Anchor text must use target keyword or close variant (no "click here")
• Link placement: within body content, not just navigation/sidebar

Generate the link matrix as a JSON adjacency list:

{
  "links": [
    { "from": "pillar", "to": "cluster-0-post-0", "type": "mandatory", "anchor": "keyword" },
    { "from": "cluster-0-post-0", "to": "pillar", "type": "mandatory", "anchor": "keyword" }
  ]
}

Step 6: Interactive Cluster Map

Generate cluster-map.html using the template at templates/cluster-map.html.

1. Read the template file
2. Build the CLUSTER_DATA JSON object from the cluster plan:

   {
     pillar: { title, keyword, volume, template, wordCount, url },
     clusters: [{ name, color, posts: [{ title, keyword, volume, template, wordCount, url, status }] }],
     links: [{ from, to, type }],
     meta: { totalPosts, totalClusters, totalLinks, estimatedWords }
   }
   

3. Replace the CLUSTER_DATA placeholder in the template with the actual JSON
4. Write the completed HTML file to the output directory
5. Inform user: "Open cluster-map.html in a browser to explore the interactive cluster map."

---

Strategy Import

When invoked with --from strategy:

1. Look for the most recent /seo plan output in the current directory (search for files matching *SEO*Plan*, *strategy*, *content-strategy*)
2. Parse markdown tables for: keywords, page types, content pillars, URL structures
3. Validate extracted data: check for duplicates, missing keywords, incomplete entries
4. Enrich with SERP data: run SERP overlap analysis on extracted keywords
5. Build cluster plan using the imported keywords as the starting set (skip Step 1)

If no strategy file is found, prompt the user: "No existing SEO plan found in the current directory. Run /seo plan first, or provide a seed keyword for fresh clustering."

---

Execution Workflow

When /seo cluster execute is invoked:

Check for claude-blog

Test: Does ~/.claude/skills/blog/SKILL.md exist?

If claude-blog IS installed:

1. Load references/execution-workflow.md for the full algorithm
2. Read cluster-plan.json from the current directory
3. Check for resume state: scan output directory for already-written posts
4. Execute in priority order: pillar first, then spokes by volume (highest first)
5. For each post, invoke the blog-write skill with cluster context:
   • Cluster role (pillar or spoke)
   • Position in cluster (cluster index, post index)
   • Target keyword and secondary keywords
   • Template type and word count target
   • Internal links to include (with anchors)
   • Links to receive from future posts (placeholder markers)
6. After each post is written, scan previous posts for backward link placeholders and inject the new post's URL
7. After all posts are written, generate the cluster scorecard

If claude-blog is NOT installed:

1. Generate detailed content briefs for each post in the cluster plan
2. Each brief includes:
   • Title and meta description
   • Primary keyword and secondary keywords
   • Template type and suggested structure (H2/H3 outline)
   • Word count target
   • Internal links to include (with anchor text)
   • Key points to cover
   • Competing pages to differentiate from
3. Write briefs to cluster-briefs/ directory as individual markdown files
4. Inform user: "Install claude-blog (https://github.com/AgriciDaniel/claude-blog) to auto-create content. Briefs saved to cluster-briefs/."

---

Cluster Scorecard

Post-execution quality report. Run automatically after /seo cluster execute or on demand via analysis of the output directory.

---

Map Regeneration

When /seo cluster map is invoked:

1. Read cluster-plan.json from the current directory
2. Scan output directory and update post statuses (planned vs written)
3. Regenerate cluster-map.html with updated statuses
4. Report: posts written vs planned, link completion percentage

---

Output Files

All outputs are written to the current working directory:

---

Cross-Skill Integration

After cluster planning or execution completes, offer: "Generate a PDF report? Use /seo google report"

---

Error Handling

---

Security

• All URLs fetched via python3 scripts/render_page.py --mode auto (SPA-aware SSRF protection via url_safety)
• No credentials stored or transmitted
• Output files contain no PII or API keys
• DataForSEO cost checks run before every API call

FLOW Framework Integration

For prompt-guided keyword research and gap analysis, use /seo flow find [url|topic] — FLOW's 5 find-stage prompts complement the SERP-overlap clustering methodology with structured discovery prompts.

---

Supporting file: references/execution-workflow.md

Execution Workflow

Overview

The execution phase transforms a cluster-plan.json into actual content. It handles priority ordering, context injection for the blog writer, backward link updates, resume capability, and post-execution quality scoring.

Priority Algorithm

Content is created in this strict order:

1. Pillar page first -- The hub must exist before any spokes can link to it
2. Spokes by search volume (descending) -- Highest-volume spokes first for maximum early impact
3. Within same volume, by cluster index -- Process Cluster 0 before Cluster 1
4. Within same cluster, by post index -- Process Post 0 before Post 1

Rationale: The pillar establishes the topical authority foundation. High-volume spokes generate the most organic traffic, so they should be published earliest for faster compounding returns.

Cluster Context Injection

When invoking blog-write for each post, pass a structured context block:

{
  "cluster_context": {
    "role": "pillar|spoke",
    "pillar_title": "The Complete Guide to ...",
    "pillar_url": "/guide/...",
    "cluster_name": "Cluster Name",
    "cluster_index": 0,
    "post_index": 0,
    "primary_keyword": "target keyword",
    "secondary_keywords": ["variant 1", "variant 2"],
    "template": "how-to",
    "word_count_target": 1500,
    "outgoing_links": [
      { "url": "/pillar-url", "anchor": "main topic guide", "type": "mandatory" },
      { "url": "/sibling-post", "anchor": "related subtopic", "type": "recommended" }
    ],
    "incoming_link_placeholder": "",
    "differentiation_note": "This post should focus on X, while sibling post covers Y"
  }
}

Context Fields Explained

Backward Link Injection

After each new post is written, update previously written posts to link to it:

Process

1. Read the link matrix from cluster-plan.json
2. Identify all posts that should link TO the newly written post
3. For each of those posts (that is already written): a. Open the post file b. Search for the placeholder comment: `` c. Replace the placeholder with an actual contextual link d. If no placeholder found, append a contextual link in the most relevant section
4. Log all backward links added

Placeholder Format

This is inserted during content creation at a contextually appropriate location. When the target post is later written, the placeholder is replaced with:

For a deeper dive, see our guide on <a href="/target-url">anchor text</a>.

Resume Capability

Execution can be interrupted and resumed. The resume algorithm:

Detection

1. Read cluster-plan.json from the current directory
2. Scan the output directory for existing post files
3. Match found files against the plan using:
   • Filename patterns (slug derived from title or keyword)
   • Content inspection (check for primary_keyword in frontmatter or first H1)
4. Mark matched posts as "status": "written" in the plan

Resume Logic

1. Load the plan with updated statuses
2. Filter to "status": "planned" posts only
3. Apply the priority algorithm to the remaining posts
4. Continue execution from the next unwritten post
5. Run backward link injection for any links between newly written and previously written posts

Edge Cases

• If the pillar is missing but spokes exist, write the pillar first and then inject backward links into existing spokes
• If a spoke file exists but is incomplete (under 50% of target word count), treat it as unwritten and recreate
• If cluster-plan.json has been modified since last execution, re-validate the plan before resuming

Scorecard Metrics

After execution completes (or on demand), generate cluster-scorecard.md:

Metric Definitions

Scorecard Output Format

# Cluster Scorecard: [Seed Keyword]

## Summary
- Posts: X/Y written (Z%)
- Total words: N (estimated: M)
- Internal links: L (density: L/Y per post)

## Metrics
| Metric | Score | Status |
|--------|-------|--------|
| Coverage | 100% | PASS |
| Link Density | 3.2/post | PASS |
| ...

## Issues Found
- [List any FAIL or WARN metrics with remediation steps]

## Next Steps
- [Actionable items to reach 100% on all metrics]

Quality Gates

Before marking execution as complete, verify:

1. Every spoke links to the pillar (mandatory)
2. The pillar links to every spoke (mandatory)
3. No post has fewer than 3 incoming internal links
4. No two posts share the same primary keyword
5. No orphan pages exist
6. All posts meet minimum word count (80% of target)

If any gate fails, flag it in the scorecard and provide specific remediation instructions. Do NOT silently pass a failing cluster.

---

Supporting file: references/hub-spoke-architecture.md

Hub-and-Spoke Content Architecture

Structure Overview

A hub-and-spoke cluster consists of one pillar page (the hub) connected to multiple spoke clusters, each containing 2-4 individual posts. The pillar provides broad coverage; spokes provide deep dives into subtopics.

                    [Spoke 1a] --- [Spoke 1b]
                         \       /
                      [Cluster 1]
                           |
[Spoke 2a] -- [Cluster 2] -- [PILLAR] -- [Cluster 3] -- [Spoke 3a]
[Spoke 2b] /                                        \ [Spoke 3b]
                           |
                      [Cluster 4]
                         /       \
                    [Spoke 4a] --- [Spoke 4b]

Pillar Page Specifications

Spoke Page Specifications

Cluster Constraints

Template Auto-Selection by Intent

Selection logic:

1. Match the keyword's classified intent to the table above
2. If multiple templates match, prefer the one whose SERP results show the most similar content format (e.g., if top results are all listicles, use listicle)
3. Avoid duplicate templates within the same cluster unless justified by intent

Internal Link Rules

Mandatory Links

• Every spoke MUST link to the pillar (at least once in body content)
• The pillar MUST link to every spoke (in its relevant section)
• These are non-negotiable -- a cluster without these links is structurally broken

Recommended Links

• Spoke-to-spoke within the same cluster: 2-3 links per post
• Use contextual anchor text (target keyword or close variant)
• Place links within body paragraphs, not just in "related posts" sections

Optional Links

• Cross-cluster spoke-to-spoke: 0-1 links per post
• Only when there is a genuine topical bridge between clusters
• Avoid forcing cross-links that do not add reader value

Minimum Link Requirements

• Every post must have at least 3 incoming internal links
• No orphan pages (every page reachable from pillar within 2 clicks)
• Anchor text diversity: no single anchor text used for more than 40% of links to a page

Cannibalization Prevention

1. No two posts share the same primary keyword. Period.
2. If SERP overlap between two keywords is 7+, merge into a single post
3. After clustering, verify uniqueness: list all primary keywords and check for near-duplicates (e.g., "best CRM" and "top CRM software")
4. If near-duplicates found, either merge the posts or differentiate by intent (e.g., one as "best-of" list, another as "comparison")

JSON-LD Schema Templates

Pillar Page

[
  { "@type": "Article", "headline": "...", "author": {...}, "datePublished": "..." },
  { "@type": "BreadcrumbList", "itemListElement": [
    { "@type": "ListItem", "position": 1, "name": "Home", "item": "..." },
    { "@type": "ListItem", "position": 2, "name": "Pillar Title", "item": "..." }
  ]},
  { "@type": "ItemList", "name": "Topic Cluster", "itemListElement": [
    { "@type": "ListItem", "position": 1, "url": "spoke-1-url" }
  ]}
]

Spoke Page

[
  { "@type": "Article", "headline": "...", "author": {...}, "isPartOf": { "@id": "pillar-url" } },
  { "@type": "BreadcrumbList", "itemListElement": [
    { "@type": "ListItem", "position": 1, "name": "Home", "item": "..." },
    { "@type": "ListItem", "position": 2, "name": "Pillar Title", "item": "pillar-url" },
    { "@type": "ListItem", "position": 3, "name": "Spoke Title", "item": "..." }
  ]}
]

cluster-plan.json Schema

{
  "version": "1.9.0",
  "seed_keyword": "string",
  "created_at": "ISO-8601",
  "pillar": {
    "title": "string",
    "keyword": "string",
    "volume": 0,
    "template": "ultimate-guide",
    "wordCount": 4000,
    "url": "string",
    "status": "planned|written"
  },
  "clusters": [
    {
      "name": "Cluster Name",
      "posts": [
        {
          "title": "string",
          "keyword": "string",
          "volume": 0,
          "template": "string",
          "wordCount": 1500,
          "url": "string",
          "status": "planned|written"
        }
      ]
    }
  ],
  "links": [
    { "from": "pillar", "to": "cluster-0-post-0", "type": "mandatory", "anchor": "keyword" }
  ],
  "serp_matrix": {
    "keywords": ["string"],
    "scores": [[0]]
  },
  "scorecard": {
    "coverage": 0.0,
    "linkDensity": 0.0,
    "orphanPages": 0,
    "cannibalization": 0,
    "contentGaps": 0
  }
}

---

Supporting file: references/serp-overlap-methodology.md

SERP Overlap Methodology

Core Principle

Two keywords that return the same Google results should be targeted by the same page. Two keywords that return completely different results need separate pages. This is the foundation of SERP-based clustering -- using Google's own ranking decisions to determine content architecture rather than relying on keyword text similarity or stemming.

Scoring Algorithm

Step 1: Collect SERP Data

For each keyword in the candidate set, retrieve the top 10 organic results:

• Use WebSearch or DataForSEO serp_organic_live_advanced
• Extract only organic result URLs (ignore ads, featured snippets, PAA, knowledge panels)
• Normalize URLs: strip protocol, trailing slash, and query parameters (except meaningful ones)
• Store as a set of 10 URLs per keyword

Step 2: Pairwise Comparison

For each pair of keywords (A, B):

1. Retrieve the URL sets: urls_A and urls_B
2. Compute overlap: shared = urls_A intersection urls_B
3. Score: overlap_score = len(shared)

Step 3: Apply Thresholds

Step 4: Handle Ambiguous Scores (3-4 Range)

Scores in the 3-4 range require tiebreaking:

1. Check domain overlap (same domains but different pages = closer relationship)
2. Check intent alignment (same intent category = lean toward same cluster)
3. Check volume ratio (if one keyword has 10x+ more volume, it likely deserves its own post)
4. When in doubt, keep in same cluster with separate posts (err toward cohesion)

Optimization Strategy

Full pairwise comparison of N keywords requires N*(N-1)/2 SERP fetches. For 40 keywords, that is 780 comparisons. Optimize by reducing unnecessary checks:

Pre-Grouping

1. Classify all keywords by intent (Informational, Commercial, Transactional)
2. Group keywords that share the same head term (e.g., "CRM software" variants)
3. Only run pairwise SERP comparison within pre-groups
4. Cross-check boundary keywords (highest volume in each group) across groups

Skip Rules

• If keywords A and B are both long-tail variants of the same head term AND share the same intent, assume overlap 4-6 (same cluster) without checking SERP
• If keywords are in different intent categories, assume overlap 0-2 unless they share a head term
• Verify assumptions with spot-check SERP comparisons (sample 20% of skipped pairs)

Scoring Matrix Format

Store the overlap data as a symmetric matrix in cluster-plan.json:

{
  "serp_matrix": {
    "keywords": ["keyword-a", "keyword-b", "keyword-c"],
    "scores": [
      [10, 5, 1],
      [5, 10, 3],
      [1, 3, 10]
    ]
  }
}

Diagonal is always 10 (a keyword overlaps perfectly with itself).

Anti-Patterns

1. Never cluster by text similarity alone. "Dog training tips" and "dog training classes" may have completely different SERPs despite similar text.
2. Never use stemming-only grouping. "Run" and "running" may target different intents entirely.
3. Never assume related searches belong in the same cluster. Verify with SERP data.
4. Never ignore SERP feature differences. If keyword A triggers a local pack and keyword B triggers a featured snippet, they likely need different content types even with moderate URL overlap.
5. Never treat all domains equally. Wikipedia and Reddit appear in many SERPs. Consider filtering out ubiquitous domains (top 5 most common) before scoring, or weighting domain-specific results higher.

Data Source Priority

1. DataForSEO (if available): Most reliable, consistent SERP data. Use serp_organic_live_advanced with location_code: 2840 (US) and language_code: "en".
2. WebSearch (fallback): Adequate for clustering but results may vary by session. Run multiple searches for the same keyword and use the most common result set.

Caching

Within a single clustering session, cache all SERP results. If keyword A's results are fetched for the A-B comparison, reuse them for the A-C comparison. This halves the number of actual SERP fetches needed.