GoodSpace grew 20× on content alone
I've spent the last couple of days reading GoodSpace, an AI recruitment platform out of India. It started as idle curiosity about a domain I recognised and turned into the most interesting growth story I've looked at all year.
Here's what pulled me in. Their organic traffic:
Sep 2025 4,638
Oct 2025 30,335
Nov 2025 47,827
Dec 2025 223,911
Aug 2026 74,355Two flat years, then a wall. Ranking keywords went from 954 to 6,030 in the same window. You can't buy that, and you can't link your way there in sixty days. Somebody built a content engine and it worked.
What they built
Four templated silos, all aimed squarely at Indian job-hunting: leave application formats, offer acceptance letters, CTC explainers, interview questions by technology, resume examples by role. The vocabulary is exact. Casual leave. Experience certificate. Freshers.
The traffic split confirms the aim. 93,000 monthly visits from India in English,
18,700 in Hindi, then Philippines, Pakistan, the UK. The US contributes 770,
despite the .ai domain.
What makes it impressive is the authority they did it on. Their backlink profile looks large at 33,000 links, but most of it is inherited noise pointing at a different organisation entirely. Strip that out and roughly 120 genuine referring domains remain. They're pulling 74,000 visits a month on a link profile that belongs to a site a tenth their size. The content is carrying all of it.
The gap
Their two best pages after the homepage are the free tools: an ATS resume
checker at /premium/ats, and an AI mock interview. Those are where a reader
becomes a user.
I checked what links to them.
homepage 6,163 backlinks 1,587 referring domains
/premium/ats 5 backlinks 5 referring domains
every blog post 0 0Which shows up in the rankings:
| keyword | volume | position |
|---|---|---|
| ats resume | 18,100 | 47 |
| ats cv checker | 12,100 | 29 |
| ats resume checker | 12,100 | 26 |
| resume score | 9,900 | 22 |
They have around four thousand pages about resumes and interviews, and none of them link to the tool that scans your resume. The writing and the product are run as separate concerns, so the traffic arrives and then has nowhere obvious to go.
That's a good problem to find. The hard part is already done. What's left is plumbing, and plumbing is buildable.
linkforge
So I built one. linkforge↗ reads a sitemap, fetches pages politely, pulls out every paragraph, and scores each one against each target page. Where a paragraph is genuinely about ATS scanning and already contains a phrase like "applicant tracking system", it proposes wrapping those words in a link.
One restriction does most of the work: it never invents copy. The anchor is
always text the author already wrote. Their /guides/ats opens with "An ATS
(Applicant Tracking System) is the software recruiters use to collect, parse,
and rank job applications." The tool wraps Applicant Tracking System. The
sentence stays exactly as written and three words become a link.
That's what makes the output reviewable. You can approve or reject three hundred of these in a sitting, because nothing is being rewritten.
Four real paragraphs from their site, and the four links linkforge proposed for them:
An ATS (Applicant Tracking System) is the software recruiters use to collect, parse, and rank job applications before a human sees them.
/premium/atsTake a well lit photo and then let software do the polishing. The Goodspace AI Headshot Generator is the fastest way to turn a normal phone photo into a clean, professional portrait.
/premium/headshot-generatorWhich fields to include, what to cut, and how to rehearse it out loud. The GoodSpace AI Mock Interview is where you practise it, a live AI interviewer across 10,000+ roles.
/premium/ai-mock-interviewA declaration is neutral text that neither helps nor hurts your ATS score. If your goal is to rank well in an ATS, focus on relevant keywords and a parseable layout instead.
/premium/ats
Anchors, targets and scores are from the real run. Score is TF-IDF cosine between the paragraph and the target page; the rules that pick what survives are further down.
Constraints, and why they're the interesting part
Automated internal linking done badly is worse than none. Repeated exact-match anchors and link-stuffed pages are a recognisable spam pattern, and a tool that produces them at scale is a liability rather than a feature.
So the rules do the real work. Nothing goes inside headings, existing links, code, nav, or tables. No page links to itself. The output is a JSON patch with byte offsets, so a human approves before anything changes. The rest is a single loop over candidates sorted by relevance, and it is short enough to read in full:
for (const c of candidates) {
if (onThisPage >= maxLinksPerPage) break; // 3 per page
if ((perTarget.get(c.target.url) || 0) >= maxLinksPerTargetPerPage) continue;
if (usedParagraphs.has(c.paragraph.index)) continue; // one per paragraph
// Prefer an anchor phrasing we have not leaned on already.
const anchor = c.anchors.slice().sort(
(a, b) =>
(anchorUsage.get(a.text.toLowerCase()) || 0) -
(anchorUsage.get(b.text.toLowerCase()) || 0) ||
b.specificity - a.specificity
)[0];
const used = anchorUsage.get(anchor.text.toLowerCase()) || 0;
// Hard ceiling on any single exact anchor string across the whole site.
if (used >= 8) {
skipped.push({ url: page.url, reason: 'anchor-overused', anchor: anchor.text });
continue;
}
…
}Those four guards are the whole anti-spam story. The sort is the one that surprised me: because a phrasing already used is pushed to the back, the run produces anchor variation on its own, and nobody has to write variation rules.
Scoring is TF-IDF cosine similarity with IDF computed across the whole crawl, so terms that happen to be common on that particular site get down-weighted for free. I considered embeddings and skipped them. Every score here traces back to shared terms, and when a person reviews each row, an explainable score beats a slightly better one.
The bug that taught me something
First run of linkforge against their blog put an ATS link on an article about workplace kindness. The matched text was "Compare the best free ATS resume checkers in 2026", which really is about ATS, so the score was correct.
It was their related-posts widget, rendered as <p> tags. The same teaser
appeared on four unrelated pages.
I reached for a structural fix first: detect the container, match the class name. That works on this site and breaks on the next one. Repetition is the more durable signal. Text appearing verbatim across three or more pages is chrome, whatever markup wraps it. That is twelve lines, and it took most of the false positives with it:
export function dropBoilerplate(pages, threshold = 3) {
const seen = new Map();
for (const page of pages) {
for (const key of new Set(page.paragraphs.map((p) => norm(p.text)))) {
seen.set(key, (seen.get(key) || 0) + 1);
}
}
for (const page of pages) {
page.paragraphs = page.paragraphs.filter(
(p) => (seen.get(norm(p.text)) || 0) < threshold
);
}
}
const norm = (s) => s.toLowerCase().replace(/\s+/g, ' ').trim().slice(0, 200);It counts distinct pages rather than occurrences, so a phrase repeated three times on one long page survives, and a teaser repeated once each on three pages does not.
What it found
linkforge crawled seventy-four pages, one request every 1.2 seconds, robots.txt
honoured.
15 insertions across 14 pages
9 → /premium/ats
3 → /premium/headshot-generator
3 → /premium/ai-mock-interviewEight distinct anchor phrasings came out of it: "Applicant Tracking System", "Resume Scanner", "ATS score", "check your resume", "Mock Interview". Nobody wrote that spread. It falls out of the reuse ceiling.
You can watch the whole run on the demo page↗, which replays the crawl and shows each proposed link live in its sentence.
Here's one as it would render:
…Which words to put in your resume is a separate job, covered in the ATS resume keywords guide, and running your file through our scanner to get a score…
The sentence is already about the scanner. The link should exist and doesn't.
What I'm not claiming
Fifteen from seventy-four doesn't extrapolate across a sitemap of 1,388 URLs. Guide pages match far better than blog posts, since guides are already about the tools, and the blog run converted at less than half the rate. Better to say that now than have somebody quote my number and find out later.
I also have no performance data, no view of their code, and no idea what's on their roadmap. This is a site read from the outside with public data, and they may have shipped all of it last week.
Where I'd go next
The linking layer is the first piece. After that: a slug pipeline, since their URLs still say 2025 while the titles say 2026 and somebody is fixing that by hand every January. Schema generation from existing content. Quality gates on the programmatic templates.
All of it is the same shape of work, sitting in the seam between a content operation that works and a product surface that deserves the traffic.
The source
Seven files, Node with no dependencies, about six hundred lines all told. These are the files linkforge actually runs, read straight off disk — open one:
cli.jsthe run itself, and what gets written out
#!/usr/bin/env node
import { readFile, writeFile, mkdir } from 'node:fs/promises';
import { resolve, join } from 'node:path';
import { get, sleep, loadRobots, allowed } from './fetch.js';
import { readSitemap, filterUrls } from './sitemap.js';
import { paragraphs, title, dropBoilerplate } from './extract.js';
import { buildIdf, makeScorer, tokenize } from './match.js';
import { planInsertions } from './plan.js';
import { htmlReport } from './report.js';
const args = parseArgs(process.argv.slice(2));
if (args.help || !args.config) {
console.log(`
linkforge — proposes internal links from content pages to conversion pages.
node src/cli.js --config targets.goodspace.json [options]
--config <file> target definition (required)
--limit <n> max pages to fetch (default 50)
--delay <ms> delay between requests (default 1200)
--out <dir> output directory (default ./out)
--min <n> minimum relevance, overrides config
--quiet suppress progress output
Outputs plan.json (exact offsets, machine-applicable) and report.html.
Nothing is ever written to the target site.
`);
process.exit(args.help ? 0 : 1);
}
const cfg = JSON.parse(await readFile(resolve(args.config), 'utf8'));
const limit = Number(args.limit ?? 50);
const outDir = resolve(args.out ?? './out');
const rules = { ...cfg.rules, ...(args.min ? { minRelevance: Number(args.min) } : {}) };
const log = args.quiet ? () => {} : (...a) => console.error(...a);
const origin = new URL(cfg.site).origin;
log(`\n ${cfg.site}`);
// 1. robots.txt
const robots = await loadRobots(origin);
const delay = Math.max(Number(args.delay ?? 1200), robots.crawlDelay ?? 0);
log(` robots: ${robots.disallow.length} disallow rules · ${delay}ms between requests`);
// 2. sitemap
const all = await readSitemap(cfg.sitemap);
if (!all.length) {
console.error(' ✗ no URLs found in sitemap — check the sitemap URL');
process.exit(1);
}
const candidates = filterUrls(all, cfg)
.filter((u) => allowed(new URL(u).pathname, robots))
.slice(0, limit);
log(` sitemap: ${all.length} URLs → ${candidates.length} in scope (limit ${limit})\n`);
// 3. fetch, sequentially and slowly
const pages = [];
for (const [n, url] of candidates.entries()) {
const res = await get(url);
if (res.ok) {
const paras = paragraphs(res.body);
if (paras.length) {
pages.push({
url: res.url,
path: new URL(res.url).pathname,
title: title(res.body),
paragraphs: paras,
});
}
log(` ${String(n + 1).padStart(3)}/${candidates.length} ${paras.length ? '·' : '∅'} ${new URL(url).pathname.slice(0, 72)}`);
} else {
log(` ${String(n + 1).padStart(3)}/${candidates.length} ✗ ${res.status} ${new URL(url).pathname.slice(0, 72)}`);
}
if (n < candidates.length - 1) await sleep(delay);
}
if (!pages.length) {
console.error('\n ✗ no readable pages — the site may render content client-side');
process.exit(1);
}
// 4. strip nav/related-post chrome that survived the markup filters
const boiler = dropBoilerplate(pages);
if (boiler.removed) {
log(`\n boilerplate: dropped ${boiler.removed} repeated paragraphs (${boiler.distinctBoilerplate} distinct)`);
}
// 5. score
const corpus = pages.flatMap((p) => p.paragraphs.map((x) => tokenize(x.text)));
const scorer = makeScorer(buildIdf(corpus));
const { insertions, skipped, anchorUsage } = planInsertions({
pages,
targets: cfg.targets,
scorer,
rules,
site: cfg.site,
});
// 5. write
await mkdir(outDir, { recursive: true });
const stats = {
generated: new Date().toISOString().slice(0, 16).replace('T', ' '),
pagesFetched: pages.length,
paragraphs: corpus.length,
pagesAffected: new Set(insertions.map((i) => i.page)).size,
uniqueAnchors: anchorUsage.size,
};
await writeFile(
join(outDir, 'plan.json'),
JSON.stringify({ site: cfg.site, stats, rules, insertions, skipped }, null, 2)
);
await writeFile(
join(outDir, 'report.html'),
htmlReport({ site: cfg.site, stats, insertions, targets: cfg.targets })
);
// 6. summary
const tally = new Map();
for (const i of insertions) tally.set(i.target, (tally.get(i.target) || 0) + 1);
log(`\n ${insertions.length} insertions across ${stats.pagesAffected} pages\n`);
for (const [t, c] of [...tally].sort((a, b) => b[1] - a[1])) {
log(` ${String(c).padStart(4)} → ${t}`);
}
if (skipped.length) log(`\n ${skipped.length} skipped (anchor reuse ceiling)`);
log(`\n ${join(outDir, 'plan.json')}\n ${join(outDir, 'report.html')}\n`);
function parseArgs(argv) {
const o = {};
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (!a.startsWith('--')) continue;
const k = a.slice(2);
const next = argv[i + 1];
if (!next || next.startsWith('--')) o[k] = true;
else { o[k] = next; i++; }
}
return o;
}
fetch.jspolite HTTP — one at a time, robots.txt, crawl-delay
// Polite HTTP: one request at a time per host, fixed delay, robots.txt respected.
// Deliberately slow. We are crawling someone else's site uninvited.
const UA = 'linkforge/0.1 (internal-link research; contact via site)';
export async function get(url, { timeout = 15000 } = {}) {
const ctrl = new AbortController();
const t = setTimeout(() => ctrl.abort(), timeout);
try {
const res = await fetch(url, {
headers: { 'user-agent': UA, accept: 'text/html,application/xhtml+xml,application/xml' },
signal: ctrl.signal,
redirect: 'follow',
});
if (!res.ok) return { ok: false, status: res.status, url };
return { ok: true, status: res.status, url: res.url, body: await res.text() };
} catch (err) {
return { ok: false, status: 0, url, error: err.message };
} finally {
clearTimeout(t);
}
}
export const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
/**
* Minimal robots.txt check. Reads Disallow rules for `*` and our UA.
* Not a full spec implementation — errs toward refusing.
*/
export async function loadRobots(origin) {
const res = await get(new URL('/robots.txt', origin).href);
if (!res.ok) return { disallow: [], crawlDelay: null }; // no robots.txt => no restrictions
const disallow = [];
let crawlDelay = null;
let applies = false;
for (const raw of res.body.split('\n')) {
const line = raw.split('#')[0].trim();
if (!line) continue;
const [rawKey, ...rest] = line.split(':');
const key = rawKey.trim().toLowerCase();
const value = rest.join(':').trim();
if (key === 'user-agent') {
applies = value === '*' || value.toLowerCase().includes('linkforge');
} else if (applies && key === 'disallow' && value) {
disallow.push(value);
} else if (applies && key === 'crawl-delay') {
const n = Number(value);
if (!Number.isNaN(n)) crawlDelay = n * 1000;
}
}
return { disallow, crawlDelay };
}
export function allowed(pathname, robots) {
return !robots.disallow.some((rule) => pathname.startsWith(rule));
}
sitemap.jssitemap parsing and path filters
import { get } from './fetch.js';
const locs = (xml) =>
[...xml.matchAll(/<loc>\s*([^<]+?)\s*<\/loc>/gi)].map((m) => decodeXml(m[1]));
const decodeXml = (s) =>
s
.replace(/</g, '<')
.replace(/>/g, '>')
.replace(/"/g, '"')
.replace(/'/g, "'")
.replace(/&/g, '&');
/**
* Reads a sitemap, following one level of sitemap-index nesting.
* Returns a de-duplicated list of page URLs.
*/
export async function readSitemap(url, { depth = 0 } = {}) {
const res = await get(url);
if (!res.ok) return [];
const isIndex = /<sitemapindex[\s>]/i.test(res.body);
const found = locs(res.body);
if (!isIndex) return found;
if (depth > 1) return [];
const out = [];
for (const child of found.slice(0, 50)) {
out.push(...(await readSitemap(child, { depth: depth + 1 })));
}
return [...new Set(out)];
}
export function filterUrls(urls, { sourceInclude = [], sourceExclude = [] }) {
return urls.filter((u) => {
let path;
try {
path = new URL(u).pathname;
} catch {
return false;
}
if (sourceExclude.some((p) => path.includes(p))) return false;
if (sourceInclude.length && !sourceInclude.some((p) => path.includes(p))) return false;
return true;
});
}
extract.jsparagraphs with byte offsets, and the boilerplate check
// Extracts linkable paragraphs from raw HTML, with byte offsets so a proposed
// insertion can be expressed as an exact, verifiable replacement.
const BLOCKED_CONTAINERS = [
'script', 'style', 'noscript', 'svg', 'nav', 'header', 'footer',
'aside', 'pre', 'code', 'form', 'button', 'table', 'figure',
];
/** Character ranges we must never write into. */
function blockedRanges(html) {
const ranges = [];
for (const tag of BLOCKED_CONTAINERS) {
const re = new RegExp(`<${tag}\\b[^>]*>[\\s\\S]*?<\\/${tag}>`, 'gi');
for (const m of html.matchAll(re)) ranges.push([m.index, m.index + m[0].length]);
}
// Headings: never place a link inside one.
for (const m of html.matchAll(/<h[1-6]\b[^>]*>[\s\S]*?<\/h[1-6]>/gi)) {
ranges.push([m.index, m.index + m[0].length]);
}
return ranges.sort((a, b) => a[0] - b[0]);
}
const inside = (pos, ranges) => ranges.some(([s, e]) => pos >= s && pos < e);
export function title(html) {
const m = html.match(/<title[^>]*>([\s\S]*?)<\/title>/i);
return m ? decode(stripTags(m[1])).trim() : '';
}
export function stripTags(s) {
return s.replace(/<[^>]+>/g, '');
}
export function decode(s) {
return s
.replace(/ /g, ' ')
.replace(/&/g, '&')
.replace(/</g, '<')
.replace(/>/g, '>')
.replace(/"/g, '"')
.replace(/&#(\d+);/g, (_, d) => String.fromCharCode(+d));
}
/**
* @returns {Array<{index:number, start:number, end:number, html:string, text:string,
* words:number, safeSpans:Array<[number,number]>}>}
* `start`/`end` bound the paragraph's inner HTML within the document.
* `safeSpans` are offsets *within the inner HTML* that contain no markup and
* no existing anchor — the only places we are willing to insert.
*/
export function paragraphs(html) {
const blocked = blockedRanges(html);
const out = [];
for (const m of html.matchAll(/<p\b[^>]*>([\s\S]*?)<\/p>/gi)) {
if (inside(m.index, blocked)) continue;
const innerStart = m.index + m[0].indexOf(m[1]);
const inner = m[1];
const text = decode(stripTags(inner)).replace(/\s+/g, ' ').trim();
const words = text ? text.split(' ').length : 0;
if (!words) continue;
out.push({
index: out.length,
start: innerStart,
end: innerStart + inner.length,
html: inner,
text,
words,
safeSpans: plainTextSpans(inner),
});
}
return out;
}
/**
* Drops boilerplate: paragraphs whose exact text recurs across several pages.
*
* Found this the hard way. Their blog template renders a "related posts" widget
* as <p> tags, so teaser copy like "Compare the best free ATS resume checkers"
* appears on pages about workplace kindness — and scored well, because the teaser
* really is about ATS. A link placed there would sit inside a recommendations
* carousel rather than prose.
*
* Structural detection (class names, containers) is brittle across sites.
* Repetition is not: text that appears verbatim on N different pages is chrome,
* whatever markup wraps it.
*
* @param {Array<{paragraphs:Array}>} pages mutated in place
* @param {number} threshold distinct pages before text is chrome
*/
export function dropBoilerplate(pages, threshold = 3) {
const seen = new Map();
for (const page of pages) {
for (const key of new Set(page.paragraphs.map((p) => norm(p.text)))) {
seen.set(key, (seen.get(key) || 0) + 1);
}
}
let removed = 0;
for (const page of pages) {
const kept = page.paragraphs.filter((p) => (seen.get(norm(p.text)) || 0) < threshold);
removed += page.paragraphs.length - kept.length;
page.paragraphs = kept;
}
return { removed, distinctBoilerplate: [...seen.values()].filter((n) => n >= threshold).length };
}
const norm = (s) => s.toLowerCase().replace(/\s+/g, ' ').trim().slice(0, 200);
/**
* Spans of the paragraph's inner HTML that are plain text: outside every tag and
* outside any existing <a>...</a>. Nested links are invalid HTML, so anchors are
* excluded wholesale rather than trimmed.
*/
function plainTextSpans(inner) {
const forbidden = [];
for (const m of inner.matchAll(/<a\b[^>]*>[\s\S]*?<\/a>/gi)) {
forbidden.push([m.index, m.index + m[0].length]);
}
for (const m of inner.matchAll(/<[^>]+>/g)) {
forbidden.push([m.index, m.index + m[0].length]);
}
forbidden.sort((a, b) => a[0] - b[0]);
const spans = [];
let cursor = 0;
for (const [s, e] of forbidden) {
if (s > cursor) spans.push([cursor, s]);
cursor = Math.max(cursor, e);
}
if (cursor < inner.length) spans.push([cursor, inner.length]);
return spans.filter(([s, e]) => e - s > 0);
}
match.jsTF-IDF, cosine, and finding a usable anchor phrase
// TF-IDF relevance between a paragraph and a target page.
// Deterministic and explainable — every score can be traced to shared terms,
// which matters more than raw accuracy when a human reviews each suggestion.
const STOP = new Set(
`a an and are as at be but by for from has have how i if in into is it its of on or
that the their then there these they this to was were what when where which who will
with you your our we can do does not no yes about more most some such than too very
just also here his her them out up down over under again once only own same so`
.split(/\s+/)
);
export const tokenize = (s) =>
s.toLowerCase().replace(/[^a-z0-9\s'-]/g, ' ').split(/\s+/)
.filter((t) => t.length > 1 && !STOP.has(t));
function tf(tokens) {
const m = new Map();
for (const t of tokens) m.set(t, (m.get(t) || 0) + 1);
return m;
}
/** IDF computed across every paragraph in the crawl. */
export function buildIdf(docTokenLists) {
const df = new Map();
for (const tokens of docTokenLists) {
for (const t of new Set(tokens)) df.set(t, (df.get(t) || 0) + 1);
}
const n = docTokenLists.length || 1;
const idf = new Map();
for (const [t, d] of df) idf.set(t, Math.log((n + 1) / (d + 1)) + 1);
return idf;
}
function vector(tokens, idf) {
const v = new Map();
for (const [t, count] of tf(tokens)) {
v.set(t, (1 + Math.log(count)) * (idf.get(t) ?? 1));
}
return v;
}
export function cosine(a, b) {
let dot = 0;
const [small, large] = a.size < b.size ? [a, b] : [b, a];
for (const [t, w] of small) {
const o = large.get(t);
if (o) dot += w * o;
}
if (!dot) return 0;
const norm = (m) => Math.sqrt([...m.values()].reduce((s, w) => s + w * w, 0));
return dot / (norm(a) * norm(b) || 1);
}
export function makeScorer(idf) {
const cache = new Map();
const vec = (key, tokens) => {
if (!cache.has(key)) cache.set(key, vector(tokens, idf));
return cache.get(key);
};
return {
targetVector: (target) =>
vec(`t:${target.url}`, tokenize(`${target.name} ${target.keywords.join(' ')}`)),
paragraphVector: (key, text) => vec(`p:${key}`, tokenize(text)),
cosine,
};
}
/**
* Locates every place a target keyword literally appears in the paragraph's
* plain text, restricted to spans that carry no markup.
* We wrap words the author already wrote rather than injecting new copy —
* safer to review, and it keeps anchor text naturally varied.
*/
export function anchorCandidates(paragraph, target) {
const found = [];
for (const kw of target.keywords) {
const re = new RegExp(`(?<![\\w-])${escapeRe(kw)}(?![\\w-])`, 'gi');
for (const m of paragraph.html.matchAll(re)) {
const s = m.index;
const e = s + m[0].length;
const safe = paragraph.safeSpans.some(([ss, se]) => s >= ss && e <= se);
if (!safe) continue;
found.push({ keyword: kw, start: s, end: e, text: m[0], specificity: kw.split(' ').length });
}
}
// Prefer longer, more specific phrases: "ats resume checker" over "resume".
return found.sort((a, b) => b.specificity - a.specificity || a.start - b.start);
}
const escapeRe = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
plan.jsthe rules, and the patch it emits
// Turns scored matches into a reviewable set of insertions.
// Every rule here exists because automated internal linking done carelessly is
// worse than none: over-optimised anchors and link-stuffed pages are the exact
// footprint that gets a site discounted.
import { anchorCandidates } from './match.js';
export function planInsertions({ pages, targets, scorer, rules, site }) {
const {
maxLinksPerPage = 3,
maxLinksPerTargetPerPage = 1,
minRelevance = 0.18,
minParagraphWords = 25,
skipFirstParagraphs = 0,
} = rules;
const anchorUsage = new Map(); // exact anchor text -> times used across the run
const targetTally = new Map(); // target url -> links gained
const insertions = [];
const skipped = [];
for (const page of pages) {
let onThisPage = 0;
const perTarget = new Map();
// Score every (paragraph, target) pair, then take the best first.
const candidates = [];
for (const p of page.paragraphs) {
if (p.index < skipFirstParagraphs) continue;
if (p.words < minParagraphWords) continue;
for (const target of targets) {
// Never link a page to itself.
if (page.path === target.url) continue;
const rel = scorer.cosine(
scorer.paragraphVector(`${page.url}#${p.index}`, p.text),
scorer.targetVector(target)
);
if (rel < minRelevance) continue;
const anchors = anchorCandidates(p, target);
if (!anchors.length) continue;
candidates.push({ paragraph: p, target, relevance: rel, anchors });
}
}
candidates.sort((a, b) => b.relevance - a.relevance);
const usedParagraphs = new Set();
for (const c of candidates) {
if (onThisPage >= maxLinksPerPage) break;
if ((perTarget.get(c.target.url) || 0) >= maxLinksPerTargetPerPage) continue;
if (usedParagraphs.has(c.paragraph.index)) continue; // one link per paragraph
// Prefer an anchor phrasing we have not leaned on already.
const anchor =
c.anchors.slice().sort(
(a, b) =>
(anchorUsage.get(a.text.toLowerCase()) || 0) -
(anchorUsage.get(b.text.toLowerCase()) || 0) ||
b.specificity - a.specificity
)[0];
const key = anchor.text.toLowerCase();
const used = anchorUsage.get(key) || 0;
// Hard ceiling on any single exact anchor string across the whole site.
if (used >= 8) {
skipped.push({ url: page.url, reason: 'anchor-overused', anchor: anchor.text });
continue;
}
const href = new URL(c.target.url, site).href;
const before = c.paragraph.html;
const after =
before.slice(0, anchor.start) +
`<a href="${href}">${anchor.text}</a>` +
before.slice(anchor.end);
insertions.push({
page: page.url,
pagePath: page.path,
pageTitle: page.title,
target: c.target.url,
targetName: c.target.name,
href,
anchorText: anchor.text,
matchedKeyword: anchor.keyword,
relevance: Number(c.relevance.toFixed(4)),
paragraphIndex: c.paragraph.index,
// Offsets are into the fetched document, so the patch is verifiable.
docOffsetStart: c.paragraph.start,
docOffsetEnd: c.paragraph.end,
original: before,
replacement: after,
excerpt: contextLine(c.paragraph.text, anchor.text),
});
anchorUsage.set(key, used + 1);
targetTally.set(c.target.url, (targetTally.get(c.target.url) || 0) + 1);
perTarget.set(c.target.url, (perTarget.get(c.target.url) || 0) + 1);
usedParagraphs.add(c.paragraph.index);
onThisPage++;
}
}
return { insertions, skipped, targetTally, anchorUsage };
}
function contextLine(text, anchor) {
const i = text.toLowerCase().indexOf(anchor.toLowerCase());
if (i === -1) return text.slice(0, 160);
const s = Math.max(0, i - 70);
const e = Math.min(text.length, i + anchor.length + 70);
return (s ? '…' : '') + text.slice(s, e) + (e < text.length ? '…' : '');
}
report.jsthe standalone review page
const esc = (s = '') =>
String(s).replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>').replace(/"/g, '"');
function mark(excerpt, anchor) {
const i = excerpt.toLowerCase().indexOf(anchor.toLowerCase());
if (i === -1) return esc(excerpt);
return (
esc(excerpt.slice(0, i)) +
'<mark>' + esc(excerpt.slice(i, i + anchor.length)) + '</mark>' +
esc(excerpt.slice(i + anchor.length))
);
}
export function htmlReport({ site, stats, insertions, targets }) {
const byTarget = targets.map((t) => ({
...t,
count: insertions.filter((i) => i.target === t.url).length,
})).sort((a, b) => b.count - a.count);
return `<title>Internal link plan — ${esc(site)}</title>
<style>
:root{--bg:#fbfaf8;--fg:#1a1a1a;--muted:#6b6b6b;--line:#e4e1dc;--card:#fff;--accent:#b4421f;
--mono:ui-monospace,SFMono-Regular,Menlo,monospace}
body{background:var(--bg);color:var(--fg);font:15px/1.6 ui-sans-serif,-apple-system,"Segoe UI",sans-serif;margin:0}
.wrap{max-width:900px;margin:0 auto;padding:48px 24px 80px}
h1{font-size:1.75rem;letter-spacing:-.02em;margin:0 0 4px}
.sub{color:var(--muted);margin:0}
.meta{font:12px var(--mono);color:var(--muted);margin-top:12px}
h2{font-size:1.1rem;margin:40px 0 4px;padding-bottom:8px;border-bottom:1px solid var(--line)}
.kpi{display:grid;grid-template-columns:repeat(auto-fit,minmax(120px,1fr));gap:10px;margin:24px 0}
.kpi div{background:var(--card);border:1px solid var(--line);border-radius:8px;padding:12px}
.kpi b{display:block;font-size:1.5rem;letter-spacing:-.02em}
.kpi span{font-size:.7rem;color:var(--muted);text-transform:uppercase;letter-spacing:.05em}
table{width:100%;border-collapse:collapse;font-size:.88rem;margin:12px 0}
th,td{text-align:left;padding:8px 10px;border-bottom:1px solid var(--line);vertical-align:top}
th{font-size:.7rem;text-transform:uppercase;letter-spacing:.06em;color:var(--muted)}
.card{background:var(--card);border:1px solid var(--line);border-radius:10px;padding:16px 18px;margin:12px 0}
.row{display:flex;justify-content:space-between;gap:12px;flex-wrap:wrap;font:12px var(--mono);color:var(--muted)}
.row a{color:var(--muted)}
mark{background:#ffe9a8;padding:1px 2px;border-radius:3px}
.ex{margin:10px 0 8px;font-size:.95rem}
.to{font:12px var(--mono)}
.to b{color:var(--accent);font-weight:600}
.scroll{overflow-x:auto}
code{font:12px var(--mono);background:#f1efeb;padding:1px 5px;border-radius:4px}
.note{background:#fff8e6;border:1px solid #e8d9a0;border-radius:8px;padding:12px 16px;font-size:.88rem;margin:20px 0}
</style>
<div class="wrap">
<h1>Internal link plan</h1>
<p class="sub">${esc(site)} — proposed insertions, none applied.</p>
<p class="meta">${esc(stats.generated)} · ${stats.pagesFetched} pages fetched · ${stats.paragraphs} paragraphs scanned</p>
<div class="note"><b>Proposals only.</b> Every insertion below wraps text the author already wrote — no copy is invented. Review before applying; the JSON patch carries exact document offsets for each change.</div>
<div class="kpi">
<div><b>${stats.pagesFetched}</b><span>Pages</span></div>
<div><b>${insertions.length}</b><span>Insertions</span></div>
<div><b>${stats.pagesAffected}</b><span>Pages changed</span></div>
<div><b>${stats.uniqueAnchors}</b><span>Distinct anchors</span></div>
</div>
<h2>Links gained per target</h2>
<div class="scroll"><table>
<tr><th>Target</th><th>Path</th><th>New inbound links</th></tr>
${byTarget.map((t) => `<tr><td>${esc(t.name)}</td><td><code>${esc(t.url)}</code></td><td><b>${t.count}</b></td></tr>`).join('\n')}
</table></div>
<h2>Proposed insertions</h2>
${insertions.map((i) => `<div class="card">
<div class="row">
<a href="${esc(i.page)}">${esc(i.pagePath)}</a>
<span>relevance ${i.relevance.toFixed(3)} · ¶${i.paragraphIndex}</span>
</div>
<p class="ex">${mark(i.excerpt, i.anchorText)}</p>
<div class="to">→ <b>${esc(i.target)}</b> · matched <code>${esc(i.matchedKeyword)}</code></div>
</div>`).join('\n')}
</div>`;
}
Everything interesting is a constraint, which is why extract.js and plan.js
are the two worth opening. The other five are plumbing: fetch one URL at a time
and obey crawl-delay, parse a sitemap, count terms, write an HTML table.
It proposes; nothing is applied, and nothing is ever written to the site being crawled.