Hreflang at Scale: The Sitemap-Driven Approach That Survived 40 Locales
Inline hreflang tags break once you hit a dozen locales. Here's the sitemap-driven setup we use on programmatic sites with 40+ locales, including the validation pipeline that catches drift before Google does.
The first time we rolled out hreflang on a programmatic marketplace with 40 locales, we did it the textbook way: inline <link rel="alternate"> tags in the <head>. It worked until it didn't. Pages started dropping out of the return tags, canonicals drifted, and Search Console lit up with "no return tags" errors across half the catalogue.
This is the setup we use now. It's boring, which is the point.
Why inline hreflang falls apart past ~10 locales
Inline hreflang has three failure modes that get worse with scale:
- Page weight. 40 locales means 40
<link>tags per page, minimum. On a template that renders 2M URLs, that's 80M tags Googlebot has to parse. It's not a dealbreaker, but it's wasted crawl surface. - Cache invalidation. When you add locale #41, every page in your CDN needs to be purged so the new tag appears everywhere. Miss one, and you've got a half-broken cluster that Google will happily interpret as "these pages don't reference each other."
- Drift between templates. If your PDP template, category template, and blog template each render hreflang independently, they will diverge. One team will fix a bug in one place and forget the others. We've seen this on every large site we've audited.
The sitemap-based approach centralises the mapping in one place that's cheap to regenerate and easy to validate.
When inline still makes sense
If you have fewer than ~8 locales and a single rendering layer, inline is fine. The operational cost of a sitemap pipeline isn't worth it. The breakpoint in our experience is somewhere around 10 locales or when you start running multiple rendering stacks (say, Next.js for marketing and a separate service for the catalogue).
The data model
Everything starts with a canonical identifier that's independent of URL. On an e-commerce catalogue, that's usually the product group ID. On an editorial site, it's the story ID. The rule: one entity, many URLs, one hreflang cluster.
CREATE TABLE hreflang_cluster (
entity_id TEXT NOT NULL,
locale TEXT NOT NULL, -- e.g. 'en-gb', 'de-de', 'x-default'
url TEXT NOT NULL,
is_indexable BOOLEAN NOT NULL,
last_seen_at TIMESTAMPTZ NOT NULL,
PRIMARY KEY (entity_id, locale)
);
CREATE INDEX ON hreflang_cluster (entity_id);
CREATE INDEX ON hreflang_cluster (url);
The is_indexable flag matters more than people think. A URL that's noindex, 404, 301, or canonicalised elsewhere must not appear in a hreflang cluster. Google treats hreflang as a bidirectional signal, and a single broken member can invalidate the cluster for the rest.
We populate this table from whatever source of truth owns the URLs — usually the CMS or catalogue service — and refresh it nightly. For high-velocity sites, you can tail the change feed and update in near-real-time, but nightly has been enough for every client we've shipped this for.
Generating the sitemap
Google supports hreflang inside sitemap XML, and this is where the scale win lives. Each URL gets a <url> block with sibling <xhtml:link> entries for every locale in its cluster.
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:xhtml="http://www.w3.org/1999/xhtml">
<url>
<loc>https://example.com/en-gb/product/widget-42</loc>
<xhtml:link rel="alternate" hreflang="en-gb"
href="https://example.com/en-gb/product/widget-42"/>
<xhtml:link rel="alternate" hreflang="de-de"
href="https://example.com/de-de/produkt/widget-42"/>
<xhtml:link rel="alternate" hreflang="fr-fr"
href="https://example.com/fr-fr/produit/widget-42"/>
<xhtml:link rel="alternate" hreflang="x-default"
href="https://example.com/en/product/widget-42"/>
</url>
</urlset>
A few hard rules we enforce at generation time:
- Every URL in the cluster appears in every locale's sitemap block for that entity. Hreflang is bidirectional; one-way references get dropped.
x-defaultis required. Pick the locale that handles unmatched users — usually your global English page or a language selector.- Only include URLs where
is_indexable = true. Exclude the whole entity if fewer than two indexable locales remain. - Keep each sitemap file under 50MB uncompressed and under 50,000 URL entries. For large catalogues, shard by entity ID hash and reference them from a sitemap index.
Sharding strategy
We shard sitemaps by entity, not by locale. A common mistake is generating one sitemap per locale (sitemap-en-gb.xml, sitemap-de-de.xml). This duplicates every hreflang block N times and triples your sitemap storage. Instead:
sitemap-index.xml
→ sitemap-products-0001.xml (entities 0000-0999)
→ sitemap-products-0002.xml (entities 1000-1999)
...
Each shard contains all locales for its slice of entities. Googlebot fetches less, you store less, and the mapping stays coherent inside a single file.
The validation pipeline
This is where most teams skip steps and pay for it six months later. Before publishing a sitemap, run four checks:
1. Reciprocity. For every (A → B) hreflang reference, verify (B → A) exists. A simple SQL check:
SELECT a.entity_id, a.locale AS from_locale, b.locale AS to_locale
FROM hreflang_cluster a
JOIN hreflang_cluster b ON a.entity_id = b.entity_id AND a.locale <> b.locale
LEFT JOIN hreflang_cluster c
ON c.entity_id = a.entity_id
AND c.locale = a.locale
AND EXISTS (SELECT 1 FROM hreflang_cluster d
WHERE d.entity_id = a.entity_id AND d.locale = b.locale)
WHERE c.url IS NULL;
2. Indexability. HEAD request every URL in a sample (we do 1% daily, 100% weekly) and confirm 200 status and no noindex. Flag anything that drifts.
3. Canonical consistency. The URL in the hreflang block must equal the self-canonical on the destination page. If the page canonicalises elsewhere, the cluster is lying to Google.
4. Locale code validity. Only ISO 639-1 language codes with optional ISO 3166-1 region codes. en-uk is a classic bug — it should be en-gb. Validate against a whitelist.
We run this as a pre-publish job. If reciprocity failures exceed 0.5% of clusters, we block the sitemap push and alert. It has caught silent breakages from CMS migrations more than once.
Monitoring in production
Sitemap-based hreflang moves the signal out of the page, which means you can't just view-source to debug. You need three things in place:
- Search Console International Targeting report, checked weekly. Watch "no return tags" counts by locale.
- Server logs filtered for sitemap fetches by Googlebot. If fetch frequency drops, something broke at the index level.
- A diff job that compares yesterday's sitemap to today's and alerts if cluster sizes change by more than an expected delta. Entity churn is normal; a 30% drop in cluster count overnight is a bug.
What Search Console won't tell you fast enough
GSC's hreflang errors lag by days or weeks. Don't rely on it as a primary alert. Treat it as confirmation of problems your pipeline should have caught the night before. If you're learning about a hreflang issue from GSC first, your validation layer is incomplete.
Edge cases that bite
A few patterns we've learned to handle explicitly:
- Shared content across regions. If
en-usanden-capoint to the same URL, that's fine — but you still need both hreflang entries pointing at that single URL, and the page's self-canonical must match. - Unavailable products. When a SKU goes out of stock in one region, don't just drop it from the cluster silently. Decide: is it
noindex(remove from cluster), or still indexable with a "notify me" state (keep in cluster)? Document the rule. - Blog content. Most blog posts aren't localised. Don't force them into the hreflang system. Hreflang is for genuinely equivalent content, not translations-in-progress.
- Pagination. Hreflang each paginated URL to its equivalent paginated URL in other locales.
page=2in English maps topage=2in German, not topage=1.
Where we'd start
If you're staring at a sprawling hreflang setup that's already in production, don't rip it out on day one. Do this instead:
- Stand up the
hreflang_clustertable and populate it from your current source of truth — even if you're still rendering tags inline. - Run the four validation checks against your current state. You'll find bugs. Fix those first; they're costing you traffic right now.
- Build the sitemap generator in parallel and submit it alongside your existing setup. Google will reconcile.
- Once Search Console shows the sitemap-based clusters being honoured, remove the inline tags in one locale, monitor for two weeks, then roll out across the rest.
The boring version wins here. Hreflang isn't a place for cleverness — it's a place for a data model, a validator, and a cron job. If you want a hand auditing an existing international setup, that's the kind of work our SEO and growth engineering team tends to enjoy more than we probably should.
Want a team like ours?
72Technologies builds production software for the kind of teams who actually read this blog.
Start a projectKeep reading
Query Clustering with Embeddings: Building the Topic Map That Killed Our Cannibalization
Keyword cannibalization on a 12k-page site isn't a content problem — it's a clustering problem. Here's the embeddings pipeline we built to merge, redirect, and consolidate pages that were fighting each other in Google.
Content Decay Detection with GSC + GA4: The Refresh Queue That Actually Moves Rankings
Most 'content refresh' programmes are vibes. Here's the data pipeline we use to detect decaying URLs, score them by recoverable traffic, and feed a refresh queue writers actually ship against.
Faceted Navigation Without the Index Bloat: A Rules Engine That Keeps Google Happy
Faceted navigation is where programmatic sites quietly burn crawl budget and dilute rankings. Here's the rules engine we use to decide which facet combinations deserve an index, a noindex, or a hard block.
