Skip to content
Marc Nashaat Consulting Start a conversation

How to run an enterprise content audit for sites over 5,000 URLs

At enterprise scale a content audit is a data reconciliation problem long before it's an editorial one. This is the method I use, from building the inventory to getting the deletions signed off.

On a 200-page site a content audit is a reading exercise. On a 5,000-URL site it’s a data problem, and the teams that treat it as one are still auditing nine months later with a third of the sheet blank.

The method below is built around that. Get the inventory right, apply verdict criteria mechanically to the bulk, and spend your scarce human judgment on the few hundred URLs where the data is genuinely ambiguous. Budget four to six weeks for a first pass at this scale, and expect most of that to go on step one.

Build the content inventory from three sources and reconcile the gaps

No single export knows about every URL. The CMS knows what got published, the sitemap knows what you claim, and the crawl knows what’s reachable. Where the three disagree is where your problems live.

Export every URL from the CMS with content type, author, publish date, last-modified date and workflow status. Take it from the database or the API rather than the admin interface (which filters). Pull every XML sitemap referenced in robots.txt plus anything in a sitemap index, including the ones nobody has regenerated in years. Then run a full crawl seeded from the homepage, the sitemaps and the CMS export together, with rendering on, capturing status code, canonical, word count, internal links in and out, and publish date wherever the template exposes it.

Join all three on the URL, add a source column, and read the disagreements. URLs in the sitemap but absent from the crawl are orphans. URLs in the crawl but absent from the CMS usually come from a template nobody remembers commissioning. URLs in the CMS and nowhere else are unpublished or unlinked, and both matter.

Now append performance data: the sixteen months of page-level clicks, impressions and average position that Search Console holds, plus twelve months of analytics sessions, conversions and assisted revenue. Join both on the same URL key, with trailing slashes and parameters normalized first, or a fifth of your rows won’t match.

An important detail is that Search Console aggregates page-level data by canonical URL, while your analytics tool reports the URL the visitor landed on. A page can therefore look dead in one system and healthy in the other. That isn’t a data error, it’s two tools answering different questions, and ignoring it gets pages deleted that were earning.

Normalizing the URL key sounds trivial and it’s a common reason an audit produces wrong verdicts. Do it once, deliberately, and write the rule down.

The metrics that decide each content audit verdict

Every URL gets one of six verdicts: keep as is, update, consolidate, redirect, remove, or leave alone because it isn’t editorial content at all. The metrics exist to assign most of them without a meeting.

Organic clicks over twelve months, not sessions. Search Console is judging the page that ranked, which is the thing you’re forming an opinion about anyway.

Impressions with near-zero clicks. This flags a page ranking on page two for something real, and it’s almost always an update rather than a deletion.

Conversions and assisted conversions. The pages that never rank but keep appearing in closed-won paths are exactly the ones a traffic-only audit deletes, and you’ll hear about it from sales.

Referring domains to the URL. This turns a remove into a redirect, and occasionally into a rewrite in place.

Internal links in (a measure of how embedded the page is, and therefore how much work removing it creates).

Last meaningful update, which isn’t last-modified. A template change touches every timestamp on the site and tells you nothing about the content.

Set your thresholds before you look at the data, or you’ll calibrate them around the pages you already like. A workable default: zero clicks and zero conversions in twelve months with no referring domains is a remove candidate, meaningful impressions with poor click share is an update candidate, and anything with referring domains is never a plain remove.

Find keyword cannibalization before you assign any verdicts

Do this before verdicts, because it changes them. Pull the Search Console query export, group by query, and find the queries where more than one URL has taken impressions over the period. Watch for queries where the ranking URL changed from month to month (the clearest symptom of a site competing with itself).

Then go and read the pages. Genuine cannibalization is two pages serving the same intent, and the fix is consolidation. What looks like cannibalization but isn’t is a category page and a guide ranking for the same head term with different intents behind them, and consolidating those loses one of the two jobs. The distinction is editorial and it’s the part of the audit nobody can automate.

Two URLs alternating in the results for one query isn’t a ranking problem. It’s a decision nobody has made about which page is meant to win.

How to consolidate pages without losing the equity

Consolidation is the most valuable verdict and the one most likely to cause damage, because it’s a small migration executed by people who don’t think of it as one.

The sequence runs like this. Choose the survivor on existing authority and internal links rather than on which page the team prefers. Move the genuinely unique material from the losing pages into it and improve that material rather than concatenating it. Redirect each retired URL to the survivor with a 301. Update every internal link that pointed at the retired URLs (the crawl already told you where they are) and drop those URLs from the sitemap in the same release.

The risks are specific and avoidable. Redirect chains accumulate fast when a page you consolidate this year was already a redirect target from two years ago, so check every hop resolves in one step. Don’t send the retired URLs somewhere convenient either. Google’s migration guidance says not to redirect many old URLs to one irrelevant destination such as the homepage, because that can be treated as a soft 404. You then lose the equity you were trying to preserve. The same page says to keep redirects in place for at least a year, which you want to know before someone proposes cleaning up the rules next quarter.

Don’t reach for a canonical tag instead of a redirect either. Google describes rel="canonical" as a strong signal rather than an instruction, and picks its own canonical when it disagrees. Right tool for genuine duplicates, wrong tool for retirement.

Finally, stage it. Consolidating ten pages into one over a weekend moves rankings before the survivor has settled, and you’ll have no idea which change did what. Batches of 20 to 50 URLs per release let you attribute what happened.

Keep a record of every retired URL and its destination. Somebody will ask in eighteen months, usually halfway through a replatform.

Getting content audit deletions signed off

Most content audits I’ve run have stalled at deletion. Somebody wrote those pages, somebody’s team is measured on volume, and “remove” reads as a judgment on a colleague. The process has to make the decision impersonal and finite, or it won’t happen at all.

What works: one named decision-maker per content type, a review window of ten working days after which the default verdict applies, batches of 100 to 200 URLs rather than the full sheet, and a statement that removal means a 301 to the best alternative. That last phrasing is far easier to agree to than deletion, and it’s also the correct technical answer. Publish the verdicts and the data behind them so nobody gets surprised in a meeting.

On a regulated site, legal review of removals isn’t optional and it’s slow. Build it into the schedule rather than discovering it in week three.

How to start a content audit this week

Build the joined inventory before you form a single opinion. Pull the CMS export, the sitemaps and the crawl, normalize the keys, join Search Console and analytics onto them, and count the rows that disagree between sources. That number tells you how large the audit really is, and it’s reliably larger than the project plan assumed.

If you’d rather not spend six weeks on it in-house, that reconciliation is the first thing I do on a content audit engagement, and the verdicts feed straight into the strategy work that follows.

Written by

Marc Nashaat

Toronto-based Enterprise SEO & Growth consultant with 15+ years working with Fortune 50–500 brands including FedEx, Toyota, Adobe and Public Storage. Google Partner; featured in Forbes, CBC News and CIO.