A competitor's press release tells you what they want the market to believe. Their site structure tells you what they actually paid for. Every top-level menu item and every directory of templated URLs stands for budget, headcount and an internal argument somebody won.
This is a six-pass teardown that fits into one afternoon and ends in IA decisions and testable hypotheses, not a slide of screenshots or a plan to copy someone else's menu.
Why site structure is strategy
Copy can be rewritten before lunch; moving a section or rebuilding a menu takes design, development and redirects. That cost makes structure an honest signal:
- Navigation order is a priority ranking. The first GNB slot is the most contested real estate on the site, and what got demoted to the footer lost an argument.
- Depth is a budget. Depth-1 pages are the ones the business wants found; depth-4 pages are the ones it must have but won't promote.
- A dedicated page is a commitment. A paragraph about healthcare is a mention.
/industries/healthcare/with its own case studies is a sales motion. - Structure moves before announcements. A B2B SaaS rival that quietly adds six industry pages under a new "Solutions" menu is telling you where its next sales hires will sit.
One caveat: structure shows intent and investment, not results. You can see what a competitor built, not whether it works.
Scope it before you crawl
Pick three to five sites
| Role | Why it's in the set | Illustrative example |
|---|---|---|
| Direct competitor | Same buyers, same deals: the table stakes | The rival your sales team loses to most |
| Category leader | The category at scale, and where it is heading | The vendor every analyst report names first |
| Adjacent category | Patterns your category hasn't adopted yet | A payroll tool, if you sell HR software |
Write the questions first
Without questions, a teardown becomes a screenshot album. Write five first, each tied to a decision you are about to make:
- Do leaders organise solutions by industry, role or product? (Our Solutions menu.)
- How many clicks to pricing, and is it public at all? (Pricing placement.)
- Do they publish comparison pages, and against whom? (A comparison hub.)
- Which markets have they localised, and how deeply? (Localisation roadmap.)
- Do docs live inside the marketing IA or on a subdomain? (Docs architecture.)
If a question doesn't map to a decision, drop it. Capturing everything is how one afternoon becomes a week.
The six-pass teardown
Run the passes in order; each builds on the first.
Pass 1: Inventory, sitemap versus links
Capture: every URL in sitemap.xml, every URL found by following links from the homepage, and the overlap. The gap is the insight:
| Set | What it usually means |
|---|---|
| In the sitemap, not linked | Orphans, campaign landers, programmatic pages for search engines, stale sections nobody retired |
| Linked, not in the sitemap | Sitemap hygiene gaps, or sections on a separate system (help centre, community, docs) |
| In both | The maintained core |
/lp/ or /go/ prefixes among sitemap-only URLs usually mean paid acquisition; hundreds of near-identical ones usually mean programmatic SEO.
If the link crawl returns a homepage and a dozen links for a company of four hundred people, the navigation is probably rendered by JavaScript and invisible to a plain HTML crawl (see JavaScript rendering and crawlability). In AttractiveWebAI, DeepAnalyzer seeds its crawl from both the sitemap and discovered links, and Strong Mode renders pages in a headless browser for exactly this case.
Pass 2: Depth map
Capture: pages per click depth, maximum depth, and click distance to money pages: pricing, demo or contact, top product pages, sign-up.
Normalise before comparing. A 4,000-page site will always have more depth-3 pages than a 150-page one, so compare the share at each depth. Click distance needs no normalisation; two clicks is two clicks.
Read distances as strategy. Pricing one click from home on Rival A and three clicks behind "Talk to sales" on Rival B is a self-serve business next to a sales-led one, not a matter of taste. Note which link creates the shortest path, too: a 60-link footer makes any site look shallow. The fundamentals are in how to analyze website information architecture.
Pass 3: Navigation and labels
Capture: GNB items in order, mega-menu groupings, footer columns and utility links. Copy labels verbatim; paraphrasing destroys the evidence.
Then name each menu's organising principle: product ("Invoicing, Payments"), audience or role ("For Finance Teams"), task ("Get paid faster") or industry ("Healthcare, Retail"). Mature sites usually combine two, with product in the primary menu and industry or role under "Solutions".
Last, judge clarity: brand coinages that need explaining ("Nexus", "Pulse") versus words buyers actually search for. A competitor that renamed "Platform" to "Invoicing" has decided who it is selling to.
Pass 4: Page-type mix
Capture: classify every URL by type and count each; URL patterns do most of the work.
| Page type | How to spot it | Heavy investment signals |
|---|---|---|
| Product / feature | /product/, /features/ |
Product-led messaging |
| Solution / industry | /solutions/, /industries/ |
Vertical or enterprise sales |
| Case study | /customers/ |
Proof-driven, sales-assisted buying |
| Comparison | /vs/, /alternatives/ |
Fighting for switchers |
| Integration | /integrations/{partner} |
Ecosystem play, often programmatic SEO |
| Pricing | /pricing |
Self-serve, if public and shallow |
| Docs | /docs/, docs. subdomain |
Developer adoption as a channel |
| Blog / resources | /blog/, /resources/ |
Editorial acquisition |
Ratios reveal the acquisition engine. A rival with 300 integration pages and 30 blog posts grows through its ecosystem and search templates; the reverse ratio means editorial. If those integration pages differ only in the partner name, check how much unique content each carries.
Pass 5: Templates, metadata and stack
Sample five to ten pages per important type and capture:
- Title and H1 templates. "{Partner} integration for {Category} | Brand" is a keyword strategy in plain sight.
- Structured-data types per template, such as
Product,FAQPage,BreadcrumbListandOrganization: the pages they want rich results for (see the structured data guide). - hreflang markets. The languages they bothered to localise rank their market priorities. A localised homepage with English-only docs is a toe in the water, not a market entry.
- Tech stack. CMS, analytics, tag manager, experimentation, chat. An experimentation tool means the page you see may be one variant of a live test.
AttractiveWebAI's single-page analysis reports metadata, headings, links, images and the detected stack from one URL, which turns this pass into sampling rather than a view-source marathon.
Pass 6: Internal linking
Capture: where links come from (navigation, footer, breadcrumbs, related-content modules, body copy) and which pages receive the most.
Link equity concentrates on whatever a site wants to rank. A footer with 60 links to industry pages is an SEO decision, not a navigation one; blog posts that consistently link to comparison pages show where content traffic is routed. The mechanics match auditing your own internal links; here you read the pattern for intent rather than fix it.
Synthesis: from observations to IA decisions
Build the comparison matrix
One row per dimension, one column per site, including yours. The example below is illustrative only: fictional sites and numbers, not real data.
| Dimension | Us | Rival A (direct) | Rival B (leader) | Rival C (adjacent) |
|---|---|---|---|---|
| URLs in sitemap | 180 | 420 | 3,900 | 260 |
| GNB organising principle | Product | Product | Product + industry | Task |
| Clicks to pricing | 2 | 1 | 2 | Not public |
| Comparison pages | 0 | 6 | 24 | 0 |
| Integration pages | 12 | 40 | 310 | 8 |
| hreflang markets | EN, KO | EN | EN, DE, FR, JA | EN |
| Schema on product pages | None | Product, FAQPage | Product, BreadcrumbList | Organization |
Write both gap lists
They have, we lack: comparison pages, industry landing pages, breadcrumbs. These are candidate investments.
We have, they lack: a Korean localisation nobody else offers, public pricing while Rival C hides theirs. Protect these: redesigns that "normalise" toward the category tend to delete them.
Turn gaps into hypotheses
Rewrite each gap as: because we observed X, we believe changing Y for audience Z will produce W, measured by M. Then find the cheapest test.
| Gap | IA hypothesis | Cheapest test |
|---|---|---|
| Leader has 24 comparison pages; we have none | Buyers evaluating alternatives never reach us | Three comparison pages against the rivals sales meets most; track entrances and demo requests for a quarter |
| Leaders organise Solutions by industry; we use product | Enterprise buyers can't find themselves in our menu | Entry points for the two industries with case studies; track menu clicks and paths to demo |
| Rival A has pricing at depth 1; ours is at depth 2 | Self-serve buyers leave before seeing a price | Pricing in the GNB for a month; watch pricing views and sign-up starts |
Don't cargo-cult
Copying a competitor's structure is the most common way to waste a teardown:
- They may be wrong. Their menu may be the residue of a reorganisation, or the losing variant of a test nobody cleaned up.
- Structure without the engine doesn't perform. Three hundred integration pages work because a partnership and content pipeline sits behind them. Copy only the URL pattern and you publish three hundred thin pages.
- A different go-to-market needs a different IA. A sales-led leader hides pricing for reasons that don't apply to a self-serve challenger.
Borrow a pattern only when you can explain why it works for your users. Everything else goes into the hypothesis table.
Etiquette and limits
A teardown reads public structure. It is not an intrusion exercise.
- Public pages only. Never crawl behind a trial login, a paywall or a gated form.
- Respect robots.txt and identify your crawler honestly.
- Rate-limit. Low concurrency, delays between requests, a page cap.
- Don't republish. Record structure and labels; keep screenshots internal and never lift copy.
- Check the terms of service, which sometimes restrict automated access. This is not legal advice; if in doubt, ask your legal team.
The one-afternoon agenda
| Time | Block | Output |
|---|---|---|
| 0:00–0:20 | Choose sites, write five questions | Question list |
| 0:20–1:00 | Start crawls; transcribe menus while they run | Inventories, navigation transcript |
| 1:00–1:40 | Passes 1–2: inventory gap, depth map | Gap table, depth shares |
| 1:40–2:20 | Passes 3–4: navigation, page types | Page-type ratios |
| 2:20–3:00 | Passes 5–6: templates, stack, linking | Template notes |
| 3:00–3:40 | Matrix, gap lists, hypotheses | Hypothesis table |
| 3:40–4:00 | Write the summary | One-pager |
The deliverables
- A one-page summary: the five questions answered, the top three gaps in each direction, and three hypotheses with an owner and a measure each.
- An IA spreadsheet: one tab per site (URL, depth, page type, template, title, H1, schema, screen ID) plus the matrix. Screen IDs let the team say "RVB-EN-P-04" instead of "that pricing page with the toggle". Lay it out like your own IA Excel document; DeepAnalyzer's IA export drafts each tab with screen IDs assigned, so your time goes into annotation, not data entry.
Then keep the spreadsheet. Re-run the crawls next quarter and diff them: new directories and renamed menus appear there before they appear in press releases. That is how a one-off teardown becomes competitive intelligence.
FAQ
Is it legal to crawl a competitor's website?
Crawling public pages at a polite rate is common practice, but it depends on your jurisdiction and the site's terms of service; this is not legal advice. Stay on public pages, respect robots.txt, never bypass logins or paywalls, and don't republish content.
How many competitors should I analyse?
Three to five. With fewer, you cannot tell a category convention from one company's quirk; with more, the afternoon becomes a week and the synthesis gets shallow.
What if the competitor's site is a JavaScript single-page app?
A plain HTML crawl may find little beyond the homepage, because the navigation renders in the browser. Use a headless-browser crawl and check its coverage against the sitemap.
How do I know which page types matter most?
Count them, then weight by placement. A type with many URLs and a GNB slot is a strategic bet; a type reachable only through the sitemap is usually aimed at search engines rather than visitors.
Should we copy the market leader's structure?
No. Borrow the patterns you can explain, and test them. The leader's IA reflects its content engine, go-to-market and history, including legacy it would remove if it could.