Technical SEO | Crawl · Render · Index

A page has to be crawled, rendered and indexed
before its content gets a vote.

Keywords and content decide where you place. The technical layer decides whether you are in the race at all. We audit that layer against what search engines and AI crawlers actually do with your site, then ship the fixes ourselves.

  • Fixes shipped, not filed
  • In-house engineers
  • Every fix recrawled and verified
  • 10+ years of engineering heritage
Get a free technical audit →

Full crawl · prioritised fix list · yours to keep either way

Crawl Render Index Canonicalise RankEvery stage is a place a page can quietly drop out.

What we fix

The layer a content audit reports around, not on.

Six areas account for most of the ranking capacity a site is leaving unused. They are related: a rendering problem creates an indexation problem, and an architecture problem wastes the crawl budget that would have found the fix. We work through them in the order the dependencies run.

01

Core Web Vitals

  • LCP, INP and CLS read from field data in CrUX, not lab scores alone
  • Render-blocking scripts, font loading and image delivery
  • Third-party tags measured for what they cost the main thread
  • Templates fixed at the source rather than page by page

02

Crawlability

  • robots.txt directives reviewed per user agent, Google and otherwise
  • Crawl traps from filters, sorting parameters and infinite pagination
  • Server logs read to see which URLs Googlebot requests in practice
  • Crawl budget pointed at pages that can earn, away from ones that cannot

03

Indexation

  • Pages that should be indexed and are not, with the reason for each
  • Pages that are indexed and should not be, thinning the site's signal
  • Canonical tags checked against what Google selected, which often differs
  • Parameter and duplicate URLs consolidated onto one address

04

Rendering and JavaScript

  • Rendered DOM compared against raw HTML for what survives the difference
  • Client-side routes that never produce a crawlable, linkable URL
  • Server-side rendering or prerendering where content depends on hydration
  • Mobile-first indexing checked on the mobile render, since that is the one used

05

Site architecture

  • Click depth from the homepage to every page that needs to rank
  • Orphan pages that exist in the sitemap and nowhere in the navigation
  • Internal linking that routes authority toward commercial pages
  • URL structure and redirect chains cleaned up before they compound

06

Structured data

  • Schema types matched to what each template genuinely is
  • Markup validated against Google's requirements, not just JSON syntax
  • Eligibility for rich results checked and monitored after deployment
  • Entity detail that gives answer engines something precise to quote

None of this is a substitute for content, and we do not present it as one. It is what makes content countable. A page that is blocked, canonicalised elsewhere, or reachable only after hydration can carry the best article in your category and never register. Fix the technical layer and the content you already own starts working.

Why we're different

Findings are cheap. Shipping the fix is the work.

Most technical SEO engagements end at a findings document. The document is usually accurate. It then enters a development backlog, gets sized against product priorities, and waits. Months later the crawl looks much the same and the audit is out of date, because the site moved and the report did not.

We work inside a business that has been building and maintaining production websites for over a decade, so the engineers who ship code every day are the ones correcting your render path, restructuring your internal links and deploying your schema. Nothing is lost in a handoff, because there is no handoff between finding the problem and fixing it.

In-house engineersDirect codebase or CMS accessQA before deployRecrawled and verified

Scope of work

What a full technical audit covers.

The complete scope, grouped the way the dependencies run rather than alphabetically. Not every site needs every item, and the audit says which ones yours does.

Crawl control

Deciding what search engines are allowed to spend their requests on, and confirming what they actually do.

  • robots.txt directives audited per user agent
  • Crawl budget allocation across templates
  • Log file analysis of real Googlebot behaviour
  • Faceted navigation and pagination traps

Indexation

Making sure the set of indexed pages matches the set of pages that should be earning impressions.

  • Canonicalisationchecked against Google's selection
  • Duplicate content and parameter consolidation
  • Redirect management and chain removal
  • Index bloat from thin and templated URLs

Rendering

Confirming the content you wrote is the content a crawler receives, on the render it is judged by.

  • JavaScript SEO and rendered DOM comparison
  • Server-side rendering or prerendering where needed
  • Mobile-first indexing parity checks
  • Crawlable URLs for client-side routes

Performance

Core Web Vitals treated as an engineering problem with a measurable threshold, not a score to chase.

  • LCP, INP and CLS on CrUX field data
  • Render-blocking scripts, styles and fonts
  • Image and asset delivery at template level
  • Third-party tags weighed against what they cost

Structure

Routing authority and crawl paths toward the pages with commercial intent behind them.

  • Site architecture and click depth
  • Internal linking strategy and orphan recovery
  • XML sitemaps accuracy and segmentation
  • URL design and hierarchy

Semantics and reach

Telling machines what a page is, and keeping the connection between them trustworthy.

  • Structured data and schema validation
  • hreflang for international and multi-region sites
  • HTTPS and security header configuration
  • Monitoring with alerts on regressions

Tooling

Not a performance claim, just the software behind the work, so you know what is running against your site. Findings from any of these are cross-checked against your own Search Console data before they reach the fix list.

Google Search ConsoleScreaming FrogPageSpeed Insights & CrUXServer log analysisGA4SemrushSEOCrawl AI

Machine readability

Googlebot is no longer the only crawler that matters.

Answer engines and AI assistants now sit between a question and your website, and they read it under stricter conditions than Google does. The technical work that makes a page indexable is the same work that makes it quotable, but the tolerances are tighter.

01

Crawler access, decided deliberately

  • GPTBot, ClaudeBot, PerplexityBot and Googlebot-Extended each read their own robots.txt rules
  • A site can be fully open to Google and quietly closed to every one of them
  • Access set as a business decision rather than inherited from a default file

02

Content that survives without JavaScript

  • Most AI crawlers do not execute JavaScript the way Googlebot's rendering service does
  • Copy that appears only after hydration is often invisible to them entirely
  • Server-rendered HTML is the difference between being read and being skipped

03

Structure a model can quote accurately

  • Clean heading hierarchy and self-contained answers that survive being extracted
  • Schema that states entities, prices, locations and authorship without ambiguity
  • Facts kept consistent across the pages that reference them

This is not a separate discipline sold as an add-on. It is the same crawl, render and index work, held to the tolerance the newer crawlers require. A page that is easy for a machine to parse is easy for all of them to parse.

Our process

Audit, prioritise, ship, verify, monitor.

Five stages, run in that order. The order carries the work: prioritising before the crawl is complete produces a list of the problems that were easiest to find rather than the ones costing the most.

01

Full technical crawl

Every reachable URL, crawled rendered and raw, cross-referenced against Search Console coverage, CrUX field data and your server logs. Logs are the part most audits skip, and the only source that shows which pages Googlebot genuinely spends its requests on.

02

Prioritised fix list

A crawler flags hundreds of items and most of them do not matter. Each finding is scored on the traffic it is blocking against the effort to resolve it, so the list opens with what changes the numbers and states plainly which items are safe to leave alone.

03

Implementation

Fixes shipped into your codebase or CMS by engineers, through your review process and your branch conventions, with every change documented. Template-level problems are corrected at the template so they stay corrected as the site grows.

04

Recrawl and verify

The site is recrawled, URL Inspection is run against affected templates, index coverage is rechecked, and vitals are watched on field data as the 28-day window turns over. A change that deployed cleanly but never reached the index is still an open item.

05

Ongoing monitoring

Technical SEO does not stay fixed. A release ships a noindex, a plugin rewrites a canonical, a new script pushes INP past its threshold. Scheduled crawls and alerts on indexation, status codes and vitals surface it in days rather than at the next quarterly review.

Want to see what a full crawl turns up on your site?

The audit and the prioritised fix list are yours to keep, whether you implement them with us, with your own team, or on your own schedule.

Get a free technical audit →

Track record

The track record predates the company.

The team behind Wegile DGTL has worked together for over a decade, running the complete marketing engine at its parent company, a software development company, and delivering campaigns for its clients before that. The results below were earned by this team across those years, on accounts where the technical work and the content work were run together.

Home Services · California · Landscape & Christmas Lighting · Landscaping · Turf Installation · Irrigation · Outdoor Lighting

Elevated Seasons: from a wasted budget to market visibility.

8.76x

ROMI in 2 years

11.26x

ROAS (ad spend only)

2.28x

Lower cost per lead

3.82x

Lower customer acquisition cost

Elevated Seasons came to us after a previous agency spent their budget and delivered almost nothing. Sound familiar? We rebuilt the brand, launched Google Search and Performance Max, and ran the SEO strategy that took their revenue from a near standstill to consistent growth, still climbing two years in.

Brand repositioningGoogle Search adsPerformance MaxSEO strategy2yr ongoing partnership

eCommerce · Hair & Beauty · SEO + Meta Ads + Google Shopping

Hairbarnyc: technical foundation first, then content and spend on top of it.

320%

Organic revenue increase

38%

Customer acquisition cost reduction

8mo

To hit results

SEO, Meta Ads and Google Shopping run as one coordinated budget. Over eight months that combination drove a 320% increase in organic revenue and cut customer acquisition cost by 38%, with the crawl, indexation and product template issues resolved before any of the content or spend work was scaled.

Technical SEOIndexation cleanupProduct template fixesGoogle ShoppingMeta Ads
"Working with them transformed our business. Their creative testing, Meta retargeting and dedicated tech support boosted our product sales and appointments, helping us scale revenue and grow our brand. Their strategic insight and execution have been truly exceptional."

★★★★★ 5.0 · Beny, Hairbarnyc

Digital Products & eCommerce · Paid Social Launch

Deliciously Fit, with Chris Powell.

2,000

Books sold

<3mo

Time to sell out the run

A digital recipe book with hundreds of high-protein recipes built for weight-loss and GLP-1 audiences. Our team ran the paid social strategy behind the launch, testing hooks and formats against a cold audience, selling 2,000 copies in under three months.

Paid social launchCreative testingCold audience acquisition2,000 copies sold
Chris Powell, Deliciously Fit
"I've been working with them for nearly 10 years, and they've been an incredible partner every step of the way. From developing my fitness app and nonprofit app, to helping maintain and grow my website, their team has consistently delivered with professionalism and precision. They're responsive, reliable, and deeply knowledgeable, always guiding projects with care and expertise. I trust them fully and plan to continue working with them for years to come."

★★★★★ 5.0 · Chris Powell, Deliciously Fit

Technical SEO questions

Straight answers. No sales fluff.

The questions developers, in-house SEO teams and business owners ask before commissioning technical work, answered plainly and in full.

A full crawl of every reachable URL, then a review of what search engines can do with them. Crawl budget and robots.txt directives, indexation and canonical signals, how each page renders once JavaScript executes, Core Web Vitals on field data rather than lab scores alone, internal linking depth and orphan pages, sitemap accuracy, redirect chains, duplicate and parameter URLs, hreflang if you run multiple regions, HTTPS and security headers, and structured data validity.

Server log files are then read alongside all of it, because logs show which of those pages Googlebot is actually requesting. That is usually where the surprises are.

Content and links compete for position. Technical SEO decides whether a page is eligible to compete at all. A page that is blocked, canonicalised elsewhere, rendered client-side with no server-rendered fallback, or buried six clicks deep can carry excellent content and never rank for it.

Technical work does not replace content, it makes content countable. The two compound, which is the argument for fixing the technical layer first: everything you publish afterwards inherits the improvement.

Ranking for some terms usually means part of the site is crawled and indexed cleanly, not all of it. The common pattern is a strong homepage and a handful of pages carrying the entire site, while templated pages, product variants, paginated archives or a JavaScript-driven section sit partially indexed.

An audit shows the gap between pages that exist, pages that are indexed, and pages that earn impressions. That gap is normally where the available growth already is, without publishing anything new.

It depends on the fix and on how often Google recrawls your site. Indexation fixes, such as unblocking valid URLs or correcting a wrong canonical, can show up within days of a recrawl. Core Web Vitals move on field data, which uses a 28-day rolling window, so improvements surface gradually across roughly a month.

Architecture and internal linking changes compound over several months as authority redistributes. Each fix is recrawled and verified rather than left for a ranking report to imply it worked.

We implement them. Fixes go into your codebase or CMS by the same engineers who ship production code, then each one is QA checked and recrawled before it is marked done.

A findings document with no implementation path is the most common reason technical SEO stalls, because the work ends up queued behind product priorities in a development backlog and ages out before it ships.

Yes, and it is a common setup. In-house teams usually own content, keyword strategy and stakeholder relationships, and want engineering capacity for the technical layer rather than another strategy document.

We work to your ticketing process, document every change, and hand over the crawl configuration and monitoring so your team keeps the visibility after the engagement ends.

It does, and the requirements are stricter. Most AI crawlers do not execute JavaScript the way Googlebot's rendering service does, so content that appears only after hydration is often invisible to them. They also respect their own user agents in robots.txt, which means a site can be fully open to Google and quietly closed to GPTBot, ClaudeBot or PerplexityBot.

Clean server-rendered HTML, accurate structured data and a deliberate crawler access policy are what make a page usable as a citation rather than merely indexable.

One last thing

The audit costs nothing, and the fix list is yours either way.

A full crawl of your site, the findings that matter ranked against the ones that do not, and an honest view of how much of your current traffic ceiling is technical. Implement it with us, hand it to your own developers, or keep it on file. The document is yours regardless.

Get your free technical audit

Full crawl · prioritised fix list · no obligation