Technical optimisation

The technical audit is the stage at which I scan the site and analyse the areas that affect indexing, crawl budget, the quality of server responses and rendering. The basis is a crawl of the site, that is, retrieving the list of URLs together with response statuses, metadata and information on internal linking. This allows me to identify, among other things, errors, duplication and blocks that make it harder for the search engine to access the content. I gather the audit results in a report with example URLs, the consequences of the problem, a recommendation and an indication of the place on the site or in the code where action is needed.

As part of the audit I carry out an indexing analysis to establish which page types are indexed and which are excluded, and what the reasons are (e.g. noindex, canonical, blocks, parameters or duplication). I verify robots and the sitemap for rules that block resources or sections and for the quality of the sitemap: completeness, freshness, URL types and consistency with canonical and HTTP statuses. I also check the information architecture and linking (click depth, orphan pages, link loops, navigation and pagination), because this directly affects the discovery of key pages. In addition, I analyse HTTP statuses and redirects to detect 4xx/5xx, soft 404s and redirect chains and loops, including http/https and www/non-www inconsistencies.

The audit also covers duplication and canonical, including duplicates arising from parameters, variants, filters, sorting or language versions, and verification of the consolidation signals. I check rendering and resources to make sure that the key content and links are available after rendering, that resources are not blocked and that the JS implementation does not hinder indexing. I check performance and stability (delays, resource weight, cache, compression, images and server errors) and the mobile aspects: consistency of content and metadata between mobile and desktop, and UX problems that may block access to the content. If applicable, I also verify structured data and language versions (language/region mapping, hreflang, canonical and redirects between versions).

• Site crawl: URLs, response statuses, metadata and internal linking.
• Indexing analysis: noindex, canonical, blocks, parameters and duplication as reasons for exclusions.
• Robots and sitemap: completeness, freshness and consistency with canonical and HTTP statuses.
• Architecture and linking: click depth, orphan pages, loops, navigation and pagination.
• HTTP statuses and redirects: 4xx/5xx, soft 404s, chains/loops, http/https and www/non-www.
• Rendering and resources: availability of content/links and the impact of JS and blocked resources.
• Performance and stability: cache, compression, images, delays and server errors.
• Mobile: consistency of content and metadata, and elements interfering with indexing.
• Structured data and language versions (if applicable): correctness of the implementation and consistency with the content.

Implementation and optimisation is the stage at which fixes are carried out in the CMS, code or server configuration in line with the technical backlog. The work is done iteratively, with risk control, in order to limit the possibility of side effects on indexing, navigation and performance. The scope of implementation follows from the planned tasks and their dependencies, which makes it possible to implement the changes in a logical order. In practice this means carrying out those fixes that are feasible in the given environment and within the available implementation capabilities.

The way the work is carried out is affected by access to a test environment and by the agreed deployment windows, because testing before publication reduces the risk of problems in production. The absence of these conditions may limit the scope of changes or increase the number of iterations needed for a safe implementation. The implementation stage is also sometimes limited by technological and organisational dependencies, such as the capabilities of the CMS, a lack of access to the server or code, external components, release priorities, the risk of changes to templates and the availability of the dev team. These conditions are taken into account when choosing the order and form in which the backlog tasks are carried out.

• Indexability: corrections to noindex/canonical and other signals that affect indexing.
• Robots and sitemaps: tidying up the blocks and adapting the sitemap to the current URL structure.
• Redirects and statuses: removing 4xx/5xx errors, soft 404s, and redirect chains and loops.
• Parameters and duplication: limiting duplicates arising from filters, sorting and parameters.
• Pagination and internal linking: improving the discovery of key pages.
• Performance and rendering: changes that affect speed, stability and the availability of content/links in rendering.
• Structured data and language versions (if applicable): corrections to the markup and to the settings between versions.

Monitoring and iterations is the stage at which, after the implementation has been signed off, I observe how the site behaves in terms of indexing and crawl and whether new errors appear. I focus on the signals that point to problems with the availability of URLs, exclusions or errors that may occur after changes to the site. This makes it possible to catch quickly the problems that were not visible during the earlier verification. Monitoring also serves to keep the site technically in order as further modifications appear.

I translate the results of the observation into further tasks and update the technical backlog, so that the priorities match the current situation. New problems may arise both from subsequent implementation changes and from modifications within sections or templates, which is why the iterations are planned as the next specific steps to be carried out. Updates to the backlog include refining the URL/section scope and the sign-off criteria, so that subsequent implementations are verifiable. Depending on what was agreed about the implementation model, the tasks may take the form of recommendations or of items to be carried out directly, if the appropriate permissions are available.

• Observing indexing and crawl errors after implementation and identifying new problems after changes to the site.
• Updating the technical backlog with new tasks arising from the monitoring.
• Refining the priorities, dependencies and acceptance criteria for the next iterations.