Site architecture design

Site architecture optimisation is a structured process running from defining the scope to quality control after implementation. The work begins with determining which sections of the site are covered and what the implementation constraints and the rules for approving changes are. Next come the inventory and the crawl, which feed the analyses of indexing, accessibility, navigation depth, URL structure, taxonomy, filters, pagination, internal linking, and canonicalisation and duplication signals. The result of these analyses is a diagnosis of the architecture problems and decisions on the target division into sections and hierarchy.

The final part of the process concentrates on designing the target architecture and translating it into implementation requirements. In practice, this produces URL and indexing rules and control over page generation, as well as an implementation specification for the dev team. If the structure changes, a redirect and URL migration plan is prepared, with old->new URL mappings and rules for handling removals and merges. After implementation, post-publication QA is carried out, and the results of the work go into the final report and the backlog of further actions, together with priorities.

• Start, and agreeing the scope and the approval workflow for changes
• Access and input materials: indexing and traffic data, and information about the CMS
• Site inventory and crawl (URLs, templates, page types, areas of scale)
• Analyses: indexing and accessibility, navigation depth, URLs, taxonomy, filters, pagination, linking, canonicalisation and duplication
• Diagnosis of the problems and decisions on the target hierarchy and the page types to be indexed
• Target architecture design, URL/indexing rules, implementation specification, redirect plan
• Implementation support and consultations, QA after implementation, final report and backlog

Access and input materials are needed so that the work on the site architecture can be based on indexing and traffic data and on the real rules by which the CMS operates. In practice this is about being able to check how crawlers reach individual page types and which areas of the site are key from the point of view of the structure. Information about how the site generates pages and URLs is just as important, because it affects the scope of the crawl and the later decisions on indexing control. If a test environment is available, it allows changes and their interpretation to be verified more safely before publication.

For this service, specific inputs are needed on the client’s side that directly make it possible to carry out the inventory and the crawl of the site. The input materials are used to build the list of URLs and page types and to establish which sections have the highest priority in the analyses. This makes it possible to work on a consistent data set and limits the risk of overlooking important areas generated dynamically by the CMS.

• access to indexing and traffic data
• a list of key sections and/or URLs to be included in the work
• information about the CMS and the rules for generating pages and URLs
• the possibility of working on a test environment (if one is available)

Analysing the indexing and accessibility of pages means checking which page types are indexed, which are blocked and where there are problems with crawler access. I verify errors, redirects and potential loops that can hinder the crawl and distort the interpretation of the site structure. The reference point is the previously identified URL types and templates, so the assessment concerns specific groups of pages and not individual cases. The results of this analysis are organised so that it is clear which problems require a priority response.

In this part of the work I compare how different areas of the site behave in terms of access and indexing, in order to point out the places that create a risk for the quality of the index. The irregularities identified are then aggregated into the diagnosis of the architecture problems together with their consequences, including in the context of wasted crawl budget, duplication and poor distribution of link equity. On that basis it is easier to establish which page types should be developed and indexed and which should be restricted. The conclusions from this stage are also the input for further decisions on indexing rules and control over page generation.

• checking the indexing status of page types and URL patterns
• identifying blocks and barriers to crawler access
• detecting errors, redirects and loops that affect the crawl
• organising the problems for further diagnosis and prioritisation