What is the .htaccess file and how can you use it in SEO?

A website may stop working after making certain edits to the .htaccess file, but there is no need to panic. This happens when SEO specialists make technical fixes. 

.htaccess files are both a blessing and a curse for SEOs. They help with adjusting a site’s functionality, from protecting files with a password, through 301 redirects, to setting up a custom error page. .htaccess files are useful when the site runs on Apache Web Server (Apache web server).

What is .htaccess? 

The term .htaccess refers to a configuration file that can be read and executed by Apache Web Server software in order to enable/disable additional functionality and features.

The .htaccess file exists in a Unix-based environment and is rendered at directory level. It overrides the general server settings, allowing a custom configuration of the site.

1. 01Configuration fileRead by Apache
2. 02Directory levelOverrides server settings
3. 03Access controlManages user permissions

A key configuration file for server customisation and improving SEO.

What does the name .htaccess mean? 

The name .htaccess is short for “hypertext access”. The name was created when the file became popular among developers, who used it to modify user access functions.

The .htaccess file uses Apache server http.config directives to allow or restrict access to directories for people using usernames and passwords. In the case of SEO, .htaccess plays a much more important role.

What is .htaccess used for? 

If you have a site with constantly changing requirements, then .htaccess will be a huge asset. In SEO, the .htaccess file is useful because it can be used to pass server directives regarding 301 redirects, enable caching, update HTTP headers, control crawling, create SEO-friendly URL headers, etc.

1. 01Root directory & subdirectoriesConfiguration at each folder level.
2. 02Multiple WordPress sitesPresent in the directory of each site.
3. 03Note for beginnersDo not edit without testing (demo site).
4. 04Access via FTPKnowledge required (e.g. FileZilla).
5. 05WordPress plugins (e.g. Yoast)Make safe editing easier.

Key point: the .htaccess file is ubiquitous, but it requires caution and the right tools (FTP or plugins) to edit, especially for beginners.

Where can you find the .htaccess file? 

The .htaccess file refers to directory-level configuration, so you can find it in almost all folders in the web directory. If you have one web directory with many subdirectories (sites), the .htaccess file can be found in the root directory as well as in each subdirectory.

Note: .htaccess is a file that beginners should not work on. It is therefore recommended to test it on a demo site. You must also have knowledge of using Filezilla, Total Commander or others in order to access the root folder if you are blocked as an Admin in WordPress. 

Editing .htaccess in the Yoast plugin

If you run a WordPress-based site, there are quite a few plugins that support editing the .htaccess file. Most WordPress users use Yoast as the default SEO plugin. Here are the steps for editing the .htaccess file in the Yoast panel in WordPress:

• step 1 – log in to your WordPress admin panel,
• step 2 – open Yoast settings,
• step 3 – open Tools, 
• step 4 – while in the tools section, select “File Editor”,
• step 5 – edit the .htaccess file and save the changes.

On most WordPress-based sites, the .htaccess file has the following default configuration: 

# BEGIN WordPress
RewriteEngine On
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
# END WordPress

1. 01FTP access (FileZilla)Log in to the web directory
2. 02Download the .htaccess fileDownload it from the directory to your drive
3. 03Update the fileFind and remove the faulty code

Don’t worry, the lockout problem can be fixed easily. Use FTP, download the file and remove the incorrect code before uploading it again.

What to do if the website does not have a .htaccess file?

Because .htaccess is an automatically generated configuration file, it will be available in 99% of cases. Usually, however, it is a hidden file in directories, and webmasters think it is missing. Just use the “show hidden files” option to find it.

If you still cannot find the .htaccess file, you can create it yourself and upload it to your web server.

Stages of creating your own .htaccess file

• step 1 – open Notepad,
• step 2 – enter the configuration (for testing, it is worth using the default configuration shown above),
• step 3 – save the file in ASCII under the name .htaccess,
• step 4 – make sure the file has not been saved in .txt format,
• step 5 – use Filezilla to upload the .htaccess file to your web directory.

Note: if you are a WordPress user and have an empty field for the .htaccess file, simply add the configuration and save the changes.

What to do if you are locked out of the panel after updating the .htaccess file? 

It is possible to lock yourself out of and be thrown out of the panel after updating the .htaccess file using Yoast or the WP file manager. Do not panic, as this problem can be easily fixed.

If you are an SEO specialist, get FTP access to your website from the developer who manages the server. If you need specialist help, the developer will be the best choice.

• step 1 – log in to your web directory using FileZilla, 
• step 2 – download the .htaccess file from the directory,
• step 3 – update the .htaccess file using the default configuration and save it,
• step 4 – replace the old .htaccess file in the directory with the new one.

Note: if your priority is to restore the website as quickly as possible, use the default code. If you have made many changes to the .htaccess file, try to find the faulty code. Remove it before uploading the file to the server again.

Why is an error visible after updating the .htaccess file? 

If you are an SEO specialist who wants to gain in-depth knowledge of .htaccess, there is a chance you will make a few mistakes at the beginning. This is not a problem if you understand why the web server reported an error.

There are several common problems that SEO specialists encounter when it comes to updating the .htaccess file. Here is a list of the most common .htaccess problems:

• disabled overrides: for your .htaccess file to work properly, you must first enable the “AllowOverride” option (allow overrides). If this option is disabled, all configurations set in the .htaccess file will be inactive. To make sure the override option is enabled, take the following steps:

• step 1 – open the Apache configuration file (http.conf),
• step 2 – set the override permission directive to: “AllowOverride All” (allow all overrides),
• step 3 – save the Apache configuration file and restart the Apache server.

• incorrect file name: this is the second most common mistake made by SEO specialists. The .htaccess file is a Unix-based configuration file saved in ASCII, so a mistake in its name leads to an error. If the file does not start with a “.” or has been uploaded in a format other than .txt, the Apache server will ignore such a file and the configuration you set, 

• .htaccess settings hierarchy: .htaccess files are executed based on their hierarchy. Several rules placed at the beginning of the .htaccess file may cancel rules appearing at later stages of the configuration. If the initial rules are crucial, try moving the specific configuration higher up,

• multiple .htaccess files: .htaccess files can be used in directories, so there is a chance that your site uses many such files. In such cases, one file may contradict the configuration set in another file, which leads to errors. The problem can be solved by deactivating individual .htaccess files,

• syntax error: the .htaccess file functions entirely based on the syntax used to configure your website. A syntax error in the file can cause the site to go offline. It is therefore essential to understand the syntax before updating .htaccess files.

What are the common error messages that may appear after updating the .htaccess file? 

Every time a user visits your website, he or she interacts with the web server directly or indirectly. Pages that load after clicking on illustrations and other viewed resources are fetched by the web server. As a webmaster, you can limit this.

Some websites use the .htaccess file to set up authentication before accessing pages. When it comes to SEO specialists, they use this file to make sure that users and search engine bots have easy access to the most important pages. If a page does not provide the required information, the web server will generate an error based on the configuration set in the .htaccess file.

Here is a list of error codes displayed by web servers when they are unable to fetch the required data.

Client request errors

• 400 — bad request: invalid URL structure. The server is unable to understand the user’s request, 
• 401 — authorisation required: this message is displayed when access to the page has been restricted by webmasters, 
• 402 — payment required (not yet used): if payment cannot be initiated, the following code will appear,
• 403 — forbidden – the reason this 403 error appears is an attempt to access a resource with restricted permission. A page displays 403 errors when users try to access a page requiring authentication, 
• 404 — not found: 404 is a clear indication to the user that the required URL is unavailable on the site. This error may be caused by entering the URL incorrectly or by removing the page from the site,
• 405 — method not allowed: this HTTP response status code indicates that the server rejected the request method despite understanding its purpose, 
• 406 — not acceptable (encoding): usually this problem occurs when the server cannot respond to the accept request,
• 407 — proxy authentication required: this error indicates that the request cannot be completed due to a lack of proxy server authentication between the browser and the server,
• 408 — request timeout: this is one of the common HTTP errors encountered by webmasters when the server does not receive a complete client request within the allotted timeout period,
• 409 — conflict: this error appears when the state of the target resource conflicts with the current state. To resolve this issue, identify the conflict and send the request again, 
• 410 — gone: this error code informs the user that access to the requested resource has been completely removed from the server and will remain so,
• 411 — length required: this error means the server is unable to accept the request due to a problem defining the content-length header,
• 412 — precondition failed: this is an error caused by a security issue in one or more security configurations deployed on your server,
• 413 — request entity too large: when the requested resource is too large for the server to load, the user may see a 413 error, 
• 414 — request URL too long: this refers to a URL structure exceeding 2048 characters. The server is unable to decipher this, which leads to a 414 error, 
• 415 — unsupported media type:  this error appears when the server refuses to load a resource that has an unsupported media format.

Server errors

• 500 — Internal Server Error (internal server error),
• 501 — Not Implemented (not implemented),
• 502 — Bad Gateway (“bad gateway”),
• 503 — Service Unavailable (service unavailable),
• 504 — Gateway Timeout (“gateway timeout”),
• 505 — HTTP Version Not Supported (unsupported HTTP version).

What is the .htaccess file used for? 

Redirects

Have you changed the domain name of your website? As an SEO specialist, you certainly do not want users to see a 404 error page, and you do not want the site to lose its authority and earned trust.

The .htaccess file is the solution to both of these problems. Adding a redirect to the .htaccess file will help redirect traffic and the site’s authority to the new domain.

It is interesting that you can do the same when it comes to the URLs of your site. By creating a 301 redirect, you move users and search engine bots wanting to access the old URL to the new page within your site.

An example of a redirect using .htaccess at domain level:

# This allows the entire site to be redirected to another domain 

Redirect 301 / http://przyklad.com/

An example of a URL redirect using .htaccess: 

RedirectMatch 301 ^/old-url.html$ /new-url.html

SEO-friendly URLs (URLs practical for SEO)

Is the structure of your URLs disorganised? Are users and search engine bots unable to understand what is on the site? This happens to many webmasters.

Failing to pay attention to URL structure at the outset becomes a major problem once the site has gained authority. By using the .htaccess file you can establish the correct URL structure on your site.

In addition, any extension paired with the URL (e.g. .html or .php) can be easily removed by adding instructions in the .htaccess file.

Example 1: using .htaccess to remove extensions from URLs

Removing the .php extension

RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME}.php -f
RewriteRule ^(.*)$ $1.php [NC,L]

Removing the .html extension

RewriteEngine on 
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME}.html -f
RewriteRule ^(.*)$ $1.html [NC,L]

Example 2: using the .htaccess file to create lowercase URLs

Method 1

RewriteCond %{REQUEST_URI} [A-Z]
RewriteRule . ${lc:%{REQUEST_URI}} [R=301,L]

The code above affects the entire URL structure, including the domain name.

Method 2

RewriteCond %{REQUEST_URI} [A-Z]
RewriteRule ^.+.html$ ${lc:%{REQUEST_URI}} [NC,R=301,L]

The example above only affects HTML file names.

Example 3: using .htaccess to dynamically rewrite URLs 

RewriteEngine On
RewriteRule /(.*)/(.*)/$ page.php?category=$1&product=$2

Bad URL: site.com/page.php?category=2&product=54

Good URL:  site.com/sandwiches/rueben-sandwich/

Redirecting to the latest version of the site

For some reason, a search engine may index multiple versions of your homepage. To avoid confusion on the search engines’ side and send users to the latest version of the website, you should inform search engine bots where they should go by using a 301 redirect.

To change the www version. Using a 301 redirect, use the following code in the .htaccess file:

RewriteCond %{HTTP_Host} ^site.com$ [NC]
RewriteCond ^(.*)$ https://www.site.com/$1[R+301,L]
RewriteCond %{THE_REQUEST} ^.*/index
Rewriter ^(.*)index$ https://www.site.com/$1 [R=301, L]

If you prefer the URL without www., use the above code without www.

Using the .htaccess file to speed up the website

Page speed is very important for webmasters. That is hardly surprising, because Google regards speed as one of the more important ranking factors. It has a significant impact on a website’s position on the search results page (SERP).

Google does not want pages at the top of the list to cause problems and poor user experiences. In addition, slow loading consumes a lot of crawling limit (crawl budget).

One of the easier and safer ways to improve page speed is configuring the .htaccess file. 

Enabling caching

By enabling caching in the .htaccess file, website assets will be stored in the user’s browser, allowing them to load quickly.

You can enable caching in two ways. 

“ExperiesByType” – using this method in the .htaccess file, you can set cache timeframes, i.e. the expiry period for each asset on your website. 

Example:

<ifModule mod_headers.c>
# YEAR
<FilesMatch ".(ico|gif|jpg|jpeg|png|flv|pdf)$">
Header set Cache-Control "max-age=29030400"
</FilesMatch>
# WEEK
<FilesMatch ".(js|css|swf)$">
Header set Cache-Control "max-age=604800"
</FilesMatch>
# 45 MIN
<FilesMatch ".(html|htm|txt)$">
Header set Cache-Control "max-age=2700"
</FilesMatch>
</ifModule>

“Cache-Control Header” – the cache-control header uses the maximum validity period of assets before they expire.

Example: 

# One month for the most static assets

<filesMatch ".(css|jpg|jpeg|png|gif|js|ico)$">
Header set Cache-Control "max-age=2628000, public"
</filesMatch>

Enabling Gzip compression

One of the reasons for a website’s slower speed is the size of its assets. By enabling gzip in the .htaccess file, you can reduce the size of images, the file size and the amount of data sent to the recipient’s browser.

Enabling gzip is one of the easier guidelines in the .htaccess file.

Here is an example:

<ifModule mod_gzip.c>
mod_gzip_on Yes
mod_gzip_dechunk Yes
mod_gzip_item_include file .(html?|txt|css|js|php|pl)$
mod_gzip_item_include handler ^cgi-script$
mod_gzip_item_include mime ^text/.*
mod_gzip_item_include mime ^application/x-javascript.*
mod_gzip_item_exclude mime ^image/.*
mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*
</ifModule>

Enabling deflate

Some web servers do not support gzip, so the site may display errors. In such cases, it is recommended to use deflate in the .htaccess file.

Here is an example: 

<IfModule mod_deflate.c>
# Compress text, HTML, JavaScript, CSS, XML
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/x-font
AddOutputFilterByType DEFLATE application/x-font-opentype
AddOutputFilterByType DEFLATE application/x-font-otf
AddOutputFilterByType DEFLATE application/x-font-truetype
AddOutputFilterByType DEFLATE application/x-font-ttf
AddOutputFilterByType DEFLATE application/x-javascript
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE font/otf
AddOutputFilterByType DEFLATE font/ttf
AddOutputFilterByType DEFLATE image/svg+xml
AddOutputFilterByType DEFLATE image/x-icon
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/xml

# The following lines are intended to avoid errors in some browsers

BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4.0[678] no-gzip
BrowserMatch bMSIE !no-gzip !gzip-only-text/html
BrowserMatch bMSI[E] !no-gzip !gzip-only-text/html

Using the .htaccess file to improve crawling and indexing

You may already be using the robots.txt file to allow or block search engines from crawling and indexing your website. However, you may have assets other than web pages on your site. In such cases, the robots.txt file may not work.

If you want certain assets, such as PDF or Word documents, not to be indexed, the best way is to set the X-robots tag in the .htaccess file.

A custom .htaccess header can work with all indexing guidelines.

Example: 

<FilesMatch ".(docx|pdf)$">
    Header add X-robots-tag "noindex, noarchive, nosnippet"
</FilesMatch>

In this case, each file with the .doc and .pdf extension will be treated as “noindex, noarchive, nosnippet” (no indexing, archiving or snippets).

Guidelines for bots

You may or may not want Google to access all pages on your site for indexing. Restrictions in robots.txt and “no follow” links are not always an option available in some content management systems. If these options are unavailable, you need to use the .htaccess file to place “No Index Meta Robots” tags on the pages that are to be restricted.

For example, if you do not want search engines to show the PHP files you have generated, place the following code in the header file:

Header(“X-Robots-Tag: no index”, true);

To make the links on the page “no follow,” use code:

Header(“X-Robots-Tag: no index, no follow”, true);

To directly configure the web server, use:

<Files Match “robots.text”>

Header set X-Robots-Tag “no index”

</FilesMatch>

Adding rel=“canonical” to PDF files and headers

In 2011, Google announced support for rel=”canonical” associations with specific http headers to signal the canonical URLs of HTML documents, including PDF files. Beyond placing it in the top section of an HTML page, this is an alternative method of pointing to images and PDF files in HTML versions, such as pages with downloadable PDF documents.

When it comes to PDF files using HTML headers, you should use the code below to point the PDF file to the HTML page using the URL /page.html.

<Files “file.PDF”>

Header and Link “<htttp://www.site.com/page.html>; rel=”canonical””

</Files>

Advantages of the .htaccess file

• it is able to read all requests,
• immediate implementation of changes without the need to restart the server,
• effective access management for users according to preferences,
• sets configuration at directory level,
• a huge benefit for SEO specialists.

Disadvantages of the .htaccess file

• .htaccess files may increase the security risk to the website,
• it is slower than server-level configuration because the .htaccess file is searched for and read every time a page is loaded,
• it affects the speed of the site, which has a significant impact on traffic.

.htaccess files are not recommended as a method of server configuration due to security and performance issues.

Summary

Almost all Apache servers have a pre-defined configuration file. However, it applies to the entire site, and it is difficult to make configuration at directory level.

In such cases, the .htaccess file is considered a blessing. It allows you to configure at directory and subdirectory level, overriding Apache configuration settings.

Additionally, you can use simple configuration codes to set up authentication. This is very useful if you share hosting with multiple sites.