Why Search Engine Optimisation (SEO)

What is SEO?

Search Engine Optimisation is the process of configuring a website to appear highly in the natural or free search engine results for searches relevant to your business. These are the results to the left of the Google results page generated by the Google search algorithm. Google takes no payment for positioning in these results, and rank is based purely on Google’s assessment of a page’s relevance to a query.

A website will need to appear at or near the top of the first page of the search results to get significant traffic. Obtaining a high search result for a business is hugely valuable as it has the potential to dramatically increase website traffic and, therefore, sales and brand awareness.

Google does not publish its algorithm, and techniques for improving website performance are a matter of much speculation. This lack of clarity has led many charlatans to sell expensive, ineffective solutions for improving rankings to unsuspecting business.

Benefits of SEO

SEO is a vital part of any effective online marketing strategy. Organic search results are more trusted by consumers and receive more clicks than ads. Quality content that ranks well for important keywords will be a gift that keeps on giving, while advertising needs constant funding to send traffic to your site.

In summary, the benefits of SEO are:

  • Primary source of traffic. The Google natural search results are the largest source of traffic in every sector (source: Growth Badger)
  • Cost-effective. If managed effectively, SEO is a cost-effective method of traffic generation. The traffic is ‘free’, though there may be costs involved in generating traffic and getting the right advice.
  • Targets researching consumers. SEO gets your business get in front of your target audience as they are actively searching for information. This is the preferred first port of call for 62% of consumers (source: Hubspot).
  • Branding. Visibility on searches related to your business can have a positive impact your brand. Your business can become associated with and trusted by searchers asking questions as they conduct research leading to a purchase.
  • Sustainability. Unlike paid search, organic traffic does cease the moment you stop paying.

Search Engine Marketing: Natural/Organic Search Engine Optimisation (SEO) vs Paid Search

Search Engine Marketing:  Search Engine Optimisation and Paid Search

The opportunities for businesses to promote their businesses via search engines are Search Engine Optimisation (SEO) and Paid Search.

Search Engine Optimisation

Search Engine Optimisation is the process of configuring a website to appear higher in the natural or free search engine results for searches relevant to your business. These are the results to the left of the Google results page generated by the Google search algorithm. Google takes no payment for positioning in these results, and rank is based purely on Google’s assessment of a page’s relevance to a query.

A website will need to appear at or near the top of the first page of the search results to get significant traffic. Obtaining a high search result for a business is hugely valuable as it has the potential to dramatically increase website traffic and, therefore, sales and brand awareness.

Google does not publish its algorithm, and techniques for improving website performance are a matter of much speculation. This lack of clarity has led many charlatans to sell expensive, ineffective solutions for improving rankings to unsuspecting businesses.

While only a few people know exactly how Google and the other engines produce their results, the influencing factors are broadly accepted. These techniques can dramatically affect traffic (see the section on ranking factors below).

Search Engine Optimisation is often overlooked in favour of Paid Search as increasing natural search engine ranking is a lengthy and uncertain process. However, overall, it has the potential to be one of the most cost-effective and important sales channels for your business.

Paid Search 

Paid Search refers to the adverts which appear to the top, bottom and right-hand side of the natural search results. Paid Search programs, such as Google Ads, have the potential to provide highly targeted, measurable and cost-effective traffic to your website. Typically, the adverts are charged on a pay-per-click basis, meaning that the advertiser only pays for traffic driven to their site.

To place an ad, an advertiser specifies a list of targets (traditionally keywords but these days also products and webpages) for which they want the advert to appear, the advert’s text and the price they are happy to pay for a click. When a user enters a search term, Google displays a list of adverts next to the natural search results, triggered by the user’s search term. The adverts’ order is decided by the cost per click the advertiser is willing to pay and the advert’s past performance. This is known as the quality score.

Unlike Search Engine Optimisation, Paid Search campaigns can be set up quickly and generate sales and traffic within hours. Furthermore, after a few days, there will be enough data to tweak the adverts to improve performance. We will consider Paid Search in detail in the next chapter.

SEO Vs Paid Search

Research has shown that whilst users tend to click on the natural search results about two-thirds of the time, companies tend to spend two-thirds of their search engine marketing budget on Paid Search (Source: Moz). This is for three reasons:

  • Speed of results. Paid search campaigns can generate traffic quickly, while natural search campaigns can take months to show any results.
  • Transparency. Whilst natural search traffic is at the grace of the search engine’s algorithms, with Paid Search, the system is transparent with a direct relationship between spend and traffic.
  • Tracking. With Paid Search, it is easier to calculate the return on investment of the advertising spend as it is possible to track sales from when someone clicks on an advert through to checkout completion. This allows return on investment and cost per conversion to be calculated with great accuracy.

I would encourage companies not to forgo investing in Search Engine Optimisation in favour of Paid Search as, in the long term, it can provide an excellent return on investment. Search Engine Optimisation is cost-effective as the search engines do not charge for inclusion in their listings. Costs associated with natural search are due to the cost of the initial optimisation and ongoing optimisation costs. However, investment in SEO should be a gift that keeps on giving as performance will improve over time. Furthermore, these costs are fixed, while Paid Search is a variable cost.

The following table summarises the differences between paid and natural search.

 Natural SearchPaid Search
CostFixed: Cost of optimisation. No charge for listing.Variable: Charged per click
Timescale for resultsMonthsImmediate
Control of traffic volumeUnpredictable – depends on search engine algorithmDependable – traffic volume depends on budget
Technical requirementsYes – Require changes to website and site architecture.None – No changes to site needed
FlexibilityInflexible – Any changes made to a page will take several days for the search engines to registerFlexible – changes to a campaign appear at once
Return on investmentDifficult to measureEasy to measure

Search Engine Optimisation and Paid Search can also be helpful for different things. SEO is good for gaining traffic from generic terms that are often searched and can generate a high traffic volume, e.g., ‘convertible car’. These key phrases tend to be unsuitable for Paid Search as they have a high cost per click and do not often convert into sales due to their general nature.

Paid Search can be effective for targeting many low volume keywords. Popular searches like ‘car’, known as the search head, get a significant traffic proportion. However, the sum of all the less frequent but more specific searches such as ‘BMW 3 series convertible’ make up a similar or greater volume. This is known as the ‘Long tail’.

Keyphrases in the long tail are likely to have a lower cost per click and have a higher conversion rate as they are more specific. (compare ‘car’ with ‘2006 MW 3 series’).

It is impractical to optimise a website for the wide variety of key phrases that customers might use when looking for your products or services. However, with Paid Search, there is no (practical) limit on the number of keywords that can be targeted. Using Paid Search to target the search tail can be a very economical way of driving targeted traffic to your site.

Off-Page SEO – Building Website Authority

Google’s great insight that allowed them to build a better search engine was the value of links as a mechanism for differentiating between sites. Whereas earlier search engines just looked at the text on a page, Google considered the relationships between sites. Each link was considered a vote, and sites with many incoming links (often referred to as backlinks) were given higher search rankings.

Whilst Google has changed a lot since its early days, links are still among the most important ranking factors for SEO. However, whilst in the past, getting a lot of low-quality links or buying links would work as a strategy, these days, it is essential only to get high-quality links relevant to your business. Avoid inferior quality links as they will not help a site rank better and harm your natural search performance. Best practices for link building are as follows:

  • Build relevant, quality links.
  • Create great content which people want to link to
  • Aim for a range of links from various sources with different anchor text.
  • Do not buy links.
  • Avoid low-quality sites.

Link Building Strategies

Building links is more complex than it used to be, and as many tactics which worked well in the past, e.g., directory submission or blog comments, no longer carry any weight. Focus instead on building links organically from high-quality sources.

Create High-Quality Content

Quality content provides more content for the search engines to index and provides ‘linkbait’. Sites with excellent quality content will naturally receive links from other sites whose owners believe the linked content is a useful resource for their visitors. Quality content can take the format of text, applications and resources such as how-to guides.

Links from partners

Other websites will be happy to link to high-quality sites which share similar or complementary content. Your strategy here is not to get thousands of random links but high-quality links from relevant sites.

Sources of links can include:

  • Business partners. Ask your business partners to recommend your services by placing a link to you on their site.
  • Suppliers. Many suppliers will have a list of resellers on their sites.
  • Quality directories. There are often specific local business directories that promote nearby businesses.

An excellent source of links is websites that link to your competitors. There is a free backlink checked on Neil Patel’s site.

Online PR

Generating stories about your business is an excellent way of getting excellent quality links to your site. One way to create media interest is to submit press releases to relevant media websites. Media websites tend to be well indexed and highly ranked by search engines as they contain a large volume of quality content that attracts links and is often updated. The following types of media sites can be targeted when distributing new stories:

  • Blogs. Many bloggers are hugely influential in their fields and may be interested in writing about your product and services.
  • Press release sites. Websites that accept and distribute press releases. Some of these sites are free and others charge a small fee.
  • Online magazines. Online magazine and newspapers often write stories after direct approaches and press releases.

Technical SEO – Javascript, Scheme, Usability and Mobile

Google aims to serve its users with high-quality sites which quickly answer their query. Consequently, it has started to consider a range of factors related to the site’s performance and technical architecture. Since the technical structure can have a massive impact on its performance, it is essential to understand the factors that can affect performance.

JavaScript

Most websites use JavaScript to add interactivity and improve the user experience. It is commonly used in menus as well as pulling in products or prices. The dynamic nature of JavaScript, however, has historically caused issues with search engine crawling. If Google cannot access the menus on a site, it may have difficulties reading the content.

Google has stated that, if you are not blocking the Googlebot from crawling your JavaScript files, they can read JavaScript as a browser can. However, it can take Google longer to index JavaScript content. It is best to check that Google can read your pages correctly. This can be done from within Google Search Console using their URL Inspection Tool.

Schema

Schema is a way to label or organise your content so that search engines can better understand what some aspects of your web pages are. Schema provides structure to your data, which is why it is often referred to as ‘structured data’. The process of structuring data is often referred to as ‘markup’ because you are marking up your content with organisational code.

JSON-LD is Google’s preferred schema markup which Bing also supports. To view a complete list of the thousands of available schema markups, visit Schema.org. Besides helping bots like Google understand what a particular piece of content is about, schema markup can also enable rich snippets (mentioned above) to accompany your pages in the SERPs.

Page Load Speed and Usability

As well as being vital factors in your site’s conversion rate, page speed and website usability are also ranking factors considered by Google. These factors are combined into Google’s Core Web Vitals measures which available in the Google search console.

To benchmark your site’s performance, use Google’s PageSpeed Insights tool and measure your page speed and identify improvement opportunities.

Mobile Friendliness

In 2020 mobile internet usage exceeded desktop usage for the first time. In response to this, as of March 2021, Google will stop indexing desktop versions of a website and only index mobile-friendly sites. Making your website compatible with mobile screens is necessary for maintaining search performance and is good for users who access sites via mobiles. To ensure a good mobile experience, Website owners should implement the following:

  • Responsive design. Responsive websites are designed to adjust to the size of the user’s screen.

AMP. Accelerated Mobile Pages (AMP) are used to deliver content to mobile visitors at greater speeds

Search Engine Optimisation (SEO): Website Crawling

As we have discussed previously, producing search engine results involves several distinct processes, and many factors must be considered to build and maintain a high performing website.

  • Crawling. Can search engines find your pages and read their contents?
  • On-Page SEO. Creating compelling content which answers searcher’s queries and ranks highly for relevant keyphrases
  • Technical SEO. Building your site in a way that maximises natural search performance.
  • Link building and establishing authority. Also known as off-page SEO. Building the authority of your site by linking building and online PR.

Crawling: Can Search Engines Find Your Pages?

Before your website can appear in the search engine results, it must appear in the search engine’s index. The first task of optimising your site should be to establish if any issues prevent your site from being fully indexed. Many sites are constructed in a way that prevents search engines from indexing some or all the pages on the site.

Analysing Current Performance

It is essential to establish the proportion of your site’s pages that are indexed. Do this by comparing the number of pages on your website (as measured by your content management system) with the number appearing in the search engines index. To find the number of pages from your website which Google has indexed, consult the Google Search Console (see below). To get an estimate to the proportion of indexed pages, divide the number of indexed pages by the total number of pages.

It takes time for a search engine to refresh its index, and so if your site is large or often changes, 100% inclusion is unlikely. However, if the number of pages indexed is way below the expect this should be investigated (see below). If your site is not appearing at all, then there could be a few reasons for this:

  • New site that has not yet been crawled.
  • No links from external websites.
  • Site navigation makes it difficult for a robot to crawl it.
  • ‘Crawler directives’ are blocking search engines. E.g., robots.txt file
  • Site penalised by Google for ‘Black Hat’ tactics

Google Search Console and Google Sitemaps

Google has created a suite of tools called Google Search Console (GSC) with the mission of ‘helping you measure your site’s search traffic and performance, fix issues and make your site shine in Google Search results. These tools supplied include:

  • Coverage reports. See the number of pages that Google is indexing.
  • Search performance. Natural search clicks and performance as measure by Google.
  • Website analysis. Google’s analysis of the quality of your website, e.g., usability and page speed.
  • Google sitemaps. When submitted to Google, an XML file will provide them with details of the pages on your site.

Submitting a Google sitemap is crucial to improve your indexing and be a priority soon after a site launch. These will be generated automatically from all decent CMS and eCommerce systems and can be quickly submitted to GSC.

Indexing problems

If your site is not being indexed or has a low percentage of indexed pages, one of the following problems may be present:

Robots.txt file

There are some good reasons why you would not want a search engine to index sections of your site, for example, admin and checkout pages and duplicate content.

Robots.txt files can be found in the root directory of each website (e.g., vendlab.com/robots.txt) and tells Google which parts of your site it should and should not crawl. A problem with the robots.txt may cause Google to stop crawling your site.

Inaccessible Content

Your content could be inaccessible to search engine crawlers for several reasons:

  • Search required. If the content is only accessible via a search, the content will not be accessible to the crawler.
  • Non-text content. Do not use non-text media formats (images, video, GIFs, etc.) to display text that you wish to be indexed.
  • Non-HTML menus. Search engines have difficulty reading menus not written using HTML, e.g., JavaScript.
  • Navigation dead ends. Crawlers only find pages linked to other pages.

Server Errors

When crawling the pages on your site, a crawler may encounter errors. Google Search Console’s ‘Crawl Errors’ report lists URLs on which there are server errors and not found errors.  A moved page will lose its ranking unless the site sets a ‘301 redirect’ to direct the page visitor to the new location. A redirect will ensure that the new page is indexed correctly and that its authority is maintained.

Black Hat and White Hat SEO

Search Engine Optimisation techniques divide into two types:  Black Hat and White Hat SEO.

Black Hat SEO

Black Hat SEO techniques try to trick search engines into thinking that a website is relevant for a particular search term by manipulating weakness in the search engine’s algorithm. One example of an old black hat technique is using white text on a white background to increase a particular term’s occurrence. Black Hat techniques depend on reverse-engineering the search engine algorithm and are therefore sensitive to algorithm changes and short term.

White Hat SEO

White Hat SEO techniques aim to build long term traffic to a website through quality content and relevant links from other highly ranked sites. The aim is to create a site optimised for search engines and provides a pleasant experience for users.

Whilst all of Search Engine Optimisation is an attempt at reverse engineering, White Hat SEO is not about ‘chasing the algorithm’ but building the prominence of a website whilst staying with the search engines guidelines set by Google and the other major search engines.

White Hat or Black Hat?

Whilst there is no denying that black hat techniques can generate traffic in the short term, they are hypersensitive to changes in the search algorithm. Furthermore, using techniques not approved by Google puts websites at tremendous risk of being penalised or de-indexed (removed from search results). This can be disastrous if your business depends on natural search traffic for a sizable proportion of your sales.

White hat techniques play by the rules to create a site that will receive traffic over a long period, building an asset for its owners. Google has published a list of excellent guidelines for optimising websites, and if you do what they say, you cannot go wrong!  Their recommendations include:

  • Make pages firstly for users, not for search engines.
  • Do not deceive your users.
  • Avoid tactics intended to boost search engine rankings. A helpful test is to ask, does this help my users? Would I still do this if search engines did not exist?
  • Consider what makes your website special. Make your website distinct from others in your field.

Interview with Jamie Riddell from Escaping Gravity

Jamie Riddell has been at the forefront of Search Engine Optimisation for over 20 years and his latest venture is Escaping Gravity, a Global SEO agency for Challenger brands.

SEO was one of the first things which got me excited about eCommerce, so it was great to discuss the latest developments with Jamie.

Amongst other things, we chatted about:

  • Over the last few years, paid search results are taking up more space on the SERPs.  Does that mean that natural search is less important for retailers?
  • I notice that Amazon seems to appear prominently in the SERPs for product-related searches.  Is it becoming more difficult for ‘challenger brands’ to make an impact?  Is it a case of the Matthew effect (‘For to everyone who has will more be given, and he will have abundance; but from him who has not, even what he has will be taken away.’)
  • Should SEO have a higher ROI than other (paid) forms of online promotion?
  • How important is getting excellent quality links vs creating great content?
  • Google is implementing core web metrics soon, how much effect do you think this will have on retailers, may of which have slow sites 
  • What has inspired you recently?

Interview with Kaitlyn Study of South Street and Co Marketing Agency Ecommerce Odyssey Podcast

In this week's edition of the eCommerce Odyssey Podcast, Trevor talks to Kaitlyn Study of South Street and Co, a marketing agency based in Orlando, Florida, USA.South Street & Co. was established in 2015 and helps small business owners reach more customers through great digital marketing, design and automation.Kaitlyn Study original set her business up as a side hustle, and it quickly grew into a full-time business.We discussed:How Kaitlyn started South Street and coThe importance of local searchMarketing automationThe platforms they useThe importance of coachingKaitlyn's advice for starting your side hustle!What has inspired her recentlyKaitlyn talked to Trevor Ginn from www.vendlab.com, an eCommerce agency specialising in online marketplaces and e-commerce sales platforms.
  1. Interview with Kaitlyn Study of South Street and Co Marketing Agency
  2. Interview with Juan Ignacio García Braschi, Founder of Boopos
  3. Interview with Issac Kuhlman from Real Coaching
  4. Interview with Lesley Hensell from Riverbend Consulting
  5. Interview with Simon Severino, CEO and founder of Strategy Sprints

How Search Engines Work

Search engines perform a marvelous feat. Based on a text query, they produce a list of (usually) relevant results from the billions of pages on the World Wide Web. Moreover, they do this at lightning speed, with search results appearing as your type.

Crawling and Indexing

Search engines do not achieve this by searching the web, but a massive database of the web’s content called the search engine’s ‘index’. This database holds information about the content of millions of websites (e.g., text, images, videos) and their links. The search engines use automated programs called robots (a.k.a. spiders, bots or crawlers) to investigate new websites and record any changes which have occurred to pages already in their index. This process is known as crawling. New content will be discovered by following links.

Ranking

When someone performs a search, search engines will interrogate their index for the most relevant content for that query and present that to the user. The ordering of results by relevance is referred to as ranking. A search engine’s method of indexing the web and producing a list of results for a given query is referred to as its ‘algorithm’ and is a closely guarded secret. Algorithms are complex and look at multiple factors when deciding on the ranking for each query. Industry experts now believe that Google uses over 200 ranking factors when compiling results. Ranking factors include:

Quality and Quantity of Links

Using link relationships was Google’s great insight that enabled it to build a much better search engine than the competition. Instead of just looking at the content of a site, Google considered links. A link was treated liked a vote, with content with more incoming links ranking higher. 

Links are not created equal, and links from more established sites will improve ranking more. All other things being equal, the more natural backlinks you have from trusted, high-authority websites, the better your chances of ranking higher within search results.

Content Relevancy

Your page’s rank for a query is decided by how well your page’s content matches Google’s perception of the searcher’s intent. The process of ‘On-Page Optimisation’ involves researching the best keywords to target your site pages and creating content that utilises these keywords.

Performance Metrics

Google is constantly revising its search results in light performance data. It uses performance metrics such as click-through rate and time on site and machine learning techniques to tweak its search results in real-time to produce more relevant results.

For example, if a lower result is getting more clicks than the higher result, it will judge that it is more relevant and move that result up the list to respond to user behaviour.

The Search Engine Results Page (SERPs)

Search Engine Results Pages (SERPs) are web pages displayed to users when they search online using a search engine, such as Google. The user enters their search query, and the search engine then presents them with a Search Engine Results Page (SERP).

Before optimising your website, you should understand how it might be displayed in the search results. Understanding the layout of a search result page can help you create content that encourages users to click your link. The SERPs now have dozens of features, but the most critical distinction is between ‘paid’ and ‘organic’ results and paid results.

The natural (or organic) search results are the listings of web pages generated by the search engine’s algorithm. The paid results are from advertisers bidding on keywords on Google Ads. Although Google Ads take ad relevancy into account, how much the advertiser is willing to pay is the most significant factor.

In the following figure, the results on the left are all organic:

On the right is the ‘knowledge graph’ (see below). A standard organic search result includes:

  • Page title (title tag)
  • Page URL
  • Meta description

It may also include specific organic snippets such as sitelinks.

Title

This is the title tag for the page which is defined in the page’s HTML between the <title></title> tags. The title is the first and often only part of your search result that is read. It should short and relevant while still giving enough information to provide the user with a good grasp of what the page is about. Keep it short as Google will trim title tags over 70 characters.

Sitelinks

Sitelinks are links that sometimes appear in the results below a listing’s URL and meta description. Each Sitelink links to a leaf page within the current website and has a title and description. Sitelinks are more likely to appear in searches suggesting clear brand intent (e.g., searches for domains or brand names).

You cannot directly control the appearance of Sitelinks. Google’s algorithm decides whether there are relevant Sitelinks on a website and whether to show them.

URL

Your URLs should be relevant and short whilst still supplying enough information to display what the page is about.

Snippet

The snippet is the page’s description and is limited to about 156 characters. While the snippet may be the page’s meta description, Google sometimes compiles the snippet from page content. This enables Google to customise the snippet to each search query.

Rich Snippets

Rich Snippets provide extra information to answer a query, such as a photo or a star rating. Adding structured data to your site can increase your chances of having a rich snippet displayed alongside your search results. A rich snippet looks like this:

This snippet contains a picture of the ice cream, a list of ingredients and other details.

SERP Features

SERP features can be paid, natural, or pulled directly from Google’s Knowledge Graph. Popular features include:

  • Featured snippets. These show a content snippet from a top-ranking web page. They appear at the top of the search results.
  • Knowledge card. This appears at the top of the results and offer a short, answer to a query.
  • Knowledge Panel. Panels provide information about the main topic of the query. They appear near the top of the results on mobile and the right-hand side on the desktop.
  • Image Pack. Image Packs show a number of thumbnails and clicking on them takes you to Google Images.
  • Top stories. Top stories carousels show recently published articles, live blogs, and videos.

Google maps

Google Maps provides local information for places around the world. Google maps data often appears in search results, especially for local queries. It includes information such as aerial maps, road maps, street view maps taken from Google vehicles.

Adding your business to Google My Business is free and especially important for companies with local customers such as physical shops. By creating a Google My Business profile, your business information will appear in relevant local searches, generating free traffic for your business.

Why Search Engine Optimisation is important

SEO is an vital part of any effective online marketing strategy. Organic search results are more trusted by consumers and receive a lot more clicks than ads. Quality content that ranks well for important keywords will be a gift that keeps on giving, while advertising needs constant funding to send traffic to your site.

In summary, the benefits of SEO are:

  • Primary source of traffic. The Google natural search results are the largest source of traffic in every sector (source: Growth Badger)
  • Cost-effective. If managed effectively, SEO is a cost-effective method of traffic generation. The traffic is ‘free’, though there may be costs involved in generating traffic and getting the right advice.
  • Targets researching consumers. SEO gets your business get in front of your target audience as they are actively searching for information. This is the preferred first port of call for 62% of consumers (source: Hubspot).
  • Branding. Visibility on searches related to your business can have a positive impact your brand. Your business can become associated with and trusted by searchers asking questions as they conduct research leading to a purchase.
  • Sustainability. Unlike paid search, organic traffic does cease the moment you stop paying.