Search engines perform a marvelous feat. Based on a text query, they produce a list of (usually) relevant results from the billions of pages on the World Wide Web. Moreover, they do this at lightning speed, with search results appearing as your type.
Crawling and Indexing
Search engines do not achieve this by searching the web, but a massive database of the web’s content called the search engine’s ‘index’. This database holds information about the content of millions of websites (e.g., text, images, videos) and their links. The search engines use automated programs called robots (a.k.a. spiders, bots or crawlers) to investigate new websites and record any changes which have occurred to pages already in their index. This process is known as crawling. New content will be discovered by following links.
Ranking
When someone performs a search, search engines will interrogate their index for the most relevant content for that query and present that to the user. The ordering of results by relevance is referred to as ranking. A search engine’s method of indexing the web and producing a list of results for a given query is referred to as its ‘algorithm’ and is a closely guarded secret. Algorithms are complex and look at multiple factors when deciding on the ranking for each query. Industry experts now believe that Google uses over 200 ranking factors when compiling results. Ranking factors include:
Quality and Quantity of Links
Using link relationships was Google’s great insight that enabled it to build a much better search engine than the competition. Instead of just looking at the content of a site, Google considered links. A link was treated liked a vote, with content with more incoming links ranking higher.
Links are not created equal, and links from more established sites will improve ranking more. All other things being equal, the more natural backlinks you have from trusted, high-authority websites, the better your chances of ranking higher within search results.
Content Relevancy
Your page’s rank for a query is decided by how well your page’s content matches Google’s perception of the searcher’s intent. The process of ‘On-Page Optimisation’ involves researching the best keywords to target your site pages and creating content that utilises these keywords.
Performance Metrics
Google is constantly revising its search results in light performance data. It uses performance metrics such as click-through rate and time on site and machine learning techniques to tweak its search results in real-time to produce more relevant results.
For example, if a lower result is getting more clicks than the higher result, it will judge that it is more relevant and move that result up the list to respond to user behaviour.
