A search engine is an online tool that helps people find information on the internet. A typical example? Google.
And the truth is, Google is also probably the only example you need to know. Just look at the chart of the worldwide search engine market share (data by Statcounter):
So when we talk about search engines in this guide, we mostly mean Google. Other search engines work on similar principles and as long as your website is optimized for Google, you should be all set up for others too.
Tip: Find out more about the most popular search engines in our SEOpedia article on this topic.
The process in which search engines work consists of these main steps:
…and finally, showing the search results to the user.
The process looks something like this:
Crawling is the process in which search engines scan all the internet webpages continuously.
They use small pieces of programs (called crawlers or bots) to follow all the hyperlinks and discover new pages (as well as updates to the pages they discovered before).
Martin Splitt, Google Webmaster Trend Analyst, describes the crawling process quite simply:
“We start somewhere with some URLs, and then basically follow links from thereon. So we are basically crawling our way through the internet (one) page by page, more or less.”
Once the website is crawled, the information is indexed. The search engines try to analyze and understand the pages, categorize them, and store them in the index.
The search engine index is basically a gigantic library of all the crawled websites with a single purpose – to understand them and have them available to be used as a search result.
Tip: If you want to find out whether your page was crawled and indexed, you can simply go to your Google Search Console (more on the tool in the last chapter) and use the URL Inspection Tool:
You’ll see when the page was last crawled as well as warnings about any potential crawling and indexing issues Google may have with your page.
Read more in our detailed post on Crawling & Indexing.
Once the internet user submits a search query, the search engine digs into the index and pulls out the best results. The list of the results is known as a SERP (Search Engines Results Page).
In the following paragraphs, we’ll take a closer look at Google’s Search Algorithm.
Google’s Search Algorithm is used as an umbrella term to refer to all the individual algorithms, machine learning systems and technologies Google uses to rank websites.
To provide the best results, they consider various factors, namely:
As with any other complex system, the Google algorithm needs to be updated and tweaked regularly.
Besides minor algorithm updates that happen on a daily basis, Google usually rolls a couple of core algorithm updates per year.
They are officially announced by Google and create a lot of buzz in the SEO community.
Later today, we are releasing a broad core algorithm update, as we do several times per year. It is called the December 2020 Core Update. Our guidance about such updates remains as we’ve covered before. Please see this blog post for more about that:https://t.co/e5ZQUAlt0G
— Google SearchLiaison (@searchliaison) December 3, 2020
Going through a list of the most important core algorithm updates (e.g. Panda, Penguin, Hummingbird,…) can be a great way to get a quick overview of how Google Search and SEO evolved over the years.
To learn more, read our detailed post on the Google algorithm.
Besides the algorithms, Google also uses human data input.
There are thousands of external Google employees called Search Quality Raters who follow strict guidelines (available to the public), evaluate actual search results and rate the quality of the ranked pages.
A typical example of pages that undergo this kind of strict evaluation are the so-called YMYL (Your Money, Your Life) pages – pages dealing with important topics that can impact someone’s happiness, health, safety or financial wellbeing.
Quality Raters don’t influence the rankings directly, but their data is used to improve the search algorithm.
Of course, search engines keep the exact calculations of their algorithms in secret. Nonetheless, many ranking factors are well-known.
Ranking factors are a very discussed topic in the world of SEO.
Many of them have been officially confirmed by Google but many remain in the realm of speculations and theories. From the practical point of view, it’s important to focus on factors that have a proven impact but also try to keep a “good score” across all the areas.
Further readingWhat are SEO ranking factors? (+8 confirmed factors explained)
Of course, not everything people think is a ranking factor is actually used by search engines (if something correlates with the higher rankings, it is not necessarily something Google uses in their algorithm). On the other hand, some confirmed ranking factors only have a very small impact on the rankings.
Cyrus Sheppard from Zyppy made a nice list of Google success factors (the ones that correlate with higher rankigs the most). Here are the 10 critical ones:
Note: Quality of content is undeniably the most important SEO factor (notice that 5 out of 10 critical factors are related to content). To learn more about content optimization for SEO, jump to the 4th chapter.
Other important factors that may have a positive impact on your rankings:
Director of SEO at Shopify, Kevin-Indig.com
There are many search engines, but only Google has made by far the most advances in Information Retrieval, Natural Language Understanding, and Natural Language Processing.
In the last 25 years, search engines moved from pure text-based evaluation to the machine learning age.
Today, Google iterates on user intent every month and is able to detect small nuances in the true desires of searchers: content quality, product offering, design, user experience. There are really no limits.
As a result, SEO has changed from optimizing fixed criteria to working toward optimal user experience. Smart SEOs understand that they have to go beyond backlinks and content. They have to understand the needs of searchers in the context of a keyword.