How do search engines work?

How do search engines work?

To understand what SEO, short for ‘search engine optimization’, is, we have to take a look at what search engines do first. In this video, we are going to talk about how search engines get the information they need to make their lists of results. So, let’s take a look at some of the search engines. Of course, there’s Google.

Google is the market leader in search engines, with a reported global market share of 90% in 2019. There’s also Bing, which is mostly popular in the States. About 6% of American searchers choose Bing. Also, about 4% of them uses Yahoo.

Russian searchers are much more likely to use Yandex, which accounts for 45% of Russian search traffic. And Baidu takes the lead in China, with a 65% market share. Some users might use a more privacy-oriented search engine, such as DuckDuckGo, or Startpage. And these are just a pick of the bunch.

Research your audience to discover which search engine they might be inclined to use. Of course, in a lot of countries, Google is the default choice of search engine. But, this might not be the case everywhere. So, how do these search engines get their results?

Well, search engines consist of three parts: 1. the crawler, which is also called a spider or a bot; 2. the index; and 3. the algorithm.

The process begins with a spider.

– In this example, we’ll call it a spider, because that’s the easiest image to remember – A spider travels the web through links. Links connect pages to each other. The spider then looks for data, kind of like a field researcher who sails out to travel the world looking for knowledge. It particularly looks for content, headings and links, which contain information about what a page is about, and how it’s structured.

The spider goes around the internet 24/7. When it passes through a website, it saves the HTML version of a page in an enormous database, and this database is called the index. The spider also finds and indexes news items, images, and videos for the search engine to embed in the search results. The index is updated every time the spider comes around again and finds new information.

In Google’s case, the spider revisits your site more or less often, depending on how often you change things and how important Google thinks your site is. So now the contents of the page have been put into the index! But how does that translate to ranking? Search engines use an algorithm for this.

The algorithm takes the data from the index and makes a calculation on a bunch of factors that predict whether a result will be useful for the searcher. Then it uses this calculation to see which order of pages the search engine results page should show. The search engine results page, or SERP, is simply the webpage where the results are shown. Like when you use Google to find out more about delicious popcorn.

We know the most important elements the algorithm takes into account because of our experience with SEO, and researching, and reading about what Google says about their algorithm. For example, we know that site speed, great content and the security of sites are important, amongst many others. But Google makes minor changes to the importance of all these different factors almost every day. They also release bigger updates every few months.

These changes depend on what Google thinks best serves their users. We’ll tell you more about what the algorithm looks for in a later video. Also, you can learn more about some of the big algorithm updates Google released in the past in our reading materials. But how do you get your website indexed to be shown by the search engine?

Well, spiders need to land on your page from another link. If there is no link, the spiders won’t know your site exists. This means that if you launch a new website, it won’t be spidered immediately. First, an indexed site needs to establish a link to one of your pages.

You can think of the indexed sites as a network like this. The spiders need a road to the new site in order to reach and index it. There! Now it’s part of the system!

But then, is everything a done deal after your site is indexed for the first time? No! Spiders follow all the links on the web that they have access to continuously. This way they can find new sites to index and change outdated information.

Like we maintain our websites to stay relevant, Google never stops reevaluating their indexing database and algorithm. While we’re talking about links, both internal links – which are links from one part of your website to another part of your website – and external links – which are links from a different website to yours – have a tremendous effect on search engine rankings. This effect can be positive or negative, depending on how much sense it makes. For example, a page about baking delicious cakes linking to a page about cement does not make any sense.

Linking to a page about different flavors of cake frosting matches the subject. In conclusion, these are our key takeaways for this video: 1. Search engines consist of crawlers – which are also called spiders or bots – and an index, and an algorithm. 2.

Spiders follow links to collect info about websites, which they put in an index. Search engines then use an algorithm to turn the index contents into search results. And 3. A spider needs a link from another website to yours to be able to find it.

Credit: Yoast Academy