How Search Engines Work


Search engines consist of two main types.† They are crawler-based search engines and human powered directories.† The main differences is that crawler-based engines have automatic lists where as human powered ones require that information be submitted so websites are found that match close to the exact description.

Crawler-Based Search Engines

There are three basic elements of crawler-based engines.† The crawler is the element that goes to the web page and reads it and makes the connections to similar websites.† The crawler, also known as the spider, returns to the web pages regularly to look for any changes.† The second part of the search engine is the index.† The index includes everything that the spider finds which keeps it like a big organized book of web pages.† Lastly, there is the search engine software.† This software is very important because it goes through all the web pages and ranks them in order of importance for someone when they are trying to perform a search.† An example of a crawler-based search engine is HotBot.†

How a crawler-based search engine ranks websites

The search engines have to get the most relevant information filed out from the information that may not be as relevant.† Algorithm is an important set of rules followed by search engines. The general rules include:
        The location/frequency method depending on the keywords
        HTML pages are usually more important and useful
        Where the keywords appear on the web page
        How often keywords appear on a specific web page
        Meta tags are also used and sometime propel certain pages to the top/relevant positions
        Links to other pages are also considered

Tips to get crawler-based websites in the top rankings

        Pick keywords carefully by imagining what words you would think of if you were performing a specific search.
        Make sure your keywords appear on the web pages in the most important places such as titles or high up on the page.
        Make sure you have good references and reliable content
        Make a site map as well as good links within your pages and include hyper links that take a searcher to the main pages.
        Try to avoid symbols as part of the URL.
        Try to use meta tags.
        Build good links into your page.
        Continue to check on links so the page stays up to date.

Human Powered Directories

In using human powered directories, a person types in a description for the web sites you want to locate.† The directory only matches websites that directly relate to the description the person typed in originally.† Yahoo is one example of this kind of search engine.

Different Types of Search Engines

        Major Search Engines
        Specialty Search Engines
        Regional Search Engines
        News Search Engines
        Kids Search Engines
        Multimedia Search Engines
        Search Utilities

Major Search Engines

Major search engines are classified based on how well known they are or how often they are used.† These are usually more dependable for searchers also.†† Listed below are the eighteen major search engines and an interesting fact about most of them:
        AOL Search-does not list AOL only content
        Alta Vista-consists of the most indexed pages
        Ask Jeeves-aims to direct someone to the exact page that fulfills the needs
        Direct Hit-provides research to HotBot and MSN
        Fast Search-has large wireless web/mobile indexes
        Google-known for itís high relevancy
        HotBot-has many power searching features including Direct Hit
        Iwon-backed by CBS television network
        Inktomi-no way to search this engine directly; only using itís partners will get you here
        LookSmart-websites compiled by humans
        MSN Search
        Netscape Search-lists many ďofficialĒ websites
        Northern Light-able to cluster documents within specific topics
        Open Directory-
        Real Names
        Yahoo-oldest and most popular search engine as well as the largest human-compiled guide as far as employers used for categorizing various web sites/pages

Recent Tragedy uses Search Engines

The terrorist attacks on September 11 resulted in people all over the country using the Internet and search engines to gain information on what was going on, missing people, etc.† The top ten searches according to google after the attacks were CNN, World Trade Center, BBC, Pentagon, MSNBC, Osama Bin Laden, Nostradamus, American airlines, FBI, and Barbara Olson.† People started their own personal websites to try and keep records of people missing or found so that family members could be informed.† Big databases were compiled with names of thousands of people that had been found dead or alive, or people that were still missing.† Itís amazing to me how helpful search engines are, especially in a time of national tragedy.


Search engines are very useful to almost anyone that uses a computer.† They are used to locate sites that may be pertinent and of most relevancies to what information someone is trying to find.† A reader might want to use the websites to get research on a particular thing, learn about something new, or look up anything that may be questionable.†† They are very common and the majority of computer users know about engines such as AOL and Yahoo.† Search engines will continue to grow and become even more helpful in years to come.

Related Links

Search Engine Watchís General information on Search Engines

Search Engines & Terrorist Attacks: The Use of Google on September 11th

Listings of Various Search Engines