Fundamental Differences Between Search Engines And Web Directories

"Search engine" and "Web directory" are two different search services available to the Web community; although they are often mistakenly confused. Search engines have indices that are built up by robots or crawlers; whereas Web directories build up their indices through human editors. Many search engines and directories contain both a computer-generated index and a human generated index, and are referred to as hybrids.
Google, Inktomi, AltaVista, AlltheWeb and the like are all forms of search engines. These search engines write programs known as robots, crawlers and/or spiders that have the following functions: (1) to locate Web pages, (2) to read the contents of the Web pages and (3) report its findings back to the search engine's indices or databases. Many search engines update their index either on a bi-monthly or monthly basis. When Web searchers use a search engine to locate Web sites that are relevant to the keyword search, they are searching the search engine's index. A search engine with a larger and more up-to-date index is a better representation of the information available in the Web.
Yahoo!, Open Directory Project (dmoz.org), Gipsy and the like are all forms of Web directories. These directories use human editors to review sites that are submitted for submission to the directory. Directories, unlike search engines, use a hierarchical tree structure to organize their database. Another common distinction is that a directory tends to list Web sites (root directory of a site or homepage) whereas a search engine will list Web pages (individual pages of a Web site). Due to the manual process of adding sites to a directory, directories often have to supplement their search results with a search engine partner to increase the relevancy of the produced search results.
Now that we have a better overall understanding of the common differences between search engines and Web directories, we will discuss the details about search engines and how they work. Search engines all have their own confidential algorithms that determine which Web pages are to be shown first. The algorithms assign weights to certain components or variables that it finds within a page.

For example, many search engines consider the text within the title of the page to be deemed very important. The title of a page is considered important to search engines and is given higher weight because (1) it is displayed on the top of the menu bar in your browser, (2) the title is displayed in the search engine results page and (3) the title is displayed in your browser bookmarks when you add that page to your "favorites" or bookmarks.

These weights provide the search engine's algorithms with a method to show one Web page over another. The specific variables and the suggested weights assigned to those variables are for a separate article. If you want users to find your site, you need to make sure that a search engine spider can access your site and read your code. Then you must ensure that the content is written well and contains targeted keyword specific language.
Web directories should be browsed through their hierarchical structure and not searched. Humans assign titles and descriptions that might not be within the source code of the page. Also, there is normally a submission fee to be added to a directory. The submission fee is to cover the costs of having the human editors review the site. Most directories will only add a new site if it has unique content that is presented in a professional manner by a legitimate company.
Most users tend to use search engines by typing in keywords into a search box. It is a quick and easy way to find specific information. I use search engines to locate technical information quickly, and Web directories to locate a listing of sites that offer similar services. For example, I was in need of an office sign for my company. I went to the Yahoo! directory and located the most appropriate category for sign manufacturers and contacted a handful for quotes.  

Author:
Barry Schwartz
RustyBrick, Inc.
  barry@rustybrick.com
  www.rustybrick.com
  845-369-6869 x201