Search Engines in the Old Days

Early search engines were not nearly as intricate and advanced as our present options, and although the first search engine is up for debate, Archie was probably the first. A shortened form of archives, Archie was developed at McGill University in Montreal. The idea behind these search engines was to help organize the few hundred or so web sites, predominantly run by universities and colleges, into useable data streams that could be categorized and searched. By using a script based data retriever, what we would now call spiders, in association with an expression matcher, Archie could be queried to find particular web filenames held in its database.

Sources of Early Search Engines and Gopher Sites:
• A brief overview of
Gopher sites and how they work.
• Telnet, FTP and
Gopher descriptions and definitions.
• Information on the
early search engines and what happened to them.
Search engine history simplified.
• A basic description of early
META search engines and their processes.

From Archie, several other search engines were developed as the World Wide Web was added to the internet. Such engines were called Veronica and Jughead and are some of the most well known of the early search engines. Archie, Veronica, and Jughead all predate the World Wide Web and predominantly indexed gopher sites. While these engines served the early internet well, when the World Wide Web exploded in the early nineteen nineties a new engine became the cornerstone of the searchable internet, called Excite.

Excite utilized keywords and URLs as its method of indexing sites and pages, and was the forerunner of indexing used today by search engines like Google. Excite used titles and the keywords within the titles to rank pages on their content. Words found within the title and the frequency that they appeared within the content allowed for higher page ranks. In addition, the shorter the URL, the higher the page rank for any indexed page. Excite was slower at indexing than today's crawlers and search engines--it took an average rate of two months to index new sites. Excite utilized a description web crawler and took no note of the META tags in a site; this has changed and Excite now indexes META tags as well.

Excite and Magellan utilized the same back end programming, but different interfaces and designs for the search results. Magellan was acquired by Excite in nineteen ninety-six, followed by the acquisition of WebCrawler. Soon Magellan fell to the wayside and support for it stopped. All of the search results found with Magellan were the same as Excite's. During this same time, Lycos and Yahoo entered the scene with their search engines and have survived the search engine industry to this day, but with less usage for Lycos. Today the World Wide Web search engine and searches are predominantly handled by Google and Yahoo, with Ask and Live Search as close seconds.