The success of the Google search engine was mainly due to its powerful PageRank algorithm and its simple, easy-to-use interface.

A search engine is a program designed to help find information stored on a computer system such as the World Wide Web, or a personal computer. The search engine allows one to ask for content meeting specific criteria (typically those containing a given word or phrase) and retrieving a list of references that match those criteria. Search engines use regularly updated indexes to operate quickly and efficiently. Google being used to search for Wikipedia This is a screenshot of a copyrighted website, video game graphic, computer program graphic, television broadcast, or film. ... Google being used to search for Wikipedia This is a screenshot of a copyrighted website, video game graphic, computer program graphic, television broadcast, or film. ... Googles main pages unusually spartan design, uncluttered appearance and quick loading time have contributed greatly to the sites mass appeal. ... PageRank is a family of algorithms for assigning numerical weightings to hyperlinked documents (or web pages) indexed by a search engine. ... A computer program (often simply called a program) is an example of computer software that prescribes the actions (computations) that are to be carried out by a computer. ... A computer system consists of a set of hardware and software which processes data in a meaningful way. ... Graphic representation of the world wide web around Wikipedia The World Wide Web (WWW, or simply Web) is an information space in which the items of interest, referred to as resources, are identified by global identifiers called Uniform Resource Identifiers (URI). ... A phrase is a group of words that functions as a single unit in the syntax of a sentence. ...

Without further qualificiation, search engine usually refers to a Web search engine, which searches for information on the public Web. Other kinds of search engine are enterprise search engines, which search on intranets and personal search engines, which search individual personal computers. An intranet is a local area network (LAN) used internally in an organization to facilitate communication and access to information that is sometimes access restricted. ...

Some search engines also mine data available in newsgroups, large databases, or open directories like DMOZ.org. Unlike Web directories, which are maintained by human editors, search engines operate algorithmically. The Open Directory Project (ODP), also known as DMoz (for Directory. ... A web directory is a directory on the World Wide Web that specializes in linking to other web sites and categorizing those links. ...

History

The first Web search engine was "Wandex", a now-defunct index collected by the World Wide Web Wanderer, a web crawler developed by Matthew Gray at MIT in 1993. Another very early search engine, Aliweb, also appeared in 1993 and still runs today. One of the first engines to later become a major commercial endeavor was Lycos, which started at Carnegie Mellon University as a research project in 1994. Also referred to as just the Wanderer, this was a perl based web crawler that was first deployed in June, 1993 to measure the size of the World Wide Web. ... See WebCrawler for the specific search engine of that name. ... The Massachusetts Institute of Technology, or MIT, is a research institution and university located in the city of Cambridge, Massachusetts directly across the Charles River from Bostons Back Bay district. ... 1993 is a common year starting on Friday of the Gregorian calendar and marked the Beginning of the International Decade to Combat Racism and Racial Discrimination (1993-2003). ... Aliweb is a Web search engine that was the first for the web. ... Lycos is an Internet search engine and web directory. ... Carnegie Mellon University is a private research university located in Pittsburgh, Pennsylvania. ... 1994 was a common year starting on Saturday of the Gregorian calendar, and was designated the International year of the Family. ...

Soon after, many search engines appeared and vied for popularity. These included WebCrawler, Hotbot, Excite, Infoseek, Inktomi, Open Text, Northern Light, and AltaVista. In some ways they competed with popular directories such as Yahoo!. Later, the directories integrated or added on search engine technology for greater functionality. A person, action, decision or thing becomes popular if many people like or use that entity/object. ... See WebCrawler for the specific search engine of that name. ... Hotbot Hotbot was one of the early Internet search engines and was launched in May 1996 as a service of Wired Magazine. ... Excite Excite is an Internet portal with an included search engine. ... Infoseek was a search engine founded in 1994 by Steve Kirsch. ... Inktomi was created by Eric Brewer, who was an assistant professors of computer science at University College of Berkeley and Paul Gauthier, who was a graduate student of the same college. ... Open Text Corporation is a Canadian high-tech company based in Waterloo, Ontario, Canada. ... AltaVista home page, 2004 The name AltaVista refers both to an Internet search engine company and to that companys search engine product. ... ...

In 2002, Yahoo! acquired Inktomi and in 2003, Yahoo! acquired Overture, which owned AlltheWeb and Altavista. In 2004, Yahoo! launched its own search engine based on the combined technologies of its acquisitions and providing a service that gave pre-eminence to the Web search engine over the directory. Inktomi was created by Eric Brewer, who was an assistant professors of computer science at University College of Berkeley and Paul Gauthier, who was a graduate student of the same college. ... Screenshot of AlltheWeb AlltheWeb is a major search engine. ... AltaVista home page, 2004 The name AltaVista refers both to an Internet search engine company and to that companys search engine product. ...

Search engines were also known as some of the brightest stars in the Internet investing frenzy that occurred in the late 1990s. Several companies entered the market spectacularly, recording record gains during their initial public offerings. Some have taken down their public search engine, and are marketing Enterprise-only editions, such as Northern Light (http://www.northernlight.com/) which used to be part of the 8 or 9 early search engines after Lycos came out. 1990 is a common year starting on Monday of the Gregorian calendar. ... In financial markets, an initial public offering (IPO) is the first sale of a companys common shares to public investors. ...

Before the advent of the Web, there were search engines for other protocols or uses, such as the Archie search engine for anonymous FTP sites and the Veronica search engine for the Gopher protocol. In computing, a protocol is a convention or standard that controls or enables the connection, communication, and data transfer between two computing endpoints. ... Archie is a search engine designed to index FTP archives, allowing people to find specific files. ... Anonymous FTP is a FTP server, which does not require user authorization. ... Veronica is a search engine system for the Gopher protocol, developed in 1992 by Steven Foster and Fred Barrie at the University of Nevada. ... Gopher is a distributed document search and retrieval network protocol designed for the Internet. ...

Osmar R. Zaïane's From Resource Discovery to Knowledge Discovery on the Internet (http://citeseer.ist.psu.edu/117999.html) details the history of search engine technology prior to the emergence of Google.

Recent additions to the list of search engines include a9.com, AlltheWeb, Ask Jeeves, Clusty, Gigablast, Ez2Find, Laplounge (http://www.laplounge.com/), Teoma, WiseNut, GoHook, Walhello, Kartoo, Snap, and Mamma . Screenshot of the Beta A9. ... Screenshot of AlltheWeb AlltheWeb is a major search engine. ... ask. ... Categories: Computer stubs ... Gigablast Screenshot Gigablast is a relatively recent search engine. ... Teoma. ... Screen shot of WiseNut WiseNut is a crawler-based search engine now owned by Looksmart. ... GoHook is a crawler-based search engine. ... Walhello is a spider based search engine developed in the Netherlands for the Internet. ... A next generation search engine which shows results in visual interface. ... A search engine which shows results with a logo of the top level domain & information about popularity of that web site on SNAP. SNAP Search Engine External links SNAP Search Engine ...

Google

Around 2001, the Google search engine rose to prominence. Its success was based in part on the concept of link popularity and PageRank. How many other web sites and web pages link to a given page is taken into consideration with PageRank, on the premise that good or desirable pages are linked to more than others. The PageRank of linking pages and the number of links on these pages contribute to the PageRank of the linked page. This makes it possible for Google to order its results by how many web sites link to each found page. Google's minimalist user interface was very popular with users, and has since spawned a number of imitators. Googles main pages unusually spartan design, uncluttered appearance and quick loading time have contributed greatly to the sites mass appeal. ... Link popularity (link pop) is a measure of the quantity and quality of other web sites that link to a specific site on the World Wide Web. ... PageRank is a family of algorithms for assigning numerical weightings to hyperlinked documents (or web pages) indexed by a search engine. ... The user interface is the part of a system exposed to users. ...

Google and most other web engines utilize not only PageRank but more than 150 criteria to determine relevancy. The algorithm "remembers" where it has been and indexes the number of cross-links and relates these into groupings. PageRank is based on citation analysis that was developed in the 1950s by Dr. Eugene Garfield at the University of Pennsylvania. Google's founders cite Garfield's work in their original paper. In this way virtual communities of webpages are found. Teoma's search technology uses a communities approach in its ranking algorithm. NEC Research Institute has worked on similar technology. Web link analysis was first developed by Dr. Jon Kleinberg and his team while working on the CLEVER project at IBM's Almaden research lab. Categories: Chemistry stubs | Chemistry ... The University of Pennsylvania (commonly referred to as Penn or UPenn, although the former is the preferred and recognized nickname of the University) is a private university in Philadelphia, Pennsylvania and a member of the Ivy League. ... A virtual community is a group whose members are connected by means of information technologies, typically the Internet. ... NEC Corporation is a multi-national information technologies company headquarterd in Minato-ku, Tokyo, Japan. ...

Challenges faced by search engines

• The web is growing much faster than any present-technology search engine can possibly index (see distributed web crawling).
• Many web pages are updated frequently, which forces the search engine to revisit them periodically.
• The queries one can make are currently limited to searching for key words, which may result in many false positives.
• Dynamically generated sites may be slow or difficult to index, or may result in excessive results from a single site.
• Many dynamically generated sites are not indexable by search engines; this phenomenon is known as the invisible web.
• Some search engines do not order the results by relevance, but rather according to how much money the sites have paid them.
• Some sites use tricks to manipulate the search engine to display them as the first result returned for some keywords. This can lead to some search results being polluted, with more relevant links being pushed down in the result list.

Distributed web crawling is a distributed computing technique whereby Internet search engines employ many computers to index the Internet via web crawling. ... Searching for keywords is used as the basic functionality to search in large collections of documents, including the web. ... A false positive, also called false alarm, exists when a test reports, incorrectly, that it has found a signal where none exists in reality. ... The deep web (or invisible web or hidden web) is the name given to the publicly accessible pages on the World Wide Web that are not indexed by search engines. ...

How search engines work

Web search engines work by storing information about a large number of web pages, which they retrieve from the WWW itself. These pages are retrieved by a web crawler (sometimes also known as a spider) — an automated web browser which follows every link it sees. The contents of each page are then analyzed to determine how it should be indexed (for example, words are extracted from the titles, headings, or special fields called meta tags). Data about web pages is stored in an index database for use in later queries. Some search engines, such as Google, store all or part of the source page (referred to as a cache) as well as information about the web pages, whereas some store every word of every page it finds, such as Altavista. This cached page always holds the actual search text since it is the one that was actually indexed, so it can be very useful when the content of the current page has been updated and the search terms are no longer in it. This problem might be considered to be a mild form of linkrot, and Google's handling of it increases usability by satisfying user expectations that the search terms will be on the returned web page. This satisfies the principle of least astonishment since the user normally expects the search terms to be on the returned pages. This relevance to the search makes these cached pages very useful, even beyond the fact that they may contain data that may no longer be available elsewhere. A webpage or Web page is a page or file of the World Wide Web, usually in HTML/XHTML format (the file extensions are typically htm or html) and with hypertext links to enable navigation from one page or section to another. ... See WebCrawler for the specific search engine of that name. ... Meta tags are used to provide structured data about data. ... Google, Inc. ... In computer science, a cache (pronounced kăsh) is a collection of data duplicating original values stored elsewhere or computed earlier, where the original data are expensive (usually in terms of access time) to fetch or compute relative to reading the cache. ... AltaVista home page, 2004 The name AltaVista refers both to an Internet search engine company and to that companys search engine product. ... Link rot is the process by which links on a website gradually become more irrelevant or broken as time goes on, because the websites that are linked to disappear, change content or redirect to a new location. ... Usability is the study (and measure) of the ease with which people interact with man-made tools and systems. ... User expectations refers to the consistency that users expect from products. ... In science, the informal principle of least astonishment, also known as the principle of maximum boredom, states that the explanation which is the least astonishing and which is the most boring is usually (but not always) the right one. ...

When a user comes to the search engine and makes a query, typically by giving key words, the engine looks up the index and provides a listing of best-matching web pages according to its criteria, usually with a short summary containing the document's title and sometimes parts of the text. Most search engines support the use of the boolean terms AND, OR and NOT to further specify the search query. An advanced feature is proximity search, which allows you to define the distance between keywords. In general, a query is a form of questioning, in a line of inquiry. ... In computer science, a keyword is an identifier which indicates a specific command. ... Proximity Search is an advanced search option used by the Yahoo! and Walhello search engine. ...

The usefulness of a search engine depends on the relevance of the results it gives back. While there may be millions of Web pages that include a particular word or phrase, some pages may be more relevant, popular, or authoritative than others. Most search engines employ methods to rank the results to provide the "best" results first. How a search engine decides which pages are the best matches, and what order the results should be shown in, varies widely from one engine to another. The methods also change over time as Internet usage changes and new techniques evolve. This article needs cleanup. ...

Most Web search engines are commercial ventures supported by advertising revenue and, as a result, some employ the controversial practice of allowing advertisers to pay money to have their listings ranked higher in search results. Generally speaking, advertising is the paid promotion of goods, services, companies and ideas by an identified sponsor. ...

The vast majority of search engines are run by private companies using proprietary algorithms and closed databases, the most popular currently being Google, MSN Search, and Yahoo!. There does exist open-source search engine technology such as ht://Dig, Nutch, Egothor, and OpenFTS [1] (http://www.searchtools.com/tools/tools-opensource.html), but there is no publically-available world wide web search server using this technology. Google, Inc. ... MSN Search is a search engine that allows you to search for web pages, news, products, groups (see MSN Groups), images and Microsofts encyclopedia, Encarta. ... ... Open source refers to projects that are open to the public and which draw on other projects that are freely available to the general public. ... Htdig is a search engine issued under the GPL License. ... Nutch is an effort to build an open source search engine. ... Egothor is an open source search engine implementation written entirely in Java to ensure cross platform compatibility. ... OpenFTS is an open source database search engine based on PostgreSQL. It is distributed under the GNU General Public License. ...

Keyword Searching

Searching for keywords is used as the basic functionality to search in large collections of documents, including the web. A user specifies keywords and the search engine tries to find the documents that best matches these keywords. In addion to keywords also often the boolean terms AND, OR and NOT can be used to further specify the search query. Boolean search is supported by most search engines. An advanced feature is proximity search, which allows you to define the distance between keywords. Proximity Search is an advanced search option used by the Yahoo! and Walhello search engine. ...

See also

• Data mining
• History of the Internet
• List of search engines
• Metasearch engine
• Search engine optimization
• Search engine spammer
• Web indexing

Data mining, also known as knowledge-discovery in databases (KDD), is the practice of automatically searching large stores of data for patterns. ... ... This is a list of notable World Wide Web search engines A9 Alexa Internet AltaVista AllTheWeb Ask Jeeves Australian Search Engine (WebSearch. ... A metasearch engine is a search engine that sends user requests to several other search engines and returns the results from each one. ... Search engine optimization (SEO) is a set of methodologies aimed at improving the visibility of a website in search engine listings. ... A search engine spammer is a person who uses techniques that are deemed unfair in order to increase his/her web sites position within the search results of a search engine. ... Web indexing (or Internet indexing) includes back-of-book-style indexes to individual websites or web documents (these are relatively rare), and the creation of metadata (subject keywords) to provide a more useful vocabulary for Internet search engines. ...

External links

• Articles and information on search engines and how they work (http://www.searchengineforums.com)
• On Search, the Series (http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC), Tim Bray, 2003 — A series of essays on search engine techniques.
• searchtools.com, a comprehensive list of free and commercial search software (http://www.searchtools.com/)
• Directory of Internet search engine resources (http://www.searchenginefinder.com/)