The funny thing about the Web as a medium is that it's always been plagued by too much content, rather than too little. A now-infamous report issued last summer by the NEC Research Institute announced that for all their inflated stock prices, the major search engines were only covering at best a fraction of the Web. Forrester Research estimates the size of the web at 500-600 million pages; AltaVista, which claims to index the most pages of any major search engine, only covers 150 million. The best bet, the report stated, may be search engines like Metacrawler, which simply aggregate search results from other engines.MORE >>
The truth is out there -- the real question is, where the hell is it in all this crap? Search has always been an integral part of how people use the Web, since its earliest days, and the Web's first big brands -- Yahoo, Excite, Lycos -- were all search engines. But can they still cut the mustard? A new generation of search services is springing up, with names like Google and FAST, armed with next-generation technology, and they say they have the power to supplant their elders and finally make sense of the Web.