Wikipedia founder Jimmy Wales
Can a guy called "Jimbo," along with an army of volunteer editors and an open source software client named after a worm, take on Google, the most successful Internet company in history?
Before you answer that question, consider that "Jimbo" is Jimmy Wales, the founder of Wikipedia, the most visited reference site on the Internet and the second most visited domain after a Google search. The success of Wikipedia should give us pause before dismissing latest challenge to seemingly unbeatable Google.
In the last two years, visits to Wikipedia have grown over 600%. Hitwise tracks 3,098 educational reference sites, and Wikipedia accounted for over 30% of visits to all sites in the category. Even more intriguing is the interdependence between Wikipedia and Google. Wikipedia is the second most visited site from Google searches (MySpace.com is number one), due mostly to the sheer volume and prominent placement of Wikipedia listings in Google search-engine results. Google is responsible for 46% of Wikipedia's traffic. How could Jimbo possibly best Google, which was, according to Hitwise, responsible for 64.4% of all Internet searches in the U.S. last week?
In a July 27 speech at the O'Reilly Open Source Convention, Wales announced his plans to take on the current state of search, which he believes is in need of transparency rather than "black box" algorithms to determine what results are presented to users. As he explains on his company's site, Wikia.com, search "is part of the fundamental structure of the Internet and, it is currently broken." For-profit Wikia was started in 2004 as the commercial version of not-for-profit Wikipedia, but has now shifted its focus to search. Community involvement through volunteer editors, the backbone of Wikipedia, is clearly in Wales' plans. He revealed a second part of his strategy through his acquisition from Looksmart of the curiously named Grub, an open source distributed search crawler index.
Reminiscent of the early days of Google's index, built by thousands of daisy-chained personal computers, the Grub Project works by using participants' idle computer power. If enough users volunteer their idle computer time to index the Web, hundreds of thousands of PCs could possibly match the indexing power of first-tier search engines such as Google. The idea of this kind of distributed computing first gained notoriety with the SETI@home project, launched in late 1999. The project set out to find extraterrestrial life through a novel program that launched a screensaver when you weren't using your computer. Behind the scenes, the program was retrieving data files to analyze, returning the results to a main server. Since its inception, the SETI@Home project has accumulated over five million volunteer clients, with over one million active as of the beginning of this year.
If Wales can grow his volunteer base of previously idle computers as he has grown the editorial manpower of Wikipedia, perhaps a feasible search index is within grasp. Indexing the Internet, however, is the least of Jimbo's problems. Search engines rely on their algorithms, or complex formulas, to determine what listings to return for a searcher's query. Wales' answer to a better search experience is to combine a computer algorithm with editors who monitor what results should be returned for any given search. But can a viable search engine rely on the altruistic motives of its volunteer keepers? As we discussed here, the anonymity of contributors to such a model can create a product vulnerable to sabotage and subterfuge.
Problems aside, you have to admire Jimbo's chutzpah in challenging the state of the search industry. Jimmy, I only wish I had some idle computer time to donate, unfortunately I'm still busy searching for extraterrestrials.
Bill Tancer is general manager of global research at Hitwise.