Data Miners

(2 of 3)

The biggest winner so far appears to be Autonomy, which recently won a contract initially worth about $3 million to provide the software for information collection, analysis and routing for the 22 agencies that fall under the new Department of Homeland Security (DHS). But few other contracts have been signed thus far, as the feds slog through the swampy process of budgeting, appropriating and procuring. "We've all been waiting for the wheelbarrows of money to show up," says Leonard Pomata, president of the government division of webMethods, based in Fairfax, Va. "Aside from the emergency funding that has been spent on guards and gates and guns ... there hasn't been a significant amount for new initiatives."

That should change in the coming months, once DHS Secretary Tom Ridge has time to survey his new dominion. As details of the government's actions before 9/11 continue to unfold, two immediate needs become clear. The U.S. needs better ways of uncovering hidden connections within the masses of information it collects from different sources. And it needs to make sure that information stored within one agency's database can be shared with the appropriate officials elsewhere.

Just a few years ago, such goals would have been laughable. But innovations in artificial intelligence let the government recognize and quantify links between disparate forms of data. New software languages automatically translate information from different systems into a common tongue. Both advances have vastly expanded the tools available for the war against terrorism.

Throughout the '90s, data mining spread from one industry to the next, enabling companies to know more about customers' needs and to zero in on the characteristics that distinguish the customers they want from those they do not. A credit-card company using a system designed by Teradata, a division of NCR, found that customers who fill out applications in pencil rather than pen are more likely to default. A major hotel chain discovered that guests who opted for X-rated flicks spent more money and were less likely to make demands on the hotel staff, according to privacy consultant Larry Ponemon. These low-maintenance customers were rewarded with special frequent-traveler promotions. Victoria's Secret stopped uniformly stocking its stores once MicroStrategy showed that the chain sold 20 times as many size-32 bras in New York City as in other cities and that in Miami ivory was 10 times as popular as black. Aspect Communications, based in San Jose, Calif., sells a program that identifies callers by purchase history. The bigger the spender, the quicker the call gets picked up. So if you think your call is being answered in the order in which it was received, think again.

The technology that underlies these applications, known as customer-relationship management (CRM), is a subset of data mining and can be used by any organization that needs to quickly analyze massive amounts of data. "More and more often," says Michael Schiff, vice president of Current Analysis, a business-intelligence firm based in Sterling, Va., "you see it becoming 'X'RM, where the relationship being managed could be with a customer or supplier or stockholder or, to go out on a limb here, terrorist."

The potential has not been lost on the intelligence community. When CIA agents came calling on digiMine, a retail data-mining specialist based in Bellevue, Wash., they carried a list of 900 companies that were pursuing data-mining capabilities, says CEO Usama Fayyad. Through a nonprofit venture-capital fund that it finances, called In-Q-Tel, the CIA has invested in several data-mining companies that serve both the private sector and the CIA.

One such firm, Systems Research and Development (SRD), based in Las Vegas, uses an algorithm that looks for what are known as nonobvious relationships to flag a casino if one of its employees appears to have a connection to a known cheater. After cleansing databases of misspelled names and aliases, the software looks through the casino's customer databases, as well as public records it acquires, which may contain criminal charges, addresses and phone numbers. Within 90 days of implementation, a Louisiana riverboat casino caught four employees who were helping friends and family members cheat, including a roulette-wheel spinner who had shared a phone number with someone the casino had caught placing bets after the ball had landed. The same data-sifting capabilities, says SRD founder Jeff Jonas, can help police and intelligence agencies make sense of Arabic names that may be transliterated in a dozen ways.

The CIA's investments, typically no more than a couple of million dollars a year, give the agency a chance to tailor products to its needs while boosting the survival rate among small but innovative firms. A CIA investment in Intelliseek, based in Cincinnati, Ohio, accelerated the company's ability to give its software multilingual capabilities. In addition to creating a more useful product for CIA analysts, the investment allowed Intelliseek to enter international markets two years faster than it had planned.