Face the Music

For police officers and poker players, there are few things worse than a cool liar. While most of us have little trouble coming up with a good fib when we need one, no sooner do we start to tell it than we give ourselves away, exhibiting all sorts of facial tics and nervous mannerisms that reveal just how uncomfortable we are with the story we're telling. Truly gifted dissemblers, however, reveal very little of this, lying so easily and skillfully that even the most well-trained eye wouldn't notice a thing. If cops and card players can be fooled, though, it's now possible that another type of sophisticated lie detector can't: the computer.

Scientists at the Salk Institute in La Jolla, Calif., have developed a computer system that has learned to read the rapidly changing expressions in a human face and may one day be able to draw conclusions about the emotions behind them. Get such a system out of the lab and into a police station, and the business of lie detection and law enforcement could change for good.

The trick of developing a computer that can understand faces was not to try to replicate the elusive mental processes human beings use to make judgments about one another. Despite the computer's ability to calculate the trajectories of spacecraft or pick the next move in a chess game, the machines have until now been flummoxed by crude recognition tasks that even a baby can perform, often failing to distinguish between a beach ball and a cabbage, to say nothing of picking out a familiar face in a photo album filled with strangers. Such a pattern-recognition talent, says Salk Institute neuroscientist Terrence Sejnowski, in whose lab the lie detection work was done, "is a survival skill humans probably had even before they acquired language. For computers, it's a major challenge."

To get around this, the Salk researchers based the design of their face-recognizing computer on the one thing all computers do well: acquiring, storing and analyzing masses of information at lightning speed. In the 1970s, psychologist Paul Ekman and his colleagues at the University of California, San Francisco, developed a classification of 46 muscle movements that appear to account for the entire spectrum of human facial expressions. The movements--or action units, as Ekman called them--range from the slight crow's-feet crinkling around the eyes that accompanies a smile to the contraction of forehead muscles that is an integral part of a scowl. "Some of these movements are so difficult that they're impossible to fake unless you're a very skilled actor," says Marian Bartlett, a post-doctoral fellow in Sejnowski's lab.

To help catch these actors in the act of deception, Bartlett scanned pictures of faces into the computer and wrote instructions that taught the machine to recognize six of Ekman's coded movements: the fleeting grimace or scowl, for example, that may precede a liar's counterfeit smile. When the computer was later presented with other, unfamiliar pictures and videotapes, it showed a remarkable ability to apply what it had learned, detecting similar flickers in the new pictures and even outperforming human volunteers who competed with the machine to spot the same telltale twitches.

Of course, identifying facial twitches is not the same as reading states of mind, particularly when the computer can make sense of only half a dozen expressions. But the newly trained machine should get a lot smarter in the not-too-distant future. In the next round of experiments, the scientists plan to expand the computer's recognition skills by teaching it to identify all of the 46 muscle actions in the Ekman catalogue. They will then program the computer to recognize the various combinations of these movements, pouring live video images of human volunteers directly into the machine.

The research at Sejnowski's and other labs could lead to a lie detector more reliable and less intrusive than existing polygraphs, which measure reactions like heartbeat and perspiration that clever subjects can control. Says Bartlett: "It would spot in an instant any facial movement that indicated a conflicting emotion, like a beginning of a scowl quickly covered up by a smile." The CIA is funding a study linking the Salk Institute's efforts with similar work at the University of Pittsburgh.

Where the technology could ultimately lead is difficult to say, but Sejnowski anticipates big things. Ekman often used videotapes to gauge the emotional states of subjects, once detecting a brief flicker of sadness in the face of a patient who later turned out to be suicidal. A computer like Sejnowski's could have made the diagnosis in real time. Farther down the road could be a host of other emotion-measuring computer systems, ranging from smart atms that can shut down if they spot a suspicious patron to television systems that can determine if a finger-wagging politician is telling the truth. Privacy advocates will no doubt have much to say about all this, but the technology may be on the way. "You can be sure it's coming," Sejnowski says flatly. Whether humanity is ready for it is another question.