Joe Frazier, the former heavyweight boxing champ who died of liver cancer on Nov. 7 at 67, won't go down in history as the greatest fighter of all time. Muhammad Ali, the man with whom Frazier sparred so epically, both inside and outside the ropes, owns that distinction. Frazier's role in his rival's outsize life will always define his own legacy: it's impossible to mention "Smokin' Joe" without summoning Ali a few seconds later.
But if Ali defined Frazier, well, Frazier made Ali too. If not for Frazier's greatness his left hook crumbled opponents, and he defended his heavyweight title four times from 1970 to '73 Ali could never have been called the Greatest. And though the annals of boxing won't remember him as the better fighter, at times Frazier could be the bigger man.
Ali feared Frazier, and that insecurity brought out the worst in him. During the height of their rivalry in the racially charged postcivil rights 1970s, Ali belittled Frazier whenever he could. He'd call Frazier an "Uncle Tom," "ignorant," "the Gorilla." In black communities, Ali characterized Frazier as the white man's champ. "I'm not just fightin' one man," Ali bellowed before their first bout, in 1971, the "Fight of the Century" at New York City's Madison Square Garden. "I'm fightin' a lot of men, showin' a lot of 'em here is one man they couldn't conquer. My mission is to bring freedom to 30 million black people. I'll win this fight because I've got a cause. Frazier has no cause. He's in it for the money alone." (Frazier won the bout in a 15-round decision.)
Frazier, who was inelegant, introspective and prone to mood swings that he called the slouchies, rarely rose to Ali's bait. "I don't want to be no more than no more than what I am," he once said. Friends wondered whether Frazier paid any mind to the social injustices that Ali harped on. Ali relished his role as cultural provocateur; his preaching, as much as his pugilism, is why he is revered. Still, Ali never had reason to use Frazier as a comic foil, especially since the shots he took were far from funny. "Ali can't touch me," Frazier said, "in ability or decency."
Joe Frazier grew up in Beaufort, S.C., where he was raised in a four-room shack on a farm, the second youngest of 13 children. He threw his first punches against a feed bag stuffed with rags, hung from an oak tree. Frazier told his siblings he'd be the next Joe Louis. "I'd hit that heavy bag for an hour at a time," he once said. "I'd wrap my hands with a necktie of my Daddy's, or a stocking of my Momma's or sister's, and get to it." At school, kids would give him a quarter or a sandwich to walk with them as a repellent against bullies.
Ali portrayed Frazier as some sort of puppet of the white man, but in truth, Jim Crow sent Frazier fleeing from South Carolina. "Son," Frazier's mother told him, "if y'all can't get along with the white man in the South, y'all better leave home." A teenage Frazier hitchhiked to Charleston and, as he said, "caught the first thing smokin' that was goin' north." Frazier settled in Philadelphia, where he took a job as a butcher in a kosher slaughterhouse. He caught the eye of a fight manager at a local Police Athletic League, and lost only one of his amateur fights, to Buster Mathis at the trials for the 1964 Olympics. Mathis got hurt, however, and the trip to the Tokyo Games fell to Frazier. Despite fighting his final match with a broken thumb, Frazier came home with the heavyweight gold.
The medal didn't make Frazier rich: after Tokyo, he took a job as a janitor in a North Philadelphia Baptist church. But he soon found some financial backing and turned pro in 1965. With Ali stripped of his boxing license because of his refusal to serve in Vietnam, Frazier soared through the heavyweight ranks and won the world title in 1970. But that same year, Ali returned to the ring; their first face-off the Fight of the Century came on March 8, 1971.
TIME wrote before the fight: "No amount of bluster is likely to deter Smokin' Joe, a raging, bobbing, weaving, rolling swarmer who moves in one basic direction-right at his opponent's gut. A kind of motorized Marciano, he works his short arms like pistons, pumping away with such mechanical precision that he consistently throws between 54 and 58 punches each round. He works almost exclusively inside, crouching and always moving in to slam the body. When the pummeling begins to slow his opponent, when the guard drops to protect the stomach, Frazier tosses a murderous left hook to the head. His coup de grâce is lethal. 'Getting hit by Joe,' says Light Heavyweight Ray Anderson, one of Frazier's sparring partners, 'is like getting run over by a bus.' Some of his victims, like Light Heavyweight Champion Bob Foster, literally have no recollection of what hit them."
In typically understated fashion, Ali labeled the fight "the biggest sporting event in the history of the whole planet earth." It was the first time two undefeated heavyweight champs had met for the title. Ed Sullivan, Alan Shepard, Bill Cosby, Michael Caine, Hubert Humphrey and Burt Bacharach were among the luminaries at ringside. Frank Sinatra took pictures for LIFE magazine. The fight lived up to the billing. Frazier, the body puncher, came out swinging for Ali's head. Ali, the ring dancer, tried matching Frazier hook-for-hook. Ali turned up the showmanship: he invited Frazier to swing at his gut, and when Frazier connected, he'd shake his head, as if a little kid were punching him. "Nooo contest," Ali crowed at one point.
In the 11th round, however, Frazier pummeled Ali with two left hooks. Ali staggered and barely survived the round. In the 15th and final stanza, Frazier landed one more roundhouse left, sending Ali to the canvas. He got back up, but by that point it was finished: Frazier won the fight on a unanimous decision.
It was the only time he beat Ali. Frazier lost his championship belt to George Foreman, who knocked Frazier down six times before the ref stopped their 1973 title fight in the second round ("Down goes Frazier! Down goes Frazier! Down goes Frazier!" Howard Cosell memorably cried.) The next year, Ali got his rematch with Frazier, and won it in a decision to set up their rubber match, in Manila, on Oct. 1, 1975. The "Thrilla in Manila" took place in 100°F heat before an estimated 700 million closed-circuit and television viewers in some 65 countries. It became the duo's most famous brawl. Frazier refused to wear down, but by the 14th round, Ali was pounding him at will. Frazier's eyes were almost swollen shut. Frazier's trainer, Eddie Futch, threw in the towel at the end of the round. "I want him, boss," Frazier screamed. Futch refused. "It's all over," Futch replied. "No one will forget what you did here today." He was right. Afterward, Ali said he had never felt closer to death. He described Frazier as "the greatest fighter of all time, next to me."
Frazier lost to Foreman one more time, in 1976, and attempted an early 1980s comeback, thankfully short-lived. He started a musical act, Smokin' Joe and the Knockouts; that didn't last long either. He opened up a gym in North Philadelphia, and like too many ex-fighters he fell on hard times. "Over the years, Frazier has lost a fortune through a combination of his own generosity and naïveté," read a 2006 profile in the New York Times, "his carousing, failed business opportunities and deep hatred for his former chief boxing rival, Muhammad Ali."
After their fighting days, Frazier matched Ali's past unseemliness with some hurtful remarks of his own. "Look at him now," Frazier told writer Thomas Hauser for his 1992 book on Ali. "He's damaged goods. I know it; you know it. Everyone knows it ... He was always making fun of me. I'm the dummy; I'm the one getting hit in the head. Tell me now, him or me: Which one talks worse now?" In 1996, after Ali lit the Olympic torch at the Atlanta Games, Frazier told a group of reporters, "I wish Ali had fallen into [the flame]. If I had the chance, I'd have pushed him in." Such comments did not endear Frazier to any corporate sponsors.
But in recent years, Frazier's bitterness faded. "Nobody has anything but good things to say about Muhammad now," Frazier told SI.com in 2009. "I'd do anything he needed for me to help." A few years ago, the pair conducted a photo shoot together at Frazier's gym, which is now shuttered. The day before Frazier's death, Ali said in a statement: "My family and I are keeping Joe and his family in our daily prayers. Joe has a lot of friends pulling for him, and I'm one of them."
Frazier lost this last fight. But in so many others, he thrilled the world.