Originally an alternate for the Games, Frazier knocked out his first three opponents then won the gold-medal bout despite a broken thumb

By Zach Miles

Joe Frazier was on the cusp of the biggest step of his young boxing career. The youngest of 12 children from a quiet town on the South Carolina coast, the 20-year-old had advanced to the finals of the 1964 U.S. Olympic Trials, one win away from competing in the Tokyo 1964 Olympic Games.

It was a long way from the Philadelphia slaughterhouses, where Frazier had worked since age 16, occasionally sparring with sides of beef. It was even farther from being in the Frazier family’s living room and mimicking his boxing idols during their televised bouts.

But in the heavyweight final, Frazier ran into a wall – 300-pound Buster Mathis, the only boxer to beat Frazier during his amateur career. Frazier lost on a decision. Mathis earned Team USA’s berth in the Tokyo Games; Frazier would make the trip as well, but only as a reserve.

While training in Tokyo, however, Mathis, broke his knuckle. Frazier would get his chance to compete in the Olympics after all.

Frazier won his opening two bouts with first-round knockouts. In the semifinals, he knocked out the Soviet Union’s Vadim Yemelyanov in the second round, but suffered a broken left thumb in the process. So close to Olympic glory, though, there was not time for any discussion – Frazier would have to battle through the injury.

The gold-medal fight proved daunting for Frazier. His signature left hook, perhaps lacking its normal power because of the broken thumb, did not phase 30-year-old German Hans Huber. For the first time in Tokyo, Frazier was forced to go the distance but he gutted out a 3-2 judges’ decision to win.

“I got what I came to Tokyo for,” Frazier said, though it was clear he felt he had some unfinished business even after winning Team USA’s only boxing gold in Tokyo.

“Could I take Mathis? You bet I could. I could take him tonight or tomorrow.”

The fairy tale of winning Olympic gold, however, wore off as soon as Frazier arrived home in Philadelphia.

“Joe came back from Tokyo, and there was no one there to greet him at the airport,” longtime boxing writer Bernard Fernandez said. “He had a broken hand and had to go right back to working in the slaughterhouse … which certainly wouldn’t happen today.”

Getting back to work was nothing new to Frazier. The son of sharecroppers, he had worked in the fields with his parents and siblings while growing up, moving to New York City to live with an older brother at age 15, then to Philadelphia a year later.

Now he was an Olympic champion, working in the slaughterhouse while his broken thumb healed.

Frazier turned professional the following year, backed by a collection of 80 investors that called itself the Cloverlay Group. It was a play on words: cloverleaf for good luck and overlay, a betting term for having favorable odds. The agreement stipulated that Frazier would take home 70 percent of his winnings and the investors would split the remaining 30 percent.

In 1968, Frazier finally got another shot at Mathis, knocking him out to improve to 20-0 as a pro and win the first title of his pro career, the New York State Athletic Commission heavyweight belt. Three years later, Frazier took on Muhammad Ali in the what became known as the Fight of the Century at New York’s Madison Square Garden. Both fighters entered undefeated in their pro careers, but it was Frazier who scored a unanimous decision in a 15-round bout that went the distance, winning $2.5 million and returning quite an investment for the members of the Cloverlay Group.

Seven years after returning home from Tokyo to an empty Philadelphia airport, Frazier stood atop the boxing world as the heavyweight champion on America’s biggest boxing stage.

“There was never a doubt in any of our minds that Joe would win,” Cloverlay member Joe Hand Sr. said. “For him to lose, you’d have had to stab him or shoot him … all that anger he’d built up from Ali’s insults. Ali might have been the best fighter, but Joe had the biggest heart.”

Frazier finished his career with only four losses, two to Ali and two to George Foreman – both, like Frazier, Olympic champions and members of the U.S. Olympic & Paralympic Hall of Fame.

Following his own career, Foreman became a boxing trainer and guided two of his children, son Marvis Frazier and daughter Jacqui Frazier-Lyde, as they pursued their boxing careers. Frazier-Lyde starred in the first-ever pay-per-view event headlined by a women’s bout, facing Laila Ali, the daughter of Muhammad Ali.

Frazier was diagnosed with liver cancer and passed away at home in Philadelphia in 2011. His private funeral drew a crowd of 3,500 people, according to Fernandez, including Ali and basketball star Magic Johnson.

Hand started a campaign started to erect a statue of Frazier in Philadelphia.  After some delays, the statue was unveiled in September 2015. 

Frazier’s legacy lives on in Philadelphia, in Olympic history and in the hearts of his competitors, as well. The tribute was much deserved according to one of Frazier’s biggest opponents.

“I am very happy to know Joe is being honored and memorialized in the city he loved, something that is long overdue,” Ali said in a statement to reporters at the time. “Joe was a great boxer and a worthy opponent in the ring. He always brought his best whenever he stepped inside the ropes.”

Zach Miles, a student in the University of Georgia Sports Media Certificate program, is a digital content intern for the U.S. Olympic & Paralympic Museum.