Ken Norton Jr., a coach with the Seattle Seahawks, confirmed the death to The Associated Press before handing the phone to his wife, too distraught to talk.
Norton, the only heavyweight champion never to win the title in the ring, had been in poor health for the last several years after suffering a series of strokes, a friend of the fighter said.
"He's been fighting the battle for two years," said Gene Kilroy, Ali's former business manager. "I'm sure he's in heaven now with all the great fighters. I'd like to hear that conversation."
Norton broke Ali's jaw in their first bout, beating him by split decision in 1973 in a non-title fight in San Diego. They fought six months later, and Ali won a split decision.
They met for a third time on Sept. 28, 1976, at Yankee Stadium and Ali narrowly won to keep his heavyweight title.
Norton would come back the next year to win a heavyweight title eliminator and was declared champion by the World Boxing Council. But on June 9, 1978, he lost a bruising 15-round fight to Larry Holmes in what many regard as one of boxing's epic heavyweight bouts and would never be champion again.
"Kenny was a good, good fighter. He beat a lot of guys," said Ed Schuyler Jr., who covered many of Norton's fights for The Associated Press. "He gave Ali fits because Ali let him fight coming forward instead of making him back up."
Norton finished with a record of 42-7-1 and 33 knockouts. He would later embark on an acting career, appearing in several movies, and was a commentator at fights.
"So saddened by the passing of Ken Norton Sr. and sending condolences to the Norton family," Seahawks coach Pete Carroll tweeted. "This hits close to home for all of us here."
Ken Norton Jr. was a linebacker for 13 years in the NFL, playing for Dallas and San Francisco, and coaches the position for the Seahawks. He and his father were estranged for a time in the 1990s before finally reconciling.
Norton always gave his father credit for his career, saying he learned how to train hard by watching him go for early morning runs when he was a child.
"It's been noted that my father and I are on speaking terms and everything's back together now," Norton Jr. said in 1995. "It's part of what I do. No matter what I do, I can't get away from boxing."
Norton, born Aug. 9, 1943, in Jacksonville, Ill., started boxing when he was in the Marines, and began his pro career after his release from duty in 1967. He lost only once in his early fights but had fought few fighters of any note when he was selected to meet Ali.
At the time, Ali was campaigning to try to win back the heavyweight crown he lost to Joe Frazier in 1973.
Few gave Norton, who possessed a muscular, sculpted body, much of a chance against Ali in the fight, held at the Sports Arena in San Diego, where Norton lived. But his awkward style and close-in pressing tactics confused his opponent, and Norton broke Ali's jaw on the way to the decision that put him in the top echelon of heavyweight fighters.
"Ali thought it would be an easy fight," Kilroy said. "But Norton was unorthodox. Instead of jabbing from above like most fighters he would put his hand down and jab up at Ali."
Kilroy said after the fight Norton visited Ali at the hospital where he was getting his broken jaw wired. Ali, he said, told him he was a great fighter and he never wanted to fight him again.
Instead, they would meet two more times, including the final fight at Yankee Stadium on a night when police were on strike and many in the crowd feared for their safety. The fight went 15 rounds and Ali won a decision.
Kilroy said Ali and Norton never had any animosity toward each other and became good friends over the years. Still, Norton always thought he had won all three fights.
Norton would come back in 1977 to win an eliminator against Jimmy Young and was declared champion by the WBC when Leon Spinks was stripped of the title after deciding to fight Ali in a rematch instead of defending his new title against the mandatory challenger.
His fight against Holmes in 1978 at Caesars Palace was his last big hurrah, with the two heavyweights going back and forth, trading huge blows inside a steamy pavilion in the hotel's back lot. The fight was still up for grabs in the 15th round and both fighters reached inside themselves to deliver one of the more memorable final rounds in heavyweight history.
When the decision was announced, two ringside judges favored Holmes by one point while the third favored Norton by a point.
Norton was badly injured in a near fatal car accident in 1986. He recovered but never regained his full physical mobility.
"The doctors said I would never walk or talk," Norton said at an autograph session in 2011 in Las Vegas, lifting his trademark fedora to show long surgical scars on his bald head.
Kilroy said Norton was visited at the veteran's hospital in the Las Vegas suburb of Henderson by former fighters, including Mike Tyson, Earnie Shavers and Thomas Hearns.
Norton fought only five more times after losing his title to Holmes. His final fight came Nov. 5, 1981, when he was knocked out in the first round by Gerry Cooney at Madison Square Garden.
Information on services and other survivors was not immediately released by the family.
