Palmer, 77, died Saturday at George Washington University Hospital of pulmonary fibrosis, according to his wife, Nancy.
"God bless John Palmer, tireless reporter, always a gentleman, loving husband and doting father," former NBC News anchor Tom Brokaw wrote on Twitter. He said the death of his friend of nearly 50 years was "heartbreaking."
Palmer worked for NBC from 1962 to 1990, and then returned to the network from 1994 until 2002. He became a familiar face to viewers of the "Today" show during much of the 1980s, delivering the news in a straightforward, no-nonsense manner at a time when the program often led in the ratings.
NBC News praised Palmer in a statement Saturday as a "brilliant, brave, and tireless journalist who guided viewers through many of the most significant events of the past half-century - from the early days of the civil rights movement through the tragedy of 9/11."
"He covered five presidents and traveled to every corner of the world, always showing the empathy and compassion that helped set him apart," the statement said. "His kindness is remembered by all of us, and it built lasting bonds throughout our news division."
A native of Kingsport, Tenn., Palmer was a graduate of Northwestern University and held a master's degree from Columbia University.
He got his start as a reporter in Atlanta in 1960, according to a news segment on Palmer that aired on the NBC Nightly News on Saturday, and two years later moved up to the network.
In the 1970s, he was based in Beirut, covering the 1973 Arab-Israeli war, the war in Cyprus and the civil war in Angola. He later served as a correspondent in Paris and at the White House.
In April 1980 he landed one of his biggest scoops, breaking the news of the Carter administration's failed attempt to rescue the American hostages being held in Iran. Eight U.S. servicemen died when a helicopter crashed into a C-130 transport plane at a staging area in Iran.
"The rescue mission had been aborted and eight Americans had died," Palmer told viewers.
His reporting on the story brought him the Merriman Smith Memorial Award for excellence in presidential news coverage, making him the first broadcast journalist to receive that honor.
"John Palmer brought to the White House beat his foreign policy experience and a steady reassuring voice, in good times and in bad," NBC News correspondent Andrea Mitchell said in the tribute to Palmer that aired Saturday night.
It was also at NBC's Washington news bureau that Palmer met his wife, Nancy, a Nightly News producer.
In 1982 he became news anchor on the "Today" show during the highly successful run of Jane Pauley and Bryant Gumbel as co-hosts. He remained there until 1989, when he was abruptly replaced by Deborah Norville, who was being groomed for a co-host role, and handed her old job on the show that preceded it, "NBC News at Sunrise." Norville succeeded Pauley shortly afterward but was herself replaced after the show's ratings plummeted.
In 1986, Palmer anchored the first hours of NBC's coverage of the Challenger space shuttle disaster, as well as other special reports.
Palmer left the network in 1990 to anchor a syndicated news program called "Instant Recall," interviewing figures such as Carter, Presidents Gerald Ford and Ronald Reagan, Anwar Sadat, Jonas Salk and Chuck Yeager.
He also hosted the weekly "Discovery Journal" on The Discovery Channel, and anchored a daily newscast on the TV channel of the Christian Science Monitor.
In 1994, he was invited back to NBC as a Washington-based national correspondent.
Speaking to anchor Brian Williams on NBC sister network MSNBC upon his retirement in 2002, Palmer looked back on his tenure with satisfaction, including the access it gave him to a succession of the nation's chief executives.
"I was enriched as a kid from the East Tennessee mountains," said the Kingsport, Tenn., native, "to be able to go fishing with Jimmy Carter, to go to the movies with Ronald Reagan, and to play golf with Bill Clinton."
After exiting NBC, he continued to work in journalism, including through hosting roles on Retirement Living TV, a network dedicated to seniors.
Several of Palmer's colleagues praised him on Twitter on Saturday evening.
Major Garrett, chief White House correspondent for CBS News, described Palmer as "a brilliant, competitive, ethical and gentle man."
Susan Page, Washington bureau chief of USA Today, said she first met Palmer at the Reagan White House and described him as "one of the kindest/most joyful reporters on the beat."
NBC News reporter Chuck Todd said Palmer "epitomized professionalism at NBC News. And he was always generous with advice and counsel. A great news man."
On Saturday's Nightly News segment, Andrea Mitchell observed that her longtime colleague "loved Sinatra, golf and fishing," but most treasured being a husband and father.
He and his wife Nancy have three grown daughters, one of whom is a producer for the Today show, one who works in the entertainment industry and one who is in Washington pursuing a journalism career.
AP writer Norman Gomlak in Atlanta contributed to this report.